Tiled region adjacency graph computation via pixel-region adjacency graph

Abstract

To provide a system, a computer-implemented method, and a computer program product for facilitating tiled region adjacency graph computation via pixel-region adjacency graphs.SOLUTION: A computer-implemented method includes: accessing an image generated by a scientific instrument; performing second operations constructing, in tile-wise fashion, a region adjacency graph for the image, based on a plurality of pixel-region adjacency graphs; and performing marker-based watershed segmentation on a region adjacency graph of the image. The region adjacency graph can be constructed from a plurality of pixel-region adjacency graphs respectively corresponding to a plurality of tiles of the image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Non-Provisional No. 18 / 605,200, filed March 14, 2024, the disclosure of which is incorporated herein by reference in its entirety. [Background technology]

[0002] Various scientific instruments can capture images of samples. It may be desirable to segment such images. Segmentation can become increasingly difficult as the size of such images increases. Summary of the Invention

[0003] The following presents a summary to provide a basic understanding of one or more embodiments. This summary is not intended to identify key or critical elements or to delineate the scope of any particular embodiments or the claims. Its sole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. In one or more embodiments described herein, a device, system, computer-implemented method, apparatus, or computer program product that facilitates computation of a tiled region adjacency graph via a pixel region adjacency graph is described.

[0004] According to one or more embodiments, a system is provided. The system may include a non-transitory computer-readable memory capable of storing computer-executable components. The system may further include a processor operably coupled to the non-transitory computer-readable memory and capable of executing the computer-executable components stored in the non-transitory computer-readable memory. In various embodiments, the computer-executable components may include an access component capable of accessing an image generated by a scientific instrument. In various aspects, the computer-executable components may include an execution component capable of performing marker-based watershed segmentation on a region adjacency graph of the image, where the region adjacency graph may be constructed from a plurality of pixel region adjacency graphs corresponding to a plurality of tiles of the image, respectively.

[0005] According to one or more embodiments, a computer-implemented method is provided. In various embodiments, the computer-implemented method can include accessing, by a device operatively coupled to a processor, an image generated by a scientific instrument, and performing, by the device, marker-based watershed segmentation on a region adjacency graph of the image, where the region adjacency graph can be constructed from a plurality of pixel region adjacency graphs corresponding respectively to a plurality of tiles of the image.

[0006] According to one or more embodiments, a computer program product is provided for facilitating computation of a tiled region adjacency graph via a pixel region adjacency graph. In various embodiments, the computer program product may comprise a non-transitory computer-readable memory having program instructions embedded therein. In various aspects, the program instructions may be executable by a processor to cause the processor to access an image captured by a charged particle microscope and to construct a region adjacency graph for the image in a tile-by-tile manner based on a plurality of pixel region adjacency graphs. [Brief explanation of the drawings]

[0007] Various embodiments will be readily understood from the following detailed description taken in conjunction with the accompanying drawings. To facilitate this description, like reference numerals refer to like structural elements. The embodiments are illustrated in the drawings by way of example, not limitation. The drawings are not necessarily drawn to scale. [Figure 1] FIG. 1 shows an exemplary, non-limiting block diagram of a scientific instrument module according to various embodiments described herein. [Figure 2] 1 illustrates an exemplary, non-limiting flow diagram of a computer-implemented method according to various embodiments described herein. [Figure 3] FIG. 1 illustrates a block diagram of an exemplary, non-limiting system that facilitates computation of a tiled region adjacency graph via a pixel region adjacency graph, in accordance with one or more embodiments described herein. [Figure 4] FIG. 1 illustrates a block diagram of an exemplary, non-limiting system including multiple tiles that facilitates computation of a tiled region adjacency graph via a pixel region adjacency graph, in accordance with one or more embodiments described herein. [Figure 5] 1 shows an exemplary, non-limiting block diagram of multiple tiles according to one or more embodiments described herein. [Figure 6] 1 shows an exemplary, non-limiting block diagram of multiple tiles according to one or more embodiments described herein. [Figure 7] 1 shows an exemplary, non-limiting block diagram of multiple tiles according to one or more embodiments described herein. [Figure 8] FIG. 1 illustrates a block diagram of an exemplary, non-limiting system including a region adjacency graph that facilitates computation of a tiled region adjacency graph via a pixel region adjacency graph, in accordance with one or more embodiments described herein. [Figure 9] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 10] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 11] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 12] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 13] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 14] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 15] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 16] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 17] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 18] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 19] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 20] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 21] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 22]1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 23] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 24] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 25] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 26] 1 shows an exemplary, non-limiting block diagram for constructing a region adjacency graph in accordance with one or more embodiments described herein. [Figure 27] FIG. 1 shows a block diagram of an exemplary, non-limiting system including one or more marker-based watershed segmentation basins that facilitate the computation of a tiled region adjacency graph via a pixel region adjacency graph, according to one or more embodiments described herein. [Figure 28] 1 illustrates an exemplary, non-limiting algorithm that facilitates computation of a tiled region adjacency graph via a pixel region adjacency graph, according to one or more embodiments described herein. [Figure 29] 1 illustrates an exemplary, non-limiting algorithm that facilitates computation of a tiled region adjacency graph via a pixel region adjacency graph, according to one or more embodiments described herein. [Figure 30] 1 illustrates an exemplary, non-limiting algorithm that facilitates computation of a tiled region adjacency graph via a pixel region adjacency graph, according to one or more embodiments described herein. [Figure 31] FIG. 1 shows an exemplary, non-limiting block diagram of a graphical user interface that may be used in implementing some or all of the methods or techniques disclosed herein, according to various embodiments described herein. [Figure 32]FIG. 1 illustrates an exemplary, non-limiting block diagram of a computing device in which some or all of the methods or techniques disclosed herein may be implemented, according to various embodiments described herein. [Figure 33] 1 shows an exemplary, non-limiting block diagram of a scientific instrument support system in which some or all of the methods or techniques disclosed herein may be implemented, according to various embodiments described herein. [Figure 34] 1 illustrates a block diagram of an exemplary non-limiting operating environment that can facilitate one or more embodiments described herein. [Figure 35] 1 illustrates an exemplary networking environment operable to perform various implementations described herein. DETAILED DESCRIPTION OF THE INVENTION

[0008] The following detailed description is merely illustrative and is not intended to limit the embodiments or the application / uses of the embodiments, nor is it intended to be bound by any expressed or implied information presented in the preceding "Background" or "Summary" sections, or in the "Detailed Description" section.

[0009] One or more embodiments will now be described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a more thorough understanding of one or more embodiments. It will be apparent, however, that in various instances, one or more embodiments may be practiced without these specific details.

[0010] Various operations may be described sequentially as multiple separate actions or operations in a manner that is most helpful for understanding the subject matter disclosed herein. However, the order of description should not be construed as implying that these operations are necessarily order dependent. In particular, these operations may be performed in an order different from the order presented. The operations described may be performed in an order different from the described embodiment. In additional embodiments, various additional operations may be performed or described operations may be omitted.

[0011] Although some elements may be referred to in the singular (e.g., "processing device"), any suitable element may be represented by multiple instances of that element, and vice versa. For example, a set of operations described as being performed by a processing device may be implemented with different ones of the operations performed by different processing devices. As used herein, the phrase "based on" should be understood to mean "based at least in part on," unless otherwise specified.

[0012] A scientific instrument (e.g., mass spectrometer, charged particle electron microscope) may be any suitable computerized device capable of capturing or generating electronic measurements (e.g., capturing or generating spectroscopic images or compositional spectra) in a scientific, laboratory, research, or clinical operating situation. To facilitate the capture or generation of such electronic measurements, the scientific instrument may affect a complex configuration of actuatable components (e.g., ion sources, ion lenses, heaters, coolers, fluid valves, fluid pumps, circuit switches, sample stages, apertures), sensors (e.g., ion detectors, voltmeters, thermistors, potentiometers, pressure gauges), or consumables (e.g., carrier fluids, calibrants, filters).

[0013] Various scientific instruments (e.g., scanning or transmission electron microscopes, electron energy loss microscopes) can utilize their component actuatable parts to capture images of a sample (e.g., electron tomography images, x-ray tomography images, confocal microscopy images). In some cases, the images may be two-dimensional arrays of pixels. In other cases, the images may instead be three-dimensional arrays of voxels. In either case, the images can be very large, depending on the particular imaging modality used to capture them. For example, scientific instruments often generate images ranging in size from tens to hundreds of gigabytes.

[0014] It may often be desirable to perform segmentation on such images to extract or identify desired structural information about any samples depicted in the images. While convolutional neural networks have demonstrated remarkable success for a variety of image segmentation tasks, there are nonetheless various situations in which non-machine learning segmentation algorithms are preferred or desirable. Indeed, such non-machine learning segmentation algorithms are more transparent and interpretable (e.g., activation maps generated by hidden layers of convolutional neural networks often do not hold easily understandable or explainable meaning) and do not require the collection or annotation of training data (e.g., very large amounts of training data may be required for convolutional neural networks to achieve satisfactory levels of segmentation accuracy). One such non-machine learning segmentation algorithm is the marker-based watershed algorithm.

[0015] Unfortunately, marker-based watershed algorithms have been found to consume excessive computational resources (e.g., processing power, memory storage, and computation time) when implemented on very large images produced by scientific instruments. With respect to various other non-machine learning segmentation algorithms, such excessive consumption of computational resources can be mitigated through tiling, i.e., by dividing the large image into tiles, applying any algorithms in question locally to each tile, and then merging the local algorithm results from each tile into a global algorithm result for the large image. However, tiling cannot currently be used reliably with marker-based watershed algorithms. Specifically, it has been observed that there is often a discrepancy between the local selection of marker-based watershed separation lines when the marker-based watershed algorithm is implemented on individual tiles and the global selection of marker-based watershed separation lines when the marker-based watershed algorithm is implemented on a given image where no tiles exist. In other words, marker-based watershed algorithms, when used in conjunction with tiling, have been found to produce inconsistent segmentation basins (e.g., local marker-based watershed separation lines or boundaries that are assumed to be continuous may become discontinuous after merging tiles). These inconsistencies or inconsistencies may be referred to as "marker-based watershed leakage."

[0016] Rather than applying a marker-based watershed algorithm to an entire large image (e.g., which consumes too many resources) or to individual tiles of a large image (e.g., which causes leakage), the marker-based watershed algorithm can instead be performed on a simplified version of the large image. In particular, it has been found that performing a marker-based watershed algorithm on superpixels of a region adjacency graph (RAG) corresponding to the large image can facilitate accurate segmentation without excessively consuming computational resources. However, because scientific instruments often generate images of very large size, any computerized workstations supporting or otherwise associated with those scientific instruments are often only capable of analyzing or manipulating images in a tiled manner (e.g., such workstations do not have sufficient processing power to perform any type of computationally intensive operation on images that occupy tens or hundreds of gigabytes of storage). Unfortunately, existing techniques are unable to generate RAGs for large images in a tiled context.

[0017] Therefore, systems or techniques that can ameliorate one or more of these technical problems may be desirable.

[0018] Various embodiments described herein may address one or more of these technical problems. One or more embodiments described herein may include a system, computer-implemented method, apparatus, or computer program product that can facilitate the computation of a tiled region adjacency graph via a pixel region adjacency graph. In other words, the inventors of various embodiments described herein have devised various techniques for computing or otherwise generating a RAG for large images in a tiled context. In particular, the inventors have devised a new type of graph structure called a pixel-region adjacency graph (PRAG), and various embodiments described herein can leverage such new type of graph structure to create a RAG for a tiled image. More specifically, various embodiments described herein may include converting each tile of a given image into a pixel adjacency graph (PAG) and converting each PAG into a minimum spanning forest, where trees in the minimum spanning forest are flagged or designated as either boundary trees or interior trees. A boundary tree (which may also be referred to as a boundary region) may be any tree having at least one node located on a boundary of a tile, and an interior tree (which may also be referred to as an interior region) may be any tree having no nodes located on a boundary of a tile. In various aspects, various embodiments described herein may further include converting each minimum spanning forest into a PRAG by maintaining or preserving the boundary regions of the minimum spanning forest and by replacing each interior region of the minimum spanning forest with a single respective node. In various cases, for any two tiles adjacent to each other, at least some of the boundary regions of the PRAGs of the two tiles may be merged together to create new interior regions, and these new interior regions may be replaced by a respective single node as described above.When all of the PRAGs are merged together, the resulting graph structure cannot have any remaining border regions and can therefore be considered the RAG for the given image. Therefore, marker-based watershed segmentation can be performed on the RAG. In this manner, various embodiments described herein can be considered to facilitate the computation or calculation of RAGs in a tiled context or tile-by-tile manner, which is not known to be achieved with existing techniques.

[0019] Various embodiments described herein can be considered as computerized tools (e.g., any suitable combination of computer-executable hardware or computer-executable software) that can facilitate the computation of a tiled region adjacency graph via a pixel region adjacency graph. In various aspects, such computerized tools can comprise an access component, a tile component, a graph component, or an execution component.

[0020] In various embodiments, a scientific instrument may be present. In various aspects, the scientific instrument may be any suitable computerized device capable of electronically capturing or generating images of any suitable sample (e.g., thin section). As some non-limiting examples, the scientific instrument may be a scanning electron microscope, a transmission electron microscope, or an electron energy loss microscope.

[0021] In various aspects, the image can show or otherwise visually depict any suitable physical structure or physical property of the sample, or any suitable portion thereof. In some cases, the image can be a two-dimensional array of pixels, with each pixel having a respective intensity value. In other cases, the image can instead be a three-dimensional array of voxels, with each voxel having a respective intensity value.

[0022] In either case, it may be desirable to perform marker-based watershed segmentation on the image. However, the image may be large or may otherwise consume a substantial amount of computerized memory or storage space (e.g., the image may be tens to hundreds of gigabytes in size). Therefore, it may be desirable to perform such marker-based watershed segmentation on the image on a tile-by-tile basis. The computerized tools described herein can facilitate such tile-by-tile marker-based watershed segmentation.

[0023] In various embodiments, an access component of a computerized tool can access the images electronically. For example, the access component can receive, search, or otherwise retrieve the images from any suitable centralized or distributed data structure (e.g., a graph data structure, a relational data structure, a hybrid data structure). As a non-limiting example, the access component can receive, search, or retrieve the images from the scientific instrument itself. In either case, the access component can be considered a conduit through which other components of the computerized tool can electronically interact with (e.g., read, write, edit, copy, manipulate) the images.

[0024] In various embodiments, a tiling component of a computerized tool can electronically decompose an image into multiple tiles. In various aspects, a tile of an image can be any suitable contiguous collection of pixels (or voxels, as appropriate) of the image. In other words, any two pixels (or voxels) of a tile can be either adjacent to each other or joined together by a chain of one or more other adjacent pixels (or voxels) that are also within the tile. In further words, a tile can be considered a coherent multi-pixel (or multi-voxel) fragment, section, or piece of an image. In various cases, a tile can assume any suitable shape (e.g., its pixels or voxels can be arranged so that the tile has a rectilinear or triangular shape) and any suitable size (e.g., a tile can be composed of any suitable number of pixels or voxels). In various cases, any of the multiple tiles can have the same or different shapes or sizes as one another. In various aspects, the multiple tiles can be disjoint from one another. That is, any of the multiple tiles may not overlap one another. In such cases, the tiles can be considered to fit together like pieces of a jigsaw puzzle to form the image. In other aspects, the tiles may instead not be separate but still be individually unique. That is, at least some of the tiles may partially overlap one another. In such cases, the tiles can be considered to fit together like pieces of a jigsaw puzzle to form the image, provided that overlapping portions of the tiles are not double-counted. In either case, each of the tiles can be smaller than the image itself (e.g., in some cases, by an order of magnitude or more). Thus, analyzing or manipulating individual tiles of the tiles can be considered less computationally intensive than analyzing or manipulating the image as a whole.

[0025] In various embodiments, a graph component of a computerized tool can electronically generate a RAG for an image. In various aspects, the graph component can achieve such generation by utilizing multiple PRAGs, each corresponding to a respective tile of the image. In other words, the graph component can construct a respective PRAG for each of the tiles, and the component can construct a RAG by combining all of the PRAGs.

[0026] More specifically, for each given tile of the plurality of tiles, the graph component can convert the given tile into a pixel adjacency graph (PAG). In particular, the PAG can be a graph data structure including nodes and edges. In various aspects, each pixel (or voxel) of a given tile can be represented by a respective node of the PAG, such that the value of each node can be equal to the intensity value of its respective pixel (or voxel). In various cases, the PAG can include a unique edge between any two nodes representing adjacent pixels (or adjacent voxels) of the tile. In various cases, nearest neighbors can be implemented for a pixel. That is, a particular pixel can be considered to be adjacent to up to four other pixels: every pixel (if any) immediately above the particular pixel, every pixel (if any) immediately below the particular pixel, every pixel (if any) immediately to the right of the particular pixel, and every pixel (if any) immediately to the left of the particular pixel. However, in other cases, next-nearest neighbors can be implemented for a pixel. That is, a particular pixel can be considered to be neighbors to up to eight other pixels: every pixel (if any) immediately above the particular pixel, every pixel (if any) immediately below the particular pixel, every pixel (if any) immediately to the right of the particular pixel, every pixel (if any) immediately to the left of the particular pixel, every pixel (if any) immediately to the top right of the particular pixel, every pixel (if any) immediately above and to the left of the particular pixel, every pixel (if any) immediately to the bottom right of the particular pixel, and every pixel (if any) immediately below and to the left of the particular pixel. Similarly, nearest neighbors can be implemented for voxels such that any particular voxel can be neighbors to up to six other voxels (e.g., above, below, front-back, left, and right).In other cases, next-nearest neighbors may be implemented for voxels, such that any particular voxel can be neighbored by up to 26 other voxels (e.g., a particular voxel can be considered the center of a 3x3x3 cube of voxels, and each of the 26 other voxels in that 3x3x3 cube can be considered neighbors of that particular voxel). It should be understood that any other intermediate definition of adjacency can be used, such that any given pixel (or voxel) can be considered neighbored not only by its nearest neighbor pixel but also by fewer than all of its next-nearest neighbor pixels (or voxels). In either case, all pairs of nodes in the PAG that represent adjacent pixels (or adjacent voxels) can be connected by a respective edge, and the weight of that edge can be equal to any suitable function of the values ​​of that pair of nodes (e.g., it can be equal to the reciprocal of the absolute difference in intensity values ​​of the adjacent pixels or adjacent voxels represented by that pair of nodes).

[0027] In various aspects, for any given PAG, the graph component can transform the given PAG into a minimum spanning tree (MST). In various instances, the graph component can achieve this by applying or otherwise implementing Brufka's algorithm (or any other suitable MST discovery algorithm) to the given PAG. In various cases, the MST for a given PAG can include all of the nodes of the given PAG, but the MST can include only a strict subset of the edges of the given PAG. In particular, the MST can include only edges of the given PAG for which the following occurs: for each pair of nodes in the MST, a path exists in the MST joining the pair of nodes, no cycles (e.g., circular paths) exist in the MST, and the sum of the weights of the edges in the MST is minimized.

[0028] In various aspects, for any given MST, the graph component can transform the given MST into a minimum spanning forest (MSF). In various instances, the graph component can accomplish this transformation for any desired set of roots. In various cases, the root can be any node in the given MST that is designated as a root node. It should be understood that a root can also be referred to as a marker. In some aspects, root or marker designation can be facilitated in any suitable manner, such as via an extended minimum operation, via random selection, or via any other suitable paradigm. In either case, given a set of roots, the graph component can transform the given MST into an MSF. Specifically, for each pair of roots in the given MST, the graph component can identify a path in the MST that joins the pair of roots, and the graph component can remove any edge in that path with the highest or largest weight. Once this removal is performed for each pair of roots, the remainder of the given MST can be considered an MSF. In various aspects, an MSF can include all of the nodes of a given MST, but an MSF can include only a strict subset of the edges of a given MST. In particular, an MSF can include multiple trees such that each of the multiple trees includes exactly one of a set of roots, such that for any two nodes that belong to the same tree, there exists a path in the MSF joining those two nodes, and for any two nodes that do not belong to the same tree, there does not exist a path in the MSF joining those two nodes.

[0029] It should be understood that any suitable technique may be implemented to track which nodes belong to which tree of the MSF. As a non-limiting example, each unique or separate tree of the MSF may be considered to have a respective identifier or name (e.g., tree A, tree B, tree C), and each given node of the MSF may be assigned a respective label that indicates or specifies the identifier or name of the tree to which the given node belongs (e.g., a label indicating that node X belongs to tree A, a label indicating that node Y belongs to tree B, a label indicating that node Z belongs to tree C, etc.). It should be understood that such labels may be stored in any suitable electronic manner (e.g., they may be stored in an auxiliary table separate from the MSF).

[0030] In various instances, for any given MSF, the graph component can generate flagged MSFs. In various cases, the graph component can accomplish this as follows: For any tree within a given MSF, the graph component can designate or flag that tree as a boundary tree if that tree contains at least one node that is located on the boundary of every tile to which the given MSF corresponds. Conversely, for any tree within a given MSF, the graph component can designate or flag that tree as an interior tree if that tree does not contain a node that is located on the boundary of every tile to which the given MSF corresponds. In either case, the flagged MSF can be considered to have the same trees as the given MSF, with some such trees flagged or otherwise designated as boundary trees and others such trees flagged or otherwise designated as interior trees. In other words, each tree of a flagged MSF can be considered to be labeled with its own respective identifier or name (e.g., tree A, tree B, tree C), and each tree of a flagged MSF can also be considered to be flagged as either a boundary tree or an internal tree (e.g., tree A can be flagged as a boundary tree, tree B can be flagged as a boundary tree, and tree C can be flagged as an internal tree). Again, it should be understood that such labels and flags can be stored in any suitable electronic manner (e.g., they can be stored in one or more auxiliary tables separate from the flagged MSF).

[0031] In various aspects, for any given flagged MSF, the graph component can generate a flagged MST. In various cases, the graph component can accomplish this as follows: As described above, a given flagged MSF can be viewed as being generated from a given MSF, which itself can be generated from a given MST based on a given set of roots. In particular, maximum-weight edges can be removed from a given MST to form the given MSF based on paths connecting each pair of root nodes. In various cases, the graph component can reinsert those removed edges into the given flagged MSF. Such reinsertion can be viewed as converting the given flagged MSF into a flagged MST. Note that the flagged MST can be viewed as containing the same nodes and edges that constitute the boundary and interior trees of the given flagged MSF. However, the reinsertion of edges can cause those trees to no longer be disconnected from one another. Thus, rather than being referred to as "trees," they may be referred to herein as "regions" (e.g., tree A may be referred to herein as region A, tree B may be referred to herein as region B, and tree C may be referred to herein as region C). That is, the boundary trees of a flagged MSF may be referred to herein as boundary regions of a flagged MST, and the internal trees of a flagged MSF may be referred to herein as internal regions of a flagged MST. Thus, the labels above indicating which nodes belong to which trees can be considered instead to indicate which nodes belong to which regions (e.g., node X belongs to region A, node Y belongs to region B, node Z belongs to region C), and the flags above designating each tree as either a boundary tree or an internal tree can be considered instead to designate each region as either a boundary region or an internal region.

[0032] In various aspects, for any given flagged MST, the graph component can convert the given flagged MST to a PRAG. In various instances, the graph component can accomplish such conversion by preserving the bounding region of the given flagged MST and by condensing the interior regions of the given flagged MST into respective region nodes (also referred to as per-region nodes). More specifically, for each interior region of the given flagged MST, the graph component can replace the interior region with a single node. In various instances, the single node can be any one of the nodes of the interior region. For purposes of illustration or visualization, the single node can be positioned or located at the geometric centroid of the interior region. In either case, the single node can be considered to represent or otherwise represent any spatial area covered by the interior region. Thus, the single node can be referred to as a region node (or per-region node). After such compression of all interior regions into respective region nodes, any remainder of a given flagged MST can be considered a PRAG. In particular, a PRAG can include all of the boundary regions of a given flagged MST, and the PRAG can include respective region nodes in place of the interior regions of the given flagged MST. In this way, some nodes of the PRAG can represent individual pixels (e.g., nodes of boundary regions), while other nodes of the PRAG can instead represent entire spatial areas (e.g., region nodes replacing interior regions). Thus, a PRAG can be considered a type of hybrid graph structure.

[0033] In this manner, the graph component can generate a respective PRAG for each of multiple tiles of the image. In various aspects, the graph component can merge all of such PRAGs together, and the result of such merging can be considered a RAG for the image. More specifically, for any two tiles that are adjacent to one another (e.g., share a boundary), the graph component can combine the PRAGS of those two tiles. In various instances, such combining can include identifying which boundaries are shared between the two tiles, identifying one or more boundary regions from each of the two PRAGs that correspond to the shared boundaries, and inserting edges (e.g., null-weight edges or minimum-weight edges) between adjacent, duplicated, or otherwise matching nodes of those identified boundary regions. After such edge insertion, those identified boundary regions can now be considered to be one or more newly formed interior regions of the union or union of the PRAGs of the two tiles. In various cases, the graph component may then condense these newly formed interior regions into respective region nodes as described above (e.g., in some aspects, such condensation may include re-performing the various operations described above on the merged or combined version of the two PRAGs, such as re-applying Brufka's algorithm, removing maximum-weight edges based on routing, and flagging border and interior regions). In either case, each time the graph component merges two or more adjacent PRAGs, it may incrementally reduce the total cardinality of the border regions across all of the PRAGs. When all of the PRAGs are merged together, the final result may be a graph data structure that does not include border regions; instead, such final graph data structure may include only region nodes and their associated edges. In various aspects, such final graph data structure may be considered a RAG of the image.

[0034] In this way, the graph component can compute the RAG of the image on a tile-by-tile basis.

[0035] In various embodiments, the execution component of the computerized tool can electronically perform any suitable downstream analysis on the RAGs of the image. As a non-limiting example, the execution component can perform marker-based watershed segmentation on the RAGs of the image, thereby resulting in one or more marker-based watershed segmentation basins corresponding to the image. Note that in various aspects, such marker-based watershed segmentation can be performed despite potentially enormous image sizes. In essence, the RAGs of the image can be considered a smaller, simplified, less computationally intensive version of the image that can be analyzed or otherwise manipulated without excessively consuming computational resources. In various instances, the execution component can visually render the one or more marker-based watershed segmentation basins on any suitable computer screen or monitor. In various instances, the execution component can electronically transmit the one or more marker-based watershed segmentation basins to any other suitable computing device.

[0036] Thus, the computerized tools described herein can be considered to facilitate marker-based watershed image segmentation in a tiled context. In various aspects, the computerized tools described herein can facilitate such marker-based watershed image segmentation by constructing RAGs based on PRAGs that respectively correspond to image tiles.

[0037] Various embodiments described herein can be used using hardware or software to solve problems that are highly technical in nature (e.g., to facilitate the computation of tiled region adjacency graphs via pixel region adjacency graphs), are not abstract, and cannot be implemented as a set of mental operations by a human. Furthermore, some of the processes performed can be performed by a dedicated computer (e.g., a scanning or transmission electron microscope) to perform the defined operations associated with marker-based watershed segmentation.

[0038] For example, such defined operations may include: accessing, by a device operatively coupled to the processor, an image generated by the scientific instrument; and performing, by the device, marker-based watershed segmentation on a region adjacency graph of the image, the region adjacency graph constructed from a plurality of pixel region adjacency graphs corresponding to a plurality of tiles of the image. In various aspects, such defined operations include: decomposing, by the device, the image into a plurality of tiles; generating, by the device, a plurality of pixel adjacency graphs based on the plurality of tiles, where, for a first pixel adjacency graph corresponding to a first tile, a node of the first pixel adjacency graph represents each pixel or voxel of the first tile; and generating, by the device (e.g., via Brufka's algorithm), a plurality of minimum spanning forests based on the plurality of pixel adjacency graphs, where, for the first minimum spanning forest corresponding to the first pixel adjacency graph, the first minimum spanning forest includes one or more boundary trees and one or more interior trees. generating, by the device, a plurality of pixel region adjacency graphs based on the plurality of minimum spanning forests, where for a first pixel region adjacency graph corresponding to the first minimum spanning forest, the first pixel region adjacency graph includes one or more boundary regions each corresponding to the one or more boundary trees and a set of per-region nodes into which the one or more interior trees are condensed; and merging, by the device, the plurality of pixel region adjacency graphs into the region adjacency graph by combining boundary regions of adjacent tiles, re-flag such combined boundary regions as new interior regions, and condense such new interior regions into new per-region nodes.

[0039] Such defined operations are inherently computerized. Indeed, scientific instruments such as charged particle electron microscopes or electron energy loss microscopes are highly technical computerized devices equipped with specific computerized hardware (e.g., temperature sensors, pressure sensors, voltage sensors, ion beam emitters, ion focusing lenses, mass analyzers, ion detectors, beam apertures, fluid valves). Scientific instruments and the operations they perform cannot be implemented in any reasonable or practical manner without a computer, by the human mind, or by a human using pen and paper. Furthermore, images captured by scientific instruments (e.g., electron tomography images, X-ray tomography images, confocal microscopy images) are arrays of pixels or voxels that collectively depict or illustrate an analyzed sample. Such images cannot be generated or captured in any reasonable or practical manner without a computer, by the human mind, or by a human using pen and paper. Still further, marker-based watershed segmentation is a specific type of computing task that can be performed on images produced by scientific instruments. In particular, marker-based watershed segmentation is an inherently computational task in which separation lines or boundaries between different visual objects depicted in an image are determined based on intensity values ​​of the pixels or voxels of the image (e.g., in some cases, the separation lines or boundaries may be located between pixels or voxels). It makes no sense to consider the computational task of marker-based watershed image segmentation outside of the context of computing. Furthermore, PAGs, RAGs, and the novel PRAGs described herein are specific types of data structures that serve as alternative computerized representations of pixel arrays or voxel arrays. It makes no sense to consider the creation or manipulation of PAGs, RAGs, or PRAGs outside of the context of computing.

[0040] Furthermore, various embodiments described herein can integrate various teachings regarding the computation of tiled region adjacency graphs via pixel region adjacency graphs into practical applications. As explained above, images captured by scientific instruments can often be enormous in size (e.g., consuming hundreds of gigabytes of memory space). Such enormous size often far exceeds the processing capabilities of computerized workstations supporting the scientific instruments. Therefore, to facilitate the analysis or manipulation of such enormous images, those computerized workstations can consider such enormous images in a tiled manner or otherwise operate on such large amounts of images (e.g., by considering individual tiles of the image rather than considering the entire image at once). Marker-based watershed segmentation is a commonly known and useful analysis that is often desired to be performed on images captured by scientific instruments. However, marker-based watershed segmentation cannot be facilitated in a tiled manner due to marker-based watershed leakage (e.g., due to inconsistency between local and global selection of marker-based watershed separation lines or boundaries). Marker-based watershed segmentation can be facilitated on a simplified version of a larger image, such as a RAG of the larger image. However, existing techniques do not know how to generate RAG in a tiled context. Therefore, existing techniques may be considered at a disadvantage.

[0041] Various embodiments described herein may help to ameliorate one or more of such technical problems. In particular, various embodiments described herein may facilitate the computation of RAGs in a tiled context, as described above. In various aspects, various embodiments described herein may do so by leveraging a new type of graph structure devised by the inventors, PRAGs. More specifically, given an image, various embodiments described herein may include dividing the image into tiles, computing a PAG for each tile, computing an MST for each PAG (e.g., via Brufka's algorithm), computing an MSF for each MST (e.g., by deleting the highest weight edge based on a given root pair), flagging or designating the boundary tree and interior tree of each MSF, generating a flagged MST for each flagged MSF (e.g., by reinserting the deleted edges), converting each flagged MST into a PRAG (e.g., by preserving the boundary regions but condensing interior regions into respective region nodes), and merging all of the PRAGs together into a single RAG (e.g., by combining the boundary regions of each of adjacent PRAGs to form new interior regions and condensing such new interior regions into new region nodes). When all of the PRAGs are merged together, the resulting graph structure may retain only region nodes and their associated edges, with all boundary regions previously in the individual PRAGs converted to new interior regions via merging and then potentially condensed into new region nodes. In this manner, various embodiments described herein may be thought of as computing the RAGs in a tiled manner (e.g., based on tiles of the larger image, rather than based on the entire larger image itself). After the RAGs are computed, various embodiments described herein may include performing marker-based watershed segmentation on the RAGs.Thus, marker-based watershed segmentation can be facilitated even when an image is divided into tiles. In other words, various embodiments described herein permit or enable a computer to perform an activity or function that was previously not possible, namely, RAG construction in a tiled manner or context, and thus marker-based watershed segmentation. In short, existing techniques cannot accurately perform marker-based watershed segmentation on image tiles. While existing techniques can accurately perform marker-based watershed segmentation on RAGs of an image, the existing techniques only know how to generate RAGs from an entire image; the existing techniques do not know how to generate RAGs from image tiles. Therefore, various embodiments described herein can be considered to address or ameliorate various technical problems or disadvantages plaguing existing techniques. For at least these reasons, various embodiments described herein can be considered concrete and tangible technical improvements in the field of marker-based watershed segmentation. Therefore, various embodiments described herein certainly qualify as useful and practical applications of computers.

[0042] Furthermore, various embodiments described herein can control real-world tangible devices based on the disclosed teachings. For example, various embodiments described herein can electronically start, stop, or otherwise actuate real-world hardware (e.g., ion beam emitters, ion focusing lenses, carrier fluid valves / pumps) of real-world scientific instruments (e.g., electron energy loss microscopes), perform real-world analysis on real-world data captured by those real-world scientific instruments (e.g., calculate marker-based watershed segmentation watersheds for images captured by the scientific instruments), and electronically render the results of such real-world analysis on a real-world computer screen (e.g., visually render the calculated segmentation watersheds for viewing by a user or technician).

[0043] FIG. 1 illustrates an exemplary, non-limiting block diagram of a scientific instrument module 102 according to various embodiments described herein.

[0044] In various embodiments, the scientific instrument module 102 may be implemented by circuitry (e.g., including electrical or optical components) such as a programmed computing device. The logic of the scientific instrument module 102 may be contained on a single computing device or may be distributed across multiple computing devices that communicate with each other as needed. Examples of computing devices that may implement the scientific instrument module 102, alone or in combination, are discussed herein with reference to FIGS. 32 and 34, and examples of systems or networks of interconnected computing devices in which the scientific instrument module 102 may be implemented across one or more of the computing devices are discussed herein with reference to FIGS. 33 and 35.

[0045] The scientific instrument module 102 may include first logic 104, second logic 106, and third logic 108. As used herein, the term "logic" may include an apparatus that performs a set of operations associated with the logic. For example, any of the logic elements included in the scientific instrument module 102 may be implemented by one or more computing devices programmed with instructions that cause one or more processing devices of the computing devices to perform a set of associated operations. In particular embodiments, a logic element may include one or more non-transitory computer-readable media having instructions that, when executed by one or more processing devices of the one or more computing devices, cause the one or more computing devices to perform the associated set of operations. As used herein, the term "module" may refer to a collection of one or more logic elements that together perform a function associated with the module. Different logic elements within a module may take the same form or different forms. For example, some logic within a module may be implemented by a programmed general-purpose processing device, while other logic within the module may be implemented by an application-specific integrated circuit (ASIC). In another example, different ones of the logic elements in a module may be associated with different sets of instructions executed by one or more processing devices. A module may omit one or more of the logic elements depicted in an associated drawing; for example, a module may include a subset of the logic elements depicted in an associated drawing if that module performs a subset of the operations discussed herein with reference to that module.

[0046] In various embodiments, there may be a scientific instrument corresponding to the scientific instrument module 102. In various aspects, the scientific instrument may be any suitable computerized device capable of electronically measuring some scientifically relevant, clinically relevant, or research-related characteristic, property, or attribute of an analytical sample (e.g., of a known or unknown mixture, compound, or collection of substances). As a non-limiting example, the scientific instrument may be a mass spectrometer operably coupled to a gas chromatograph or liquid chromatograph. In such a case, the scientific instrument may measure or determine the ion spectrum (e.g., relative ion abundance as a function of mass-to-charge ratio) of the analytical sample. As another non-limiting example, the scientific instrument may be a scanning electron microscope. In such a case, the scientific instrument may measure or determine the surface topography of the analytical sample. As yet another non-limiting example, the scientific instrument may be a transmission electron microscope. In such a case, the scientific instrument may measure or determine details of the internal structure of the analytical sample. As a more general, non-limiting example, the scientific instrument may be any suitable type of charged particle microscope (e.g., some types of microscopes may use a beam of non-electron ions to capture images).

[0047] In various embodiments, the first logic 104 can access an image captured or otherwise generated by a scientific instrument. In various aspects, the image can be any suitable pixel array or voxel array capable of depicting or illustrating any suitable analytical sample.

[0048] In various embodiments, the second logic 106 can construct a region adjacency graph for the image on a tile-by-tile basis. In various aspects, such construction can be based on multiple pixel region adjacency graphs, each corresponding to a plurality of tiles of the image. More specifically, the second logic 106 can include decomposing the image into multiple tiles and generating a respective pixel adjacency graph for each tile, thereby resulting in the multiple pixel adjacency graphs. In various instances, the second logic 106 can include converting each of the multiple pixel adjacency graphs into a respective minimum spanning tree via application of Brufka's algorithm. This can result in the multiple minimum spanning trees. In various cases, the second logic 106 can include converting each of the multiple minimum spanning trees into a respective minimum spanning forest by deleting the highest-weight edge located along the path joining the respective root nodes. This can result in the multiple minimum spanning forests. In various aspects, the second logic 106 may include flagging or otherwise designating boundary trees and interior trees of each of the plurality of minimum spanning forests, thereby resulting in a plurality of flagged minimum spanning forests. In various aspects, the second logic 106 may include converting each of the plurality of flagged minimum spanning forests into a respective flagged minimum spanning tree by reinserting any previously removed edges based on the root path. This may generate a plurality of flagged minimum spanning trees. In various cases, the second logic 106 may include converting each of the plurality of flagged minimum spanning trees into a respective one of a plurality of pixel region adjacency graphs by replacing interior regions with individual region nodes. In various cases, the second logic 106 may include merging the plurality of pixel region adjacency graphs together by combining boundary regions of adjacent pixel region adjacency graphs that share a boundary.Such merging can result in new interior regions, which can then be replaced with new region nodes. After all such merging is complete, the resulting graph structure can be considered to be the region adjacency graph of the image.

[0049] In various embodiments, the third logic 108 can apply marker-based watershed segmentation to a region adjacency graph of the image, thereby resulting in one or more marker-based watershed segmentation basins. In some aspects, the third logic 108 can include transmitting or sharing the one or more marker-based watershed segmentation basins with any other computing device. In other aspects, the third logic 108 can include visually displaying the one or more marker-based watershed segmentation basins on any suitable computer screen or computer monitor. In either case, the one or more marker-based watershed segmentation basins can be considered to be generated in a tiled context that existing techniques do not know how to accurately achieve.

[0050] Thus, the scientific instrument module 102 can facilitate marker-based watershed segmentation through the computation of a tiled region adjacency graph, which is achieved by leveraging pixel region adjacency graphs.

[0051] 2 is an exemplary, non-limiting flow diagram of a computer-implemented method 200 according to various embodiments described herein. The operations of computer-implemented method 200 may be used in any suitable context to perform any suitable operations (e.g., performed by or used in conjunction with any of the various modules, computing devices, or graphical user interfaces described with respect to FIGS. 1, 31, 32, 33, 34, and 35). Although the operations are shown in FIG. 2 once each and in a particular order, the operations may be reordered or repeated as desired and necessary (e.g., different operations performed may be performed in parallel, as appropriate).

[0052] In various aspects, operation 202 may include performing, by a device operably coupled to the processor, a first operation of accessing an image produced by the scientific instrument. In various cases, first logic 104 may perform or otherwise facilitate operation 202.

[0053] In various cases, operation 204 may include performing, by the device, a second operation of constructing, on a tile-by-tile basis, a region adjacency graph for the image based on the plurality of pixel region adjacency graphs. In various cases, second logic 106 may perform or otherwise facilitate operation 204.

[0054] In various aspects, operation 206 may include segmenting, by the device, the image based on a region adjacency graph. In some instances, such segmentation may be achieved via a marker-based watershed segmentation algorithm. In various cases, third logic 108 may perform or otherwise facilitate operation 206.

[0055] Thus, the computer-implemented method 200 can facilitate marker-based watershed segmentation through the computation of a tiled region adjacency graph, which is achieved by leveraging pixel region adjacency graphs.

[0056] FIG. 3 illustrates a block diagram of an exemplary, non-limiting system that can facilitate computation of a tiled region adjacency graph via a pixel region adjacency graph, in accordance with one or more embodiments described herein.

[0057] In various embodiments, a scientific instrument 302 may be present. In various aspects, the scientific instrument 302 may be as described above. That is, the scientific instrument 302 may be any suitable computerized device capable of electronically measuring any suitable scientifically relevant, clinically relevant, or research-related characteristic, attribute, or property of any suitable analytical sample. In particular, the scientific instrument 302 may be a charged particle microscope. In such a case, the scientific instrument 302 may utilize its constituent hardware (e.g., electron source, anode, condenser lens, condenser aperture, scanning coil, objective lens, objective aperture, deflector, condenser, stigmator, electron detector, x-ray detector, actuable sample stage) to electronically generate or capture an image 304 of any given analytical sample.

[0058] In various aspects, the image 304 can assume any suitable format, size, or dimensions. As a non-limiting example, the image 304 can be an x ​​x y pixel array, for any suitable positive integers x and y. In such a case, the image 304 can be considered to have a total of x y pixels, with each pixel having a corresponding intensity value. As another non-limiting example, the image 304 can be an x ​​x y x z voxel array, for any suitable positive integers x, y, and z. In such a case, the image 304 can be considered to have a total of x y z voxels, with each voxel having a corresponding intensity value.

[0059] In either case, the system 306 may be electronically integrated with the image 304 and the scientific instrument 302 (e.g., via any suitable wired or wireless electronic connection). In various aspects, because the image 304 may visually illustrate or depict the analytical sample, it may be desirable to perform marker-based watershed segmentation on the image 304 to derive, glean, or otherwise reveal structural information about the analytical sample. However, in some cases, the image 304 may be very large in size. By way of non-limiting example, the image 304 may occupy tens, hundreds, or thousands of gigabytes of computer memory. Therefore, it may be desirable to perform marker-based watershed segmentation on the image 304 in a tiled manner. As described herein, the system 306 can facilitate such marker-based watershed segmentation.

[0060] In various aspects, the system 306 may comprise a processor 308 (e.g., a computer processing unit, a microprocessor) and non-transitory computer-readable memory 310 operatively, operatively, or communicatively connected or coupled to the processor 308. The non-transitory computer-readable memory 310 may store computer-executable instructions that, when executed by the processor 308, cause the processor 308 or other components of the system 306 (e.g., the access component 312, the tile component 314, the graph component 316, the execution component 318) to perform one or more operations. In various embodiments, the non-transitory computer-readable memory 310 may store, and the processor 308 may execute, the computer-executable components (e.g., the access component 312, the tile component 314, the graph component 316, the execution component 318).

[0061] In various embodiments, the system 306 can include an access component 312. In various aspects, the access component 312 can electronically access the scientific instrument 302. That is, the access component 312 can electronically communicate with or otherwise interact with (e.g., send electronic instructions or commands, receive electronic data from) the scientific instrument 302. Thus, the access component 312 can be considered a proxy or conduit through which other components of the system 306 can interact with, communicate with, or otherwise manipulate the scientific instrument 302. In various instances, the access component 312 can electronically access the images 304. That is, the access component 312 can electronically receive, search, or otherwise obtain the images 304 from any suitable electronic source or database (not shown). As a non-limiting example, the access component 312 can electronically receive, search, or otherwise obtain the images 304 from the scientific instrument 302. In either case, the access component 312 can be thought of as a proxy or conduit through which other components of the system 306 can interact with or otherwise manipulate the image 304. However, these are merely non-limiting examples. In other cases, the access component 312 can be omitted, and any other component of the system 306 can communicate with or interact directly with the scientific instrument 302 or the image 304.

[0062] In various embodiments, the system 306 can include a tiling component 314. In various aspects, the tiling component 314 can decompose the image 304 into multiple tiles as described herein.

[0063] In various embodiments, the system 306 can include a graph component 316. In various instances, the graph component 316 can generate a region adjacency graph for the image 304 based on multiple tiles, as described herein. In various instances, the graph component 316 can facilitate such generation by utilizing multiple pixel region adjacency graphs, each corresponding to a respective one of the multiple tiles.

[0064] In various embodiments, the system 306 can include an execution component 318. In various aspects, the execution component 318 can perform marker-based watershed segmentation on the region adjacency graph, as described herein, thereby resulting in one or more marker-based watershed segmentation basins of the image 304.

[0065] It should be noted that in various instances, the access component 312, the tile component 314, the graph component 316, and the execution component 318 may be collectively considered to be one or more software components 311 of the system 306. It should be understood that in various aspects, the one or more software components 311 are described herein as including primarily four components (e.g., the access component 312, the tile component 314, the graph component 316, and the execution component 318) for ease of explanation and illustration. However, the one or more software components 311 are not limited to being implemented as exactly such four components in all embodiments. Indeed, in some embodiments, the functionality described herein of such four components may be combined in any suitable manner to be implemented in or by fewer than four components (e.g., in some cases, a single component may perform all of the functionality described herein with respect to the access component 312, the tile component 314, the graph component 316, and the execution component 318). In other embodiments, the functionality described herein for such four components may instead be distributed, separated, divided, or fragmented in any suitable manner to be implemented in or by more than four components (e.g., two or more components may facilitate the functions performable by the access component 312, two or more components may facilitate the functions performable by the tile component 314, two or more components may facilitate the functions performable by the graph component 316, and two or more components may facilitate the functions performable by the execution component 318).

[0066] FIG. 4 illustrates a block diagram of an exemplary, non-limiting system including multiple tiles that can facilitate computation of a tiled region adjacency graph via a pixel region adjacency graph, in accordance with one or more embodiments described herein.

[0067] In various embodiments, the tile component 314 can electronically decompose or otherwise separate the image 304 into multiple tiles 402, which can include any suitable number of tiles. Various non-limiting aspects are described with respect to FIGS. 5-7.

[0068] 5-7 illustrate exemplary, non-limiting block diagrams of a number of tiles 402 according to one or more embodiments described herein.

[0069] Consider first Figure 5. In various embodiments, as shown, the plurality of tiles 402 may include n tiles, i.e., tile 402(1) through tile 402(n), for any suitable positive integer n>1. In various aspects, each of the plurality of tiles 402 may be a contiguous set, group, or subset of pixels (or voxels) of the image 304. Thus, each of the plurality of tiles 402 may be considered a discrete multi-pixel (or multi-voxel) fragment or section of the image 304.

[0070] As a non-limiting example, tile 402(1) may include multiple pixels 404(1). While pixels are described herein for ease of explanation and illustration, it should be noted that similar or equivalent embodiments may apply to voxels. In various instances, the multiple pixels 404(1) may include m pixels, i.e., pixel 404(1)(1) through pixel 404(1)(m), for any suitable positive integer m>1. In various instances, each of the multiple pixels 404(1) may be a unique or distinct pixel from image 304. In various aspects, the multiple pixels 404(1) may be contiguous. In other words, all of the multiple pixels 404(1) may be either directly or indirectly in contact with one another as positioned within image 304. In other words, for any two pixels in the plurality of pixels 404(1), those two pixels may either touch or be adjacent to one another, or may be connected together by a chain, string, or line of other pixels in the tile 402(1), with each contiguous pair touching or being adjacent to one another. In various instances, the tile 402(1) may assume any suitable shape. That is, the plurality of pixels 404(1) may be positioned within the image 304 to form any suitable geometric arrangement or layout. Indeed, in some cases, the plurality of pixels 404(1) may be positioned within the image 304 to form any suitable linear arrangement or layout (e.g., such that the tile 402(1) is square or rectangular). In such situations, the tile 402(1) may be considered an a×b pixel array, where ab=m, for any suitable positive integers a and b. However, in other cases, multiple pixels 404(1) may be positioned within image 304 to form any suitable non-linear arrangement or layout (e.g., such that tiles 402(1) are triangular, any other suitable polygonal, or any other suitable irregular shape).

[0071] As another non-limiting example, tile 402(n) may include a plurality of pixels 404(n). In various instances, the plurality of pixels 404(n) may include m pixels, i.e., pixel 404(n)(1) through pixel 404(n)(m). As noted above, each of the plurality of pixels 404(n) may be a unique or distinct pixel from image 304, and the plurality of pixels 404(n) may be contiguous. That is, all of the plurality of pixels 404(n) may either directly or indirectly touch one another when positioned within image 304 (e.g., for any two pixels in the plurality of pixels 404(n), those two pixels may either touch or be adjacent to one another, or may be joined together by a chain, string, or line of other pixels within tile 402(n), with each contiguous pair of pixels touching or being adjacent to one another). Also, as noted above, tiles 402(n) may take on any suitable shape (e.g., multiple pixels 404(n) may be positioned within image 304 to form any suitable linear arrangement or layout, or multiple pixels 404(n) may be positioned within image 304 to form any suitable non-linear arrangement or layout).

[0072] 5 depicts the multiple tiles 402 as all having the same number of pixels (e.g., m), it should be noted that this is merely a non-limiting example for ease of illustration and description. In various embodiments, any of the multiple tiles 402 may be comprised of the same or different numbers of pixels. Relatedly, any of the multiple tiles 402 may have or exhibit the same or different shapes as one another.

[0073] In a non-limiting example, each of the multiple tiles 402 can be a unique or distinct a×b rectilinear pixel array of the image 304, for any suitable positive integers a and b, where a×b=m. In some situations, such a×b rectilinear pixel arrays may be disjoint from one another. In such cases, none of the multiple tiles 402 can overlap one another (e.g., each pixel of the image 304 can belong to exactly one of the multiple tiles 402), and the union of the multiple tiles 402 can be equivalent to the image 304 (e.g., the total number of pixels in the image 304 can be equal to m). However, in other situations, such a×b rectilinear pixel arrays may be disjoint from one another. Indeed, in some cases, any two tiles that are vertically adjacent to one another in image 304 may share rows of pixels (e.g., the bottom row of pixels in the top tile may be the same as the top row of pixels in the bottom tile), and any two tiles that are horizontally adjacent to one another in image 304 may share columns of pixels (e.g., the rightmost column of pixels in the left tile may be the same as the leftmost column of pixels in the right tile). In such cases, various ones of the multiple tiles 402 may overlap one another (e.g., at least some pixels of image 304 may belong to more than one of the multiple tiles 402), and a symmetric difference (e.g., a type of union operator that avoids double counting) of the multiple tiles 402 may be equivalent to image 304 (e.g., if each pair of adjacent tiles shares one or more rows or columns of pixels, the total number of pixels in image 304 may be less than nm).

[0074] In either case, each of the plurality of tiles 402 may be smaller than the image 304. Indeed, in some cases, each of the plurality of tiles 402 may contain an order of magnitude or more fewer pixels than the image 304. Thus, the plurality of tiles 402 may be analyzed, manipulated, or otherwise examined individually or independently with the expenditure of significantly fewer computational resources than would be required to analyze, manipulate, or otherwise examine the image 304 as a whole.

[0075] Consider now FIGS. 6 and 7. FIG. 6 illustrates a captured image 602. In various aspects, captured image 602 can be considered a non-limiting example of image 304. FIG. 7 illustrates a non-limiting example of how captured image 602 can be decomposed or otherwise fragmented into four tiles (e.g., n may equal 4): tile 702, tile 704, tile 706, and tile 708. In the non-limiting example of FIG. 7, tiles 702-708 are each square, contiguous pixel arrays that can be considered a respective puzzle piece of captured image 602. Indeed, tile 702 can be considered the top-left puzzle piece of captured image 602, tile 704 can be considered the top-right puzzle piece of captured image 602, tile 706 can be considered the bottom-left puzzle piece of captured image 602, and tile 708 can be considered the bottom-right puzzle piece of captured image 602. In such a case, tile 702 can be considered adjacent to tiles 704 and 706, tile 704 can be considered adjacent to tiles 702 and 708, tile 706 can be considered adjacent to tiles 702 and 708, and tile 708 can be considered adjacent to tiles 704 and 706. As mentioned above, tiles 702-708 may, in some aspects, be disjointed from one another or not overlapping one another. That is, none of tiles 702-708 may share pixels. However, in other aspects, tiles 702-708 may instead not be disjointed from one another or overlap one another. That is, tiles 702-708 may share pixels. For example, tiles 702 and 704 may share columns of pixels (e.g., the rightmost column of tile 702 may contain the same pixels as the leftmost column of tile 704). Alternatively, tiles 702 and 706 may share rows of pixels (eg, the bottom row of tile 702 may contain the same pixels as the top row of tile 706).As yet another example, tiles 704 and 708 may share rows of pixels (e.g., the bottom row of tile 704 may contain the same pixels as the top row of tile 708). As yet another example, tiles 706 and 708 may share columns of pixels (e.g., the rightmost column of tile 706 may contain the same pixels as the leftmost column of tile 708).

[0076] 7 depicts the captured image 602 as being decomposed or fragmented into four tiles, this is merely a non-limiting example for ease of explanation and illustration. In various embodiments, the captured image 602 may instead be decomposed into any suitable number of tiles (e.g., the greater the number of tiles, the smaller and more granular each tile may be).

[0077] In either case, the tile component 314 may electronically fragment, slice, or otherwise divide the image 304 into multiple tiles 402, each of which may be smaller in size (e.g., may consume less memory space) than the image 304.

[0078] FIG. 8 illustrates a block diagram of an exemplary, non-limiting system including a region adjacency graph that can facilitate computation of a tiled region adjacency graph via a pixel region adjacency graph, in accordance with one or more embodiments described herein.

[0079] In various embodiments, the graph component 316 can electronically generate a region adjacency graph 802 (hereinafter, "RAG 802") based on the plurality of tiles 402. In various aspects, the RAG 802 can be considered a type of simplified, less memory-intensive representation of the image 304. Thus, marker-based watershed segmentation can be accurately or reliably performed on the region adjacency graph 802, regardless of whether the marker-based watershed segmentation cannot be accurately or reliably performed on the image 304 (e.g., due to the image 304 being too large) or on the plurality of tiles 402 (e.g., due to marker-based watershed leakage). In various instances, the graph component 316 can generate the RAG 802 by utilizing multiple pixel region adjacency graphs, each corresponding to a plurality of tiles 402. Various non-limiting details are described with respect to FIGS. 9-26.

[0080] 9-26 show exemplary, non-limiting block diagrams for constructing or generating a RAG 802 according to one or more embodiments described herein.

[0081] 9. In various embodiments, there may be a tile 902. In various aspects, the tile 902 may be any one of the plurality of tiles 402. Thus, the tile 902 may include a plurality of pixels 904, which may include m pixels, i.e., pixel 904(1) through pixel 904(m).

[0082] In various instances, the graph component 316 can electronically generate a pixel adjacency graph 906 (hereinafter, "PAG 906") based on the tiles 902. In various instances, the PAG 906 can include multiple nodes 908 and multiple edges 910.

[0083] In various aspects, the plurality of nodes 908 may correspond respectively to the plurality of pixels 904 (e.g., in a one-to-one manner). Thus, the plurality of pixels 904 may include m pixels, and therefore the plurality of nodes 908 may include m nodes, i.e., node 908(1) through node 908(m). In various instances, each of the plurality of nodes 908 may be considered to represent a respective pixel of the plurality of pixels 904. As a non-limiting example, node 908(1) may be considered to represent pixel 904(1). Thus, in some cases, node 908(1) may have any intensity value exhibited by pixel 904(1). As another non-limiting example, node 908(m) may be considered to represent pixel 904(m). Thus, as described above, node 908(m) may have any intensity value exhibited by pixel 904(m).

[0084] In various embodiments, the plurality of edges 910 can include any suitable number of edges such that any two of the plurality of nodes 908 representing adjacent pixels can have an edge therebetween. As a non-limiting example, assume that for any suitable positive integers 1 ≦ i < j ≦ m, pixel 904(i) and pixel 904(j) are adjacent to each other. In such a case, the plurality of edges 910 can include an edge between node 908(i) and node 908(j). It should be noted that any suitable type of adjacency rule or adjacency definition can be implemented to determine or create the plurality of edges 910. In fact, in some cases, a nearest neighbor adjacency rule or definition may be implemented. In such a situation, pixel 904(i) and pixel 904(j) can be considered adjacent if they are nearest neighbors to each other. That is, pixel 904(i) and pixel 904(j) are considered adjacent if pixel 904(i) is positioned within (and thus within image 304) the tile 902 immediately above pixel 904(j), or if pixel 904(i) is positioned within (and thus within image 304) the tile 902 immediately below pixel 904(j), or if pixel 904(i) is positioned within (and thus within image 304) the tile 902 immediately to the right of pixel 904(j), or if pixel 904(i) is positioned within (and thus within image 304) the tile 902 immediately to the left of pixel 904(j). However, in other cases, a next-nearest neighbor adjacency rule or definition may be implemented. In such a situation, pixel 904(i) and pixel 904(j) can be considered adjacent if they are nearest neighbors or next-nearest neighbors to each other.That is, pixel 904(i) and pixel 904(j) are related in the following manner: pixel 904(i) is positioned or located in the tile 902 immediately above pixel 904(j) (and thus in image 304); pixel 904(i) is positioned or located in the tile 902 immediately below pixel 904(j) (and thus in image 304); pixel 904(i) is positioned or located in the tile 902 immediately to the right of pixel 904(j) (and thus in image 304); and pixel 904(i) is positioned or located in the tile 902 immediately to the left of pixel 904(j) (and thus in image 304). Pixels 904(i) can be considered neighbors if they are positioned or located in the tile 902 immediately to the right and above pixel 904(j) (and thus in image 304), if they are positioned or located in the tile 902 immediately to the left and above pixel 904(j) (and thus in image 304), if they are positioned or located in the tile 902 immediately to the right and below pixel 904(j) (and thus in image 304), or if they are positioned or located in the tile 902 immediately to the left and below pixel 904(j) (and thus in image 304). It should be understood that any intermediate type of neighboring rule or definition can be implemented such that any given pixel in tile 902 is neighbor to any suitable combination of its nearest neighbor or next-nearest neighbor pixels.

[0085] In various aspects, the plurality of edges 910 may be weighted. In other words, each of the plurality of edges 910 may be assigned or otherwise have a scalar weight. In various instances, the scalar weight assigned to any given edge of the plurality of edges 910 may be based on the intensity values ​​of any two nodes to which the given edge connects. As a non-limiting example, assume again that pixel 904(i) and pixel 904(j) are adjacent and that pixel 904(i) and pixel 904(j) are represented by node 908(i) and node 908(j), respectively. In such a case, there may be a particular edge connecting node 908(i) to node 908(j), and the scalar weight assigned to that particular edge may be equal to or otherwise based on any suitable mathematical function of the intensity value of pixel 904(i) and the intensity value of pixel 904(j). For example, the scalar weight may be equal to the inverse of the absolute difference between the intensity value of pixel 904(i) and the intensity value of pixel 904(j) (e.g., so that the magnitude of the scalar weight increases as the intensity values ​​of pixel 904(i) and pixel 904(j) become more similar). Alternatively, the scalar weight may be equal to a power law function whose negative exponent is the absolute difference between the intensity values ​​of pixel 904(i) and pixel 904(j) (e.g., again, so that the magnitude of the scalar weight increases as the intensity values ​​of pixel 904(i) and pixel 904(j) become more similar).

[0086] Consider now FIG. 10. In various embodiments, the graph component 316 can generate a minimum spanning tree 1002 (hereinafter, "MST 1002") based on the PAG 906. In various aspects, the graph component 316 can accomplish this by performing any suitable MST discovery or computation algorithm on the PAG 906. As a non-limiting example, the graph component 316 can perform Brufka's algorithm on the PAG 906, which can be viewed as a sequence of greedy edge selections that iteratively transforms the PAG 906 into the MST 1002. However, any other suitable MST discovery or computation algorithm, such as Prim's algorithm or Kruskal's algorithm, can be used in place of (or in combination with) the Brufka's algorithm. In either case, the MST 1002 can include the same nodes as the PAG 906, but the MST 1002 can include fewer edges than the PAG 906. In particular, the MST 1002 can include multiple nodes 908, and the MST 1002 can include multiple edges 1004 instead of multiple edges 910. In various cases, the multiple edges 1004 can be a strict subset of the multiple edges 910 such that all three of the following occur: (1) for each pair of nodes in the MST 1002, a path exists in the MST 1002 connecting that pair of nodes, (2) no closed or cyclic paths exist in the MST 1002, and (3) the sum of the weights of the multiple edges 1004 is minimized (hence the term "minimum"). In other words, there can be multiple distinct subsets of the multiple edges 910 that satisfy (1) and (2) above, and the multiple edges 1004 can be the one for which any one of the multiple subsets has the lowest (e.g., minimum) sum of the weights.

[0087] Next, consider FIG. 11. In various aspects, the graph component 316 can identify a plurality of root nodes 1102 of the MST 1002. In various cases, the plurality of root nodes 1102 can include k nodes, i.e., root nodes 1102(1) to root nodes 1102(k), for any suitable positive integer 1 < k < m. In various situations, the root node 1102(1) can be the first node among the plurality of nodes 908 designated as roots by the graph component 316. Similarly, the root node 1102(k) can be the k-th node among the plurality of nodes 908 designated as roots by the graph component 316. In various aspects, the graph component 316 can identify the plurality of root nodes 1102 in any suitable manner. As a non-limiting example, the graph component 316 can identify the plurality of root nodes 1102 by performing an extended minimum operation on the MST 1002. As another non-limiting example, the graph component 316 can identify the plurality of root nodes 1102 in any other suitable random or non-random manner (e.g., the graph component 316 can iterate through the plurality of nodes 908, and for any node currently under consideration, the graph component 316 can randomly designate or consider that node as either a root or a non-root, or there may be k defined or desired locations within the tile 902, and any pixel located at those k defined or desired locations, which represents a node, may be designated or considered as a root node).

[0088] In either case, the graph component 316 can generate a minimum spanning forest 1104 (hereinafter "MSF 1104") based on both the MST 1002 and the plurality of root nodes 1102. Indeed, for each distinct pair of root nodes from the plurality of root nodes 1102, the graph component 316 can identify the path (if any) that joins that pair of root nodes together and remove from the plurality of edges 910 any edge with the highest weight on that path. By performing such removal for each unique pair of root nodes 1102, the graph component 316 can be thought of as removing the plurality of edges 910 from the plurality of edges 1108, which in turn removes the plurality of edges 1108.

[0089]

number

[0090] In either case, the MSF 1104 can include multiple trees 1106. In various instances, the multiple trees 1106 can each correspond to multiple root nodes 1102. Thus, the multiple root nodes 1102 can include k roots, and therefore the multiple trees 1106 can include k trees, i.e., tree 1106(1) through tree 1106(k). In various instances, each of the multiple trees 1106 can be a collection of connected nodes from the MST 1002 that includes a respective one of the multiple root nodes 1102 and is not coupled to any of the others of the multiple trees 1106. As a non-limiting example, tree 1106(1) can be a first subset of nodes 908 and edges 1004 (fewer than edges 1108) such that tree 1106(1) has root node 1102(1) but no other of root nodes 1102, and for each pair of nodes in tree 1106(1), there is a path in tree 1106(1) joining that pair of nodes, there are no closed or cyclic paths in tree 1106(1), and there is no path joining any node in tree 1106(1) to any node not in tree 1106(1). As another non-limiting example, tree 1106(k) can be the kth subset of nodes 908 and edges 1004 (excluding edges 1108), such that tree 1106(k) has root node 1102(k) but no other of root nodes 1102, and for each pair of nodes in tree 1106(k), there is a path in tree 1106(k) that joins that pair of nodes, there are no closed or cyclic paths in tree 1106(k), and there is no path that joins any node in tree 1106(k) to any node not in tree 1106(k).

[0091] Note that the multiple trees 1106 may be disjoint from one another, such that the union of all nodes across the multiple trees 1106 is equal to the multiple nodes 908, and the union of all edges across the multiple trees 1106 is equal to the multiple edges 1004 minus the multiple edges 1108. Additionally, note that different trees in the multiple trees 1106 can have the same or different numbers of nodes or edges.

[0092] It should be understood that any suitable electronic technique may be implemented to track which of the plurality of nodes 908 belongs to which of the plurality of trees 1106. As a non-limiting example, each of the plurality of trees 1106 may be considered to have its own respective name or identifier, and each of the plurality of nodes 908 may be considered to have a respective label that specifies or indicates which of the plurality of trees 1106 the node belongs to. For example, an auxiliary or separate data table associated with the MSF 1104 may store, for each of the plurality of nodes 908, and therefore for each of the plurality of pixels 904, the label of which tree it belongs to.

[0093] Consider now FIG. 12. In various aspects, the graph component 316 can generate a flagged minimum spanning forest 1202 (hereinafter, “flagged MSF 1202”) based on the MSF 1104. As described above, the MSF 1104 can include multiple trees 1106. In various instances, the graph component 316 can assign or provide a respective flag to each of the multiple trees 1106. In various cases, the flag assigned to any given tree can indicate either that such given tree is a boundary tree or that such given tree is instead an interior tree. In various aspects, the graph component 316 can flag any given tree as a boundary tree in response to determining that the given tree includes at least one node located on a boundary of the tile 902. In contrast, the graph component 316 can flag any given tree as an interior tree in response to determining that the given tree does not include a node located on a boundary of the tile 902. In various cases, the boundary of a tile 902 can be a distinct portion of the outermost or peripheral boundary of the tile 902 that is adjacent to or otherwise abuts another tile of the plurality of tiles 402. As a non-limiting example, assume that the tile 902 is rectilinear (e.g., the plurality of pixels 904 can be arranged in a square or rectangular layout in the image 304). In such a case, the tile 902 can be considered to have up to four boundaries: an upper boundary, a lower boundary, a right boundary, and a left boundary. In particular, the top row of pixels of the tile 902 can be considered the upper boundary of the tile 902 if another tile of the plurality of tiles 402 is located or positioned immediately above the tile 902 in the image 304. However, if another of the plurality of tiles 402 is not located or positioned immediately above the tile 902, the tile 902 can be considered to have no upper boundary.Similarly, the bottom-most row of pixels of tile 902 may be considered the bottom boundary of tile 902 if another tile of the plurality of tiles 402 is located or positioned immediately below tile 902 in image 304. However, if no other tile of the plurality of tiles 402 is located or positioned immediately below tile 902, tile 902 may be considered not to have a bottom boundary. Similarly, the right-most row of pixels of tile 902 may be considered the right-hand boundary of tile 902 if another tile of the plurality of tiles 402 is located or positioned immediately to the right of tile 902 in image 304. However, if no other tile of the plurality of tiles 402 is located or positioned immediately to the right of tile 902, tile 902 may be considered not to have a right-hand boundary. Furthermore, the left-most row of pixels of tile 902 may be considered the left-hand boundary of tile 902 if another tile of the plurality of tiles 402 is located or positioned immediately to the left of tile 902 in image 304. However, if no other one of the tiles 402 is located or positioned immediately to the left of the tile 902, the tile 902 may be considered to have no left boundary.

[0094] In either case, such flagging or designation may cause the flagged MSF 1202 to include multiple boundary trees 1204 and multiple internal trees 1206. In various aspects, the multiple boundary trees 1204 may include p trees, i.e., boundary tree 1204(1) through boundary tree 1204(p). In various cases, the multiple internal trees 1206 may include q trees, i.e., internal tree 1206(1) through internal tree 1206(q). Note that p and q may be any suitable positive integers such that p+q=k. In other words, boundary tree 1204(1) can be the first tree of the plurality of trees 1106 that graph component 316 flags or designates as a boundary tree, boundary tree 1204(p) can be the pth tree of the plurality of trees 1106 that graph component 316 flags or designates as a boundary tree, interior tree 1206(1) can be the first tree of the plurality of trees 1106 that graph component 316 flags or designates as an interior tree, and interior tree 1206(q) can be the qth tree of the plurality of trees 1106 that graph component 316 flags or designates as an interior tree. Note that such flags or designations can be electronically stored, recorded, or otherwise tracked in any suitable manner. As a non-limiting example, an auxiliary or separate data table associated with MSF 1104 may store a boundary flag or an interior flag for each of the plurality of nodes 908 (and thus for each of the plurality of pixels 904), which auxiliary or separate data table, together with MSF 1104, may collectively be considered to form flagged MSF 1202. In other words, each of the plurality of nodes 908 may be considered to have a respective label indicating which of the plurality of trees 1106 it belongs to, and each of the plurality of trees 1106 may be considered to have a respective flag indicating whether the tree is a boundary tree or, alternatively, an interior tree.

[0095] Consider now Figure 13. In various aspects, graph component 316 can generate a flagged minimum spanning tree 1302 (hereinafter "flagged MST 1302") based on the flagged MSF 1202. In various instances, graph component 316 can accomplish this by reinserting edge 1108 into flagged MSF 1202. As a non-limiting example, graph component 316 can reinsert edge 1108(1) into flagged MSF 1202, such that edge 1108(1) is located (in flagged MST 1302) between two nodes of plurality of nodes 908 that were joined before the deletion of edge 1108(1). As another non-limiting example, graph component 316 may reinsert edge 1108(l) into flagged MSF 1202, so that edge 1108(l) is located (in flagged MST 1302) between two nodes of the plurality of nodes 908 that were connected before edge 1108(l) was deleted. After such reinsertion, flagged MST 1302 may be considered to be composed of the same nodes and edges as MST 1002, the only difference being that the nodes and edges in flagged MST 1302 now have respective boundary or interior flags (e.g., again, these flags may be stored in an auxiliary or separate data table or array). Note that after the reinsertion, various pairs of trees from the plurality of boundary trees 1204 or from the plurality of interior trees 1206 may be considered to be joined or connected together by respective edges of the plurality of edges 1108. Thus, they may no longer strictly be considered or referred to as "trees." Instead, they may be considered or referred to herein as "regions."That is, the multiple boundary trees 1204 can be referred to as multiple boundary regions 1304 after edge reinsertion (e.g., boundary region 1304(1) can include the same nodes and edges as boundary tree 1204(1), and boundary region 1304(p) can include the same nodes and edges as boundary tree 1204(p). Similarly, the multiple internal trees 1206 can be referred to as multiple internal regions 1306 after edge reinsertion (e.g., internal region 1306(1) can include the same nodes and edges as internal tree 1206(1), and internal region 1306(q) can include the same nodes and edges as internal tree 1206(q). In other words, each of the plurality of nodes 908 can be considered to have a respective label indicating which of a plurality of regions (not trees) the node belongs to, and each of the plurality of regions can be considered to have a respective flag indicating whether the region is a boundary region (e.g., one of 1304) or alternatively an interior region (e.g., one of 1306).

[0096] 14. In various embodiments, the graph component 316 can generate a pixel region adjacency graph 1402 (hereinafter, "PRAG 1402") based on the flagged MST 1302. In various aspects, the graph component 316 can accomplish this by replacing multiple interior regions 1306 with multiple region nodes 1404, respectively.

[0097] As a non-limiting example, graph component 316 may remove interior region 1306(1) and replace it with region node 1404(1). In some cases, region node 1404(1) may be any node (e.g., randomly selected) within interior region 1306(1). If region node 1404(1) is to be visualized within or on tile 902, region node 1404(1) may be (but need not be) located or positioned at the geometric centroid of interior region 1306(1). In either case, region node 1404(1) may be considered a replacement or substitute for interior region 1306(1), such that any spatial area or portion of tile 902 that was previously represented in a more refined manner by interior region 1306(1) may be considered to be coarsely represented by region node 1404(1) (hence the term “region”). Note that prior to such replacement or substitution, interior region 1306(1) may have been connected by one or more of edges 1108 to one or more other of interior regions 1306 or to one or more other of boundary regions 1304. Thus, after such replacement or substitution, region node 1404(1) may be connected by one or more of edges 1108 to one or more other of interior regions 1306 (or the region nodes replacing them) or to one or more other of boundary regions 1304.

[0098] As another non-limiting example, graph component 316 can remove interior region 1306(q) and replace it with region node 1404(q). In some cases, region node 1404(q) can be any node (e.g., randomly selected) within interior region 1306(q). As noted above, if region node 1404(q) is to be visualized within or on tile 902, region node 1404(q) can be (but need not be) located or positioned at the geometric centroid of interior region 1306(q). In either case, region node 1404(q) can be considered a replacement or substitute for interior region 1306(q), such that any spatial area or portion of tile 902 that was previously represented in a finely divided manner by interior region 1306(q) can now be considered to be coarsely represented by region node 1404(q). As noted above, prior to such replacement or substitution, internal region 1306(q) may have been connected by one or more of the edges 1108 to one or more other of the internal regions 1306 (or the region nodes that replaced them), or to one or more other of the bounding regions 1304. Thus, after such replacement or substitution, region node 1404(q) may be connected by one or more of the edges 1108 to one or more other of the internal regions 1306 (or the region nodes that replaced them), or to one or more other of the bounding regions 1304.

[0099] It should be noted that area nodes 1404(1) through 1404(q) may be thought of collectively as forming multiple area nodes 1404.

[0100] In either case, PRAG 1402 may include multiple bounding regions 1304, multiple edges 1108, and multiple region nodes 1404. Thus, some nodes of PRAG 1402 may be considered to represent individual pixels of tile 902 (e.g., any nodes that make up multiple bounding regions 1304), while other nodes of PRAG 1402 may instead be considered to represent larger multi-pixel swaths or regions of tile 902 (e.g., multiple region nodes 1404). Thus, the term "pixel region adjacency graph" may be considered appropriate. Note that PRAG 1402 may be considered a novel graph data structure that did not previously exist before the inventors devised it.

[0101] 15 and 16. In various embodiments, the graph component 316 can generate the PRAG 1402 for the tile 902, as described above. In various aspects, there may be a tile 1501 (e.g., another of the plurality of tiles 402) that is adjacent to or otherwise abuts the tile 902. In various cases, the graph component 316 can generate the PRAG 1502 for the tile 1501, as described above. In various cases, the PRAG 1502 may include multiple bounding regions 1504, multiple region nodes 1506, and multiple edges 1508. In various aspects, the multiple bounding regions 1504 may include v regions, i.e., bounding region 1504(1) through bounding region 1504(v), for any suitable positive integer v. In various cases, the multiple region nodes 1506 may include w nodes, i.e., region node 1506(1) through region node 1506(w), for any suitable positive integer w. In various cases, the plurality of edges 1508 can be any edges that were deleted and then reinserted by the graph component 316 during construction of the PRAG 1502 .

[0102] In various aspects, because tile 902 and tile 1501 are adjacent or abutting one another in image 304, graph component 316 can merge or combine PRAG 1402 with PRAG 1502. In various instances, such merging or combining can result in merged pixel region adjacency graph 1510 (hereinafter, “merged PRAG 1510”). In various cases, graph component 316 can facilitate such merging or combining by inserting a minimum-weight edge between the corresponding boundary region of PRAG 1402 and the corresponding boundary region of PRAG 1502.

[0103] More specifically, because tile 902 and tile 1501 are adjacent to one another, one particular boundary of tile 902 can be considered to be adjacent to or abutting one particular boundary of tile 1501. As a non-limiting example, assume that tile 902 is immediately to the left of tile 1501. In such a case, the right boundary of tile 902 can be adjacent to or abutting the left boundary of tile 1501. As another non-limiting example, assume that tile 902 is immediately below tile 1501. In such a case, the top boundary of tile 902 can be adjacent to or abutting the bottom boundary of tile 1501. As yet another non-limiting example, assume that tile 902 is immediately above tile 1501. In such a case, the bottom boundary of tile 902 can be adjacent to or abutting the top boundary of tile 1501. As yet another non-limiting example, assume that tile 902 is immediately to the right of tile 1501. In such a case, the left boundary of tile 902 may be adjacent to or abut the right boundary of tile 1501 .

[0104] In either case, the graph component 316 may identify or otherwise know which particular boundaries of the tile 902 are adjacent to or abutting which particular boundaries of the tile 1501. For ease of explanation, these may be referred to as boundary A and boundary B, respectively. That is, boundary A of the tile 902 may be adjacent to or abut boundary B of the tile 1501. In various instances, the graph component 316 may identify or otherwise know which of the plurality of boundary regions 1304 includes any nodes that represent pixels located on boundary A of the tile 902. For ease of explanation, such regions may be referred to as one or more first identified boundary regions, and any nodes within those one or more first identified boundary regions that represent pixels located on boundary A may be referred to as first identified nodes (e.g., if the tile 902 is rectilinear and is to the left of the tile 1501, any node in the PRAG 1402 that represents the rightmost column of pixels of the tile 902 may be considered a first identified node). Similarly, in various cases, graph component 316 may identify or otherwise know which of multiple boundary regions 1504 includes any nodes that represent pixels that lie on boundary B of tile 1501. For ease of explanation, such regions may be referred to as one or more second identified boundary regions, and any node in those one or more second identified boundary regions that represents pixels that lie on boundary B may be referred to as a second identified node (e.g., if tile 1501 is rectilinear and to the right of tile 902, then any node in PRAG 1502 that represents the leftmost column of pixels in tile 1501 may be considered a second identified node).

[0105] Here, the first identified node and the second identified node can be considered to correspond to each other. Indeed, in a situation where tile 902 and tile 1501 are separated from each other or do not overlap, each of the first identified nodes can be considered to be adjacent to or next to a respective one of the second identified nodes. On the other hand, in a situation where tile 902 and tile 1501 overlap (e.g., share a column or row of pixels), each of the first identified nodes can be considered to be a duplicate or copy of a respective one of the second identified nodes. In either case, the first identified nodes can correspond to each of the second identified nodes.

[0106] In various aspects, the graph component 316 can insert a minimum weight edge (e.g., an edge having a weight of 0, or alternatively an edge whose weight is less than any suitable threshold) between a corresponding pair of first identified nodes and a corresponding pair of second identified nodes. For example, for each given one of the first identified nodes, that given first identified node can correspond to each one of the second identified nodes, and an edge of minimum weight (e.g., weight 0 or any other suitable weight) can be inserted between that given first identified node and each of its corresponding second identified nodes. Such minimum weight edge insertion can be considered as joining, merging, or combining one or more first identified boundary regions of PRAG1402 with one or more second identified boundary regions of PRAG1502. After such minimum weight edge insertion, one or more first identified boundary regions of PRAG1402 and one or more second identified boundary regions of PRAG1502 can be considered to now form one or more new regions, and such one or more new regions can be flagged as internal regions or otherwise specified. In various cases, such newly formed internal regions can be referred to as one or more new internal regions 1604. In various aspects, one or more new internal regions 1604 can include t regions, i.e., new internal regions 1604(1) to new internal regions 1604(t), for any suitable positive integer t ≦ max(p, v). In various instances, either the plurality of boundary regions 1304 and the plurality of boundary regions 1504 remaining after such minimum weight edge insertion can be considered collectively as forming one or more remaining boundary regions 1602. In various cases, one or more remaining boundary regions 1602 can include s regions, i.e., remaining boundary regions 1602(1) to remaining boundary regions 1602(s), for any suitable positive integer s < p + v.In fact, the one or more remaining boundary regions 1602 can be viewed as the union of the multiple boundary regions 1304 and the multiple boundary regions 1504, minus any of the multiple boundary regions 1304 and multiple boundary regions 1504 that have been merged via minimum weight edge insertion to create one or more new interior regions 1604.

[0107] Note that minimum-weight edge insertion may, in some cases, cause one or more of the new interior regions 1604 to initially have cycles or cyclic paths. In various aspects, such cycles or cyclic paths may be removed or eliminated by re-running Brufka's algorithm (or any other suitable MST discovery algorithm).

[0108] In either case, the creation of one or more new interior regions 1604 can be viewed as leaving intact the region nodes of PRAG 1402 and PRAG 1502 and the deleted and reinserted edges of PRAG 1402 and PRAG 1502. Thus, as shown, creating a merged PRAG 1510 as described herein can result in the merged PRAG 1510 including multiple region nodes 1404, multiple region nodes 1506, multiple edges 1108, multiple edges 1508, one or more remaining boundary regions 1602, and one or more new interior regions 1604. In various aspects, the graph component 316 can replace or substitute each of the one or more new interior regions 1604 with a respective region node, as described above. As a non-limiting example, graph component 316 can replace new interior region 1604(1) with a first new region node, which can be randomly or non-randomly selected from new interior region 1604(1). As another non-limiting example, graph component 316 can replace new interior region 1604(t) with a t new region node, which can be randomly or non-randomly selected from new interior region 1604(t).

[0109] It should be noted that the merged PRAG1510 can be considered to include fewer boundary regions than the total number of boundary regions over PRAG1402 and PRAG1502 (e.g., s < p + v). In other words, merging or combining the PRAGs described herein can be considered to incrementally or iteratively convert boundary regions into internal regions and thus into region nodes.

[0110] Figures 15 and 16 show how two adjacent PRAGs (e.g., 1402 and 1502) can be merged together to create a new PRAG. It should be understood that any suitable number of adjacent PRAGs (or new PRAGs) can be merged in this manner.

[0111] Now, consider Figure 17. In various embodiments, the graph component 316 can generate respective PAGs for each of the plurality of tiles 402 (as described with respect to Figure 9). This can result in a plurality of PAGs 1702, and the plurality of PAGs 1702 can include n PAGs, namely, PAG1702(1) generated based on tile 402(1) to PAG1702(n) generated based on tile 402(n).

[0112] In various aspects, the graph component 316 can generate respective MSTs for each of the plurality of PAGs 1702 (as described with respect to Figure 10). This can result in a plurality of MSTs 1704, and the plurality of MSTs 1704 can include n MSTs, namely, MST1704(1) generated based on PAG1702(1) to MST1704(n) generated based on PAG1702(n).

[0113] In various cases, the graph component 316 may generate a respective MSF for each of the multiple MSTs 1704 (as described with respect to FIG. 11 ), which may result in multiple MSFs 1706, which may include n MSFs, i.e., MSFs 1706(1) generated based on MSTs 1704(1) through MSFs 1706(n) generated based on MSTs 1704(n).

[0114] In various cases, the graph component 316 may generate a respective flagged MSF for each of the plurality of MSFs 1706 (as described with respect to FIG. 12 ), thereby generating a plurality of flagged MSFs 1708, where the plurality of flagged MSFs 1708 may include n flagged MSFs, i.e., flagged MSFs 1706(1) through MSFs 1708(n), generated based on MSF 1708(1).

[0115] In various aspects, the graph component 316 may generate a respective flagged MST for each of the plurality of flagged MSFs 1708 (as described with respect to FIG. 13 ), which may result in a plurality of flagged MSTs 1710, which may include n flagged MSTs, i.e., flagged MST 1710(1) generated based on flagged MSF 1708(1) through flagged MST 1710(n) generated based on flagged MSF 1708(n).

[0116] In various cases, the graph component 316 can generate a respective PRAG for each of the multiple flagged MSTs 1710 (as described with respect to FIG. 14 ), which can result in multiple PRAGs 1712, which can include n PRAGs, i.e., PRAG 1712(1) generated based on the flagged MST 1710(1) through PRAG 1712(n) generated based on the flagged MST 1710(n).

[0117] In various cases, the graph component 316 may merge or combine all of the multiple PRAGs 1712 together (as described with respect to FIGS. 15 and 16 ). As discussed above, each iteration of such merging or combining can be viewed as incrementally converting the boundary regions across the multiple PRAGs 1712 into area nodes. Thus, when all of the multiple PRAGs 1712 are merged or combined, the final resulting graph structure can be viewed as not including any boundary regions. Instead, that final resulting graph structure can include only area nodes that are joined together by any edges that were removed and then reinserted during PRAG construction. Thus, that final resulting graph structure can be viewed as a RAG 802.

[0118] 18-26 show non-limiting examples of various aspects described herein.

[0119] FIG. 18 shows tile 1800. In particular, tile 1800 is a square tile from a real-world image captured by a real-world scientific instrument. Specifically, tile 1800 was positioned or located in the upper left corner of the real-world image. Thus, no tiles were positioned or located above tile 1800, and no tiles were positioned or located to the left of tile 1800. However, there were tiles positioned or located to the right or below tile 1800. Thus, tile 1800 can be considered to have only two boundaries: a right boundary and a bottom boundary.

[0120] Figure 19 shows the MST 1900 calculated for the tile 1800. Specifically, the black dots in Figure 19 represent the nodes of the MST 1900, and the white lines represent the edges of the MST 1900. To help show how the MST 1900 has spatial significance, Figure 19 depicts the MST 1900 as being superimposed or overlaid on the tile 1800.

[0121] Figure 20 shows flagged MST 2000, calculated from MST 1900. Different shading in Figure 20 indicates edges or nodes that belong to different regions (e.g., different boundary regions, or different interior regions). Again, to help show spatial significance, Figure 20 depicts flagged MST 2000 as superimposed or overlaid on tile 1800.

[0122] FIG. 21 shows a PRAG 2100 computed from a flagged MST 2000. Again, to help illustrate spatial significance, FIG. 21 depicts the PRAG 2100 as superimposed or overlaid on the tile 1800. As shown, the PRAG 2100 includes a high density of nodes along the bottom boundary and along the right boundary of the tile 1800. Such high-density areas of nodes can be considered to belong to the boundary regions of the flagged MST 2000. Also shown, the PRAG 2100 includes a very sparse, low-density of nodes within areas of the tile 1800 that lie outside of those boundary regions. Such sparse, low-density areas of nodes can be considered region nodes into which the interior regions of the flagged MST 2000 have been replaced. In this non-limiting example, each interior region has been replaced with a random node from that interior region. It should be noted that the visibly long edges in Figure 21 can be considered to be any edges that were deleted and then reinserted during the construction of PRAG 2100. In other words, the visibly long edges in Figure 21 can be considered to be any edges that previously connected each interior region of flagged MST 2000 to other interior regions of flagged MST 2000 or to boundary regions of flagged MST 2000, and such edges can be considered to connect each region node of PRAG 2100 to other region nodes or boundary regions.

[0123] Here, Figure 22 depicts tile 2200 that was adjacent to or abutted tile 1800. In particular, tile 2200 was immediately to the right of tile 1800 in the real-world image from which tile 1800 was extracted. Other tiles were to the right of tile 2200 or below tile 2200, but no tile was above tile 2200. Thus, tile 2200 can be considered to have three boundaries: a left boundary, a right boundary, and a bottom boundary.

[0124] FIG. 23 shows a PRAG 2300 computed for tile 2200. As above, to help illustrate spatial significance, FIG. 23 depicts PRAG 2300 as superimposed or overlaid on tile 2200. As shown, PRAG 2300 contains a high density of nodes along the left boundary, along the bottom boundary, and along the right boundary of tile 2200. Such high-density areas of nodes can be considered to belong to the boundary regions of tile 2200. Also shown, PRAG 2300 contains a very sparse, low-density of nodes within areas of tile 2200 that lie outside of those boundary regions. Such sparse, low-density areas of nodes can be considered to be region nodes where the interior regions of tile 2200 have been replaced. As above, note that the visibly long edges in FIG. 23 can be considered to be any edges that have been removed and then reinserted during the construction of PRAG 2300.

[0125] 24 illustrates how PRAG 2100 and PRAG 2300 can be combined into merged PRAG 2400. Indeed, because tile 1800 may be immediately to the left of tile 2200, the right boundary of tile 1800 can be considered adjacent to (or shared with) the left boundary of tile 2200. Thus, one or more bounding regions of PRAG 2100 that include nodes located on the right boundary of tile 1800 can be identified, and one or more bounding regions of PRAG 2300 that include nodes located on the left boundary of tile 2200 can be identified, and the identified bounding regions can be merged or combined via insertion of minimum-weight edges. In some cases, the minimum-weight edges can be considered as stitches that sew PRAG 2100 to PRAG 2300. In either case, any boundary regions of PRAG2100 and PRAG2300 that are merged together can be considered to collectively form one or more new interior regions (e.g., as described above, if minimum-weight edge insertion creates cyclic paths, those cyclic paths can be removed via Brufka's algorithm). Although not shown in Figure 24, those new interior regions can then each be replaced with a new region node.

[0126] FIG. 25 illustrates a merged PRAG 2500 obtained by combining (e.g., via minimum-weight edge insertion) three PRAGs from three different tiles (e.g., a first tile in the upper left of FIG. 25, a second tile to the right of the first tile, and a third tile below the first tile). FIG. 25 can be considered to illustrate a non-limiting example in which more than two PRAGs are merged at a time. In general, any suitable number of PRAGs can be merged at any given time (e.g., not limited to merging only two PRAGs at a time).

[0127] Figure 26 shows RAG 2600, created in the tiled manner described herein, for a real-world image captured by a real-world scientific instrument. As noted above, the white lines in Figure 26 can be considered to represent edges. In particular, the white lines in Figure 26 can be considered to represent edges between respective region nodes in RAG 2600. RAG 2600 can therefore be considered a coarser, and much less memory-intensive, representation of that real-world image.

[0128] In either case, the graph component 316 can generate a RAG 802 for the image 304 based on multiple tiles 402. As described herein, the graph component 316 can achieve such generation by utilizing multiple PRAGs 1712.

[0129] FIG. 27 illustrates a block diagram of an exemplary, non-limiting system including one or more marker-based watershed segmentation basins that can facilitate the computation of a tiled region adjacency graph via a pixel region adjacency graph, according to one or more embodiments described herein.

[0130] In various embodiments, the execution component 318 can electronically perform marker-based watershed segmentation on the RAG 802. In various aspects, such application or performance of marker-based watershed segmentation can result in one or more watershed segmentation basins 2702 (also referred to as one or more marker-based watershed segmentation basins 2702). In various instances, the one or more watershed segmentation basins 2702 can be any suitable electronic data having any suitable format, size, or number of dimensions that can illustrate, depict, represent, or otherwise indicate the location of the marker-based watershed separation lines or boundaries of the RAG 802, and thus the image 304, respectively. Note that because the size of the image 304 can be very large and the processing power of a computerized workstation for the scientific instrument 302 is limited, performing such marker-based watershed segmentation on the entire image 304 may be infeasible. Also, note that due to the phenomenon of marker-based watershed leakage resulting from leaf-global mismatch, performing such marker-based watershed segmentation individually or locally on multiple tiles 402 would be unreliable. Despite these challenges, the system 306 can reliably or accurately generate one or more watershed segmentation basins 2702 through the described implementation of multiple PRAGs 1712. Indeed, existing techniques do not know how to generate RAGs for images in a tiled context. However, the inventors have devised a technique for creating such RAGs, which includes leveraging a novel graph structure that the inventors call PRAGs.

[0131] In various embodiments, the execution component 318 can electronically transmit the one or more watershed segmentation basins 2702 to any suitable computing device. In various other embodiments, the execution component 318 can electronically render the one or more watershed segmentation basins 2702 on any suitable electronic display, such as a computer screen or computer monitor. Thus, any user or technician associated with the scientific instrument 302 can examine or view the one or more watershed segmentation basins 2702.

[0132] It should be understood that the execution component 318, in various embodiments, can perform any other suitable type of analysis on the RAG 802. In other words, the execution component 318 is not limited to performing only marker-based watershed segmentation. Indeed, there may be other types of analysis or algorithms that cannot be adequately or reliably performed in a tile-by-tile manner, but can be adequately or reliably performed on the RAG. Any such analysis or algorithm can be performed or applied by the execution component 318 to the RAG 802.

[0133] Various embodiments described herein may include process flows from tile to PAG, PAG to MST, MST to MSF, MSF to flagged MSF, flagged MSF to flagged MST, and flagged MST to PRAG. It should be understood that this process flow is merely a non-limiting example for ease of explanation and illustration. Indeed, while such process flows serve to elaborate and clarify various concepts conveyed herein, it should be understood that various portions or steps in such process flows may be combined together or even omitted in various embodiments to improve computational efficiency. As a non-limiting example, it may be possible for a PAG to be directly converted to an MSF for any given set of root nodes (e.g., to skip intermediate structures of the MST). As another non-limiting example, it may be possible for a PAG to be directly converted to a flagged MST (e.g., to skip intermediate structures of the MST, MSF, and flagged MSF). In any given implementation of the various embodiments described herein, whatever particular intermediate structures are obtained or omitted, such embodiments may ultimately include computing a PRAG for each tile of the image, which PRAG may include a boundary region of the respective tile, and which PRAG may include a region node in place of an interior region of the respective tile.

[0134] 28-30 illustrate exemplary, non-limiting algorithms that can facilitate the computation of a tiled region adjacency graph via a pixel region adjacency graph, according to one or more embodiments described herein. In particular, FIGS. 28-30 illustrate Algorithm 1, Algorithm 2, Algorithm 3, Algorithm 4, and Algorithm 5, which the inventors utilized to implement various embodiments described herein. The following discussion relates to various aspects of such algorithms, as well as various mathematical notations that may be useful in reading such algorithms. Note that the mathematical notation below may utilize some variables (e.g., i, j, n, k) already utilized above, and may even utilize some variables more than once in different contexts. It should be understood that the mathematical notation is local, not global, unless otherwise specified or explicitly stated. Thus, a single variable (e.g., k) may be used in multiple different places or contexts to mean multiple different things (e.g., the number of roots in one context, a node or pixel in another context, a sum or aggregate index in yet another context). This default to local representation rather than global representation is to avoid situations where you run out of variables.

[0135] A graph G may be a pair G = (V, E), where V and E may both be finite sets. An element v∈V may be called a vertex or node of the graph G, and an element e∈E⊂{{i,j}, for i,j∈V,i≠j} may be called an edge of the graph G. An edge e i,j can be considered to connect or join node i to node j. In such cases, node i and node j can be considered to be adjacent or otherwise near each other. Given a graph G=(V,E), a graph G'=(V',E') can be said to be a subgraph of G if and only if V'⊂V and E'⊂E. An edge-weighted graph G can be a triplet G=(V,E,W), where (V,E) is the graph and W is the weight of the set of edges E.

[0136]

number

[0137]

number

[0138]

number

[0139]

number

[0140]

number

[0141] In various embodiments, the inventors devised Algorithms 1-5, as shown in Figures 28-30, and the inventors utilized such algorithms to implement various embodiments described herein. In particular, Algorithm 3 depicts a version of Brufka's algorithm that can compute an MST (e.g., one of 1704) from a given graph (e.g., one of 1702), Algorithm 4 depicts a modified and novel version of Brufka's algorithm that can compute a flagged MST (e.g., one of 1710) from a given graph (e.g., one of 1702), Algorithm 5 depicts a novel algorithm that can compute a PRAG (e.g., one of 1712) from a given labeled graph (e.g., one of 1710), and Algorithms 1 and 2 depict auxiliary procedures utilized or called by Algorithms 3, 4, or 5.

[0142] Consider Algorithm 3 first. Algorithm 3 aims to compute the MST of a given graph. The input can include a graph with vertices V and edges E, and the output can be the resulting MST. Algorithm 3 can begin by initializing the number of trees to be equal to the number of edges in the graph. Algorithm 3 can then enter a loop that continues until only one tree remains. Within each iteration, Algorithm 3 can perform the following steps. The lowest-weight edge incident to each node can be determined by calling the function “setLowestEdge(V,E)” described in Algorithm 1, and the result can be stored in an array “lowestEdge.” For each node v in V, Algorithm 3 can search the “lowestEdge” array for the lowest-weight edge incident to v. If the searched edge is not equal to −1 (indicating that the edge is found), Algorithm 3 can proceed with processing the edge. Specifically, Algorithm 3 can update the “lowestEdge” array by setting the value of the current vertex v to −1, indicating that the lowest-weight edge of this vertex has been processed. Algorithm 3 can then identify the canonical representations (root nodes) of the source and target vertices of the current edge, denoted as a and b, respectively. If the canonical representations a and b are different, Algorithm 3 can perform a union operation, which can merge the trees rooted at a and b into a single tree. The current edge can then be added to the MST, and the number of trees can be decremented by one. Algorithm 3 can continue this process until only one tree remains, indicating that the MST has been computed. The resulting MST can be stored. Overall, Algorithm 3 (Brufka's algorithm) can iteratively select the lowest-weight edge from each node and merge the corresponding trees, progressively building the MST.

[0143] Now consider Algorithm 4. Algorithm 4 has been devised by the inventors to compute a flagged MST (e.g., one 1710) for any given graph (e.g., one of 1702). The input to Algorithm 4 may include a graph having vertices V and edges E, along with a marker array label and a boundary flag border. The output of Algorithm 4 may be the resulting flagged MST. Algorithm 4 may begin by initializing the number of trees to be the same as the number of edges in the graph. Algorithm 4 may include an operation of updating the marker array label based on the boundary flag border, as shown in Algorithm 2. This operation may be specifically designed or intended to manage the boundary nodes of the graph by treating them as labeled nodes. The expected number of trees may be set to the number of nodes in the graph. Algorithm 4 may then enter a loop that continues until the number of trees in the previous iteration is the same as the number of trees in the current iteration. Within each iteration, Algorithm 4 may perform the following steps: The lowest-weight edge incident to each node can be determined by calling the function "setLowestEdge(V,E)" from Algorithm 1, and the result can be stored in the array "lowestEdge." For each node v in V, Algorithm 4 can search the "lowestEdge" array for the lowest-weight edge incident to v. If the searched edge is not equal to -1 (indicating that the edge is found), Algorithm 4 can proceed to process the edge. Specifically, Algorithm 4 can identify the canonical representations of the source and target vertices of the current edge (root nodes), denoted as a and b, respectively. If the canonical representations a and b are different and their labels are the same (same rooted tree or background), or if the label of the current node v is 0 (rainfall process), Algorithm 4 can perform a union operation using label management.This operation can merge the trees rooted at a and b into a single tree while preserving the labels. The current edge can be added to the MST, and the number of trees can be decremented by one. After computing the minimum spanning forest (MSF), Algorithm 4 can propagate the labels of the canonical representation to each node. Finally, Algorithm 4 can add the remaining edges to construct the MST, thus repeating until only one tree remains. Within each iteration, Algorithm 4 can perform similar steps to the previous one, with the difference being that the label condition can be removed. Overall, Algorithm 4 can compute a flagged MST by utilizing a modified version of Brufka's algorithm that incorporates marker and boundary management.

[0144] Once the flagged MSTs are obtained, they can be simplified into a PRAG by aggregating nodes that belong to the same non-boundary region / tree into their respective region nodes. Thus, a PRAG can be a graph containing as many nodes as there are distinct regions, in addition to any nodes that belong to the bounding tree in the flagged MST. The set of edges can include any edges connecting two nodes that belong to distinct regions / trees, as well as any edges connecting any two nodes that belong to bounding regions / trees. To maintain links between the nodes of a PRAG and image regions (e.g., pixels of any tile this PRAG represents), an array, vertexMap, can be constructed along with the PRAG. The vertexMap can indicate, for each node in the input graph (e.g., for each vertex), the index of the region / tree to which it belongs in the PRAG. This allows per-pixel labeling of tiles to be obtained from the labeling in the PRAG.

[0145] Algorithm 5 is a novel algorithm devised by the inventors to achieve this simplification. As input, Algorithm 5 can take a graph and an array that associates a respective label with each of its nodes. A node that belongs to a bounded region / tree can have the label "B". The edges of the graph can be considered sequentially by Algorithm 5. If a given node of that edge belongs to a bounded region / tree, Algorithm 5 can add the given node to the PRAG. If the given node does not belong to a bounded region / tree, Algorithm 5 can use a map to determine whether the label associated with the given node has already been discovered (e.g., lutLabel can be considered a dummy array used to track whether a label for a particular region / tree is already represented in the PRAG). If the label associated with the given node is not already represented by a region node in the PRAG, Algorithm 5 can add the given node to the PRAG, and the given node can now be considered a region node in the PRAG. A given node may be associated with any area node in the PRAG that is already associated with the given node's label if the label associated with that node is already represented by an area node in the PRAG. Once the nodes of the edge have been processed in this way, the edge itself may be added to the PRAG only if the two nodes of the edge are associated with different nodes in the PRAG (e.g., any edge whose two nodes are represented by the same area node in the PRAG may be omitted from the PRAG).

[0146] Once a PRAG is computed for each tile, all such PRAGs can be merged together, resulting in a RAG that contains only region nodes and the edges connecting them. To do so efficiently, each pair of neighboring tiles can have an overlap of one pixel row or one pixel column, as needed. This can make it possible to match two neighboring PRAGs in linear time, at the expense of maintaining a boundary list containing the indices of the regions / trees shared between the two tiles. Rather than replacing the nodes in a PRAG by duplicating them, equivalent nodes can be connected by zero-weight edges.

[0147] While the disclosure herein primarily describes various embodiments as implementing minimum spanning trees and minimum spanning forests, these are merely non-limiting examples for ease of explanation, and it should be understood that the various embodiments described herein may alternatively be facilitated by using maximum spanning trees or maximum spanning forests instead.

[0148] It should be noted that the terms "pixel adjacency graph" and "pixel region adjacency graph" are utilized herein regardless of whether image 304 is composed of pixels or voxels. This is for ease of explanation. It should be understood that a "pixel adjacency graph" may alternatively be referred to as a "voxel adjacency graph" if desired. Similarly, it should be understood that a "pixel region adjacency graph" may alternatively be referred to as a "voxel region adjacency graph" if desired.

[0149] In various embodiments, the execution component 318 can electronically calculate or compute the amount of computer memory consumed by creating the RAG 802. In particular, the inventors have implemented one or more embodiments using C++.

[0150] In such implementations, boundaries were stored as labels to reduce memory footprint. Six reserved values ​​for the labels were as follows: 0 = intersection, 1 = east boundary, 2 = west boundary, 3 = north boundary, 4 = south boundary, and 5 = background. This labeling scheme avoided storing boundaries in a separate array or table. Additionally, instead of storing a label for each edge, edges could be sorted by label. Indeed, some edges have different source and destination labels and can therefore be assigned label 0, meaning that such an edge is an intersection between two different labels. By sorting edges in this manner, a count for each label can be stored rather than a large array explicitly identifying each label for each edge. For example, two values ​​could be stored: label[0] = 123 and label[1] = 223. These could be interpreted as meaning that the first 123 edges have label 0 and the next 100 edges have label 1. Using this trick, memory consumption associated with label tracking can be significantly compressed or reduced.

[0151] In our implementation, the pipeline responsible for creating and processing tiles was defined using two parameters: tileSize and tileBatching, allowing for fine-grained control of memory consumption. This pipeline was multithreaded, using a thread pool to execute tile processing tasks. In particular, tileBatching represented the number of tiles to be processed in parallel and then merged. As an example, if 12 threads were available, tileBatching could default to 11, and the 12th thread could merge the 11 PRAGs created by the first 11 threads. Thus, if tileSize was set to 1024, 11 tiles of size 1024 x 1024 pixels could be processed in parallel, and once all tiles were processed, the merging thread could merge and simplify them. Using this implementation, the maximum theoretical size of memory consumption could be defined as follows:

[0152]

number

[0153] [Table 1] where inputTileImage can be the size in memory of the tile, and inputSeedImage can be the size in memory of any seed image associated with the tile (e.g., any larger image from which the tile is cropped or extracted). Thus, depending on the input parameters, the maximum memory consumption can be customized or adjusted by tweaking the parameters (e.g., tileSize, tileBatching) according to the memory or processing capabilities of the user's computerized workstation. The inventors have found that for an image of size 20,000 x 20,000 pixels, a tilling size of 1,024 x 1,024 pixels with a total of 12 threads minimizes memory consumption.

[0154] The scientific instrument systems, methods, or techniques disclosed herein may involve interactions with a human user (e.g., via a user local computing device 3320 discussed herein with reference to FIG. 33). These interactions may include providing the user with information (e.g., information about the operation of a scientific instrument such as the scientific instrument 3310 of FIG. 33, information about a sample being analyzed or other tests or measurements performed by the scientific instrument, information retrieved from a local or remote database, or other information), or providing options for the user to enter commands (e.g., to control the operation of a scientific instrument such as the scientific instrument 3310 of FIG. 33 or to control the analysis of data generated by the scientific instrument), queries (e.g., to a local or remote database), or other information. In some embodiments, these interactions may be performed via a graphical user interface (GUI) that includes a visual display on a display device (e.g., display device 3210 discussed herein with reference to FIG. 32) that provides output to a user and / or prompts the user to provide input (e.g., via one or more input devices such as a keyboard, mouse, trackpad, or touchscreen included in other I / O devices 3212 discussed herein with reference to FIG. 32). The scientific instrument systems, methods, or techniques disclosed herein may include any suitable GUI for interaction with a user.

[0155] 31 depicts an exemplary graphical user interface 3100 (hereinafter "GUI 3100") that may be used in implementing some or all of the support methods or techniques disclosed herein, according to various embodiments. In various aspects, the GUI 3100 may be provided on any suitable electronic display (e.g., the display device 3210 discussed herein with reference to FIG. 32) of a computing device (e.g., the computing device 3200 discussed herein with reference to FIG. 32) of a scientific instrument support system (e.g., the scientific instrument support system 3300 discussed herein with reference to FIG. 33), and a user or technician may interact with the GUI 3100 using any suitable input device (e.g., any of the other I / O devices 3212 discussed herein with reference to FIG. 32) and input technique (e.g., cursor movement, motion capture, facial recognition, gesture detection, voice recognition, button actuation).

[0156] GUI 3100 can include a data display area 3102, a data analysis area 3104, a scientific instrument control area 3106, and a settings area 3108. The particular number and arrangement of areas depicted in Figure 31 is merely illustrative, and other embodiments of GUI 3100 can include any number and arrangement of areas, including any desired features.

[0157] The data display area 3102 may display data generated by a scientific instrument (eg, the scientific instrument 3310 discussed herein with reference to FIG. 33).

[0158] Data analysis area 3104 can display any suitable data analysis results (e.g., the results of analyzing the data shown in data display area 3102 or other data). In some embodiments, data display area 3102 and data analysis area 3104 can be combined within GUI 3100 (e.g., to include both data output from a scientific instrument and some analysis of the data in a common graph or area).

[0159] The scientific instrument control area 3106 may include options that allow a user or technician to control scientific instruments (e.g., scientific instruments 3310 discussed herein with reference to FIG. 33). For example, the scientific instrument control area 3106 may include configurable parameters that govern the operation of such scientific instruments (e.g., configurable parameters that govern the voltage or current of the scientific instrument, that govern the internal temperature of the scientific instrument, or that govern the fluid flow rate of the scientific instrument).

[0160] The settings area 3108 may include options that enable a user or technician to control any feature or function of the GUI 3100 (or of other GUIs) or perform common computing operations with respect to the data display area 3102 and the data analysis area 3104 (e.g., saving data to a storage device such as the storage device 3204 discussed herein with reference to FIG. 32 , sending data to another user, labeling data).

[0161] As mentioned above, the scientific instrument module 102 can be implemented by one or more computing devices. Figure 32 is a block diagram of a computing device 3200 that can implement some or all of the scientific instrument methods or techniques disclosed herein, according to various embodiments. In some embodiments, the scientific instrument module 102 can be implemented by a single instance of the computing device 3200, or by multiple instances of the computing device 3200. Furthermore, as discussed below, the computing device 3200 (or multiple instances thereof) that implements the scientific instrument module 102 can be part of one or more of the scientific instrument 3310, user local computing device 3320, service local computing device 3330, or remote computing device 3340 of Figure 33.

[0162] Although computing device 3200 is shown as having several components, any one or more of these components may be omitted or duplicated as suitable for the application and configuration. In some embodiments, some or all of the components included in computing device 3200 may be mounted on one or more motherboards and enclosed in a housing (e.g., comprising plastic, metal, or other material). In some embodiments, several of these components may be fabricated on a single system-on-a-chip (SoC) (e.g., an SoC may include one or more instances of processing device 3202 and one or more instances of storage device 3204). 32, but may include interface circuitry (not shown) for coupling to the one or more omitted components using any suitable interface (e.g., a Universal Serial Bus (USB) interface, a High-Definition Multimedia Interface (HDMI) interface, a Controller Area Network (CAN) interface, a Serial Peripheral Interface (SPI) interface, an Ethernet interface, a wireless interface, or any other suitable interface). For example, computing device 3200 may omit display device 3210, but may include display device interface circuitry (e.g., connectors and driver circuits) to which display device 3210 may be coupled.

[0163] Computing device 3200 may include processing device 3202 (e.g., one or more processing devices). As used herein, the term “processing device” may refer to any device or portion of a device that processes electronic data from registers or memory and converts the electronic data into other electronic data that may be stored in registers or memory. Processing device 3202 may include one or more digital signal processors (DSPs), application specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptoprocessors (dedicated processors that run cryptographic algorithms in hardware), server processors, or any other suitable processing devices.

[0164] The computing device 3200 may include a storage device 3204 (e.g., one or more storage devices). The storage device 3204 may include one or more memory devices, such as random access memory (RAM) (e.g., a static RAM (SRAM) device, a magnetic RAM (MRAM) device, a dynamic RAM (DRAM) device, a resistive RAM (RRAM) device, or a conductive-bridging RAM (CBRAM) device), a hard-drive-based memory device, a solid-state memory device, a network drive, a cloud drive, or any combination of memory devices. In some embodiments, the storage device 3204 may include memory that shares a die with the processing device 3202. In such embodiments, the memory may be used as cache memory and may include, for example, embedded dynamic random access memory (eDRAM) or spin transfer torque magnetic random access memory (STT-MRAM). In some embodiments, storage device 3204 may include a non-transitory computer-readable medium having instructions that, when executed by one or more processing devices (e.g., processing device 3202), cause computing device 3200 to perform any suitable ones or portions thereof of the methods disclosed herein.

[0165] The computing device 3200 may include an interface device 3206 (e.g., one or more instances of an interface device 3206). The interface device 3206 may include one or more communication chips, connectors, or other hardware and software to manage communications between the computing device 3200 and other computing devices. For example, the interface device 3206 may include circuitry for managing wireless communications for transferring data to and from the computing device 3200. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, or communications channels that can communicate data using modulated electromagnetic radiation over a non-solid medium. This term does not imply that the associated devices do not include any wiring, although in some embodiments they may not. The circuitry included in interface device 3206 for managing wireless communications may implement any of several wireless standards or protocols, including, but not limited to, Institute for Electrical and Electronic Engineers (IEEE) standards including Wi-Fi (IEEE 802.11 family), IEEE 802.16 standards (e.g., IEEE 802.16-2005 amendment), the Long Term Evolution (LTE) project (e.g., Advanced LTE project, Ultra Mobile Broadband (UMB) project (also referred to as "3GPP®2")) with any amendments, updates, and / or revisions.In some embodiments, the circuitry included in the interface device 3206 for managing wireless communications is compatible with Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), and other standards. In some embodiments, the interface device 3206 may operate in accordance with a GSM Data Access (HSPA), Evolved HSPA (E-HSPA), or LTE network. In some embodiments, the circuitry included within the interface device 3206 for managing wireless communications may operate in accordance with an Enhanced Data for GSM Evolution (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). In some embodiments, the circuitry included in interface device 3206 for managing wireless communications can operate according to Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Telecommunication (DECT), Evolution-Data Optimized (EV-DO), and derivatives thereof, as well as any other wireless protocols designated as 3G, 4G, 5G, and beyond. In some embodiments, interface device 3206 may include one or more antennas (e.g., one or more antenna arrays) for receiving and / or transmitting wireless communications.

[0166] In some embodiments, the interface device 3206 may include circuitry for managing wired communications, such as electrical, optical, or any other suitable communications protocol. For example, the interface device 3206 may include circuitry supporting communications according to Ethernet technology. In some embodiments, the interface device 3206 may support both wireless and wired communications, or may support multiple wired or multiple wireless communications protocols. For example, a first set of circuits in the interface device 3206 may be dedicated to short-range wireless communications, such as Wi-Fi or Bluetooth, and a second set of circuits in the interface device 3206 may be dedicated to long-range wireless communications, such as global positioning system (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, or others. In some embodiments, the first set of circuits in the interface device 3206 may be dedicated to wireless communications, and the second set of circuits in the interface device 3206 may be dedicated to wired communications.

[0167] The computing device 3200 may include a battery / power circuit 3208. The battery / power circuit 3208 may include one or more energy storage devices (e.g., batteries or capacitors) or circuitry for coupling components of the computing device 3200 to an energy source (e.g., alternating current) remote from the computing device 3200.

[0168] Computing device 3200 may include a display device 3210 (e.g., multiple display devices). Display device 3210 may include any visual indicator, such as a heads-up display, a computer monitor, a projector, a touchscreen display, a liquid crystal display (LCD), a light-emitting diode display, or a flat panel display.

[0169] Computing device 3200 may include other input / output (I / O) devices 3212. Other I / O devices 3212 may include, for example, one or more audio output devices (e.g., speakers, headsets, earphones, alarms), one or more audio input devices (e.g., microphones or microphone arrays), a location device (e.g., a GPS device that communicates with a satellite-based system to receive the location of computing device 3200), an audio codec, a video codec, a printer, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, accelerometers, gyroscopes), an image capture device such as a camera, a cursor control device such as a keyboard, mouse, stylus, trackball, or touchpad, a barcode reader, a Quick Response (QR) code reader, or a radio frequency identification (RFID) reader.

[0170] The computing device 3200 may have any suitable form factor for its use and configuration, such as a handheld or mobile computing device (e.g., a mobile phone, a smartphone, a mobile internet device, a tablet computer, a laptop computer, a netbook computer, an ultrabook computer, a personal digital assistant (PDA), an ultra-mobile personal computer), a desktop computing device, or a server computing device or other networked computing component.

[0171] One or more computing devices implementing any of the scientific instrument modules, methods, or techniques disclosed herein may be part of a scientific instrument support system. Figure 33 is a block diagram of an exemplary scientific instrument support system 3300 in which some or all of the scientific instrument support methods disclosed herein may be implemented, according to various embodiments. The scientific instrument modules, methods, or techniques disclosed herein (e.g., scientific instrument module 102, computer-implemented method 200, system 306) may be implemented by one or more of the scientific instruments 3310, user local computing devices 3320, service local computing devices 3330, or remote computing devices 3340 of the scientific instrument support system 3300.

[0172] Any of the scientific instrument 3310, the user local computing device 3320, the service local computing device 3330, or the remote computing device 3340 may include any of the embodiments of computing device 3200, and any of the scientific instrument 3310, the user local computing device 3320, the service local computing device 3330, or the remote computing device 3340 may take the form of any suitable of the embodiments of computing device 3200.

[0173] The scientific instrument 3310, the user local computing device 3320, the service local computing device 3330, or the remote computing device 3340 may each include a processing device 3302, a storage device 3304, and an interface device 3306. The processing device 3302 may take any suitable form, including any form of processing device 3202, and the processing devices 3302 included in different ones of the scientific instrument 3310, the user local computing device 3320, the service local computing device 3330, or the remote computing device 3340 may take the same form or different forms. The storage device 3304 may take any suitable form, including any form of storage device 3204, and the storage devices 3304 included in different ones of the scientific instrument 3310, the user local computing device 3320, the service local computing device 3330, or the remote computing device 3340 may take the same form or different forms. The interface device 3306 may take any suitable form, including any form of interface device 3206, and the interface devices 3306 included in different ones of the scientific instrument 3310, the user local computing device 3320, the service local computing device 3330, or the remote computing device 3340 may take the same or different forms.

[0174] The scientific instrument 3310, the user local computing device 3320, the service local computing device 3330, and the remote computing device 3340 can communicate with other elements of the scientific instrument support system 3300 via communication paths 3308. The communication paths 3308 may communicatively couple the interface devices 3306 of different ones of the elements of the scientific instrument support system 3300, as shown, and may be wired or wireless communication paths (e.g., by any of the communication techniques discussed herein with reference to the interface device 3206). While the particular scientific instrument support system 3300 depicted in FIG. 33 includes communication paths between each pair of the scientific instrument 3310, the user local computing device 3320, the service local computing device 3330, and the remote computing device 3340, this “fully connected” implementation is merely illustrative, and in various embodiments, various ones of the communication paths 3308 may not be present. For example, in some embodiments, the service local computing device 3330 may lack a direct communication path 3308 between its interface device 3306 and the interface device 3306 of the scientific instrument 3310, but instead may communicate with the scientific instrument 3310 via a communication path 3308 between the service local computing device 3330 and the user local computing device 3320, and a communication path 3308 between the user local computing device 3320 and the scientific instrument 3310.

[0175] Scientific instrument 3310 may include any suitable scientific instrument, such as scientific instrument 302.

[0176] The user local computing device 3320 may be a computing device that is local to the user of the scientific instrument 3310 (e.g., according to any of the embodiments of the computing device 3200). In some embodiments, the user local computing device 3320 may also be local to the scientific instrument 3310, but need not be; for example, a user local computing device 3320 in a user's home or office may be remote from but in communication with the scientific instrument 3310 such that the user may control or access data from the scientific instrument 3310 using the user local computing device 3320. In some embodiments, the user local computing device 3320 may be a laptop, smartphone, or tablet device. In some embodiments, the user local computing device 3320 may be a portable computing device.

[0177] The service local computing device 3330 may be a computing device (e.g., according to any of the embodiments of computing device 3200) that is local to an entity that provides services to the scientific instrument 3310. For example, the service local computing device 3330 may be local to the manufacturer of the scientific instrument 3310 or a third-party service company. In some embodiments, the service local computing device 3330 may communicate with the scientific instrument 3310, the user local computing device 3320, or the remote computing device 3340 (e.g., via a direct communication path 3308 or via multiple "indirect" communication paths 3308, as discussed above) to receive data regarding the operation of the scientific instrument 3310, the user local computing device 3320, or the remote computing device 3340 (e.g., results of self-tests of the scientific instrument 3310, calibration coefficients used by the scientific instrument 3310, measurements of sensors associated with the scientific instrument 3310, etc.). In some embodiments, the service local computing device 3330 can communicate with the scientific instrument 3310, the user local computing device 3320, or the remote computing device 3340 (e.g., via a direct communication path 3308 or via multiple "indirect" communication paths 3308, as discussed above) to send data to the scientific instrument 3310, the user local computing device 3320, or the remote computing device 3340 (e.g., to update programmed instructions such as firmware in the scientific instrument 3310, to initiate the performance of a test or calibration sequence in the scientific instrument 3310, to update programmed instructions such as software in the user local computing device 3320 or the remote computing device 3340).A user of the scientific instrument 3310 may utilize the scientific instrument 3310 or user local computing device 3320 to communicate with the service local computing device 3330 to report problems with the scientific instrument 3310 or user local computing device 3320, request a technician visit to improve the operation of the scientific instrument 3310, order consumables or replacement parts related to the scientific instrument 3310, or for other purposes.

[0178] The remote computing device 3340 may be a computing device (e.g., according to any of the embodiments of computing device 3200 discussed herein) that is remote from the scientific instrument 3310 or the user local computing device 3320. In some embodiments, the remote computing device 3340 may be included in a data center or other large-scale server environment. In some embodiments, the remote computing device 3340 may include network-attached storage (e.g., as part of the storage device 3304). The remote computing device 3340 may store data generated by the scientific instrument 3310, perform analysis of data generated by the scientific instrument 3310 (e.g., according to programmed instructions), facilitate communications between the user local computing device 3320 and the scientific instrument 3310, or facilitate communications between the service local computing device 3330 and the scientific instrument 3310.

[0179] In some embodiments, one or more of the elements of the scientific instrument support system 3300 shown in Figure 33 may be omitted. Furthermore, in some embodiments, more than one of various of the elements of the scientific instrument support system 3300 of Figure 33 may be present. For example, the scientific instrument support system 3300 may include multiple user local computing devices 3320 (e.g., different user local computing devices 3320 associated with different users or in different locations). In another example, the scientific instrument support system 3300 may include multiple scientific instruments 3310 that all communicate with a service local computing device 3330 and / or a remote computing device 3340; in such an embodiment, the service local computing device 3330 may monitor these multiple scientific instruments 3310, and the service local computing device 3330 may trigger updates or "broadcast" other information to the multiple scientific instruments 3310 simultaneously. Different ones of the scientific instruments 3310 in the scientific instrument support system 3300 can be located near each other (e.g., in the same room) or far from each other (e.g., on different floors of a building, in different buildings, in different cities, etc.). In some embodiments, the scientific instruments 3310 can be connected to an Internet of Things (IoT) stack that enables command and control of the scientific instruments 3310 through web-based applications, virtual reality or augmented reality applications, mobile applications, or desktop applications. Any of these applications can be accessed by a user operating a user local computing device 3320 that communicates with the scientific instruments 3310 by way of an intervening remote computing device 3340. In some embodiments, the scientific instruments 3310 can be sold by a manufacturer as part of a local scientific instrument computing unit 3312, along with one or more associated user local computing devices 3320.

[0180] In some embodiments, different ones of the scientific instruments 3310 included in the scientific instrument support system 3300 may be different types of scientific instruments 3310. For example, one scientific instrument 3310 may be a mass spectrometer and another scientific instrument 3310 may be a chromatograph or an autosampler. In some such embodiments, the remote computing device 3340 or the user local computing device 3320 may combine data from the different types of scientific instruments 3310 included in the scientific instrument support system 3300.

[0181] In various instances, machine learning algorithms or models may be implemented in any suitable manner to facilitate any suitable aspects described herein. To facilitate some of the above-described machine learning aspects of various embodiments, consider the following discussion of artificial intelligence (AI). Various embodiments described herein may facilitate automating one or more features or functions using artificial intelligence. Components may use various AI-based schemes to perform various embodiments / examples disclosed herein. To provide or assist in many of the decisions described herein (e.g., determining, ascertaining, inferring, calculating, predicting, foreseeing, estimating, deriving, forecasting, detecting, calculating), components described herein may examine all or a subset of the data to which they are permitted access and may provide or determine inferences about the state of a system or environment from a set of observations as captured via events or data. The decisions may, for example, be used to identify a particular context or action, or may generate a probability distribution over states. The decisions may be probabilistic, i.e., the calculation of a probability distribution over states of interest based on a consideration of data and events. Determination can also refer to the techniques used to compose higher level events from a set of events or data.

[0182] Such determination can result in the construction of new events or actions from sets of observed events or stored event data, regardless of whether the events are closely correlated in time and whether the events and data originate from one or more event and data sources. The components disclosed herein can employ a variety of classification (explicitly trained (e.g., via training data) and implicitly trained (e.g., via observing behavior, preferences, historical information, receiving external information, etc.)) schemes or systems (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines, etc.) in connection with performing automated or determined actions in connection with the claimed subject matter. Thus, classification schemes or systems can be used to automatically learn and perform certain functions, actions, or decisions.

[0183] The classifier is based on an input attribute vector z = (z1, z2, z3, z4, z n ) can be mapped to a confidence that the input belongs to a class, such that f(z) = confidence(class). Such classification can use probability or statistics-based analysis (e.g., taking into account analytical utility and cost) to determine the action to be automatically taken. A support vector machine (SVM) can be an example of a classifier that can be used. SVMs operate by finding a hypersurface in the space of possible inputs, which hypersurface attempts to split triggering criteria from non-triggering events. Intuitively, this makes the classification correct for test data that is close but not identical to the training data. Other directed and undirected model classification techniques include, for example, Naive Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, or probabilistic classification models that provide independence of different patterns, any of which can be used. Classification as used herein also includes statistical regression utilized to develop models of priority.

[0184] To provide additional context for the various embodiments described herein, Figure 34 and the following discussion are intended to provide a brief, general description of a suitable computing environment 3400 in which various embodiments described herein may be implemented. While the embodiments are described above in the general context of computer-executable instructions executing on one or more computers, those skilled in the art will recognize that the embodiments may also be implemented in combination with other program modules or as a combination of hardware and software.

[0185] Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the methods of the present invention may be practiced with other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, Internet of Things (IoT) devices, distributed computing systems, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, etc., each of which may be operatively coupled to one or more associated devices.

[0186] The illustrated embodiments of the present disclosure may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0187] A computing device typically includes a variety of media, which may include computer-readable storage media, machine-readable storage media, or communication media, and these two terms are used interchangeably herein as follows. A computer-readable storage medium or machine-readable storage medium may be any available storage medium that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. By way of example and not limitation, a computer-readable storage medium or machine-readable storage medium may be implemented in connection with any method or technology for storing information, such as computer-readable or machine-readable instructions, program modules, structured or unstructured data, etc.

[0188] The computer readable storage medium includes, but is not limited to, a random access memory (RAM). The term "tangible" or "non-transitory" may include, for example, a programmable storage medium (e.g., a storage medium, a memory, a network interface, a network controller, a network controller), a programmable logic unit (PLC), a programmable logic controller ...

[0189] The computer-readable storage medium may be accessed by one or more local or remote computing devices for various operations on the information stored by the medium, for example, via access requests, queries, or other data retrieval protocols.

[0190] Communication media typically embodies computer-readable instructions, data structures, program modules, or other structured or unstructured data in a data signal, such as a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery or transmission media. The term "modulated data signal" or signal refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal or signals. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.

[0191] 34 , an exemplary environment 3400 for implementing various embodiments of the aspects described herein includes a computer 3402, which includes a processing unit 3404, a system memory 3406, and a system bus 3408. The system bus 3408 couples system components, including but not limited to the system memory 3406, to the processing unit 3404. The processing unit 3404 can be any of a variety of commercially available processors. Dual microprocessors and other multi-processor architectures can also be used as the processing unit 3404.

[0192] The system bus 3408 may be any of several types of bus structures that may be further interconnected to a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory 3406 includes a ROM 3410 and a RAM 3412. The basic input / output system (BIOS) may be stored in non-volatile memory such as a ROM, erasable programmable read-only memory (EPROM), or EEPROM, and contains the basic routines that help to transfer information between elements within the computer 3402, such as during start-up. The RAM 3412 may also include a high-speed RAM such as static RAM for caching data.

[0193] The computer 3402 further includes an internal hard disk drive (HDD) 3414 (e.g., EIDE, SATA), one or more external storage devices 3416 (e.g., a magnetic floppy disk drive (FDD) 3416, a memory stick or flash drive reader, a memory card reader, etc.), and a drive 3420, e.g., a solid state drive, optical disk drive, etc., that can read from or write to a disk 3422, such as a CD-ROM disk, DVD, BD, etc. Alternatively, if a solid state drive is included, the disk 3422 is not included unless it is separate. Although the internal HDD 3414 is shown as being located within the computer 3402, the internal HDD 3414 can also be configured for external use within a suitable chassis (not shown). Additionally, although not shown in the environment 3400, a solid state drive (SSD) can be used in addition to or in place of the HDD 3414. HDD 3414, external storage device 3416, and drive 3420 can be connected to system bus 3408 by HDD interface 3424, external storage interface 3426, and drive interface 3428, respectively. Interface 3424 for external drive implementations may include at least one or both of Universal Serial Bus (USB) and Institute of Electrical and Electronics Engineers (IEEE) 1394 interface technologies. Other external drive connection technologies are within the contemplation of the embodiments described herein.

[0194] The drives and their associated computer-readable storage media provide non-volatile storage of data, data structures, computer-executable instructions, etc. In the case of computer 3402, the drives and storage media accommodate the storage of any data in a suitable digital format. While the above description of computer-readable storage media refers to each type of storage device, it should be understood by those skilled in the art that other types of storage media that are readable by a computer, whether currently existing or developed in the future, may also be used in the exemplary operating environment, and further, any such storage media may include computer-executable instructions for implementing the methods described herein.

[0195] A number of program modules can be stored in the drives and RAM 3412, including an operating system 3430, one or more application programs 3432, other program modules 3434, and program data 3436. All or portions of the operating system, applications, modules, or data can also be cached in RAM 3412. The systems and methods described herein can be implemented using various commercially available operating systems or combinations of operating systems.

[0196] Computer 3402 may optionally include emulation technology. For example, a hypervisor (not shown) or other intermediary may emulate the hardware environment of operating system 3430, and the emulated hardware may optionally differ from the hardware shown in FIG. 34. In such an embodiment, operating system 3430 may include one of multiple virtual machines (VMs) hosted on computer 3402. Additionally, operating system 3430 may provide a runtime environment, such as the Java Runtime Environment or the .NET Framework, for application 3432. A runtime environment is a consistent execution environment that allows application 3432 to run on any operating system that includes the runtime environment. Similarly, operating system 3430 may support containers, and application 3432 may be in the form of a container, which is a lightweight, standalone executable package of software that includes, for example, code, runtime, system tools, system libraries, and settings for the application.

[0197] Additionally, computer 3402 may be enabled with a security module such as a trusted processing module (TPM). For example, with a TPM, a boot component may hash the next boot component in time and wait for the result to match a secure value before loading the next boot component. This process may occur at any layer within the code execution stack of computer 3402, and may be applied at the application execution level or the operating system (OS) kernel level, thereby enabling security at any level of code execution.

[0198] A user can enter commands and information into the computer 3402 through one or more wired / wireless input devices, such as a keyboard 3438, a touch screen 3440, and a pointing device such as a mouse 3442. Other input devices (not shown) can include a microphone, an infrared (IR) remote control, a radio frequency (RF) remote control, or other remote control, a joystick, a virtual reality controller or headset, a game pad, a stylus pen, an image input device such as a camera, a gesture sensor input device, a visual motion sensor input device, an emotion or face detection device, a biometric input device such as a fingerprint or iris scanner, etc. These and other input devices are often connected to the processing unit 3404 via an input device interface 3444, which can be coupled to the system bus 3408, but can also be connected by other interfaces, such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, a BLUETOOTH interface, etc.

[0199] A monitor 3446 or other type of display device can also be connected to the system bus 3408 via an interface, such as a video adapter 3448. In addition to the monitor 3446, computers typically include other peripheral output devices (not shown), such as speakers, printers, etc.

[0200] The computer 3402 can operate in a networked environment using logical connections via wired or wireless communications to one or more remote computers, such as a remote computer 3450. The remote computer 3450 can be a workstation, a server computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other common network node, and typically includes many or all of the elements described relative to the computer 3402, although for simplicity, only a memory / storage device 3452 is shown. The depicted logical connections include wired and wireless connections to a local area network (LAN) 3454 or larger networks, e.g., a wide area network (WAN) 3456. Such LAN and WAN networking environments are commonplace in offices and businesses and facilitate enterprise-wide computer networks, such as intranets, all of which may connect to a global communications network, e.g., the Internet.

[0201] When used in a LAN networking environment, the computer 3402 can be connected to the local network 3454 through a wired or wireless communication network interface or adapter 3458. The adapter 3458 can facilitate wired or wireless communication to the LAN 3454, and the LAN can also include a wireless access point (AP) disposed thereon for communicating with the adapter 3458 in a wireless mode.

[0202] When used in a WAN networking environment, the computer 3402 may include a modem 3460 or may be connected to a communications server on the WAN 3456 via other means for establishing communications over the WAN 3456, such as via the Internet. The modem 3460 may be internal or external, and a wired or wireless device, and may be connected to the system bus 3408 via the input device interface 3444. In a networked environment, program modules depicted relative to the computer 3402, or portions thereof, may be stored in the remote memory / storage device 3452. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between computers may be used.

[0203] When used in either a LAN or WAN networking environment, computer 3402 can access a cloud storage system or other network-based storage system in addition to, or instead of, an external storage device 3416 as described above, such as, but not limited to, a networked virtual machine that provides one or more aspects of information storage or processing. Generally, a connection between computer 3402 and a cloud storage system can be established via LAN 3454 or WAN 3456, respectively, by, for example, an adapter 3458 or modem 3460. Upon connecting computer 3402 to an associated cloud storage system, external storage interface 3426, with the aid of adapter 3458 or modem 3460, can manage the storage provided by the cloud storage system in the same way as other types of external storage. For example, external storage interface 3426 can be configured to provide access to cloud storage sources as if those sources were physically connected to computer 3402.

[0204] The computer 3402 may be operable to communicate with any wireless device or entity operably arranged in wireless communication, such as a printer, a scanner, a desktop or portable computer, a portable data assistant, a communications satellite, any equipment or location associated with a wirelessly detectable tag (e.g., a kiosk, a newsstand, a store shelf, etc.), and a telephone. This may include Wireless Fidelity (Wi-Fi) and BLUETOOTH® wireless technologies. Thus, communication may be of a predefined structure, similar to a traditional network, or simply ad-hoc communication between at least two devices.

[0205] FIG. 35 is a schematic block diagram of a sample computing environment 3500 with which the disclosed subject matter can interact. The sample computing environment 3500 includes one or more client(s) 3510. The client(s) 3510 can be hardware or software (e.g., threads, processes, computing devices). The sample computing environment 3500 also includes one or more server(s) 3530. The server(s) 3530 can also be hardware or software (e.g., threads, processes, computing devices). The server(s) 3530 can house threads for performing transformations, for example, by employing one or more embodiments as described herein. One possible communication between the client(s) 3510 and the server(s) 3530 can be in the form of a data packet adapted to be transmitted between two or more computer processes. The sample computing environment 3500 includes a communication framework 3550 that can be employed to facilitate communication between the client(s) 3510 and the server(s) 3530. The client(s) 3510 are operably connected to one or more client data store(s) 3520 that can be employed to store information local to the client(s) 3510. Similarly, the server(s) 3530 are operatively connected to one or more server data store(s) 3540 that can be employed to store information local to the servers 3530 .

[0206] Various embodiments may be systems, methods, apparatus, or computer program products integrated at any possible level of technical detail. A computer program product may include a computer-readable storage medium having computer-readable program instructions for causing a processor to perform aspects of various embodiments. A computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction-execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media may also include portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile discs (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through wires.

[0207] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers. A network adapter card or network interface within each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device. The computer-readable program instructions for performing the operations of various embodiments can be either assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry to perform various aspects.

[0208] Various aspects are described herein with reference to flowchart diagrams or block diagrams of methods, apparatus (systems), and computer program products according to various embodiments. It will be understood that each block of the flowchart diagrams or block diagrams, and combinations of blocks in the flowchart diagrams or block diagrams, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to manufacture a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowchart or block diagram. These computer-readable program instructions can also be stored on a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, or other device to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in one or more blocks of the flowchart or block diagram. The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to generate a computer-implemented process, causing a series of operational operations to be performed on the computer, other programmable apparatus, or other device, such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / operations specified in one or more blocks of the flowcharts or block diagrams.

[0209] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of a block or flowchart diagram, and combinations of blocks in a block or flowchart diagram, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or a combination of dedicated hardware and computer instructions.

[0210] Although the present subject matter has been described above in the general context of computer-executable instructions for a computer program product running on one or more computers, those skilled in the art will recognize that the present disclosure may be implemented in combination with other program modules. Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that various aspects may be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputing devices, mainframe computers, as well as computers, handheld computing devices (e.g., PDAs, phones), microprocessor-based or programmable consumer or industrial electronic devices, and the like. The illustrated aspects may also be practiced in distributed computing environments where tasks are performed by remote processing devices linked through a communications network. However, some, if not all, aspects of the present disclosure may be practiced on stand-alone computers. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0211] As used herein, terms such as “component,” “system,” “platform,” and “interface” can refer to or include computer-related entities or entities associated with an operating machine having one or more specific functions. The entities disclosed herein can be hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, a thread of execution, a program, or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components can reside within a process or thread of execution, and a component can be localized on one computer or distributed between two or more computers. In another example, each component can execute from various computer-readable media having various data structures stored thereon. Components can communicate via local or remote processes, such as following signals comprising one or more data packets (e.g., data from one component interacting with another component in a local system, a distributed system, or with another system via signals over a network such as the Internet). As another example, a component may be a device having a particular functionality provided by mechanical parts operated by electrical or electronic circuitry operated by a software or firmware application executed by a processor. In such cases, the processor may be internal or external to the device and may execute at least a portion of the software or firmware application. As yet another example, a component may be a device that provides a particular functionality through electronic parts without mechanical parts, and the electronic parts may include a processor or other means for executing software or firmware that provides at least part of the functionality of the electronic parts.In one aspect, the components may emulate electronic components, for example, via virtual machines in a cloud computing system.

[0212] Additionally, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from context, "X employs A or B" is intended to mean all natural inclusive permutations. That is, if X employs A, then X employs B, or if X employs both A and B, then "X employs A or B" is satisfied in either of the foregoing cases. As used herein, the term "and / or" is intended to have the same meaning as "or." Furthermore, the articles "a" and "an," as used in this specification and the accompanying drawings, should generally be interpreted to mean "one or more" unless otherwise specified or clear from context as indicating the singular form. As used herein, the terms "example" or "exemplary" are utilized to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples. Additionally, any aspect or design described herein as an "example" or "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs, and is not meant to exclude equivalent exemplary structures and techniques known to those skilled in the art.

[0213] The disclosure herein describes non-limiting examples. For ease of description or explanation, various portions of the disclosure herein utilize the terms "each," "every," or "all" when discussing various examples. Such use of the terms "each," "every," or "all" is non-limiting. In other words, when the disclosure herein provides a description that applies to "each," "every," or "all" of several particular objects or components, it should be understood that this is a non-limiting example, and it should be further understood that in various other examples, such a description may apply to less than "each," "every," or "all" of the particular objects or components.

[0214] As used herein, the term "processor" can refer to virtually any computing processing unit or device, including, but not limited to, a single-core processor, a single processor with software multithreading execution capabilities, a multi-core processor, a multi-core processor with software multithreading execution capabilities, a multi-core processor with hardware multithreading techniques, and a parallel platform with distributed shared memory. Additionally, a processor can refer to an integrated circuit, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Furthermore, a processor can utilize nanoscale architectures, such as, but not limited to, molecular and quantum dot-based transistors, switches, and gates, to optimize space usage or improve performance of user equipment. A processor can also be implemented as a combination of computing processing units. In this disclosure, terms such as "store," "storage," "data store," "data storage," "database," and substantially any other information storage component associated with the operation and functionality of a component are utilized to refer to a "memory component," an entity embodied in a "memory," or a component comprising a memory. It should be understood that a memory or memory component described herein may be either volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.By way of example, and not limitation, non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or non-volatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM)). Volatile memory may include, for example, RAM, which may act as external cache memory. By way of example, and not limitation, RAM is available in many forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). Additionally, the disclosed memory components of the systems or computer-implemented methods herein are intended to comprise, but are not limited to, these and any other suitable types of memory.

[0215] What has been described above includes merely example systems and computer-implemented methods. Of course, for purposes of describing this disclosure, it is not possible to describe every conceivable combination of components or computer-implemented methods, but many further combinations and permutations of the present disclosure are possible. Furthermore, to the extent that terms such as "including," "having," "possessing," and the like are used in the detailed description, claims, appendices, and drawings, such terms are intended to be as inclusive as the term "comprising," as the term "comprises" is interpreted when used as a transitional term in the claims.

[0216] The description of various embodiments is presented for purposes of illustration and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications or technical improvements to technology found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0217] Various non-limiting embodiments are described in the following examples.

[0218] Example 1: A system can include a processor that executes computer-executable components stored in non-transitory computer-readable memory, the computer-executable components including an access component that can access an image produced by a scientific instrument and an execution component that can perform marker-based watershed segmentation on a region adjacency graph of the image, where the region adjacency graph can be constructed from a plurality of pixel region adjacency graphs that respectively correspond to a plurality of tiles of the image.

[0219] Example 2: The system described in Example 1 can be implemented where the computer-executable components further include a tile component that can decompose an image into multiple tiles.

[0220] Example 3: The system described in Example 1 or 2 may be implemented in which the computer-executable component further includes a graph component capable of generating multiple pixel adjacency graphs based on multiple tiles, and for a first pixel adjacency graph corresponding to a first tile, a node of the first pixel adjacency graph may represent each pixel or voxel of the first tile.

[0221] Example 4: A system described in any of Examples 1 to 3 can be implemented in which the graph component can generate multiple minimum spanning forests based on multiple pixel adjacency graphs, and for a first minimum spanning forest corresponding to a first pixel adjacency graph, the first minimum spanning forest can include one or more boundary trees and one or more internal trees.

[0222] Example 5: The system according to any of Examples 1 to 4 may be implemented in which the graph component is capable of generating multiple minimum spanning forests via executing Brufka's algorithm.

[0223] Example 6: A system described in any of Examples 1 to 5 can be implemented in which the graph component can generate multiple pixel region adjacency graphs based on multiple minimum spanning forests, and for a first pixel region adjacency graph corresponding to a first minimum spanning forest, the first pixel region adjacency graph can include one or more boundary regions corresponding respectively to one or more boundary trees, and a set of nodes per region into which one or more internal trees are condensed.

[0224] Example 7: A system described in any of Examples 1-6 may be implemented in which the graph component can merge multiple pixel region adjacency graphs into a region adjacency graph by combining boundary regions of adjacent tiles, re-flag such boundary regions as new interior regions, and condense such new interior regions into new per-region nodes.

[0225] Example 8: The system according to any one of Examples 1 to 7 can be implemented in which the execution component can calculate the amount of memory consumption involved in creating the region adjacency graph.

[0226] In various embodiments, any combination or combinations of Examples 1 to 8 may be implemented.

[0227] Example 9: A computer-implemented method may include accessing, by a device operatively coupled to a processor, an image generated by a scientific instrument; and performing, by the device, marker-based watershed segmentation on a region adjacency graph of the image, wherein the region adjacency graph may be constructed from a plurality of pixel region adjacency graphs corresponding respectively to a plurality of tiles of the image.

[0228] Example 10: The device may implement the computer-implemented method of Example 9, further comprising decomposing the image into a plurality of tiles.

[0229] Example 11: The device may implement the computer-implemented method described in Example 9 or 10, further comprising generating a plurality of pixel adjacency graphs based on a plurality of tiles, wherein for a first pixel adjacency graph corresponding to a first tile, a node of the first pixel adjacency graph may represent each pixel or voxel of the first tile.

[0230] Example 12: The computer-implemented method of any of Examples 9 to 11 may be implemented by a device, further comprising generating a plurality of minimum spanning forests based on a plurality of pixel adjacency graphs, where for a first minimum spanning forest corresponding to a first pixel adjacency graph, the first minimum spanning forest may include one or more boundary trees and one or more interior trees.

[0231] Example 13: The computer-implemented method according to any of Examples 9 to 12 may be implemented in which a device generates a plurality of minimum spanning forests via executing Brufka's algorithm.

[0232] Example 14: The computer-implemented method of any of Examples 9 to 13 may be implemented by the device, further comprising generating a plurality of pixel region adjacency graphs based on a plurality of minimum spanning forests, wherein for a first pixel region adjacency graph corresponding to the first minimum spanning forest, the first pixel region adjacency graph may include one or more boundary regions each corresponding to one or more boundary trees, and a set of nodes per region into which one or more internal trees are condensed.

[0233] Example 15: The device may implement the computer-implemented method of any of Examples 9-14, further comprising merging multiple pixel region adjacency graphs into a region adjacency graph by combining boundary regions of adjacent tiles, re-flag such boundary regions as new interior regions, and condense such new interior regions into new per-region nodes.

[0234] Example 16: The computer-implemented method according to any of Examples 9 to 15 can be implemented, where the image can be an electron tomography image, an X-ray tomography image, or a confocal microscope image.

[0235] In various embodiments, any combination or combinations of Examples 9 to 16 may be implemented.

[0236] Example 17: A computer program product for facilitating computation of a tiled region adjacency graph via a pixel region adjacency graph may comprise a non-transitory computer-readable memory having embedded thereon program instructions. In various aspects, the program instructions may be executable by a processor to cause the processor to access an image captured by a charged particle microscope and to construct a region adjacency graph for the image in a tile-by-tile manner based on a plurality of pixel region adjacency graphs.

[0237] Example 18: Program instructions include causing a processor to: decompose an image into a plurality of tiles; generate, based on the plurality of tiles, a plurality of pixel adjacency graphs, where for a first pixel adjacency graph corresponding to a first tile, a node of the first pixel adjacency graph represents each pixel or voxel of the first tile; generate, based on the plurality of pixel adjacency graphs, a plurality of minimum spanning forests, where for the first minimum spanning forest corresponding to the first pixel adjacency graphs, the first minimum spanning forest includes one or more boundary trees and one or more interior trees; The computer program product described in Example 17 may be implemented, which may be further executable to generate a plurality of pixel region adjacency graphs, where for a first pixel region adjacency graph corresponding to the first minimum spanning forest, the first pixel region adjacency graph includes one or more boundary regions each corresponding to one or more boundary trees and a set of nodes per region into which one or more internal trees are condensed, and to merge the plurality of pixel region adjacency graphs into the region adjacency graph by combining the boundary regions of adjacent tiles, re-flag such boundary regions as new internal regions, and condense such new internal regions into new nodes per region.

[0238] Example 19: The computer program product of Example 17 or 18 may be implemented, wherein the program instructions may be further executable to cause the processor to segment the image based on a region adjacency graph.

[0239] Example 20: The computer program product of any of Examples 17-19 may be implemented, wherein the processor is capable of segmenting the image via marker-based watershed segmentation.

[0240] In various embodiments, any combination or combinations of Examples 17-20 may be implemented.

[0241] In various embodiments, any combination or combinations of Examples 1-20 may be implemented.

Claims

1. 1. A system comprising: a processor executing computer-executable components stored in non-transitory computer-readable memory, the computer-executable components comprising: an access component for accessing images produced by the scientific instrument; an execution component that performs marker-based watershed segmentation on a region adjacency graph of the image, wherein the region adjacency graph is constructed from a plurality of pixel region adjacency graphs that respectively correspond to a plurality of tiles of the image.

2. The computer-executable components include: The system of claim 1 , further comprising a tile component that decomposes the image into the plurality of tiles.

3. The computer-executable components include:

3. The system of claim 2, further comprising: a graph component that generates a plurality of pixel adjacency graphs based on the plurality of tiles, wherein for a first pixel adjacency graph corresponding to a first tile, a node of the first pixel adjacency graph represents each pixel or voxel of the first tile.

4. 4. The system of claim 3, wherein the graph component generates a plurality of minimum spanning forests based on the plurality of pixel adjacency graphs, and for a first minimum spanning forest corresponding to the first pixel adjacency graph, the first minimum spanning forest includes one or more boundary trees and one or more interior trees.

5. The system of claim 4 , wherein the graph component generates the plurality of minimum spanning forests via executing Brufka's algorithm.

6. 5. The system of claim 4, wherein the graph component generates the plurality of pixel region adjacency graphs based on the plurality of minimum spanning forests, and for a first pixel region adjacency graph corresponding to the first minimum spanning forest, the first pixel region adjacency graph includes one or more boundary regions corresponding respectively to the one or more boundary trees and a set of nodes per region into which the one or more internal trees are condensed.

7. 7. The system of claim 6, wherein the graph component merges the plurality of pixel region adjacency graphs into the region adjacency graph by combining boundary regions of adjacent tiles, re-flag such boundary regions as new interior regions, and condense such new interior regions into new per-region nodes.

8. The system of claim 1 , wherein the execution component calculates the amount of memory consumption associated with creating the region adjacency graph.

9. 1. A computer-implemented method comprising: accessing, by a device operatively coupled to the processor, images produced by the scientific instrument; and performing, by the device, marker-based watershed segmentation on a region adjacency graph of the image, wherein the region adjacency graph is constructed from a plurality of pixel region adjacency graphs corresponding respectively to a plurality of tiles of the image.

10. The computer-implemented method of claim 9 , further comprising decomposing, by the device, the image into the plurality of tiles.

11. 11. The computer-implemented method of claim 10, further comprising generating, by the device, a plurality of pixel adjacency graphs based on the plurality of tiles, wherein for a first pixel adjacency graph corresponding to a first tile, a node of the first pixel adjacency graph represents each pixel or voxel of the first tile.

12. 12. The computer-implemented method of claim 11, further comprising generating, by the device, a plurality of minimum spanning forests based on the plurality of pixel adjacency graphs, wherein for a first minimum spanning forest corresponding to the first pixel adjacency graph, the first minimum spanning forest includes one or more boundary trees and one or more interior trees.

13. The computer-implemented method of claim 12 , wherein the device generates the plurality of minimum spanning forests via executing Brufka's algorithm.

14. 13. The computer-implemented method of claim 12, further comprising generating, by the device, the plurality of pixel region adjacency graphs based on the plurality of minimum spanning forests, wherein for a first pixel region adjacency graph corresponding to the first minimum spanning forest, the first pixel region adjacency graph includes one or more boundary regions respectively corresponding to the one or more boundary trees and a set of nodes per region into which the one or more internal trees are condensed.

15. 15. The computer-implemented method of claim 14, further comprising: merging, by the device, the plurality of pixel region adjacency graphs into the region adjacency graph by combining boundary regions of adjacent tiles, re-flag such boundary regions as new interior regions, and condensing such new interior regions into new per-region nodes.

16. The computer-implemented method of claim 9 , wherein the image is an electron tomography image, an X-ray tomography image, or a confocal microscope image.

17. 1. A computer program product for facilitating computation of a tiled region adjacency graph via a pixel region adjacency graph, the computer program product comprising: a non-transitory computer readable memory having embedded thereon program instructions, the program instructions causing a processor to: accessing images captured by a charged particle microscope; A computer program product executable by the processor to cause a region adjacency graph for the image to be constructed in a tile-by-tile manner based on a plurality of pixel region adjacency graphs.

18. The program instructions cause the processor to: Decomposing the image into tiles; generating a plurality of pixel adjacency graphs based on the plurality of tiles, wherein for a first pixel adjacency graph corresponding to a first tile, a node of the first pixel adjacency graph represents each pixel or voxel of the first tile; generating a plurality of minimum spanning forests based on the plurality of pixel adjacency graphs, where for a first minimum spanning forest corresponding to the first pixel adjacency graph, the first minimum spanning forest includes one or more boundary trees and one or more interior trees; generating the plurality of pixel region adjacency graphs based on the plurality of minimum spanning forests, wherein for a first pixel region adjacency graph corresponding to the first minimum spanning forest, the first pixel region adjacency graph includes one or more boundary regions respectively corresponding to the one or more boundary trees and a set of nodes for each region into which the one or more interior trees are condensed; 18. The computer program product of claim 17, further executable to merge the plurality of pixel region adjacency graphs into the region adjacency graph by combining boundary regions of adjacent tiles, re-flag such boundary regions as new interior regions, and condense such new interior regions into new per-region nodes.

19. 20. The computer program product of claim 18, wherein program instructions are further executable to cause the processor to segment the image based on the region adjacency graph.

20. 20. The computer program product of claim 19, wherein the processor segments the image via marker-based watershed segmentation.

Images