Image compression and analysis

Abstract

Provided is the processing of a large-scale colored image dataset by efficiently performing image analysis, processing, and manipulation based on adaptive particle representation (APR). 【Solution means】The analysis of large-scale biological objects and tissue samples is performed with a calculation speed 100 times or more faster and an excellent memory storage compression ratio. The embodiment is extremely suitable for large-scale imaging projects of large three-dimensional transparent tissue samples such as whole-brain mapping initiatives and human nerve tissue pathology. The data store holds an adaptive sampling representation for tiles, individual tile positions, and other corresponding image data for the biological object of interest. By reading out a tile from the data store only once, a complete image of the maximum resolution of the biological object is output as an image.

Description

Related Applications

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 366,818, filed on June 22, 2022, and U.S. Provisional Patent Application No. 63 / 366,825, filed on June 22, 2022. The entire disclosure content of these applications is hereby incorporated herein by reference. BACKGROUND OF THE INVENTION

[0002] Biological imaging and medical imaging are a type of digital imaging, which refers to the technical field of generating and depicting images of the growth of internal parts, tissues, organs, skeletal structures, tendons, ligaments, some blood vessels, etc. For example, there are various techniques and devices for generating medical images, such as confocal light sheet microscopy, X-ray systems, CT (computed tomography) scanners, MRI (magnetic resonance imaging) devices, ultrasonic systems, PET (positron emission tomography) scanners, etc. The depicted images are digital visualizations, which enable medical staff to diagnose and treat patients, or more specifically, to detect diseases, abnormalities, and injuries. For example, a CT scan provides a series of two-dimensional slices of the body, i.e., cross-sectional images, by which medical staff can digitally view tumors, their shape, size, location, and the blood vessels nourishing the tumors on a computer display screen.

[0003] Researchers, attending physicians, etc. rely on biological images and medical images. Typical modalities of these imaging methods (more generally, digital imaging systems) include imaging, retrieving, processing (analyzing), communicating / delivering, and displaying raw digital images (such as by using any of the above devices and techniques). SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] In some biological and medical research, image data and their corresponding digital files can be relatively large in volume, potentially causing problems with the speed of computer memory storage (archiving), retrieval, processing (analysis), and display. For example, tissue samples of the entire mouse brain, color images of wide-area cross-sections of the human brain, and other annotated medical images are relatively large digital files. Various image compression algorithms exist, but their primary purpose is file storage. This is because the file contents must be decompressed to be read and understood, negating the benefits of compression. [Means for solving the problem]

[0005] Embodiments of the present invention solve the shortcomings of the art in the manipulation (processing for analysis, communication / distribution, display, etc.) and management (storage in computer memory, archiving, retrieval, communication, etc.) of specific medical image data (specifically, large-volume multidimensional image data). Advantageously, the embodiments provide faster processing and improved storage capacity for image files in computer memory without scaling up processor resources or memory.

[0006] The embodiments provide computer-based systems or methods implemented on a computer for better manipulation (e.g., processing, analysis, distribution, display, etc.) and management (e.g., storage, retrieval, communication, etc.) of multidimensional images / medical images / biomechanical images. Specifically, the embodiments achieve storage (archiving), retrieval, analysis, processing, and display of relatively large image data at acceptable speeds without (or without) additional memory or processor resources. Examples of relatively large image data include, but are not limited to, images generated by confocal / light-sheet microscopes (other optical microscopes such as wide-field microscopes, super-resolution microscopes, electron microscopes, etc.), X-rays, computed tomography (CT), positron emission tomography (PET), optical coherence tomography (OCT), MRI, ultrasound, and other digital imaging techniques, such as biological images and medical images.

[0007] The aforementioned image data may be multidimensional and may include, in addition to the digital image of the subject, color, annotations (e.g., text and / or notes and / or marks from researchers or medical professionals), graphics, a temporal dimension, or other dimensions.

[0008] Examples of the relatively large-scale image data mentioned above include, but are not limited to, tissue samples, whole mouse brains, cross-sections of human brains, brains or parts of other mammals, and other biological objects or biological materials of the subject.

[0009] In one embodiment, a computer-based image processing method acquires, or accesses or receives, a series of tomographic images (broadly speaking, two-dimensional image units) in the depth direction along a third spatial dimension axis from a given image data of a living organism. Each two-dimensional image unit (i.e., tomography) is an n×m pixel frame of each parallel plane orthogonal to the axis. In this method, each series of two-dimensional image units along each orthogonal axis is acquired in the depth direction. Each two-dimensional unit in a series is an n×m pixel frame of each parallel plane orthogonal to the corresponding axis of that series. Each orthogonal axis of different series intersects each plane at different frame positions. As a result, the method yields multiple series (so-called tiles) representing image volumes imaged in three spatial dimensions of various parts of the living organism. Each of the obtained series (i.e., tiles) represents a separate image volume obtained from the given image data.

[0010] In other words, as used herein, a "frame" refers to a 2D (two-dimensional) n x m pixel array of data. Multiple such 2D arrays (frames) are arranged consecutively along the same orthogonal axis to form a 3D data array, which is referred to herein as a "tile." The orthogonal axes of separate tiles (continuums of frames) are separate orthogonal axes formed by the continuum of each frame. In other words, one tile consists of corresponding frames arranged consecutively along one orthogonal axis. A second tile consists of corresponding frames arranged consecutively along a different orthogonal axis, a third tile consists of frames arranged consecutively along the axis of the same tile, and so on.

[0011] Next, in this method, when capturing an image, i) By compressing the image data of the tile region with less information and maintaining the maximum resolution or high resolution of the region with more information, an adaptive sampling representation of the tile is obtained. ii) Automatically determine the transformation parameters of the tile, iii) The compressed data and the raw data are automatically compared to verify that the converted data captures the necessary information. iv) Calculate the maximum intensity projection image of the tile in multiple spatial directions for each edge where an adjacent tile exists, v) Using the calculated maximum projection image, the registration of each pair of three spatial dimensions between adjacent tiles is calculated, thereby registering the tile with other adjacent tiles (in this embodiment, after image acquisition, the tile positions are globally optimized with respect to the confidence scale. The relative tile positions thus obtained make it possible to stitch together each tile with only one readout, and the image volume can be reconstructed by this stitching), vi) Storing the adaptive sampling representation for the tile, the transformation parameters obtained for the tile, and an index of the spatial position of the tile in a data store in computer memory (this storage is done such that the given image data is stored in the data store in a reduced size, along with the individual tile positions, rather than storing the raw source image data of the biological object in the data store), Each tile (series) is processed individually and digitally automatically.

[0012] In this embodiment, highly efficient image processing (using compressed image data for analysis) and scalability (through individual tile analysis), which have not been achieved in conventional techniques, are achieved as follows. In this method, when an image is captured, a classifier is applied to a given tile by (i) extracting object features and object positions from the given tile to segment the object, and (ii) aggregating and storing the segmented objects in the data store. Objects that appear two or more times in adjacent tile regions are automatically merged by a matching algorithm. The matching algorithm may include, but is not limited to, (a) finding the nearest neighbor candidate in the feature space considering the position, size, and shape of each object (in the final sample space after registration), and (b) confirming that it is a reliable match using Lowe's condition (the condition that the second nearest neighbor candidate must be separated by a predetermined margin). Other matching algorithms are also preferred. In this embodiment, the target image can then be stitched together and reconstructed by reading each tile represented in the data store, or each tile from the corresponding data in the data store (i.e., the stored compressed image data, the stored relative tile position, the compression parameters, the stored segmented object, and the stored object position), only once. In this way, the embodiment prevents the problem that can occur in conventional techniques where analysis such as object segmentation is applied to the entire image file, where the image file may be too large for processing and storage.

[0013] Finally, the method outputs a complete image of the biological object at maximum resolution by reading the tiles from the data store (i.e., compressed image data and corresponding data) only once (rather than reading them multiple times or at different times). By using the stored compressed image data, stored relative tile locations, compression parameters, stored segmented objects, and stored object locations from the data store, the need for additional computer memory and / or processor resources for storing, retrieving, communicating, distributing, processing, analyzing, displaying, or any combination thereof of the complete image is reduced.

[0014] In this embodiment, tile regions with low information content are compressed using Adaptive Particle Representation (ARP) technology.

[0015] Image processing methods implemented in computers are: (A) With respect to a given image data of a living organism, a series of 2D image units, each of which is an n×m pixel frame of each parallel plane orthogonal to the axis of a third spatial dimension, is acquired in the depth direction. Thus, each series of 2D image units along each orthogonal axis at each tile position on each plane is acquired in the depth direction, and each 2D image unit in a series becomes an n×m pixel frame of each parallel plane orthogonal to the corresponding axis of the series. Through this acquisition, multiple series that function as tiles are obtained, representing image volumes of various parts of the living organism captured in three spatial dimensions, and each of these multiple series is a separate tile representing a separate image volume.

[0016] In some embodiments, the method automatically processes each tile individually by: (B)(i) compressing image data of low-information tile regions and maintaining high (maximum) resolution of high-information regions to obtain an adaptive sampling representation of the tile; (ii) determining the tile position relative to other tiles; (iii) segmenting the adaptive sampling representation of the tile for the target object by extracting object features and corresponding object positions; and (iv) storing the adaptive sampling representation of the tile, the obtained tile position index, the segmented object, the extracted corresponding object features, and the object position in a data store in computer memory. This storage is performed such that the given image data is stored in the data store in a reduced size, rather than storing the source image of the biological object in the data store, along with the individual tile positions.

[0017] As a next step, the method includes (C) aggregating the segmented objects stored in the data store, the aggregation of which automatically merges objects that appear two or more times in adjacent tiles among the segmented objects to obtain a worklist of the segmented objects stored in the data store, and (D) reading the tiles only once based on the adaptive sampling representation for the tiles, the tile positions, and the worklist of the segmented objects stored in the data store to form a complete image of part or all of the biological object at maximum resolution. In other words, the method does not require reading tiles multiple times or at different times to facilitate storage or to perform image analysis (e.g., object segmentation). As configured, reading the tiles only once reduces the need for additional computer memory and / or processor resources for any of the storage, retrieval, communication, distribution, processing, analysis, display, and combinations thereof of the complete image.

[0018] The worklist may include the segmented objects, object positions, and indicators for their respective feature quantities.

[0019] In the embodiment, the individual processing of each tile further includes (i) automatically determining compression parameters used to compress the image data of the tile, (ii) storing the determined compression parameters in the data store, and (iii) automatically comparing the obtained adaptive sampling representation with the source image data to verify that the adaptive sampling representation captures the necessary information of the given image data.

[0020] In this embodiment, determining the relative tile position of a tile includes (i) calculating maximum projection images of the tile in multiple spatial directions for each edge of the tile where an adjacent tile exists, (ii) calculating registration parameters for each tile pair with already acquired adjacent tiles, and (iii) storing the calculated registration parameters for each tile pair in the data store. The registration parameters for each tile pair are calculated by correlating the maximum projection image calculated for each edge of the tile where an acquired corresponding adjacent tile exists with the maximum projection image calculated for the corresponding edge of the acquired corresponding adjacent tile. As a result, the accuracy of the relative tile position of the tile with respect to adjacent tiles is improved in this method.

[0021] In embodiments, the registration further includes global optimization. Specifically, after image acquisition, the method globally optimizes all stored pair registration results with respect to a confidence scale so that all tile positions are accurately corrected. The improved accuracy and correction of relative tile positions enable (a) the stitching together of image data corresponding to tiles, (b) the proper reconstruction of the entire biological object, and (c) the reliable retrieval of higher-order information about the imaged biological object, such as (but not limited to) the volume of the biological object and the number of segmented objects within it. In embodiments, process C (aggregation of segmented objects) is performed after global optimization.

[0022] In an embodiment, the given image data includes a biological image or a medical image captured and generated by any one of optical microscopy or an optical microscope (such as a wide-field microscope, a confocal microscope, a super-resolution microscope, an electron microscope, a light sheet microscope, etc., but not limited thereto), X-ray, computed tomography, positron emission tomography, optical coherence tomography, magnetic resonance imaging, ultrasonic waves, and other digital imaging technologies. For a target tile, processes A and B are performed during the imaging of the biological image or the medical image, and process C is performed after the imaging of the image corresponding to the target tile.

[0023] In an embodiment, the biological object is any one of a tissue sample, a whole mouse brain, a cross-section of a human brain, other parts of a mammalian brain, and other biological materials of the target.

[0024] In some embodiments, one or more digital processors automatically perform the steps of the method described herein. Another embodiment provides a computer-based system, a computer program product, and software as a service (SaaS). A computer-based system embodying the present invention may consist of an interface, a data store in computer memory, and a work module operably connected between the interface and the data store and executable by a processor. The interface receives or accesses image data of a target biological object. The work module is configured (programmed) to automatically perform adaptive sampling representation (e.g., APR), tile registration, and image compression by segmenting and aggregating objects on the image data in response to the interface. The data store holds, for each tile, the adaptive sampling representation, the obtained relative tile position, the segmented object, the corresponding object position, and extracted feature quantities. The system generates a complete image of part or all of the biological object at maximum resolution by reading the tiles only once from the contents of the data store. Because tile reading is performed only once, the need for additional computer memory and / or processor resources is reduced for any of the operations and management of the complete image, including storage, retrieval, communication, distribution, processing, analysis, display, and any combination thereof.

[0025] In an embodiment, for each tile, the adaptive sampling representation to be stored is subjected to merging and cropping with overlapping regions such that a physical position of the biological object is uniquely described by a particle of the adaptive sampling representation stored in the data store for the tile. In a given overlapping region where two or more tiles overlap, the adaptive sampling representation of the overlapping region merges with one of the same tiles and is cropped from the rest of these two or more tiles. The adaptive sampling representation of the merging / cropping version for the tile is stored in the data store. In this way, the (merged and cropped with overlapping regions) adaptive sampling representation for the tile brings efficiencies for storing and manipulating image data that have not been achieved heretofore.

[0026] The foregoing will become apparent from the following detailed description of the exemplary embodiments shown in the accompanying drawings. Throughout the different figures, the same reference numerals refer to the same components. The drawings are not necessarily to scale; rather, emphasis is placed on illustrating the embodiments.

Brief Description of the Drawings

[0027] [Figure 1] Graph comparing the stitching time of APR (Adaptive Particle Representation) examined by varying the operation ratio (CR), which is the ratio of the total number of voxels before compression to the total number of particles after compression, with the stitching time of TeraStitcher. [Figure 2] Graph showing the merging of segmented objects in multi-tile data for a target medical image, plotting the number of objects detected by software in the same medical image against the actual number of objects, and also reporting on the number of objects detected in the un-sliced original image data before forming the multi-tile data set (the operation ratio is the ratio of the total number of voxels before compression to the total number of particles after compression). [Figure 3A]This is a block diagram of one embodiment (with supplementary explanations (keys) provided at the bottom). [Figure 3B] This is a flowchart of one embodiment of the present invention. [Figure 4] This is a schematic diagram of the computer network environment in which the embodiment is deployed. [Figure 5] Figure 4 is a block diagram of computer nodes in a computer network. [Figure 6A] This is a schematic diagram illustrating the merging and cropping of image data of overlapping tiles in an embodiment of the present invention, showing four overlapping tiles that overlap with adjacent tiles on the left, right, top, and bottom sides. [Figure 6B] This is a schematic diagram illustrating the merging and cropping of image data of overlapping tiles in an embodiment of the present invention. In Figure 6A, the tiles are depicted by an adaptive sampling representation of the tiles after being merged and cropped with respect to the overlapping region. [Modes for carrying out the invention]

[0028] The following describes exemplary embodiments.

[0029] Adaptive particle representation (APR) is a content-adaptive, highly efficient image representation technique that also compresses data. APR adaptively represents the content of an image while maintaining image quality. While APR reduces image storage costs by compressing data, it also solves memory and processing bottlenecks by directly using the data for image processing tasks without converting it back to pixel data. See BL Cheeseman, et al, "Adaptive particle representation of fluorescence microscopy images," Nature Communications 9, no. 5160, 4 December 2018.

[0030] In APR, pixels or voxels in the target image or image volume are replaced with particles (points in space that convey intensity) placed according to the image content. Particles can be placed anywhere necessary for the image content. Different parts of the image may have different particle sizes. This change in size determines the resolution of each local representation of the image. The required resolution is expressed by an Implied Resolution Function, which assigns high resolution to image areas where the intensity in space changes rapidly (e.g., edges) and low resolution to image areas where the intensity changes little (e.g., background, uniform foreground). The Implied Resolution Function defines the radius of the neighborhood around each pixel. For any pixel position, the image intensity values ​​can be reconstructed within a user-defined error threshold range by calculating the non-negative weighted average of the intensities of the particles in its neighborhood. Since luminance conditions, i.e., intensity ranges, can vary significantly between different regions within the sample, the error is normalized based on a local intensity scale.

[0031] In other words, APR (Advanced Photon Reproducibility) is a spatially adaptive sampling method that resamples an image by relying on local information content while controlling local gain. When APR is applied to an image, the output is an image representation composed of a group of particles with associated intensity values.

[0032] The principle of the present invention is to efficiently perform image analysis and processing of relatively large image datasets using adaptive particle representation technology. In the embodiments, the analysis of large biological and tissue samples is performed with computation speeds more than 100 times faster and with excellent memory-to-storage compression ratios. The embodiments are particularly suitable for large-scale medical or biomedical imaging projects such as (but not limited to) whole-brain mapping initiatives and human neuropathology. The embodiments are also particularly suitable for continuous analysis of the same sample type (e.g., mouse brain) when experiments must be repeated for statistical purposes.

[0033] Figures 3A and 3B illustrate the data and control flow in a method or system 1000 that embodies the principles of the present invention. While the end-to-end pipeline from the source imaging device to the final usable image for distribution, storage, and communication is illustrated and described, the embodiments are not limited thereto. Other embodiments or parts thereof may function as image compression tools or devices, image storage and retrieval methods or systems, image analysis and post-processing methods or systems, etc. These are merely non-limiting examples, but Figures 3A and 3B will be described below in such a context, followed by a detailed description of the implementation.

[0034] The supplementary explanation in the lower part of Figure 3A shows hardware in blue (darkest gray or general diagonal pattern), components acting on individual tiles in orange (medium gray or general diagonal parallel pattern), higher-order components (acting on tiles and other data) in green (medium dark gray or dotted pattern), and external libraries acting on voxel data in light gray (or a white background). According to this supplementary explanation, the hardware components of the embodiment include the imaging device 101 and the data store 125. Components acting on individual tiles include the APR compression unit 121 (with automatic conversion parameters), the segmentation unit 123, and the pre-splittering unit 122 (with projection and pair-by-pair registration). Higher-order components include the visualization unit 305, the splicing unit 310 (with global optimization unit), the matching unit (smart merging of segmented objects) 130, and the reconstruction unit 320. Examples of external libraries include the atlasing unit 350 and other algorithms 355 that operate on voxel data.

[0035] In one embodiment of system 1000, a source imaging device 101, such as a microscope, captures and generates a biological image or medical image 103 of a target biological object or biological substance, such as an organ, part of a body, or tissue. Those skilled in the art will understand that various source imaging devices (confocal / light-sheet microscopes, OCT scanners, CAT scanners, PET scanners, ultrasound systems, MRI systems, X-ray systems, etc.) 101 may be used. Other optical microscopy techniques, such as (but not limited to) wide-field microscopes, super-resolution microscopes, and electron microscopes, and electron microscopy techniques can also function as the source imaging device 101 (optical microscope). Other digital imaging techniques are also suitable. The raw biological image or medical image 103 (Figure 3B) may be large in scale (or relatively large in volume compared to typical digital images). In some embodiments, the medical image or biological medical image 103 may be, for example, a wide section of the human brain, the whole mouse brain, or a part or all of another part of the mammalian brain, etc., but are not limited to these.

[0036] Source 101 captures a raw biological medical image 103 and passes it to 105 (collectively referred to as work memory; Figure 3B), such as local memory, computer storage, shared memory, or an archive. In work memory 105, users such as researchers and medical professionals can add annotations (text notes, marks, etc.) and / or graphics and / or colors to the raw biological or medical image 103. In other words, the resulting image data 110 consists of multiple information dimensions, namely, not only the raw biological medical image 103 but also annotations, graphics, and colors. Each pixel of the image data 110 is represented as an N-tuple. The data representation of the N-tuple is implemented by a data structure such as a linked list or array that stores multiple information dimensions for each pixel. Each information dimension (i.e., tuple) of each pixel may include, but are not limited to, a raw source image (digital image pixel value), annotations and graphics (overlay value), color (red, green, and blue component values), and other information layers or types such as a second or further text or graphic layer, a fluorescence channel mark, a time dimension, an indicator of blood pressure or blood flow, or other physical conditions. In one embodiment, there is only one channel or tuple under consideration, meaning that there is only one numerical value for each pixel, not multiple.

[0037] The system and method 1000 of the present invention acquires, accesses, or receives image data 110 (N-tuple data for each pixel) of a target biological object from the work memory 105 in near real-time during or after imaging by the source 101. Specifically, the interface 120 (Figure 3B) receives the image data 110 and accesses a series of depth-direction tomographic images (broadly speaking, 2D image units) along the axes of the third spatial dimension, or acquires such a series from the image data 110. Each 2D image unit is an n×m pixel frame of each parallel plane orthogonal to the axes of the third dimension. A series of frames along the axis of the target is called a tile. The interface 120 acquires various tiles, that is, separate series of tomographic images (2D image units or frames) along each axis. Each 2D image unit in a series (tile) is an n×m pixel frame of each parallel plane orthogonal to the corresponding axis of the same series. Different sequences (i.e., tiles) are located at different tile positions on different planes.

[0038] As a result, interface 120 acquires multiple such sequences (i.e., tiles). Each of these sequences / tiles represents an image volume in which the corresponding part of the target biological object in the image data 110 is captured in three spatial dimensions (3D space). Separate sequences / tiles represent separate image volumes, and separate image volumes capture separate parts of the target biological object.

[0039] The tile processing modules 121, 122, 123, and 124 are connected to communicate with interface 120. When acquiring source images, for each sequence / tile acquired by interface 120, the tile processing modules 121, 122, 123, and 124 process each tile as follows: First, in step 121, the tile processing module compresses the tile region with less information while maintaining the maximum resolution of the region with more information. This results in an adaptive sampling representation of the tile. In this embodiment, APR is used to perform such image compression and adaptive sampling of the tile. Other adaptive sampling techniques are also preferred.

[0040] Next, the tile processing module 121 automatically determines the transformation parameters used to compress the image data for the tile. In embodiments, these transformation parameters (i.e., compression parameters as defined herein) are automatically determined by algorithms and techniques described later. In some embodiments, the tile processing module 121 may apply APR individually to each channel or dimension (tuple) of the image data 110 for the tile. In a non-limiting example, the tile processing module 121 may apply APR individually to each fluorescence channel for the same tile and automatically determine the corresponding compression parameters. In steps 121 and 124 of the module, the compressed tile (i.e., the adaptive sampling representation for the tile) and the determined corresponding compression parameters are stored in the data store 125.

[0041] Module 121 further verifies the compressed representation of the tiles by automatically comparing the obtained adaptive sampling representation with the source image data. This comparison determines whether the adaptive sampling representation captures the necessary information from the source image data. Known or common techniques such as the peak signal-to-noise ratio and the structural similarity index are used in this verification comparison in step 121 of the module.

[0042] In this embodiment, the representation of compressed image data with respect to overlapping regions between adjacent tiles stored in the data store 125 can be improved. In this embodiment, module 121 stores each tile individually in the data store 125, but in the original compressed data (adaptive sampling representation) stored in the data store 125, there are overlapping regions between each adjacent tile. In this state, the physical position of a biological object (physical sample being imaged) is not always uniquely determined by a single particle (adaptive sampling representation) in the data store 125, but may be determined by two particles (adjacent regions) or four particles (overlapping regions at corners). Figure 6A shows this phenomenon with four different tiles (represented in different colors) overlapping.

[0043] The schematic diagram in Figure 6A shows the number of compressed image particles (adaptive sampling representation particles) that define the spatial position within a biological object, represented by the numbers 1, 2, and 4. Specifically, it illustrates 600 blocks of four tiles (i.e., four quadrants). In reality, there are more tiles than this, but the principle remains the same. The upper left tile 610 overlaps with its right adjacent tile 620, i.e., the upper right tile in block 600, along the length of its right edge. The upper left tile 610 overlaps with its lower adjacent tile 630, i.e., the lower left tile in block 600, along its lower edge. Similarly, the upper right tile 620 overlaps with the upper left tile 610 along its left edge. The upper right tile 620 overlaps with its lower adjacent tile 640, i.e., the lower right tile in block 600, along its lower edge. In the four quadrants (block 600), the bottom-left tile 630 overlaps with the top-left tile 610 along its upper edge. The bottom-left tile 630 overlaps with the bottom-right tile 640 along its right edge. The bottom-right tile 640 overlaps with the bottom-left tile 630 along its left edge. The bottom-right tile 640 overlaps with the top-right tile 620 along its upper edge.

[0044] Non-overlapping tile regions are marked with the number 1 to indicate that a single particle (from the adaptive sampling representation of each tile) in datastore 125 uniquely defines the spatial location to be represented within the target biological object (the physical sample being imaged). Regions where two tiles overlap are marked with the number 2 to indicate that two particles (one each from the compressed image / adaptive sampling representation of adjacent tiles) in datastore 125 define the corresponding spatial locations within the target biological object. In a non-restrictive example, the overlapping region 615 of the upper-left tile 610 (along its right edge) and the upper-right tile 620 (along its left edge), excluding or excluding the overlapping portion in the inner corners near the center of block 600, is marked with the number 2, indicating that the corresponding position or spatial location in the biological object is defined by two particles (one from each of the two corresponding sets of adaptive sampling representations for tiles 610 and 620) in the data store 125. Similarly, the overlapping regions 625 (between the upper right tile 620 and the lower right tile 640), 635 (between the upper left tile 610 and the lower left tile 630), and 645 (between the lower left tile 630 and the lower right tile 640) are marked with the number 2 to indicate that the corresponding positions (spatial locations) in the living organism are defined by two particles (one from each of the adaptive sampling representations of two adjacent tiles) in the data store 125.

[0045] The region where all four tiles 610, 620, 630, and 640 overlap (their inner corners, i.e., the lower right corner of the upper left tile 610, the lower left corner of the upper right tile 620, the upper right corner of the lower left tile 630, and the upper left corner of the lower right tile 640) is marked with the number 4 to indicate that the corresponding spatial arrangement or position in the biological object is defined by four particles, one from each particle group corresponding to the adaptive sampling representation in datastore 125 for the four adjacent tiles 610, 620, 630, and 640.

[0046] In a different embodiment, an interesting improvement is that a) particle information of overlapping tile regions is combined and the combined result is stored in the data store 125, and b) a tile (representation of a tile) is cut out such that a spatial position or the position of a point in the target sample (biological object) is uniquely determined by a particle in the stored tile, that is, uniquely determined by a particle in the data store 125. Continuing from the example in Figure 6A, in the improved embodiment shown in Figure 6B, (i) the smallest particle (representing the highest resolution) from the union of particle groups corresponding to the overlapping tile regions of the target is maintained, and the particle information (adaptive sampling or compressed image data representation) of the overlapping tile regions is combined (690). Next, in the merging method 690, (ii) for each group of particles involved in the overlap, the intensity values ​​of the particles within that group are averaged (averaged for each tile region), (iii) the maximum value of the calculated average intensity value is selected from each overlapping tile region of the target, and (iv) the result of (iii) is assigned as the intensity value to the smallest particle maintained in step (i). Finally, in method 690, the target tile (tile representation) is cut out so that all positions in the target biological object are uniquely determined by a single particle stored in the tile representation in the data store 125.

[0047] In the example in Figure 6B, method 690 maintains or merges the overlapping regions 615,635 as belonging to the upper-left tile 610, and removes the corresponding overlapping regions from the upper-right adjacent tile 620 and the lower-left adjacent tile 630. In other words, in method 690, steps (i) to (iv) above are applied to the overlapping regions 615,635 and the inner corners of tile 610, and the particles in the lightly shaded portion drawn along the right and lower edges of tile 610 are defined. In method 690, the corresponding overlapping region is cut off along the left edge of the upper-right tile 620, and the corresponding overlapping region is cut off along the upper edge of the lower-left tile 630. The compressed image data / adaptive sampling representation of tile 610 obtained by module 121 performing method 690 is stored in data store 125. Similarly, in Method 690, steps (i) to (iv) are also applied to the overlapping region 625 to define the particles within the lightly shaded area drawn along the lower edge of the upper right tile 620. In Method 690, the thus defined particles and lightly shaded area are merged with the upper right tile 620, and the corresponding overlapping region is cut off from the upper edge of the lower right tile 640. In Method 690 (Module 121), the resulting compressed image data / adaptive sampling representation for the upper right tile 620 is stored in the data store 125, and the cut-off adaptive sampling representation for the lower right tile 640 is also stored in the data store 125. In Method 690, steps (i) to (iv) are also applied to the overlapping region 645 to define the particles within the lightly shaded area drawn along the right edge of the lower right tile 630. In method 690, the thus defined particles and faintly shaded areas are merged into the lower-left tile 630, and the corresponding overlapping region is cut off from the left edge of the lower-right tile 640. In method 690 (module 121), the resulting compressed image data / adaptive sampling representation for the lower-left tile 630 and the compressed image data / adaptive sampling representation for the lower-right tile 640 are stored in the data store 125.

[0048] In the improved embodiment described above, all tiles (adaptive sampling representations of tiles) stored in the data store 125 as a result of module 121 and method 690 are the same size, except for the tiles along the rightmost edge of the acquired object and the tiles along the bottom edge of the acquired object. In Figure 6B, the areas with faint shadows indicate areas where merging and trimming have occurred (by applying method 690 with module 121). Specifically, the upper left tile 610 (including the shadowed portions 615 and 635 merged with the tiles) remains unchanged in size. The upper right tile 620 (with the shadowed portion 625 merged) has been trimmed in the width direction. The lower left tile 630 (with the shadowed portion 645 merged) has been trimmed in the length direction. The lower right tile 640 has been trimmed in both the width and length directions.

[0049] Returning to Figure 3B, in step 122, the tile processing module determines the relative tile position to other tiles. In step 122, to do this, for a given tile, the maximum projection images in multiple spatial directions of the tile are calculated for each edge where an adjacent tile exists. Next, in step 122 of the tile processing module, registration parameters for each tile pair with already acquired adjacent tiles are calculated. Specifically, in step 122, for each edge of the given tile where an acquired corresponding adjacent tile exists, the maximum projection image calculated for that edge of the tile is correlated with the maximum projection image calculated for the corresponding edge of that acquired corresponding adjacent tile. In this embodiment, by correlating the corresponding maximum projection images calculated for the edges relating a given tile and its adjacent tile, the accuracy of the relative tile position of the tile with respect to its adjacent tile is improved. In steps 122 and 124 of the module, the calculated registration parameters for each pair are stored in the data store 125. In this embodiment, the tile processing module 122 performs registration between tiles by: (a) obtaining registration for each pair of adjacent tiles using the calculated maximum value projection image when the source image is captured; and (b) performing global optimization with respect to the confidence scale after the source image is captured. This ensures that the position of each tile is optimally calculated even if each tile is read only once. Image stitching will be described in more detail later.

[0050] In some embodiments, following the processing of each tile during source image acquisition, step 123 of the tile processing module performs segmentation of the target object from the adaptive sampling representation (output from step 121) for the tile. Module 123 applies a classifier by extracting object features and object positions from a given tile to segment the object. Examples of object features include, but are not limited to, the diameter, shape, and volume of the object to be detected. In some embodiments, module 123 further calculates the position of the segmented object using the tile positions output from step 122 and stored in the data store 125. In some embodiments, the classifier may be formed by image filtering and subsequent thresholding. In other embodiments, the classifier is formed by manually annotating some tiles after compressing the tile region to train a machine learning classifier, and then directly performing segmentation for a given tile in step 123. Other classifiers are also preferred, as object segmentation will be explained in more detail later.

[0051] In a preferred embodiment, step 123 stores in datastore 125: (i) the classifier output (inclusively the segmentation of objects), which is a binary mask calculated directly (very efficiently) by APR; and (ii) indices of the physical aspects and characteristics of the segmented objects, i.e., location and feature quantities, calculated from the binary mask. Storing the binary mask together with compressed image data (adaptive sampling representation of tiles) in datastore 125 is extremely space-efficient compared to requiring twice the memory space to store the segmented objects (images) and the uncompressed target images.

[0052] In step 124 of the tile processing module, for each processed tile, (a) the adaptive sampling representation for the tile output from step 121, (b) the compression parameters obtained for the tile from step 121, (c) the position of the tile in three-dimensional space, i.e., an index of the relative tile position from step 122, and (d) the segmented object of the target output from step 123 are stored in the data store 125 in computer memory. In one embodiment, the data store 125 holds a list of the segmented objects, extracted object features, and aggregated object positions across tiles. As a result, the data store 125 does not store the raw data of the source image 103 of the target biological object, but rather holds the image data 110 in a reduced size along with the individual tile positions.

[0053] In some embodiments, as shown in Figure 3A, the data store 125 is configured to function as part of an image compression device, tool, or service 310, 130, 320 through the action of tile processing modules 121, 122, 123, 124. In embodiments, a specific portion of the image data 110 can be selectively read and displayed by the user in a compressed form stored in the data store 125 (305). See below for further details regarding the visualization unit 305. As will become clear later, in embodiments, the interface 120 and tile processing modules 121, 122, 123, 124 are executed when the source device 101 captures an image, and other processing of the system / method 1000 is performed after the source image 103 is captured.

[0054] Continuing from Figure 3B, the aggregation module 130 is communicably connected to the data store 125. The aggregation module 130 aggregates the segmented objects in the data store 125 to form a worklist, which replaces the initial aggregate list. For objects that appear two or more times in adjacent tile regions, the aggregation module 130 automatically merges these objects using a matching algorithm. In this embodiment (aggregation module 130), for example, a nearest-neighbor matching algorithm is applied within a feature space in which each object is embedded along with its position, size, and shape, but it is not limited to this. Matches can be reliably verified using the ratio of the nearest neighbor candidate to the next nearest neighbor candidate. If this ratio exceeds a predetermined threshold (usually 0.7), it is considered a reliable match, and the matched objects are merged.

[0055] Other matching algorithms and means are also suitable as matching algorithms and means for merging objects in a module or step 130.

[0056] Returning to Figure 3B, the system / method 1000 uses a merger unit 140 (also referred to as a reconstruction unit 320; Figure 3A) that responds to the aggregation module 130. The merger unit (module or step) 140, 320 reads tiles (i.e., tiles represented by data in the data store 125) only once and merges the tiles of the adaptive sampling representation to form an output assembly (file, stream, etc.) or (inclusively) an output image 150 of the target biological object with the highest resolution. The merger or reconstruction 320 by the merger unit 140 splices the tiles together. This merger is performed in a manner that supports the re-conversion of the tiles to be merged into voxel data of the image volume in the work memory 105. The approach of reading tiles only once to form the output image 150 reduces the need for additional computer memory and / or processor resources for any of the storage, retrieval, communication, distribution, processing, analysis, display, and combinations thereof of the complete image 150. Furthermore, the merging unit 140 / reconstruction unit 320 may reduce the resolution of the complete image 150 as needed, depending on the application. In another embodiment, instead, adaptive sampling representations (i.e., APR files) are directly merged. See further details later regarding image merging.

[0057] The embodiments may involve interface connections to, for example, a registration pipeline, a work algorithm, or other systems (350, 355). In the registration pipeline, images of biological objects or biological objects of the same category are aligned. Such registration aligns the output data file (complete image) 150 with similar target images in a common coordinate framework, and the segmented objects (from the aggregation module 130) are mapped to the corresponding objects in the registration pipeline. Advantageously, this alignment and mapping makes it possible to perform tasks such as counting cells in the target body part. Other uses and advantages of the embodiments will also be within the realm of those skilled in the art, based on this disclosure.

[0058] Next, further details of the implementation, in addition to the modules, steps, and components shown in Figures 3A and 3B, or as variations thereof, will be presented as another embodiment of the present invention.

[0059] (Automatic image conversion)

[0060] APR's adaptive sampling is calculated based on the gradient of the raw image signal, using a local intensity scale for error normalization as a reference. Both the gradient and the local intensity scale must be numerically estimated from the captured image. This is highly susceptible to noise, and even in regions where the underlying (true) signal is perfectly stable, high-frequency fluctuations will be added. In this implementation, the influence of noise on the gradient and local intensity scale is mitigated by using a signal smoothing B-spline approximation.

[0061] The conversion from voxel images to APR depends on several parameters. Most of these (e.g., voxel size, background intensity level, B-spline smoothing degree, etc.) can be reliably kept constant for a given optical system and experiment. However, smoothing is not sufficient to completely eliminate the effects of noise in the signal. Therefore, to avoid oversampling resulting from normalizing even small gradients, the gradient and local intensity scales are thresholded from below. This adds two additional threshold parameters, one for each quantity. The optimal values ​​of these thresholds, in contrast to the other parameters, depend heavily on the local signal intensity. Therefore, in large-scale experiments where manual adjustment is not feasible, it becomes necessary to automatically estimate these parameters.

[0062] The optimal threshold is the value that best distinguishes between the distribution of numerical values ​​(gradient and local intensity scale) derived from the actual signal and the fluctuations caused by noise. A low threshold will adapt to the noise, resulting in suboptimal compression. On the other hand, a high value will lead to undersampling and loss of information. In principle, any automated (local or global) thresholding algorithm may be used to estimate the optimal value. However, since the distribution depends heavily on the optical configuration and the local characteristics of the sample, algorithms that rely on strict assumptions about the distribution morphology are not suitable. In this embodiment, Li's (reference 10) minimum cross-entropy thresholding algorithm is used because of its proven robustness, simplicity, and computational efficiency.

[0063] (Image stitching (122,310))

[0064] The scanning and imaging of the sample is performed using a pattern such as a raster scan pattern, which is capable of covering the entire desired region by acquiring each tile sequentially. The applicant's pipeline in this application supports sparse tiling, where each tile only needs to be connected by at least one side, and this was inspired by reference 1 and applied to APR. First, an adjacency map (i.e., the sides of each tile) is calculated for each tile. Next, for each pair of adjacent tiles, the displacement amount (x,y,z) for aligning these tiles is calculated, and the maximum projection image (I x ,I y ,I z The results are calculated using the phase cross-correlation of (reference 2). Depending on the characteristics of the biological or medical images, other alignment methods may be used to improve performance. Such alignment methods include, but are not limited to, feature matching (e.g., SIFT) and machine learning approaches.

[0065] If artifacts (e.g., air bubbles trapped around the sample) are present in the volume, a masked version of the phase cross-correlation is used (see reference 3).

[0066] To enable stitching with only one readout of each tile, a maximum projection image is pre-calculated (only for the expected overlapping region to eliminate the noise of the phase cross-correlation). This calculates a total of six displacement values ​​(two for x, two for y, and two for z) for each pair of adjacent tiles. The confidence level is calculated as the least squares difference between the maximum projection images after registration, and the displacement value with the highest confidence level is maintained. The confidence level may be calculated using other measures, such as the ratio of the maximum difference between the registered and original maximum projection images to twice the maximum value of the original image (the applicant of this application has found this to be extremely effective for sparse samples), but is not limited to these measures.

[0067] These displacements are stored in three graphs (one per spatial dimension), where each vertex corresponds to a given tile and each edge corresponds to the displacement between two tiles. For each spatial dimension, a corresponding confidence graph is constructed. Finally, the splicing unit 310 globally optimizes each displacement graph using the maximum spanning tree of the confidence graphs to satisfy the internal problem constraint that each loop in the graph must be zero.

[0068] Regarding the stitching process, all steps except for calculating the phase cross-correlation are performed entirely using APR. In this process, after the maximum projection image is calculated using APR, the 2D pixel image is reconstructed. Here, regenerating the pixel data is not disadvantageous because it is simply 2D and has a small memory footprint.

[0069] Finally, as described above in Figures 3A and 3B, the tile locations are stored in the data store 125 and can be used later for calculating and visualizing the locations of segmented objects (305).

[0070] (Object segmentation)

[0071] In one embodiment, a particle classification strategy is used to segment objects such as labeled cells. A random forest with 100 estimators is trained with a small amount of data that has been manually and sparsely annotated beforehand (reference 4).

[0072] The proposed pipeline performs segmentation directly while acquiring the next tile after fully acquiring one tile, so the tiles are not merged before segmentation. Object features and locations are extracted, and a merging strategy developed based on reference 5 merges higher-order information without actually merging the APR data. For each adjacent tile, two nearest neighbor candidates (NNs) for each object in the overlapping region of the first tile are calculated within the group of objects in the same overlapping region of the second tile. If the ratio of the second NN to the first NN exceeds a threshold (typically 0.7), and the centers of the objects are closer than a distance threshold (typically 1 / 4 of the object size), the objects are considered identical and automatically merged. Objects touching the edge of the overlapping region are automatically ignored so as not to be counted twice. This strategy has also been confirmed to be effective on synthetic datasets, as shown in Figure 2.

[0073] Figure 2 is a graph showing the merging of segmented objects in multi-tile data for the target medical or biomedical image. Multi-tile composite datasets were randomly generated using varying numbers of objects and random shifts. After segmenting each tile individually, objects were merged in overlapping regions (after calculating the precise registrations described above). The number of detected objects is plotted against the actual number of objects. The graph in Figure 2 also includes, for reference, the number of objects detected in the original data (before slicing) to form the multi-tile dataset.

[0074] (Image merging)

[0075] Some applications, such as registering samples to Atlas 350 or other 355 systems, require merged volumes. Fortunately, these applications 350 and 355 typically accept low-resolution samples and do not require reconstructing the footprint of the original image data. By simply selecting an appropriate APR level, each tile is efficiently voxel-reconstructed at low resolution, with interpolation performed later if precise voxel size is required. The tiles are then merged using any merger strategy, such as calculating the average or maximum value of overlapping regions. Before transformation or merger, preprocessing (e.g., histogram equalization) may be performed to gain the benefit of improved APR calculation speed.

[0076] (Registration to Atlas (350))

[0077] From the Brainreg frontend (reference 8), the AMAP pipeline (reference 7) allows for the registration of fused mouse brains to the Allen brain mouse atlas (reference 6).

[0078] (Visualization(305))

[0079] In one embodiment, Napari (reference 9) is used as a front-end for visualization 305. Napari requires objects with slicing attributes to enable lazy loading, so it is not necessary to reconstruct the entire sample in voxel space. So-called "lazy loading" means reading and displaying the compressed image data (output from step 121 and stored in datastore 125).

[0080] (Connecting benchmarks)

[0081] Steps 1-5 (corresponding to identifying stitching parameters without merging the data) were used to compare the stitching speed with TeraStitcher (Reference 1). TeraStitcher can run multiple instances simultaneously on openmpi. Four cores were used to achieve the best performance. The applicant of this application generated a composite dataset consisting of 16 voxels (approximately 1 Gb per tile) in a 4x4 tile arrangement of 2048 x 512 x 512 units, using various computation ratios (by changing the number of objects). The computation ratio is the ratio of the total number of voxels before compression to the total number of particles after compression. The conversion from voxel data to APR was performed by automatically determining the parameters as described above (see the section on automatic image conversion).

[0082] Referring to Figure 1, the APR stitching time was investigated at various CR ratios and compared with the TeraStitcher stitching time. From the graph in Figure 1, it can be seen that the embodiment is 20 to 1000 faster in stitching time compared to TeraStitcher.

[0083] Figure 6 of the priority application, which incorporates the entire content of this specification by reference, also describes an APR conversion, splicing, segmentation, and registration to an atlas in one embodiment. The figure also describes an end-to-end pipeline for analyzing large three-dimensional cleared tissue samples of wide-area cross-sections of a whole mouse brain or a human brain. The systems and methods of this disclosure achieve computations that are more than 100 times faster.

[0084] In the end-to-end pipeline stitching process or module described above, stitching is performed directly using the APR-generated representation of the target image. The stitching module supports any tiling pattern. Advantageously, the stitching module is extremely efficient because it reads each tile only once.

[0085] The segmentation module in the end-to-end pipeline described above directly interacts with the APR-generated representation of the target image, segmenting (extracting) objects, object features, and object locations from that image. The APR-generated representation can then be efficiently and directly annotated manually by the user.

[0086] Finally, the end-to-end pipeline in this example includes registration to an atlas (350). Low-resolution pixel data can be directly reconstructed and merged from a multi-tile APR representation of the target image. This allows for the use of atlas registration pipelines such as AMAP from Brainreg (as a front-end) or remapping to APR data.

[0087] As a result, the methods and systems embodying the present invention provide a highly efficient image analysis pipeline based on APR, thereby reducing storage (memory) requirements and processing large 3D datasets at speeds 20 to 1000 times faster (improved processing speed) than existing solutions. Such embodiments are extremely suitable for large-scale imaging projects.

[0088] Further details of embodiments of the present invention are provided by the appendix of the priority application (which incorporates its entirety herein by reference).

[0089] (Computer assistance)

[0090] Figure 4 shows a digital processing environment, such as a computer network, in which embodiments of the present invention can be realized.

[0091] One or more client computers / devices 50 and one or more server computers 60 become processing units, storage devices, and input / output devices that execute application programs, core programs, etc. One or more client computers / devices 50 may consist of imaging sources such as the aforementioned medical imaging equipment 101. One or more client computers / devices 50 may also be connected to other computing devices such as further client devices / processes 50 and one or more server computers 60 via a communication network 70. The communication network 70 may be part of a remote access network, a global network (e.g., the Internet), a cloud computing server or service, a collection of computers worldwide, a local area network, a wide area network, or a gateway, all of which currently communicate with each other using their respective protocols (TCP / IP, Bluetooth®, etc.). Other electronic device / computer network architectures are also preferred.

[0092] Figure 5 is a diagram of the internal structure of the computers (e.g., client processor / device 50, server computer 60, etc.) in the computer system of Figure 4. Each computer 50, 60 is equipped with a system bus 79, which is a series of hardware lines used to transmit data between computer and processing system components. In essence, the bus 79 is a shared conduit that connects each component of the computer system (e.g., processor, disk storage, memory, I / O ports, network ports, etc.) and enables the movement of information between these components. The system bus 79 is fitted with an I / O device interface 82 for connecting various I / O devices (e.g., keyboard, mouse, display, printer, speaker, etc.) to the computers 50, 60. The network interface 86 enables the computers to connect to various other devices attached to the network (e.g., network 70 in Figure 4, etc.). Memory 90 is a volatile storage unit for computer software instructions 92 and data 94 used to implement one embodiment of the present invention (for example, the image processing method, system, technology, data store / database, program code, etc., as described in detail in Figures 3A, 3B, 6A, and 6B). Disk storage 95 is a non-volatile storage unit for computer software instructions 92 and data 94 used to implement one embodiment of the present invention. The system bus 79 is further equipped with a central processing unit 84 that executes computer instructions.

[0093] In one embodiment, the processor routines 92 and data 94 are a computer program product (generally denoted 92) such as a computer-readable medium (e.g., a removable storage medium such as at least one DVD-ROM, CD-ROM, diskette, tape, etc.) that provides at least a portion of the software instructions for the system of the present invention. The computer program product 92 may be installed by any suitable software installation procedure known in the art. In another embodiment, at least a portion of the software instructions may be downloaded via wired communication and / or wireless connection. In another embodiment, the program of the present invention is a computer program propagated signal product 107 embedded in a propagated signal on a propagating medium (e.g., radio waves, infrared waves, laser waves, sound waves, electric waves propagated by a global network such as the Internet or one or more other networks, etc.). Such a carrier medium or signal constitutes at least a portion of the software instructions for the routines / programs 92 of the present invention.

[0094] In an alternative embodiment, the propagated signal is an analog carrier wave or a digital signal carried on a propagation medium. For example, the propagated signal may be a digitized signal propagated by a network such as a global network (e.g., the Internet) or a telecommunications network. In one embodiment, the propagated signal is a signal transmitted by the propagation medium for a certain period of time, for example, an instruction for a software application transmitted in packets by the network for a period of several milliseconds, several seconds, several minutes or more. In another embodiment, the computer-readable medium of the computer program product 92 is a propagation medium that can be received and read by the computer system 50. For example, the computer system 50 receives the propagation medium and identifies the propagated signal incorporated in the propagation medium, as in the case of the computer program propagated signal product described above.

[0095] Generally speaking, the term "transient medium" or "transient carrier" encompasses the aforementioned transient signals, propagating signals, propagation media, and storage media.

[0096] In other embodiments, the program product 92 may be implemented as so-called Software as a Service (SaaS), or as other installations or communications that support the end user.

[0097] Based on the above description and details, embodiments of the present invention offer numerous advantages over conventional methods and approaches. For example, in the embodiments, the target object is segmented from the image data at the time of acquisition, rather than after the source image is acquired. This novel approach enables scalability and allows for the generation of results (image objects) that can be used immediately after image acquisition.

[0098] Advantageously, in this embodiment, the target image is stitched together and reconstructed by reading each tile (i.e., the stored compressed image data, the stored relative tile position, the compression parameters, the stored segmented object, and the stored object position) only once. In contrast, in the prior art method, the tile data is read twice: once to evaluate the registration parameters, and then once at the final stage to combine all the tiles and generate an image file. In the prior art approach, analysis such as object segmentation may be applied to the image file. However, the approach of combining all tiles is not necessarily scalable, as at some point the image file may become too large to process or store. The applicant of this application solves these shortcomings in the art by maintaining individual tiles as separate entities. The applicant's approach rather combines segmented object information (e.g., segmented objects that appear more than once in overlapping or adjacent tile regions, as described above in module 130 of Figure 3A) as needed. The applicant's approach in this application allows for good scaling, as long as the tile processing keeps pace with the data throughput of the source device 101 (which can be achieved using an adaptive sampling representation of the tiles, e.g., APR), since each tile is processed individually (or together with its neighbors).

[0099] All patents, patent application publications, and all teachings cited herein are incorporated by reference. [Prior art documents] [Non-patent literature]

[0100] [Non-Patent Document 1] [Reference 1] Bria, A. & Iannello, G. TeraStitcher - a tool for fast automatic 3d-stitching of teravoxel-sized microscopy images. BMC Bioinformatics 13 (2012). https: / / doi.org / 10.1186 / 1471-2105-13-316.

Non-licensed Document 2

Non-licensed Document 4

Non-licensed Document 5

[0101] While exemplary embodiments have been specifically illustrated and described, those skilled in the art will understand that various modifications to the form and details may be made without departing from the scope of the embodiments covered by the attached claims. Furthermore, the present invention includes the following aspects. [Aspect 1] A computer-based image processing method, A. With respect to a given image data of a living organism, a series of 2D image units, each consisting of n×m pixel frames in each parallel plane orthogonal to a third spatial dimension axis, is acquired in the depth direction, and each series of 2D image units along each orthogonal axis is acquired in the depth direction, each 2D image unit in a series being an n×m pixel frame in each parallel plane orthogonal to the corresponding axis of the series, and different series having different orthogonal axes intersecting each plane at different tile positions, and through the acquisition process, multiple series that function as tiles are obtained, representing image volumes of various parts of the living organism captured in three spatial dimensions, and each of these multiple series is a separate tile representing a separate image volume, and B. When capturing the source image, i) A sub-process to obtain an adaptive sampling representation for the tile by compressing the image data of the tile region with less information and maintaining the high resolution of the region with more information, ii) A sub-process for determining the relative position of a tile to other tiles, iii) For the tiles for the target object, the adaptive sampling representation is segmented, and this segmentation includes a sub-process of extracting object features and corresponding object locations, and iv) Instead of storing the source image of the biological object in a data store in computer memory, the given image data is stored in a reduced quantity along with the individual tile locations by a sub-process of storing the adaptive sampling representation for the tile, the obtained tile locations, the segmented object, the extracted object features, and the object locations in the data store, The process of automatically processing each tile individually by a computer processor, C. A process for aggregating the segmented objects stored in the data store, wherein the aggregation includes obtaining a worklist of the segmented objects stored in the data store by automatically merging objects that appear two or more times in adjacent tiles among the segmented objects. D. A process of reading tiles only once based on the adaptive sampling representation for the tiles, tile positions, and worklists for the segmented objects stored in the data store, and from there forming a complete image of part or all of the biological object at maximum resolution, wherein the need for additional computer memory and / or processor resources for any of the storage, retrieval, communication, distribution, processing, analysis, display, and combination thereof of the complete image is mitigated by the fact that the tiles are read only once. A method that includes [a certain feature]. [Aspect 2] In the method described in Embodiment 1, the process of processing each tile individually is further, A sub-process that automatically determines the compression parameters used to compress the image data of the tile, A sub-process for storing the requested compression parameters within the data store, A sub-process that automatically compares the obtained adaptive sampling representation with the source image data to verify that the adaptive sampling representation captures the necessary information of the given image data. Methods that include... [Aspect 3] In the method according to Embodiment 1, the sub-process for determining the relative tile positions of the tiles is, To calculate the maximum projection image of the adaptive sampling representation for each edge of the tile in multiple spatial directions, and for adjacent tiles, For each edge of the tile where a corresponding adjacent tile has already been acquired, the calculated maximum projection image of the edge of the tile is correlated with the calculated maximum projection image of the corresponding edge of the corresponding adjacent tile, thereby improving the relative accuracy of the tile with respect to its adjacent tiles, and calculating the registration parameters for each tile pair with adjacent tiles already acquired after adaptive sampling, and, The data store stores the calculated registration parameters for each tile pair. Methods that include... [Aspect 4] In the method of embodiment 3, further, An optimization process, after image acquisition, for globally optimizing all the results of the stored pairwise registrations with respect to a confidence scale, wherein the maximization process ensures that all tile positions are accurately corrected, enabling (a) the stitching together of image data corresponding to the tiles, (b) the proper reconstruction of the entire biological object, and (c) the reliable extraction of higher-order information about the imaged biological object, such as (but not limited to) the volume of the biological object and the number of segmented objects within it. A method comprising process C, which is performed after global optimization. [Aspect 5] A method according to the method of embodiment 4, wherein the splicing is performed by reading each tile only once. [Aspect 6] The method according to Embodiment 1, wherein the given image data includes a biological image or a medical image captured and generated by any of the following digital imaging techniques: optical microscopy, electron microscopy, X-ray, computed tomography, positron emission tomography, optical coherence tomography, magnetic resonance imaging, ultrasound, and other digital imaging techniques, wherein for a target tile, processes A and B are performed when capturing the biological image or medical image, and process C is performed after capturing the image corresponding to the target tile. [Aspect 7] A method according to Embodiment 1, wherein the given image data further includes any or a combination of color, annotation, text, graphics, temporal features, etc. [Aspect 8] The method according to Embodiment 1, wherein the biological material is any of a tissue sample, a whole mouse brain, a cross-section of a human brain, another part of a mammalian brain, and another biological material of the subject. [Aspect 9] In the method described in Embodiment 1, further, In addition to compressing the tile region, the process involves individually compressing different fluorescence channels of the given image data. A method that includes [a certain feature]. [Aspect 10] In the method described in Embodiment 1, further, In addition to compressing the tile region, the process involves compressing the given image data over various periods. A method that includes [a certain feature]. [Aspect 11] A method according to Embodiment 1, wherein tile regions with low information content are compressed using adaptive particle representation (APR) technology. [Aspect 12] In the method according to Embodiment 1, the segmentation applies a classifier to the adaptive sampling representation for the tile, and the classifier is (i) Image filtering process and subsequent thresholding process, (ii) Machine learning algorithms such as (but not limited to) random forests, support vector machines, and neural networks, which have been trained on similar data. A method formed by any of the following. [Aspect 13] A method according to Embodiment 1, wherein, in the process of aggregating the segmented objects, the objects are automatically merged using a matching algorithm. [Aspect 14] In the method described in Embodiment 1, further, The process of calculating the position of the segmented object using the tile positions stored in the aforementioned data store, A method that includes [a certain feature]. [Aspect 15] In the method described in Embodiment 1, further, (a) a process of displaying an image of the living organism based on the adaptive sampling representation stored for the tile and the tile position stored in the data store, (b) an image of the segmented object based on the segmented object data stored in the data store, or a combination thereof. A method that includes [a certain feature]. [Aspect 16] In the method described in Embodiment 1, further, The process of interfacing with a registration pipeline that aligns the volumes of biological objects within a common coordinate framework, and the process of mapping the segmented objects within the common coordinate framework. A method that includes [a certain feature]. [Aspect 17] A method according to Embodiment 1, wherein the complete image is displayed in a lower resolution, either in lieu of or as an option. [Aspect 18] In the method described in Embodiment 1, further, A process of reading and displaying a specific portion of the given image data from the adaptive sampling representation, A method that includes [a certain feature]. [Aspect 19] A method according to Embodiment 1, wherein the adaptive sampling representation stored for each tile is merged and cut with overlapping tile regions such that the physical location of the living organism is uniquely described by one particle of the adaptive sampling representation stored in the data store for the tile. [Aspect 20] A computer-based biomedical image processing system, A. An interface for receiving or accessing image data of a living organism, wherein the interface acquires a series of 2D image units in the depth direction, each of which is an n×m pixel frame of each parallel plane orthogonal to a third spatial dimension axis, and the acquisition includes acquiring each series of 2D image units in the depth direction along each orthogonal axis, where each 2D image unit in a series is an n×m pixel frame of each parallel plane orthogonal to the corresponding axis of the series, different series have different orthogonal axes intersecting each plane at different tile positions, and the acquisition yields a plurality of series that function as tiles, representing image volumes of various parts of the living organism captured in three spatial dimensions, where each of the plurality of series is a separate tile representing a separate image volume, and B. A tile processing module that is communicably connected to the interface, wherein the tile processing module is automatically executed by a digital processor when an image is captured. i) By compressing the image data of the tile region with less information and maintaining the maximum resolution of the region with more information, an adaptive sampling representation of the tile is obtained. ii) Determine the relative position of the tile to other tiles, iii) Segmenting the adaptive sampling representation of the tile with respect to the target object, the segmentation including extracting object features and corresponding object locations. iv) Instead of storing the source image of the biological object in a data store in computer memory, the adaptive sampling representation for the tile, the obtained tile position, the segmented object, the extracted object features, and the object position are stored in the data store in such a reduced manner that the target image data is stored together with the individual tile positions. A tile processing module that processes each tile individually, C. An aggregation module that responds to the tile processing module, wherein the aggregation module aggregates the segmented objects stored in the data store, and the aggregation includes obtaining a worklist, object position, and feature quantities for the segmented objects stored in the data store by automatically merging objects that appear two or more times in adjacent tiles among the segmented objects, D. An output assembly having content based on the adaptive sampling representation of tiles, tile positions, and the worklist for segmented objects stored in the data store, wherein the output assembly is configured to read tiles from the content only once to form a complete image of part or all of the biological object at maximum resolution, and the single tile reading reduces the need for additional computer memory and / or processor resources for any of the storage, retrieval, communication, distribution, processing, analysis, display, and combination thereof of the complete image, A computer-based biomedical image processing system equipped with [specific features / features]. [Aspect 21] A computer-based biomedical image processing system according to embodiment 20, wherein the interface receives or accesses the target image data from a source device, computer memory, archive, or any combination thereof. [Aspect 22] A computer-based biomedical image processing system according to Embodiment 20, wherein the target image data includes a biological image or medical image captured and generated by a source device using any of the following digital imaging techniques: X-ray, computed tomography, positron emission tomography, optical microscopy, electron microscopy, optical coherence tomography, magnetic resonance imaging, ultrasound, and other digital imaging techniques, and the tile processing module is configured to process the biological image or medical image in near real-time from the acquisition of the biological image or medical image. [Aspect 23] A computer-based biomedical image processing system according to Embodiment 20, wherein the target image data further includes any or a combination of color, annotation, text, graphics, temporal features, etc. [Aspect 24] A computer-based biomedical image processing system according to embodiment 20, wherein the biological material is one of a tissue sample, a whole mouse brain, a cross-section of a human brain, another part of a mammalian brain, and another biological material of the subject. [Aspect 25] A computer-based biomedical image processing system according to embodiment 20, wherein the tile processing module individually performs the adaptive sampling compression for different fluorescence channels of the target image data. [Aspect 26] A computer-based biomedical image processing system according to embodiment 20, wherein the tile processing module performs the adaptive sampling compression on the target image data over various time periods. [Aspect 27] A computer-based biomedical image processing system according to embodiment 20, wherein the tile processing module compresses image data of low-information tile regions using adaptive particle representation (APR) technology. [Aspect 28] A computer-based biomedical image processing system according to embodiment 20, wherein the system is further capable of reading and displaying a specific portion of the target image data from the adaptive sampling representation stored in the data store. [Aspect 29] A computer-based biomedical image processing system according to embodiment 20, wherein the complete image from the output assembly is optionally or alternatively displayed at a lower resolution. [Aspect 30] In the computer-based biomedical image processing system described in embodiment 20, further, (i) an interface connection to a registration pipeline that aligns the volumes of various biological objects within a common coordinate system, and (ii) a registration interface that enables mapping of the segmented objects within the common coordinate system. A computer-based biomedical image processing system equipped with [specific features / features]. [Aspect 31] In the computer-based biomedical image processing system described in embodiment 20, the tile processing module performs the following actions with respect to the adaptive sampling representation: (i) Image filtering process and subsequent thresholding process, (ii) Machine learning algorithms such as (but not limited to) random forests, support vector machines, and neural networks, which have been trained on similar data. A computer-based biomedical image processing system that segments objects by applying a classifier formed by one of the following methods. [Aspect 32] A computer-based biomedical image processing system according to embodiment 20, wherein the aggregation module automatically combines the segmented objects using a matching algorithm when aggregating them. [Aspect 33] A computer-based biomedical image processing system according to embodiment 20, wherein the tile processing module further calculates the position of the segmented object using the tile positions stored in the data store. [Aspect 34] In the computer-based biomedical image processing system described in embodiment 20, further, A display module connected to display an image of the living organism based on the stored adaptive sampling representation of the tile, the tile position stored in the data store, and the segmented object data stored in the data store. A computer-based biomedical image processing system equipped with [specific features / features]. [Aspect 35] A computer-based biomedical image processing system according to embodiment 20, wherein the tile processing module stores the adaptive sampling representations of each tile that has been merged and cut in overlapping tile regions in the data store, and each of the stored tiles uniquely represents the physical position in the biological object by a single particle in the adaptive sampling representation of the tile stored in the data store. [Aspect 36] A computer-based image processing method, A. With respect to a given image data of a living organism, a series of 2D image units, each consisting of n×m pixel frames in each parallel plane orthogonal to a third spatial dimension axis, is acquired in the depth direction, and each series of 2D image units along each orthogonal axis is acquired in the depth direction, each 2D image unit in a series being an n×m pixel frame in each parallel plane orthogonal to the corresponding axis of the series, and different series having different orthogonal axes intersecting each plane at different tile positions, and through the acquisition process, multiple series that function as tiles are obtained, representing image volumes of various parts of the living organism captured in three spatial dimensions, and each of these multiple series is a separate tile representing a separate image volume, and B. i) A sub-process to obtain an adaptive sampling representation for the tile by compressing the image data of the tile region with less information and maintaining the high resolution of the region with more information, ii) A sub-process for determining the relative position of a tile to other tiles, iii) With respect to the tiles for the target object, the adaptive sampling representation is segmented, and this segmentation includes a sub-process of extracting object features and corresponding object locations, and iv) Instead of storing the source image of the biological object in a data store in computer memory, the given image data is stored in a reduced quantity along with the individual tile locations by a sub-process of storing the adaptive sampling representation for the tile, the obtained tile locations, the segmented object, the extracted object features, and the object locations in the data store, The process of automatically processing each tile individually by a computer processor, C. A process for aggregating the segmented objects stored in the data store, wherein the aggregation includes obtaining a worklist of the segmented objects stored in the data store by automatically merging objects that appear two or more times in adjacent tiles among the segmented objects. D. A process of reading tiles only once based on the adaptive sampling representation for the tiles, tile positions, and worklists for the segmented objects stored in the data store, and from there forming a complete image of part or all of the biological object at maximum resolution, wherein the need for additional computer memory and / or processor resources for any of the storage, retrieval, communication, distribution, processing, analysis, display, and combination thereof of the complete image is mitigated by the fact that the tiles are read only once. A method that includes [a certain feature]. [Aspect 37] The method according to embodiment 36, wherein with respect to the target tile, processes A and B are performed when the source image is captured, and process C is performed after the image of the image corresponding to the target tile is captured. [Aspect 38] A computer-based biomedical image processing system, A. An interface for receiving or accessing image data of a living organism, wherein the interface acquires a series of 2D image units in the depth direction, each of which is an n×m pixel frame of each parallel plane orthogonal to a third spatial dimension axis, and the acquisition includes acquiring each series of 2D image units in the depth direction along each orthogonal axis, where each 2D image unit in a series is an n×m pixel frame of each parallel plane orthogonal to the corresponding axis of the series, different series have different orthogonal axes intersecting each plane at different tile positions, and the acquisition yields a plurality of series that function as tiles, representing image volumes of various parts of the living organism captured in three spatial dimensions, where each of the plurality of series is a separate tile representing a separate image volume, and B. A tile processing module that is communicably connected to the interface, wherein the tile processing module is automatically executed by a digital processor, i) By compressing the image data of the tile region with less information and maintaining the maximum resolution of the region with more information, an adaptive sampling representation of the tile is obtained. ii) Determine the relative position of the tile to other tiles, iii) Segmenting the adaptive sampling representation of the tile with respect to the target object, the segmentation including extracting object features and corresponding object locations. iv) Instead of storing the source image of the biological object in a data store in computer memory, the adaptive sampling representation for the tile, the obtained tile position, the segmented object, the extracted object features, and the object position are stored in the data store in such a reduced manner that the target image data is stored together with the individual tile positions. A tile processing module that processes each tile individually, C. An aggregation module that responds to the tile processing module, wherein the aggregation module aggregates the segmented objects stored in the data store, and the aggregation includes obtaining a worklist, object position, and feature quantities for the segmented objects stored in the data store by automatically merging objects that appear two or more times in adjacent tiles among the segmented objects, D. An output assembly having content based on the adaptive sampling representation of tiles, tile positions, and the worklist for segmented objects stored in the data store, wherein the output assembly is configured to read tiles from the content only once to form a complete image of part or all of the biological object at maximum resolution, and the single tile reading reduces the need for additional computer memory and / or processor resources for any of the storage, retrieval, communication, distribution, processing, analysis, display, and combination thereof of the complete image, A computer-based biomedical image processing system equipped with [specific features / features]. [Aspect 39] A computer-based biomedical image processing system according to embodiment 38, wherein the target image data includes a biomedical image or medical image captured and generated by a source device, and the tile processing module is configured to process the biomedical image or medical image in near real-time from the acquisition of the biomedical image or medical image.

Claims

1. A computer-based image processing method, A. With respect to a given image data of a living organism, a series of 2D image units, each consisting of n x m pixel frames in parallel planes orthogonal to a third spatial dimension axis, is acquired in the depth direction, and each series of 2D image units along each orthogonal axis is acquired in the depth direction, each 2D image unit in a series being an n x m pixel frame in parallel planes orthogonal to the corresponding axis of the series, and different series having different orthogonal axes intersecting each plane at different tile positions, and through the acquisition process, multiple series that function as tiles are obtained, representing image volumes of various parts of the living organism captured in three spatial dimensions, and each of these multiple series is a separate tile representing a separate image volume, and B. When capturing the source image, i) A sub-process to obtain an adaptive sampling representation for the tile by compressing the image data of the tile region with less information and maintaining the high resolution of the region with more information, ii) A side process to determine the relative position of a tile to other tiles, iii) With respect to the tiles for the target object, the adaptive sampling representation is segmented, and this segmentation includes a sub-process of extracting object features and corresponding object locations, and iv) Instead of storing the source image of the biological object in a data store in computer memory, the given image data is stored in a reduced quantity along with the individual tile locations by a sub-process of storing the adaptive sampling representation for the tile, the obtained tile locations, the segmented object, the extracted object features, and the object locations in the data store, The process of automatically processing each tile individually by a computer processor, C. A process for aggregating the segmented objects stored in the data store, wherein the aggregation includes obtaining a worklist of the segmented objects stored in the data store by automatically merging objects that appear two or more times in adjacent tiles among the segmented objects. D. A process of reading tiles only once based on the adaptive sampling representation for the tiles, tile positions, and a worklist for the segmented objects stored in the data store, thereby forming a complete image of part or all of the biological object at maximum resolution, wherein the single-tile reading process reduces the need for additional computer memory and / or processor resources for any of the storage, retrieval, communication, distribution, processing, analysis, display, and / or combination thereof of the complete image. A method that includes [a certain feature].

2. The method according to claim 1, further comprising the process of processing each tile individually, A sub-process that automatically determines the compression parameters used to compress the image data of the tile, A sub-process for storing the requested compression parameters within the data store, A sub-process that automatically compares the obtained adaptive sampling representation with the source image data to verify that the adaptive sampling representation captures the necessary information of the given image data. Methods that include...

3. In the method according to claim 1, the sub-process for determining the relative tile positions of the tiles is, To calculate the maximum projection image of the adaptive sampling representation for each edge of the tile in multiple spatial directions, and for adjacent tiles, For each edge of the tile where a corresponding adjacent tile has already been acquired, the calculated maximum projection image of the edge of the tile is correlated with the calculated maximum projection image of the corresponding edge of the corresponding adjacent tile, thereby improving the relative accuracy of the tile with respect to its adjacent tiles, and calculating the registration parameters for each tile pair with adjacent tiles already acquired after adaptive sampling, and, The data store stores the calculated registration parameters for each tile pair. Methods that include...

4. The method according to claim 3, further, An optimization process, after image acquisition, for globally optimizing all the results of the stored pairwise registrations with respect to a confidence scale, wherein the optimization process ensures that all tile positions are accurately corrected, enabling (a) the stitching together of image data corresponding to tiles, (b) the proper reconstruction of the entire biological object, and (c) the reliable extraction of higher-order information about the imaged biological object, such as (but not limited to) the volume of the biological object and the number of segmented objects within it. A method comprising process C, which is performed after global optimization.

5. The method according to claim 4, wherein the splicing is performed by reading each tile only once.

6. The method according to claim 1, wherein the given image data includes a biological image or a medical image captured and generated by any of the following digital imaging techniques: optical microscopy, electron microscopy, X-ray, computed tomography, positron emission tomography, optical coherence tomography, magnetic resonance imaging, ultrasound, and other digital imaging techniques, wherein for a target tile, processes A and B are performed when capturing the biological image or medical image, and process C is performed after capturing the image corresponding to the target tile.

7. The method according to claim 1, wherein the given image data further includes any or a combination of color, annotation, text, graphics, and temporal features.

8. The method according to claim 1, wherein the biological material is any of a tissue sample, a whole mouse brain, a cross-section of a human brain, another part of a mammalian brain, and another biological material of the subject.

9. The method according to claim 1, further, In addition to compressing the tile region, the process involves individually compressing different fluorescence channels of the given image data. A method that includes [a certain feature].

10. The method according to claim 1, further, In addition to compressing the tile region, the process involves compressing the given image data over various periods. A method that includes [a certain feature].

11. A method according to claim 1, wherein a tile region with low information content is compressed using adaptive particle representation (APR) technology.

12. In the method according to claim 1, the segmentation applies a classifier to the adaptive sampling representation for the tile, and the classifier is (i) Image filtering process, and subsequent thresholding process, (ii) Machine learning algorithms such as (but not limited to) random forests, support vector machines, and neural networks, which have been trained on similar data. A method formed by any of the following.

13. A method according to claim 1, wherein, in the process of aggregating the segmented objects, the objects are automatically merged by a matching algorithm.

14. The method according to claim 1, further, The process of calculating the position of the segmented object using the tile positions stored in the aforementioned data store, A method that includes [a certain feature].

15. The method according to claim 1, further, (a) a process of displaying an image of the living organism based on the stored adaptive sampling representation for the tile and the tile position stored in the data store, (b) an image of the segmented object based on the segmented object data stored in the data store, or a combination thereof. A method that includes [a certain feature].

16. The method according to claim 1, further, The process of interfacing with a registration pipeline that aligns the volumes of biological objects within a common coordinate framework, and the process of mapping the segmented objects within the common coordinate framework. A method that includes [a certain feature].

17. A method according to claim 1, wherein the complete image is displayed in a lower resolution, either in lieu of or optionally.

18. The method according to claim 1, further, A process of reading and displaying a specific portion of the given image data from the adaptive sampling representation, A method that includes [a certain feature].

19. A method according to claim 1, wherein the adaptive sampling representation stored for each tile is merged and cut with overlapping tile regions such that the physical location of the living organism is uniquely described by one particle of the adaptive sampling representation stored in the data store for the tile.

20. A computer-based biomedical image processing system, A. An interface for receiving or accessing image data of a living organism, wherein the interface acquires a series of two-dimensional image units in the depth direction, each of which is an n x m pixel frame of each parallel plane orthogonal to a third spatial dimension axis, and the acquisition includes acquiring each series of two-dimensional image units in the depth direction along each orthogonal axis, where each two-dimensional image unit in a series is an n x m pixel frame of each parallel plane orthogonal to the corresponding axis of the series, and different series have different orthogonal axes intersecting each plane at different tile positions, and the acquisition yields a plurality of series that function as tiles, representing image volumes of various parts of the living organism captured in three spatial dimensions, where each of the plurality of series is a separate tile representing a separate image volume, and B. A tile processing module that is communicably connected to the interface, wherein the tile processing module is automatically executed by a digital processor when an image is captured. i) By compressing the image data of the tile region with less information and maintaining the maximum resolution of the region with more information, an adaptive sampling representation of the tile is obtained. ii) Determine the relative position of the tile to other tiles, iii) The adaptive sampling representation of the tile is segmented with respect to the target object, and this segmentation includes extracting object features and corresponding object locations. iv) Instead of storing the source image of the biological object in a data store in computer memory, the adaptive sampling representation for the tile, the obtained tile position, the segmented object, the extracted object features, and the object position are stored in the data store in such a reduced manner that the target image data is stored together with the individual tile positions. A tile processing module that processes each tile individually, C. An aggregation module that responds to the tile processing module, wherein the aggregation module aggregates the segmented objects stored in the data store, and the aggregation includes obtaining a worklist, object position, and feature quantities for the segmented objects stored in the data store by automatically merging objects that appear two or more times in adjacent tiles among the segmented objects, D. An output assembly having content stored in the data store, which is based on the adaptive sampling representation of tiles, tile positions, and the worklist for segmented objects, wherein the output assembly is configured to read tiles from the content only once to form a complete image of part or all of the biological object at maximum resolution, and the single tile reading reduces the need for additional computer memory and / or processor resources for any of the storage, retrieval, communication, distribution, processing, analysis, display, and combination thereof of the complete image, A computer-based biomedical image processing system equipped with [specific features / features].

21. A computer-based biomedical image processing system according to claim 20, wherein the interface receives or accesses the target image data from a source device, computer memory, archive, or any combination thereof.

22. A computer-based biomedical image processing system according to claim 20, wherein the target image data includes a biological image or medical image captured and generated by a source device using any of the following: X-ray, computed tomography, positron emission tomography, optical microscopy, electron microscopy, optical coherence tomography, magnetic resonance imaging, ultrasound, and other digital imaging techniques, and the tile processing module is configured to process the biological image or medical image in near real-time from the acquisition of the biological image or medical image.

23. A computer-based biomedical image processing system that performs the method according to any one of claims 1 to 19.

24. A computer-based image processing method, A. With respect to a given image data of a living organism, a series of 2D image units, each consisting of n x m pixel frames in parallel planes orthogonal to a third spatial dimension axis, is acquired in the depth direction, and each series of 2D image units along each orthogonal axis is acquired in the depth direction, each 2D image unit in a series being an n x m pixel frame in parallel planes orthogonal to the corresponding axis of the series, and different series having different orthogonal axes intersecting each plane at different tile positions, and through the acquisition process, multiple series that function as tiles are obtained, representing image volumes of various parts of the living organism captured in three spatial dimensions, and each of these multiple series is a separate tile representing a separate image volume, and B. i) A sub-process to obtain an adaptive sampling representation for the tile by compressing the image data of the tile region with less information and maintaining the high resolution of the region with more information, ii) A side process to determine the relative position of a tile to other tiles, iii) With respect to the tiles for the target object, the adaptive sampling representation is segmented, and this segmentation is a sub-process that includes extracting object features and corresponding object locations, and iv) Instead of storing the source image of the biological object in a data store in computer memory, the given image data is stored in a reduced quantity along with the individual tile locations by a sub-process of storing the adaptive sampling representation for the tile, the obtained tile locations, the segmented object, the extracted object features, and the object locations in the data store, The process of automatically processing each tile individually by a computer processor, C. A process for aggregating the segmented objects stored in the data store, wherein the aggregation includes obtaining a worklist of the segmented objects stored in the data store by automatically merging objects that appear two or more times in adjacent tiles among the segmented objects. D. A process of reading tiles only once based on the adaptive sampling representation for the tiles, tile positions, and a worklist for the segmented objects stored in the data store, thereby forming a complete image of part or all of the biological object at maximum resolution, wherein the single-tile reading process reduces the need for additional computer memory and / or processor resources for any of the storage, retrieval, communication, distribution, processing, analysis, display, and / or combination thereof of the complete image. A method that includes [a certain feature].

25. The method according to claim 24, wherein with respect to the target tile, processes A and B are performed when the source image is captured, and process C is performed after the image of the image corresponding to the target tile is captured.

26. A computer-based biomedical image processing system, A. An interface for receiving or accessing image data of a living organism, wherein the interface acquires a series of two-dimensional image units in the depth direction, each of which is an n x m pixel frame of each parallel plane orthogonal to a third spatial dimension axis, and the acquisition includes acquiring each series of two-dimensional image units in the depth direction along each orthogonal axis, where each two-dimensional image unit in a series is an n x m pixel frame of each parallel plane orthogonal to the corresponding axis of the series, and different series have different orthogonal axes intersecting each plane at different tile positions, and the acquisition yields a plurality of series that function as tiles, representing image volumes of various parts of the living organism captured in three spatial dimensions, where each of the plurality of series is a separate tile representing a separate image volume, and B. A tile processing module that is communicably connected to the interface, wherein the tile processing module is automatically executed by a digital processor, i) By compressing the image data of the tile region with less information and maintaining the maximum resolution of the region with more information, an adaptive sampling representation of the tile is obtained. ii) Determine the relative position of the tile to other tiles, iii) The adaptive sampling representation of the tile is segmented with respect to the target object, and this segmentation includes extracting object features and corresponding object locations. iv) Instead of storing the source image of the biological object in a data store in computer memory, the adaptive sampling representation for the tile, the obtained tile position, the segmented object, the extracted object features, and the object position are stored in the data store in such a reduced manner that the target image data is stored together with the individual tile positions. A tile processing module that processes each tile individually, C. An aggregation module that responds to the tile processing module, wherein the aggregation module aggregates the segmented objects stored in the data store, and the aggregation includes obtaining a worklist, object position, and feature quantities for the segmented objects stored in the data store by automatically merging objects that appear two or more times in adjacent tiles among the segmented objects, D. An output assembly having content stored in the data store, which is based on the adaptive sampling representation of tiles, tile positions, and the worklist for segmented objects, wherein the output assembly is configured to read tiles from the content only once to form a complete image of part or all of the biological object at maximum resolution, and the single tile reading reduces the need for additional computer memory and / or processor resources for any of the storage, retrieval, communication, distribution, processing, analysis, display, and combination thereof of the complete image, A computer-based biomedical image processing system equipped with [specific features / features].

27. A computer-based biomedical image processing system according to claim 26, wherein the target image data includes a biomedical image or medical image captured and generated by a source device, and the tile processing module is configured to process the biomedical image or medical image in near real-time from the acquisition of the biomedical image or medical image.

Images