Method of processing digital image of a microscopic specimen

Abstract

A method of processing digital image of a microscopic specimen scanned in at least two layers which differ mutually in position of their plane of focus, the method comprising the steps of a) for individual layers of the image, a pyramid of reduced versions of the initial image is formed, wherein base level of the pyramid is formed by the respective layer of the initial digital image, and individual levels of the pyramids are divided into tiles, or individual layers of the initial digital image are divided into tiles and a pyramid of reduced images is formed for any of the individual tiles, wherein base level of any pyramid is formed by the respective tile in the respective layer of the initial digital image, b) points of interest are detected in at least one layer, c) sharpness levels of the detected points of interest in individual layers are calculated, d) information is assigned to the detected points of interest, in which layer the respective point of interest has maximum sharpness, In step e), a sparse pyramid is formed containing only such tiles from the individual layers that contain at least 1 point of interest having said maximum sharpness, and tiles that are immediately adjacent to said tiles within the individual layer.

Description

[0001] METHOD OF PROCESSING DIGITAL IMAGE OF A MICROSCOPIC SPECIMEN Field of Art

[0002] The invention relates to a method of processing digital image of a microscopic specimen captured in at least two layers which differ mutually in their position of plane of focus, relative to the plane of the specimen, wherein in this method the image is divided into tiles and at least one reduced version of them is assigned to at least some of the tiles.

[0003] Prior Art

[0004] Current technical solutions enable scanning of entire microscopic specimen (so-called WSI -Whole Slide Image). There are currently a number of commonly available WSI scanners on the market from a number of manufacturers, e.g. Hamamatsu Photonics, Roche (F. Hoffmann-La Roche Ltd), Leica Microsystems, Grundium, and others.

[0005] When e.g. histological and cytological specimens have been scanned, virtual viewing of microscopic cell and tissue samples on a computer monitor is possible, including remote viewing (so-called telepathology or telecytology), or other image analysis. See e.g. Roche's comprehensive digital pathology solution

[0006] Roche ( https. / / diagnosticsgoche,co / giobal / en / products / pj'odtict--category / digjtal--pathojo y--

[0007]

[0008] A three-dimensional microscopic specimen, for example, in the case of cytological samples (especially in the context of liquid-based cytology, where samples of cerebrospinal fluid, urine, punctures from preformed or pathological cavities, etc. are used), must be digitized by scanning in layers (meaning layers in the Z axis, perpendicular to the plane of the specimen), the thickness and number of which are selected during the actual scanning (so-called Z-stack scanning). Three-dimensional microscopic objects (e.g. cells) physically located in a specific scanned layer are then correctly focused in the digitized image only in this corresponding layer. The other layers are then abundant considering said specific object (cell). To actually view or analyze such a digitized image, it is then necessary to always select the optimal layer for a specific object of interest (e.g. cell). Such a selection of a specific layer, i.e. focusing on a specific object in the sense of classic microscopy, is then performed in a multilayer digitized image (so-called Z-Stack WSI) within the current technical solutions of digital pathology / cytology in various ways:

[0009] 1) Manually, wherein the operator / user selects the best focused layer. A disadvantage of manual focusing is that the optimal focusing layer is different in different places of the specimen and the user must focus the image (select the correct layer) quite often while viewing the entire microscopic specimen / WSI.

[0010] 2) Automatically, wherein a synthetic layer is composed of multiple layers. According to a standard solution, the image is divided into large areas and average quality of focusing of these areas is assessed, not taking into account individual particles (e.g. cells). The synthetic layer is created by joining said areas (mostly squares of a selected size, typically 256x256 to 1024x1024 pixels).

[0011] The disadvantage of that technical solution is that individual particles can lie at various depths within one area, and therefore a significant part of them can be (and often is) poorly focused. Furthermore, the boundary of the areas is chosen regardless of the particles, and therefore particles lying at the boundaries of the areas suffer from artifacts caused by joining the areas. A task of the invention is to draft a technical solution involving automated selection of a correctly focused layer for complex microscopic objects (e.g. cells in a cytological specimen) in a multilayer digitized image (Z-Stack, WSI) and subsequent joining of the images to form an optimally focused layer respecting all objects in the specimen, without losing primary image data for objects of interest (e.g. cells).

[0012] Summary of the Invention

[0013] The above task is solved by means of a computer-implemented method of processing a digital image of a microscopic specimen scanned in at least two layers which differ mutually in position of their plane of focus, the method comprising the steps of

[0014] a) for individual layers of the image, a pyramid of reduced versions of the initial image is formed, wherein base level of the pyramid is formed by the respective layer of the initial image, and individual levels of the pyramid are divided into tiles,

[0015] or individual layers of the initial image are divided into tiles and a pyramid of reduced images is formed for the individual tiles, wherein base level of any pyramid is formed by the respective tile in the respective layer of the initial digital image,

[0016] b) points of interest are detected in at least one layer,

[0017] c) sharpness levels of the detected points of interest in individual layers are calculated, d) to the detected points of interest an information is assigned, in which layer the respective point of interest has maximum sharpness,

[0018] e) a sparse pyramid is formed containing only such tiles from the individual layers that contain at least N points of interest having said maximum sharpness, and tiles that are immediately adjacent, i.e. tiles that are adjacent by their sides and tiles that are adjacent by their corners, to said tiles within the individual layer, N being equal to or greater than 1.

[0019] As follows from the mathematical methods proposed below, the term "point" in the expression "point of interest" is used in a mathematical sense, i.e. as a dimensionless entity defined by its coordinates. In general, preferably, pyramids contains at least one, preferably at least two reduced versions of the initial image.

[0020] Preferably, for each area that is defined by a tile and contains at least one point of interest detected in step b), an image of this tile is selected from a layer in which the highest number of detected points of interest with maximum sharpness is present, and optionally additionally for each area that is defined by a tile and in which no point of interest was detected in step c), an image of this tile is selected from a layer in which a point of interest that was detected closest to a center of this tile in step b) has maximum sharpness,

[0021] and a synthetic layer is composed of the images thus selected.

[0022] It is also advantageous, when in step a) the forming of reduced images comprises an application of an anti-aliasing filter with a Gaussian kernel, followed by direct subsampling. According to a preferred embodiment, in step a) the pyramid of reduced images of tiles is formed with a reduction step of 2:1 and / or in step a) the image is divided into tiles of equal size, preferably the image is divided into square tiles of 2xby 2xpixels, wherein x is an integer, preferably in the range of 8-10 inclusive.

[0023] According to an especially preferred embodiment, - the points of interest are detected by the stable wave detection method, and / or

[0024] - the points of interest are detected in one or more reduced versions of the image of one of the layers in grayscale, preferably the level or levels of the pyramid are selected based on the size of particles to be detected,

[0025] and / or

[0026] when processing a digital image containing at least three layers which differ mutually in position of their plane of focus, a pyramid from a layer that is not an outer layer is preferably selected for detection in step b), preferably a pyramid of a middle layer is selected.

[0027] After step b), the points of interest are preferably filtered based on their detector response amplitude using non-maximal suppression method, wherein point detected as having the largest amplitude is retained and the other points in its vicinity are removed.

[0028] In step c) a region of interest is defined, for instance, the region of interest having a center, which corresponds to a point of interest, after which the sharpness measures in the individual layers are calculated as the sums of brightness gradient over all pixels of the region of interest. Hence, the image is saved as a sparse pyramid, which - from the individual layers - contains only those tiles, which comprise at least N points of interest, having their maximum sharpness in the respective layer, and tiles, which are adjacent immediately to those tile, wherein N is equal to 1, or greater. Such sparse pyramid is supplemented with a synthetic layer, into which individual tiles are selected from that layer where N reaches a maximum. The principle of forming a sparse pyramid and a synthetic layer is shown in Fig. 2.

[0029] Before processing, quality of the digital image to be processed, may be evaluated, wherein the expected maximum size of the particles of interest to be viewed in the digital image is input, and for evaluation of quality of the digital image to be processed, a composite image is formed, which is composed of image slices of at least one of the outer layers, each of said slices having a center at the point of interest that has maximum sharpness in the respective outer layer, and wherein the slices have an area whose size corresponds to the size of the area of a square with a side of 1 to 3 times the expected maximum size of the particles of interest. Additionally or alternatively, the expected maximum size of the particles of interest to be viewed in the digital image is input, and for evaluation of quality of the digital image to be processed, a composite image is formed, which is composed of image slices of the layer in which the most points of interest have maximum sharpness, each of said slices having a center at the point of interest that has maximum sharpness in the respective layer, and wherein the slices have an area whose size corresponds to an area of a square with a side of 1 to 3 times the expected maximum size of the particles of interest.

[0030] Alternatively, the expected maximum size of the particles of interest to be viewed in the digital image is entered, and for evaluation of quality of the digital image to be processed, a composite image is formed for each layer, the composite image being composed of image slices of the respective layer, each of said slices having a center at the point of interest that has maximum sharpness in the respective layer, and wherein the slices have an area whose size corresponds to an area of a square with a side of 1 to 3 times the expected maximum size of the particles of interest.

[0031] The above drawbacks of prior art are also eliminated to a large extent by a computer program comprising instructions that, when executed on a computer, cause the computer to perform the above method, and also by a computer-readable medium comprising such computer program. The whole method is performed by a computer.

[0032] The method according to the invention allows to automatically find individual microscopic particles, e.g. cells, pollen grains, dust particles, in a digitized microscopic specimen and to determine in which layer each particle has optimal sharpness.

[0033] The method according to the invention is suitable for further storage and use of data from high-resolution scanning microscopes (typically 109pixels and more in each layer) scanning any specimen in multiple planes of focus (planes of focus in the specimen measured along the Z axis, which is the optical axis of the objective), wherein size of the primary image file is compressed, but without loss of image data for complex objects of interest. Lossless image compression is key for further image processing, e.g. in medical diagnostics, biology or research. Specimens can be both of an absorption or fluorescence type. The primary (reference) area of application is cytology / cytopathology, especially absorption-stained cytological specimens. Brief Description Of Drawings

[0034] The invention is further described on the basis of exemplary embodiments and with the aid of drawings, wherein Fig. 1 shows a graph of the degree of sharpness depending on the layer (the plane of focus in the specimen measured along the Z axis, which is the optical axis of the objective), and Fig. 2 shows a method of composing a synthetic layer and selection of tiles to be saved.

[0035] Exemplary Embodiments Of The Invention

[0036] First, it is necessary to prepare a microscopic specimen, as a model example, a cytological specimen is chosen, made by cytocentrifugation of a body fluid sample (Body Fluid - e.g. cerebrospinal fluid / CSF) and absorption stained. In alternative embodiments, specimens made by cytosedimentation or by smearing biological material on a microscope slide and stained with conventional staining methods (MGG, H& E, PAP / Polychrome, etc.), special methods (staining for lipid detection, for iron detection, PAS reaction, etc.) or microbiological methods (Gram staining and others), or processed for immunocytochemical detection of antigens with visualization using chromogen (ICC).

[0037] The relevant specimen is scanned and digitized with a high-resolution microscopic scanner, e.g. 109pixels (Whole-Slide Scanner) in multiple layers, i.e. focusing planes / planes of sharpness, or focus (so-called Z-Stack). In general, this method according to the invention is particularly suitable for images scanned with a resolution of at least 108, in particular at least 109, or higher.

[0038] The format for storing the image of the entire specimen (Whole-Slide Image - WSI) in multiple layers (Z-Stack) varies depending on the scanner manufacturer. In this specific exemplary embodiment, it is the Big-Tiff standard. The image is divided into square tiles. The typical tile size is 256x256 to 1024x1024 pixels.

[0039] In addition to the image having the highest resolution, a pyramid of reduced images with a step of 2:1 is also stored, in an alternative embodiment with a step of 4: 1. If the scanner does not provide said format, the image is first converted to said format by a suitable converter (recommended by the scanner manufacturer). Depending on the data source (type of scanner, or converter from proprietary to standardized format), it may be necessary to calculate the pyramid of reduced images. The pyramid advantageous for further processing has a reduction step of 2:1. The calculation of the reduced image consists of an antialiasing filter with a Gaussian kernel (5x5 pixels) followed by direct subsampling, wherein every second pixel is selected in both directions and the resulting image is therefore ¼ pixels, while the resulting pyramid of reduced images adds about 33% of the data volume to the initial image, which is an acceptable price for a significant acceleration of access during viewing and processing.

[0040] In a less advantageous alternative embodiment, a pyramid of reduced images can be created without using an antialiasing filter.

[0041] In this exemplary embodiment, a pyramid of reduced images is calculated over each scanned layer and all levels of the pyramid are divided into equally sized tiles, e.g. if the image has a size of 32k x 32k pixels (where k=1024), and the tile has a size of Ik x Ik, there are 1024 tiles in the base layer, the first reduced layer has a size of 16k x 16k and contains 256 tiles, the second reduced layer has a size of 8k x 8k and contains 64 tiles, etc.

[0042] Dividing an image into tiles is a technological matter that streamlines access to a file when viewing - they are always displayed on the same display, e.g. about 2MPix, and no matter how the magnification is set, typically 4-9 tiles and the same amount of data are still used when scrolling or zooming. The result is a constant system response time to user actions. Generally, tiles are preferably the same size at all levels of the pyramid, so that e.g. there are ¼ of them at level 2 compared to level 1.

[0043] To align corresponding tiles at different levels, a conversion between point coordinates at different levels of the pyramid(s) is preferably used and the appropriate tiles are selected accordingly, e.g. for the corresponding slice to display when zooming in on the image.

[0044] Points of interest are first detected in the image using Stable Wave Detection (SWD) method. From the general Stable Wave methodology, this implementation uses a fast spectrally selective detector of extreme, approximately symmetric regions in gray-scale images, which can be effectively implemented in a graphics processor using technologies used for neural networks. The method was first published in: J. Dupač and V. Hlaváč. Stable wave detector of blobs in images. Proceedings of the 28th DAGM (German Pattern Recognition Society) Symposium, volume 4174 of Lecture Notes in Computer Science, pages 760-769, Heidelberg, Germany, September 2006. DAGM, Springer, or as described in more detail in Dupač, Jan. Stable wave detector and tracker. Doctoral Thesis. CTU-CMP-2011-02.

[0045] This detector is not a neural network and does not require Deep Learning or a large training set. In the exemplary embodiment, the detection is performed on gray-scale images from the 3rd and 4th levels of the pyramid. Since SWD is very little affected by image blur, it is not necessary to detect points of interest in all layers, but for a typical cytological specimen scanned in the range of + / - 7 pm, detection in the middle layer is sufficient. For deeper scans, i.e. scans where the outer layers have a mutual distance greater than 14 pm, it is possible to detect points of interest in multiple layers. The output of SWD are the positions of points of interest in the image, their amplitude (response strength measuring their significance) and polarity distinguishing dark and light areas.

[0046] SWD is a very flexible method, based on an analysis of Fourier coefficients of the first harmonic wave. These are calculated in a window with a width given by the selected period, which is moved along the signal (ID) or image (2D). The implementation used works with a pair of windows (image slices) that are centered in the examined pixel. For each pixel of the image, a quadruple of Fourier coefficients is calculated according to the formula:

[0047] ' E X- u; EL-fJ(rd 4- L < <) + j) S(j U

[0048] "" EZ--W + MO + j) G ( j T

[0049] ” EL -4 EL W + tK

[0050] K El < E; ■ tL

[0051] W (W..... I) / ^ / . (T - l) / 2,

[0052] C(0 Su's KUIGM-

[0053]

[0054] -- 23T0 + 0.5) / T, i = 0, 1, . . . , T − 1.

[0055] The period T corresponds to the window length, the window width is W, for symmetry reasons T and W are odd numbers and W is chosen close to T / 2, in the implementation used T=7, W=3. Points of interest are the centers of pixels, where bh > |a | and bv> |av|, which are points representing dark areas, or -bh > |a | and -bv> |av|, which are points representing bright areas. The detector response strength - amplitude is min(|bh|, |bv|). The polarity of the point of interest is defined by the sign of bh (bh and bvhave the same sign at the point of interest), if bh > 0, the polarity is 1 (dark area), if bh < 0, the polarity is -1 (bright area).

[0056] By means of MNS method with a neighborhood of + / - (T / 2-1) weaker points of interest (measured by amplitude) are removed.

[0057] The points of interest found in this way correspond to areas of a size of approximately T / 2, in the case of T=7, areas from 1 pixel to 5 pixels can be detected. In order to detect areas corresponding to particles of larger expected size, it is possible to either change the period (with increasing period, the complexity of the calculation increases quadratically) or detect in a reduced image. If the size range is larger, as in the case of cells in the model application, this can be solved either by using more periods, ideally with a step of approximately 1:2, e.g. (7, 13, 25,...) or rather by detecting at several levels of the pyramid, which is more efficient (used in the model solution).

[0058] In the case of detection with multiple periods or at multiple levels of the pyramid, NMS filtering is again performed according to the amplitude, wherein the size of the surroundings corresponds to the expected size of the particles.

[0059] For an analysis of absorption-stained specimens, only dark areas corresponding to particles (e.g. cell nuclei, erythrocytes, or cell-like impurities in cytological specimens) are used. For the analysis of fluorescently stained specimens, only light areas would be used. The amplitude maxima are in both light and dark areas. Polarity is used to select between light and dark areas.

[0060] Points of interest are filtered according to amplitude using the Non-Maximal Suppression (NMS) method, in which the one with the largest amplitude is selected from a group of nearby points and the others are removed. The size of the surroundings for the application of NMS is chosen as the size of the smallest detected particles. NMS is also used in the case of detection in multiple layers in the case of a large spacing of the outer planes of sharpness.

[0061] Depending on the level of the pyramid at which the point of interest was detected, the approximate size of the region of interest is estimated as the ratio of the SWD period and the image reduction at the level of the pyramid at which the point of interest had the largest amplitude (this corresponds approximately to the size of the cell, if the point of interest lies in the nucleus of the cell). For example, if a point is detected at the 3rd level of the pyramid (reduction of 1 / 8) with an SWD period of 7 pixels, the estimated size of the region of interest is 56x56 pixels.

[0062] Primarily, points of interest are intended for selecting focused areas in the image when constructing a Sparse Z-Stack image representation. These points of interest can also serve in further stages of image processing as candidates for the possible position of particles (e.g. cells) during detection and segmentation. In the case of cytological specimens, the chosen algorithm (SWD) has a negligible rate of false negative cell detections, but a relatively high rate of false positive detections depending on the quality of the specimen (amount and type of impurities), which is desirable because important objects (cells) are not lost and it allows consistent focusing of areas without cells in sparse (oligocellular) specimens, thereby contributing to a better perception when viewing the specimen (a number of impurities is usually in the same optimal plane of focus as the cells, and the perception of the image is then similar to a perception from a microscope).

[0063] In an alternative embodiment, points of interest can be detected by other methods, such as the Maximally stable extremal regions (MSER) method, as described, for example, in the document by J. Matas, O. Chum, Urban M., and T. Pajdla: Robust wide baseline stereo from maximally stable extremal regions. Editors Paul L. Rosin and David Marshall, Proc, of the British Machine Vision Conference, volume 1, pages 384-393, London, UK, September 2002. BMVA.

[0064] For each of the detected points, the sharpness values in all scanned layers are calculated and for each one the optimal layer in which the respective point has maximum sharpness is selected. Different sharpness measures can be used. In the given exemplary embodiment, the sum of the brightness gradient magnitudes over all points of the square region of interest centered at the point of interest was used. Other known methods can also be used, for example the method described on the pages https: / / pyimagesearch.com / 2015 / 09 / 07 / blur-detection-with-openev / , as archived in the Internet Archive Wayback Machine on, for example, 24.1.2016, which is based on the use of the fast Fourier transform.

[0065] Fig. 1 shows images of the same cell scanned in different planes of sharpness, i.e. images of the same cell from different layers, and an associated graph, which has the layer designation on the horizontal axis and the sum of the brightness gradient magnitudes on the vertical axis. It is clear from the graph that layer 4 is selected as the best layer for the specific cell. Fig. 2 shows the method of tile evaluation / selection. The rectangle at the top indicates the specimen viewed from above, the intersection of the lines indicates the location to be displayed, and the horizontal line indicates the section of the specimen displayed below the rectangle. The section shows the scanned layers at several sharpness levels, which are indicated by horizontal lines and numbered on the right. The circles represent particles, the numbers indicate the optimal layer for a specific particle. The rectangles indicate the tiles that will be saved. The tiles drawn with a solid line lie in the layer that is optimal, i.e. has the greatest sharpness, for at least N points (in this example N = 1), and the tiles drawn with a dashed line are the tiles that are adjacent to them. Below the scanned layers, a synthetic (“focused”) layer is indicated, which is composed of tiles that are optimal for the maximum number of points, e.g. the left tile is from the first (top) layer, because it is optimal for 3 points, the second for 2 points, and the other layers for 0 points. The numbers in the tiles of the synthetic layer indicate the layer number from which the given image of the tile was selected into the synthetic layer.

[0066] The vertical line indicates how the layer is selected during viewing - the closest point of interest is found (corresponding to the closest particle) and the layer is selected based on it, in this case the second from the top.

[0067] In other words, the image is divided into tiles of a fixed size. In this exemplary embodiment, the size is 512x512 pixels, which is optimal for images of cytological specimens. Only tiles that are optimal for at least N points in the layer (N is an optional parameter) and tiles that are adjacent to them are saved, so that each particle (e.g. cell) has a sufficiently wide neighborhood in its optimal sharpness plane for further processing.

[0068] Individual tiles can be compressed using any selected compression method, ideally the same as has been used for the initial image. The default compression is JPG (this is usually used by scanner manufacturers, and it is therefore the same as in a typical initial image). If the tile size is preserved, it is possible to use compressed data directly from the initial image.

[0069] In addition to the scanned layers data, a synthetic layer is also stored, consisting of tiles from the optimal layer (i.e., from the layer that is optimal for the maximum number of points of interest lying in the tile). If the tile does not contain any focus point, the tile from the optimal layer of the point of interest closest to the center of the tile is used. The tiles in the highest resolution in the synthetic layer are not saved a second time, but are referenced by a link to the original layers. A pyramid of reduced images is calculated for each layer, including the focus layer (the default reduction step is 2:1). For scanned layers, the resulting saved pyramid is sparse, for the synthetic layer it is dense. A sparse pyramid means that in the higher layers of the pyramid only the tiles containing the best-focused points of interest and those adjacent to them are preserved (the same as in the base layer of the pyramid). In a dense pyramid, all tiles are preserved (standard image storage).

[0070] WSI images in the finest resolution typically have a size of around 109pixels, often much more, which cannot be displayed on a standard monitor (2-8 x 106pixels). It is possible to display only a slice of full-resolution view, a reduced overall view, or a reduced slice. Next, a layer must be selected. In standard state-of-the-art browsers, it is necessary to manually change the layer according to the area being viewed, similar to an optical microscope.

[0071] Images processed in accordance with the invention contain points of interest for each particle (cell) and information about the optimal layer for each point of interest. The advantage of the invention is that the browser can automatically select the optimal layer according to the particle being examined. Similarly, particles (e.g. cells) are detected and segmented in the image from the optimal layer. This prevents appearance of artifacts caused by joining into a single-layer image from multiple layers, such artifacts being present in the output of most of existing methods. These artifacts caused by joining are very disruptive both during viewing and during automatic processing.

[0072] Scanning cytological specimens is a complex process, because current scanners are not adapted to such specimens in terms of software. Their automatic focusing adapted to tissue sections usually works relatively well for specimens with high cell density, but in the case of specimens with low cell density it often fails without the standard scanning software recognizing it. Even with manual focusing, errors can occur. In addition to the problems specific to cytological specimens, quality of the scan is affected by general errors such as unevenness of the glass and dirt on the glass surface, which make positioning during scanning difficult and create false focusing targets. Therefore, it is necessary to check the scan result and, in case of insufficient quality, repeat the scan and take measures to eliminate the problem (clean the specimen, select more layers, or position the focusing point better).

[0073] Images processed according to the invention contain data that allow the quality of the scan to be visualized and thus significantly improve the quality control process. In an ideal specimen scanned by an ideal scanner, all points of interest should have the same optimal plane of sharpness corresponding to the middle layer, or possibly a few layers around the middle layer. This is the basis for the proposed visualization tools for quality control:

[0074] 1) Histogram of the optimal layer of points of interest

[0075] 2) Mosaic of points of interest filtered according to their optimal layer

[0076] 3) Spatial distribution of points of interest

[0077] 1) Assessing the quality of scanning using histogram

[0078] A histogram of optimal layers of points of interest is a column chart, the horizontal axis of which is the layer and the vertical axis is the number of focusing points for which this layer is optimal, i.e. the relevant point of interest has maximum sharpness therein. A histogram of an ideal scan has only one column with a height corresponding to the number of all points of interest in the middle layer. A quality scan has a histogram resembling the density of a normal distribution centered around the middle layer and non-zero numbers of points are only in a few layers around the middle. The flatter the histogram, the worse the scanning quality, a flat diagram usually indicates an oblique placement of the microscope slide in the scanner. If the maximum of the histogram is far from the middle layer, this indicates an inappropriately chosen focusing point.

[0079] The outer layers should not contain any points or only a negligible percentage. In samples with a larger amount of impurities, there may be a larger number of points in the outer layers; a mosaic tool is used to assess the source of the points of interest. The histogram is mainly an overview tool that quickly gives a first idea of the quality.

[0080] 2) Assessment of scanning quality using a filtered mosaic

[0081] A mosaic displays fixed-size sections from the highest resolution image centered at points of interest (these correspond to the particles under investigation, e.g. cells and impurities).

[0082] Only sections corresponding to points whose optimal layer corresponds to the selected layer or selected range of layers are displayed in the mosaic.

[0083] To assess the quality, points in the outer layers are first displayed; the resulting mosaic should not contain any particles of interest or only a negligible number and should be sharp (subjective feeling of an experienced observer). If the outer layers contain a significant amount of particles of interest, it is advisable to scan the specimen again.

[0084] If the outer layers are in order, i.e. they do not contain particles of interest or contain them only in a negligible amount, it is necessary to verify that particles are scanned in the inner layers. According to the histogram, we select the layer with the largest number of points of interest and verify whether it contains particles of interest. In the case of heavily contaminated specimens, it is necessary to go through multiple layers. If we do not find particles of interest in any layer, we assess the impurities to see if they correspond to a typical sample without particles or to impurities on the cover glass, which is a common scanning error.

[0085] 3) Assessment of scanning quality based on spatial distribution of points of interest

[0086] This tool displays a reduced view of the entire scanned part of the specimen and displays points of interest in it with markers whose color and shape correspond to the optimal layer / plane of focus of the given point. The selected image detail can be enlarged to see which particle the selected point corresponds to. For the detail, a slice from the layer that is optimal for the examined point of interest is automatically selected. The red color is reserved for the outer layers (planes of focus). The points should evenly cover the entire specimen and there should be very few red points and after enlargement they should correspond mainly to impurities. If the red dots correspond to poorly focused examined particles, the specimen must be rescanned and measures taken to eliminate the problem.

[0087] An exemplary embodiment of the method according to the invention was implemented in the form of an application for the field of cytology / cytopathology within the cAItologist software (originally called cITo- Viewer) developed by cITology s.r.o. This is an application serving for complete management of the conversion of a cytological specimen in conventional or immunocytological staining into digital form and for its subsequent viewing, storage and use for further analysis. The software also uses information about the optimal layer for each object (cell) stored in a file in the Sparse Z-Stack format for its optimal display and for further processing. Industrial Applicability

[0088] Practical applicability of this invention is quite broad - automatic segmentation and determination of the optimal plane of focus / optimal layer for viewing complex objects of interest in a multilayer microscopic image of a sample is potentially usable not only for cells in a model solution within medicine, or. cell research or diagnostics. The solution can be applied also in biological sciences in general, or in industry, e.g. for pollen grains, dust particles, etc., i.e. generally for any complex microscopic objects, where there is also a requirement for compression of the size of the primary image file, but without loss of image data for complex objects of interest. In general, the method according to the invention is usable for processing images scanned by a microscope from the fields of biology, medicine, cytology and pathology, pharmacology, botany, agronomy, industrial microscopy and others.

Claims

1. CLAIMS1. A computer-implemented method of processing digital image of a microscopic specimen scanned in at least two layers which differ mutually in position of their plane of focus, the method comprising the steps of3.a) for individual layers of the image, a pyramid of reduced versions of the initial image is formed, wherein base level of the pyramid is formed by the respective layer of the initial digital image, and individual levels of the pyramids are divided into tiles, or4.individual layers of the initial digital image are divided into tiles and a pyramid of reduced images is formed for any of the individual tiles, wherein base level of any pyramid is formed by the respective tile in the respective layer of the initial digital image,5.b) points of interest are detected in at least one layer,6.c) sharpness levels of the detected points of interest in individual layers are calculated, characterized in that it further comprises the steps of7.d) information is assigned to the detected points of interest, in which layer the respective point of interest has maximum sharpness,8.e) a sparse pyramid is formed containing only such tiles from the individual layers that contain at least N points of interest having said maximum sharpness, and tiles that are immediately adjacent to said tiles within the individual layer, N being equal to or greater than 1.

2. The method according to claim 1, characterized in that for each area that is defined by a tile and contains at least one point of interest detected in step b), an image of this tile is selected from a layer in which the highest number of detected points of interest with maximum sharpness is present, and optionally additionally for each area that is defined by a tile and in which no point of interest was detected in step c), an image of this tile is selected from a layerin which a point of interest that was detected closest to a center of this tile in step b) has maximum sharpness,10.and a synthetic layer is composed of the images thus selected.

3. The method according to any of claims 1 to 2, characterized in that in step a) the forming of reduced images comprises the application of an anti-aliasing filter with a Gaussian kernel, followed by direct subsampling.

4. The method according to any of claims 1 to 3, characterized in that in step a) pyramids of reduced images of tiles or layers are formed with a reduction step of 2:1 and / or in step a) the images of layers are divided into tiles of equal size, preferably the image is divided into square tiles of 2xby 2xpixels, wherein x is an integer, preferably in the range of 8-10 inclusive.

5. The method according to any of claims 1 to 4, characterized in that in step b)14.- the points of interest are detected by the stable wave detection method,15.and / or16.- the points of interest are detected in one or more reduced versions of the image of one of the layers in grayscale, preferably the level or levels of the pyramid are selected based on the size of particles to be detected17.and / or18.when processing a digital image containing at least three layers which differ mutually in position of their plane of focus, a pyramid from a layer that is not an outer layer is preferably selected for detection in step b), preferably a pyramid of a middle layer is selected.

6. The method according to any of claims 1 to 5, characterized in that after step b), the points of interest are filtered based on their detector response amplitude using non-maximalsuppression method, wherein point detected as having the largest amplitude is retained and the other points in its vicinity are removed.

7. The method according to any of claims 1 to 6, characterized in that in step c) a region of interest is defined, the region of interest having a center, which corresponds to a point of interest, after which the sharpness measures in the individual layers are calculated as the sums of brightness gradient over all pixels of the region of interest.

8. The method according to any of claims 1 to 7, characterized in that for evaluation of quality of the processed digital image, a graph is created showing how many points of interest have maximum sharpness in any of the individual layers.

9. The method according to any of claims 1 to 8, characterized in that the expected maximum size of the particles of interest to be viewed in the digital image is input, and for evaluation of quality of the digital image to be processed, a composite image is formed, which is composed of image slices of at least one of the outer layers, each of said slices having a center at the point of interest that has maximum sharpness in the respective outer layer, and wherein the slices have an area whose size corresponds to the size of the area of a square with a side of 1 to 3 times the expected maximum size of the particles of interest.

10. The method according to any of claims 1 to 9, characterized in that the expected maximum size of the particles of interest to be viewed in the digital image is input, and for evaluation of quality of the digital image to be processed, a composite image is formed, which is composed of image slices of the layer in which the most points of interest have maximum sharpness, each of said slices having a center at the point of interest that has maximum sharpness in the respective layer, and wherein the slices have an area whose size corresponds to an area of a square with a side of 1 to 3 times the expected maximum size of the particles of interest.

11. The method according to any of claims 1 to 8, characterized in that the expected maximum size of the particles of interest to be viewed in the digital image is entered, and for evaluation of quality of the digital image to be processed, a composite image is formed for each layer, the composite image being composed of image slices of the respective layer, each of said slices having a center at the point of interest that has maximum sharpness in the respective layer, and wherein the slices have an area whose size corresponds to an area of a square with a side of 1 to 3 times the expected maximum size of the particles of interest.

12. The method according to any one of claims 1 to 11, characterized in that for evaluation of quality of the digital image to be processed, a reduced preview of the entire digital image is displayed and the points of interest are displayed therein as signs whose color and / or shape identifies the layer in which the respective point of interest has maximum sharpness.

13. A computer program comprising instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 12.

14. A computer-readable medium comprising the computer program of claim 13.

Images