Method for compressing pyramid of high-resolution pathology slide images, method for restoring compressed pyramid of pathology slide images, and computing system for performing same

Abstract

Provided, according to an aspect of the present invention, is a method including the steps in which a computing system: acquires a pyramid of images corresponding to an original pathology slide image; generates an i-th level difference image for an arbitrary integer i, where 0<=i<K; and generates a compressed image pyramid including a 0th level difference image to a (K-1)th level difference image and a K-th level image to a (L-1)th level image in the image pyramid, wherein the step of generating the i-th level difference image includes a step of generating an i-th level approximate image having the same size as the i-th level image in the image pyramid on the basis of the K-th level image in the image pyramid, and a step of generating the i-th level difference image on the basis of value difference in each pixel of the i-th level image and the i-th level approximate image.

Description

A method for compressing a high-resolution pathology slide image pyramid, a method for restoring a compressed pathology slide image pyramid, and a computing system for performing the same.

[0001] The present invention relates to a method for compressing a high-resolution pathology slide image pyramid, a method for restoring a compressed pathology slide image pyramid, and a computing system for performing the same.

[0002]

[0003] In the medical field, pathological diagnosis, which involves interpreting actual specimens, is crucial for accurately diagnosing and predicting the prognosis of serious diseases. Conventional pathological diagnosis methods involve visual observation and interpretation of diagnostic pathology slides produced from specimens through an optical microscope by a pathologist. The dawn of digital pathology can be attributed to the use of a computer-connected microscope camera to convert pathology slides into digital images, which are then viewed and interpreted on a monitor.

[0004] Recently, digital slide scanners have become widespread, converting entire pathology slides into a single digital image, which can then be viewed and interpreted on a computer monitor. Digital pathology, a diagnostic method based on digital pathology that scans images using these slide scanners and then interprets them on a monitor, is replacing the traditional method of visually examining specimen slides through an optical microscope.

[0005] The whole slide image generated by a slide scanner needs to be scanned at a magnification of 200x or more (usually 400x) for an accurate diagnosis, and the number of pixels in the image generated in this way reaches 2.4 billion when a 2 cm x 3 cm area is scanned 200x, and 9.6 billion when scanned 400x, and the file size also reaches hundreds of MB to several GB. In the case of large medical institutions, it is known that the whole slide images generated in this way range from hundreds of TB to several PB per year, and the problem of the operating cost of storage devices / servers to store and preserve them increasing every year needs to be solved.

[0006] Pathology slide images, which are either full slide images or portions of full slide images, typically have a pyramid-shaped structure that combines images at a certain ratio with the scanned original magnification image. This is a method to support rapid image enlargement / reduction. Figure 1 illustrates an example of a pyramid structure of a pathology slide image.

[0007] Since a reduced image contains almost the same information as an enlarged image, it can be seen that a full slide image has a considerable amount of information duplication. Therefore, by effectively compressing the full slide image by taking this into account, the size of the full slide image file that needs to be stored can be reduced, thereby reducing the storage device / server operating costs of medical institutions that have introduced digital pathology.

[0008]

[0009] The technical problem to be achieved by the present invention is to provide a technical idea that enables the compressing and storing of a pyramid structure of a very large pathology slide image, thereby reducing the operating costs of storage devices / servers of medical institutions that have introduced digital pathology.

[0010]

[0011] According to one aspect of the present invention, a computing system comprises a step of obtaining an image pyramid corresponding to an original pathology slide image, wherein the image pyramid includes a 0th level image to an (L-1)th level image (L is a predefined integer greater than or equal to 2), the 0th level image of the image pyramid is the original pathology slide image, and for each integer m satisfying 1<=m<=L-1, the mth level image of the image pyramid is a reduced image of the (m-1)th level image of the image pyramid; wherein the computing system, 0<=i <K인 임의의 정수 i에 대하여, 제i레벨 차이 이미지를 생성하는 단계-여기서, 압축 기준 레벨 K는, 0<K<L을 만족하는 소정의 정수임; 및 상기 컴퓨팅 시스템이, 제0레벨 차이 이미지 내지 제(K-1)레벨 차이 이미지 및 상기 이미지 피라미드의 제K레벨 이미지 내지 제(L-1)레벨 이미지를 포함하는 압축된 이미지 피라미드를 생성하는 단계를 포함하고, 상기 제i레벨 차이 이미지를 생성하는 단계는, 상기 이미지 피라미드의 제K레벨 이미지에 기초하여 상기 이미지 피라미드의 제i레벨 이미지와 동일한 크기의 제i레벨 근사 이미지를 생성하는 단계; 및 상기 제i레벨 이미지와 상기 제i레벨 근사 이미지의 각 픽셀 단위 차이 값에 기초하여 상기 제i레벨 차이 이미지를 생성하는 단계를 포함하는 방법이 제공된다.

[0012] In one embodiment, the step of generating an i-th level approximate image having the same size as an i-th level image of the image pyramid based on the K-th level image of the image pyramid may include the step of applying bilinear interpolation or bicubic interpolation to the K-th level image of the image pyramid to enlarge it by the size of the i-th level image of the image pyramid.

[0013] In one embodiment, the step of generating an i-th level approximate image having the same size as the i-th level image of the image pyramid based on the K-th level image of the image pyramid may further include the step of inputting the K-th level image of the image pyramid, enlarged by the size of the i-th level image of the image pyramid, into an artificial neural network for image noise removal and sharpening that has been trained in advance to generate the i-th level approximate image.

[0014] In one embodiment, the step of generating an i-th level approximate image having the same size as an i-th level image of the image pyramid based on a K-th level image of the image pyramid may include the step of inputting the K-th level image, or an extended image obtained by extending the K-th level image by p pixels (wherein p is an integer greater than or equal to 1) upward, downward, left, and rightward, into a pre-trained super-resolution neural network to generate the i-th level approximate image, wherein p is a value less than or equal to Z determined by the following [Formula 1].

[0015] [Formula 1]

[0016]

[0017] (Here, C is a set containing all convolutional layers included in the super-resolution neural network, and fi is the filter size of the ith convolutional layer included in the super-resolution neural network.)

[0018] In one embodiment, the computing system, 0<=i <K인 임의의 정수 i에 대하여, 제i레벨 차이 이미지를 생성하는 단계는, 정수 i에 K-1부터 0까지 순차적으로 할당하면서 상기 제i레벨 차이 이미지를 생성하는 단계를 포함하되, 상기 이미지 피라미드의 제K레벨 이미지에 기초하여 상기 이미지 피라미드의 제i레벨 이미지와 동일한 크기의 제i레벨 근사 이미지를 생성하는 단계는, i = K-1인 경우, 상기 이미지 피라미드의 제K레벨 이미지에 기초하여 제i레벨 근사 이미지를 생성하는 단계; 및 i < K-1인 경우, 제(i+1)레벨 근사 이미지에 기초하여 제i레벨 근사 이미지를 생성하는 단계를 포함할 수 있다.

[0019] In one embodiment, the computing system, 0<=i <K인 임의의 정수 i에 대하여, 제i레벨 차이 이미지를 생성하는 단계는, 정수 i에 K-1부터 0까지 순차적으로 할당하면서 상기 제i레벨 차이 이미지를 생성하는 단계를 포함하되, 상기 이미지 피라미드의 제K레벨 이미지에 기초하여 상기 이미지 피라미드의 제i레벨 이미지와 동일한 크기의 제i레벨 근사 이미지를 생성하는 단계는, i = K-1인 경우, 상기 이미지 피라미드의 제K레벨 이미지에 기초하여 제i레벨 근사 이미지를 생성하는 단계; 및 i < K-1인 경우, 제(i+1)레벨 근사 이미지에 제(i+1)레벨 차이 이미지를 반영하여 상기 이미지 피라미드의 제(i+1)레벨 이미지를 복원하고, 복원된 제(i+1)레벨 이미지에 기초하여 제i레벨 근사 이미지를 생성하는 단계를 포함할 수 있다.

[0020] In one embodiment, the computing system, 0<=j <K인 임의의 정수 j에 대하여, 제j레벨 차이 이미지로부터 제j레벨 이미지를 복원하는 단계를 더 포함하되, 제j레벨 차이 이미지로부터 제j레벨 이미지를 복원하는 단계는, 상기 원본 이미지 피라미드의 제K레벨 이미지에 기초하여 상기 원본 이미지 피라미드의 제j레벨 이미지와 동일한 크기의 제j레벨 근사 이미지를 생성하는 단계; 상기 제j레벨 차이 이미지에 기초하여 상기 제j레벨 이미지와 상기 제j레벨 근사 이미지의 각 픽셀 단위 차이 값을 산출하는 단계; 및

[0021] It may include a step of restoring the j-level image by adding the pixel-wise difference values ​​of the j-level image and the j-level approximate image with the j-level approximate image.

[0022] In one embodiment, the step of generating the i-th level difference image based on each pixel-wise difference value between the i-th level image and the i-th level approximate image may include the step of adding 255 to each pixel-wise difference value between the i-th level image and the i-th level approximate image and dividing the sum by 2 to generate the i-th level difference image corresponding to the i-th level image, and the step of calculating the pixel-wise difference value between the j-th level image and the j-th level approximate image based on the j-th level difference image may include the step of calculating the pixel-wise difference value between the j-th level image and the j-th level approximate image by multiplying each pixel of the j-th level difference image by 2 and then subtracting 255.

[0023] In one embodiment, the step of generating the i-th level difference image based on each pixel-wise difference value between the i-th level image and the i-th level approximate image comprises: generating a first temporary image corresponding to the i-th level image by adding 255 to each pixel-wise difference value between the i-th level image and the i-th level approximate image; generating an i-th level parity image having the same size as the first temporary image, wherein each pixel of the i-th level parity image is assigned 0 if the corresponding pixel of the i-th level approximate image is even, and 1 otherwise; storing the i-th level parity image; And the step of dividing the pixel-wise difference value of the i-th level temporary image and the i-th level parity image by 2 to generate an i-th level difference image corresponding to the i-th level image, and the step of calculating the pixel-wise difference value of the j-th level image and the j-th level approximate image based on the j-th level difference image may include the step of multiplying each pixel of the j-th level difference image by 2 to generate a second temporary image corresponding to the j-th level difference image; and the step of calculating the pixel-wise difference value of the j-th level image and the j-th level approximate image by subtracting 255 from each pixel-wise sum value of the second temporary image and the j-th level parity.

[0024] According to another aspect of the present invention, there is provided a method for restoring an original image pyramid from a compressed image pyramid generated by the above-described method, comprising the steps of: obtaining a compressed image pyramid including a 0th level difference image to a (K-1)th level difference image and a Kth level image to a (L-1)th level image of the original image pyramid; and wherein the computing system, 0<=j <K인 임의의 정수 j에 대하여, 제j레벨 차이 이미지로부터 제j레벨 이미지를 복원하는 단계를 포함하되, 제j레벨 차이 이미지로부터 제j레벨 이미지를 복원하는 단계는, 상기 원본 이미지 피라미드의 제K레벨 이미지에 기초하여 상기 원본 이미지 피라미드의 제j레벨 이미지와 동일한 크기의 제j레벨 근사 이미지를 생성하는 단계; 상기 제j레벨 차이 이미지에 기초하여 상기 제j레벨 이미지와 상기 제j레벨 근사 이미지의 각 픽셀 단위 차이 값을 산출하는 단계; 및 상기 제j레벨 이미지와 상기 제j레벨 근사 이미지의 각 픽셀 단위 차이 값을 상기 제j레벨 근사 이미지와 픽셀 단위로 합산하여 제j레벨 이미지를 복원하는 단계를 포함하는 방법이 제공된다.

[0025] According to another aspect of the present invention, a computer program is provided, which is installed in a data processing device and recorded on a recording medium for performing the above-described method.

[0026] According to another aspect of the present invention, a computer-readable recording medium having recorded thereon a computer program for performing the above-described method is provided.

[0027] According to another aspect of the present invention, there is provided a computing system, comprising a processor and a memory, wherein the memory, when executed by the processor, causes the computing system to perform a method for compressing a pathology slide image pyramid, wherein the method for compressing a pathology slide image pyramid comprises: a step in which the computing system obtains an image pyramid corresponding to an original pathology slide image, wherein the image pyramid includes a 0th level image to an (L-1)th level image (L is a predefined integer greater than or equal to 2), the 0th level image of the image pyramid is the original pathology slide image, and for each integer m satisfying 1<=m<=L-1, the mth level image of the image pyramid is a reduced image of the (m-1)th level image of the image pyramid; wherein the computing system, 0<=i <K인 임의의 정수 i에 대하여, 제i레벨 차이 이미지를 생성하는 단계-여기서, 압축 기준 레벨 K는, 0<K<L을 만족하는 소정의 정수임; 및 상기 컴퓨팅 시스템이, 제0레벨 차이 이미지 내지 제(K-1)레벨 차이 이미지 및 상기 이미지 피라미드의 제K레벨 이미지 내지 제(L-1)레벨 이미지를 포함하는 압축된 이미지 피라미드를 생성하는 단계를 포함하고, 상기 제i레벨 차이 이미지를 생성하는 단계는, 상기 이미지 피라미드의 제K레벨 이미지에 기초하여 상기 이미지 피라미드의 제i레벨 이미지와 동일한 크기의 제i레벨 근사 이미지를 생성하는 단계; 및 상기 제i레벨 이미지와 상기 제i레벨 근사 이미지의 각 픽셀 단위 차이 값에 기초하여 상기 제i레벨 차이 이미지를 생성하는 단계를 포함하는 컴퓨팅 시스템이 제공된다.

[0028] According to another aspect of the present invention, there is provided a computing system, comprising a processor and a memory, wherein the memory, when executed by the processor, causes the computing system to perform a method for restoring an original image pyramid from a compressed image pyramid generated by the above-described method, wherein the method for restoring the original image pyramid comprises the steps of: obtaining a compressed image pyramid including a 0th level difference image to a (K-1)th level difference image and a Kth level image to a (L-1)th level image of the image pyramid; wherein the computing system, 0<=j <K인 임의의 정수 j에 대하여, 제j레벨 차이 이미지로부터 제j레벨 이미지를 복원하는 단계를 포함하되, 제j레벨 차이 이미지로부터 제j레벨 이미지를 복원하는 단계는, 상기 원본 이미지 피라미드의 제K레벨 이미지에 기초하여 상기 원본 이미지 피라미드의 제j레벨 이미지와 동일한 크기의 제j레벨 근사 이미지를 생성하는 단계; 상기 제j레벨 차이 이미지에 기초하여 상기 제j레벨 이미지와 상기 제j레벨 근사 이미지의 각 픽셀 단위 차이 값을 산출하는 단계; 상기 제j레벨 이미지와 상기 제j레벨 근사 이미지의 각 픽셀 단위 차이 값을 상기 제j레벨 근사 이미지와 픽셀 단위로 합산하여 제j레벨 이미지를 복원하는 단계를 포함하는 컴퓨팅 시스템이 제공된다.

[0029]

[0030] According to one embodiment of the present invention, a technical idea can be provided that enables the compressing and storing of a pyramid structure of a very large pathology slide image, thereby reducing the operating costs of storage devices / servers of medical institutions that have introduced digital pathology.

[0031]

[0032] In order to more fully understand the drawings cited in the detailed description of the present invention, a brief description of each drawing is provided.

[0033] An example of the pyramid structure of a pathology slide image is shown in Fig. 1.

[0034] FIG. 2 is a drawing showing an example of various images generated during the compression and restoration process of an image pyramid according to the technical idea of ​​the present invention.

[0035] FIG. 3 is a diagram illustrating an example of pseudo code for implementing an image pyramid compression and restoration method according to one embodiment of the present invention.

[0036] FIG. 4 is a diagram schematically illustrating the physical configuration of a computing system that performs an image pyramid compression and restoration method according to one embodiment of the present invention.

[0037] FIG. 5 is a flowchart illustrating an image pyramid compression method according to one embodiment of the present invention.

[0038] Figure 6 is a drawing illustrating step S140 of Figure 5 in more detail.

[0039] FIG. 7a and FIG. 7b are drawings each illustrating an example of step S141 of FIG. 6 in more detail.

[0040] FIG. 8a and FIG. 8b are drawings each illustrating an example of step S143 of FIG. 6 in more detail.

[0041] Figure 9 is a diagram showing the structure of an image pyramid before and after compression.

[0042] FIG. 10 is a flowchart illustrating an image pyramid restoration method according to one embodiment of the present invention.

[0043] FIG. 11a and FIG. 11b are drawings each illustrating an example of step S212 of FIG. 9 in more detail.

[0044]

[0045] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. In describing the present invention, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the gist of the present invention.

[0046] Terms such as first, second, etc. may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0047] The terminology used in this application is solely for the purpose of describing specific embodiments and is not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0048] In this specification, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0049] Additionally, in the present specification, when a component "transmits" data to another component, it means that the component may transmit the data directly to the other component, or may transmit the data to the other component via at least one other component. Conversely, when a component "directly transmits" data to another component, it means that the data is transmitted from the component to the other component without going through the other component.

[0050] Hereinafter, the present invention will be described in detail, focusing on embodiments thereof, with reference to the attached drawings. The same reference numerals in each drawing represent the same components.

[0051] Meanwhile, in this specification, the symbol '=' may be used interchangeably as a logical operator indicating equal and an operator indicating assignment, but a person having ordinary skill in the field to which the present invention pertains will be able to clearly distinguish which operator is being used depending on the context.

[0052] The image pyramid compression method according to the technical idea of ​​the present invention may be a compression method for compressing an image pyramid corresponding to a predetermined pathological slide image, and the image pyramid restoration method according to the technical idea of ​​the present invention may be a restoration method for decompressing an image pyramid compressed by the compression method.

[0053] A pathology slide image can be an image of a slide of a pathology specimen tissue stained using a predetermined staining method, or a portion thereof. The pathology specimen tissue can be a biopsy taken from various organs of the human body or a surgically excised living tissue. Meanwhile, the pathology slide image can be a digital slide image of a stained pathology specimen, or a portion thereof. The pathology specimen slide can be a sliced ​​portion of the pathology specimen. A digital slide image of a pathology specimen can be created by slicing a pathology specimen to create a glass slide, staining the slide with a predetermined staining agent, and digitizing the glass slide.

[0054] The above pathology slide image may be a full slide image obtained by digitally scanning the entire slide of the pathology specimen tissue, or in some cases, may be a tile or patch, which is a portion of the full slide image divided into grids of the same size.

[0055] In this specification, an image pyramid corresponding to a pathology slide image can be defined as follows.

[0056] An image pyramid corresponding to a given pathology slide image includes L level-specific images from level 0 to level L-1, where L may be a predetermined integer greater than or equal to 2. In other words, the image pyramid may include a level 0 image to a level (L-1) image. Hereinafter, the i-th level image included in the image pyramid will be denoted by I(i).

[0057] The 0th level image I(0) of the image pyramid corresponding to the above pathology slide image is the above pathology slide image, and for each integer m satisfying 1<=m<=L-1, the mth level image of the image pyramid is a reduced image of the (m-1)th level image of the image pyramid.

[0058] Hereinafter, the reduction ratio of the level 0 image I(0), that is, the level i image I(i) based on the pathology slide image, is denoted as D(i).

[0059] Preferably D(i) = 2 i This is usually the case, but of course, in some embodiments, a different value may be used.

[0060] If i < j, then D(i) < D(j), and D(i) = 2 i If D(j) = D(i)·2 k (k=ji). For example, D(i) = 2 i In this case, the 0th level image I(0) is the above pathology slide image, the 1st level image I(1) is an image in which the width and height of the 0th level image I(0) are each reduced by half, the mth level image I(m) is an image in which the width and height of the (m-1)th level image I(m-1) are each reduced by half, and the (L-1)th level image I(L-1) is an image in which the width and height of the (L-2)th level image I(L-2) are each reduced by half.

[0061] D(i) = 2 i , and an example with L = 4 is shown in Fig. 1. The image I(0) at the bottom of the image pyramid structure of Fig. 1 is the original pathology slide image.

[0062] Meanwhile, in order to more clearly understand the technical idea of ​​the present invention to be described later, various operations targeting images are defined as follows.

[0063] * SHRINK(I, k): An operation that reduces image I by a ratio of k (i.e., the width and height are each 1 / k).

[0064] * EXPAND(I, k): An operation that enlarges image I by a ratio of k (i.e., the width and height are each k times larger).

[0065] * DIFF(X, Y): An operation to obtain the pixel-by-pixel difference image between two images X and Y.

[0066] * ADD(X, Y): An operation to obtain the pixel-wise sum image of two images X and Y.

[0067] At this time, the i-th level image I(i) included in the image pyramid has the following relationship with the 0-th level image I(0).

[0068] I(i) = SHRINK(I(0), D(i))

[0069] I(0)≒ EXPAND(I(i), D(i))

[0070] Meanwhile, an image pyramid (image pyramid before compression) corresponding to a given pathology slide image to be compressed is created. in = {I(i) | 0 ≤ i < L}, then the compressed image pyramid I out can be defined as follows:

[0071] I out = {I'(i) | 0 ≤ i < K} ∪ {I(i) | K ≤ i < L}, 0 < K < L

[0072] I'(i) = DIFF(I(i), SHRINK -1 (I(K), D(K) / D(i)))

[0073] Here, K is a compression base level, which is a predetermined integer satisfying 0 < K < L.

[0074] Meanwhile, the image estimation operation SHRINK -1 (I, v) is an operation that estimates an image O that satisfies I = SHRINK(O, v) for an input image I, and an example of this operation is EXPAND(I, v).

[0075] The image pyramid compression method according to the technical idea of ​​the present invention stores images from the most reduced level image within the image pyramid of the image pyramid to be compressed to the compression reference level (i.e., from the (L-1)-th level image to the K-th level image) as they are, but stores only compressed information for images below the compression reference level. That is, the image pyramid compression method according to the technical idea of ​​the present invention estimates images for each level from level 0 to K-1 by utilizing the K-th level image (K is the compression reference level mentioned above), and stores a difference image between the estimation result and the original image for each level.

[0076] Meanwhile, the image pyramid restoration method according to the technical idea of ​​the present invention uses the following method during compression restoration.

[0077] I recovered = {I''(i) | 0 ≤ i < K} ∪ {I(i) | K ≤ i < L}

[0078] I''(i) = ADD(I'(i), SHRINK -1 (I(K), D(K) / D(i)))

[0079] That is, in the image pyramid restoration method according to the technical idea of ​​the present invention, the original image can be restored by estimating an image for each level from level 0 to K-1 and adding the difference image of the corresponding level to the estimation result.

[0080] As mentioned earlier, the image estimation operation SHRINK -1() As an example, the image enlargement operation EXPAND() can be used. Examples of image enlargement operations include image enlargement operations using bilinear or bicubic interpolation. In addition, by enlarging the image closer to the original image based on deep learning, which has been widely studied recently, the amount of information remaining in the difference image is reduced, so the size can be reduced during compression. This will be described later.

[0081] Meanwhile, in the case of an image in which each RGB (Red, Green, Blue) channel represents a value in the range of 0 to 255, corresponding to 1 byte per pixel, the difference between the two images is theoretically represented as a value in the range of -255 to 255 per pixel. In order to save this in a general image compression format, the range of values ​​needs to be converted to 0 to 255. There are several ways to do this, and one example is to add 255 to each pixel and then divide by 2. When creating a sum image using the difference image obtained in this way, the process of converting the values ​​of each pixel of the difference image is reversed (for example, multiply the value of each pixel by 2 and then subtract 255), and then the values ​​of each pixel of the difference image are converted and then combined with the summation target image pixel by pixel.

[0082] Figure 2 is a diagram illustrating an example of various images generated during the compression and restoration process of an image pyramid according to the technical idea of ​​the present invention. In the example of Figure 2, D(i) = 2 i , and illustrates the case where the compression criterion level K is 2.

[0083] The image I0 illustrated in Fig. 2(a) is an example of a level 0 image (i.e., an original pathology slide image) I(0), and the image I1 illustrated in Fig. 2(b) is an example of an approximate image obtained by enlarging the K-level image (i.e., a level 2 image) to the size of the level 0 image. That is, I1 = SHRINK -1 (I(2), 4). The image shown in Fig. 2(c) is the difference image I'(0) = DIFF(I0, I1) of I0 and I1, and the image shown in Fig. 2(d) shows the result of restoring the 0th level image using the difference image I'(0) and the approximate image I1, i.e., I''(0) = ADD(I'(0), I1).

[0084] Common image formats such as JPEG and PNG can be used as a format for storing the difference image. In Fig. 2, the original image I(0) is 14,599 bytes in size based on JPEG, and the difference image I'(0) is 8,352 bytes in size based on JPEG, which can achieve a compression effect of about 43% compared to the original. However, if the image format used to store the difference image uses a lossy compression algorithm, the stored image cannot be considered to be completely identical to the original image on a pixel-by-pixel basis. Accordingly, when reading the difference image stored as JPEG, etc. and restoring the original image using the method described above, the value for each pixel may fall outside the range of 0 to 255. Therefore, in actual implementation, the values ​​for each pixel must be added, and values ​​less than 0 must be set to 0, and values ​​greater than 255 must be set to 255 to restore an image close to the original.

[0085] FIG. 3 is a diagram illustrating an example of pseudo code for implementing an image pyramid compression and restoration method according to one embodiment of the present invention.

[0086] In the example of Fig. 3, I(i) represents the i-th level image of the image pyramid to be compressed, I'(i) = the difference image of the i-th level, I''(i) = the restored image of the i-th level, D(i) = the image reduction ratio of the i-th level image compared to the 0-th level image, L = the number of image pyramid levels, and K = the compression reference level.

[0087] An image pyramid compression method for compressing a pathology slide image pyramid according to the technical idea of ​​the present invention and / or an image pyramid restoration method for compressing a compressed pathology slide image pyramid can be performed by a computing system. FIG. 4 is a diagram schematically illustrating the physical configuration of a computing system (100) that performs the same.

[0088] The computing system (100) may be a data processing device having a computing capability for implementing the technical idea of ​​the present invention, and may generally include a server, which is a data processing device that can be accessed by a client through a network, as well as a computing device such as a personal computer or a mobile terminal.

[0089] The above computing system (100) may be equipped with a memory (102) in which a program for implementing the technical idea of ​​the present invention is stored, and a processor (101) for executing the program stored in the memory (102).

[0090] Depending on the embodiment of the present invention, some of the components described above may not necessarily correspond to components essential to the implementation of the present invention, and, depending on the embodiment, the computing system (100) may include more components than this.

[0091] An average expert in the technical field of the present invention will be able to easily infer that the processor (101) may be named by various names such as CPU, APU, mobile processor, etc., depending on the implementation example of the computing system (100). The processor may include a single-core CPU or a multi-core CPU. In addition, the computing system (100) may be implemented by organically combining a plurality of physical devices, and in this case, an average expert in the technical field of the present invention will be able to easily infer that the computing system (100) of the present invention may be implemented by providing at least one processor (101) for each physical device.

[0092] The above memory (102) stores the program and may be implemented as any type of storage device that the processor can access to run the program. The memory (102) may include a high-speed random access memory and may also include non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to the memory by the processor and other components may be controlled by a memory controller. In addition, depending on the hardware implementation example, the memory (102) may be implemented as a plurality of storage devices rather than a single storage device. In addition, the memory (102) may include a temporary storage device as well as a main memory device. In addition, it may be implemented as a volatile memory or a non-volatile memory, and may be defined to mean all types of information storage means that are implemented so that the program can be stored and run by the processor.

[0093] In one embodiment, the memory (102) may include a super-resolution neural network (103) used to restore a high-resolution image from a given low-resolution image, which will be described later.

[0094] The computing system (100) may further include a storage device (104) for storing various data, including an image pyramid or a compressed image pyramid, and the processor (101) may read data stored in the storage device (104) or write data to the storage device (104) through a predetermined communication interface (e.g., a bus).

[0095] In addition, the processor (101) can control various peripheral devices (105) provided to the computing device (100). The peripheral devices (105) include various data input / output devices including a keyboard, a mouse, and a display device, and may also include any device that can be connected to the computing system (100) through a predetermined connection interface.

[0096] Meanwhile, in one embodiment, the computing system (100) may be in the form of a subsystem of a predetermined mother system, unlike that illustrated in FIG. 4, and the mother system may be a server that performs functions such as performing digital image processing on pathology slide images.

[0097] Additionally, according to one embodiment, the computing system (100) may be composed of a plurality of subsystems that are connected or not connected via a network, unlike those illustrated in FIG. 4. For example, the computing system (100) may include a compression system that compresses an image pyramid and a restoration system that restores the compressed image pyramid. Each of the compression system and the restoration system may be independent computing devices, and in some cases, may be connected via a network, but this is not necessarily the case.

[0098] FIG. 5 is a flowchart illustrating an image pyramid compression method (S100) according to one embodiment of the present invention.

[0099] Referring to FIG. 5, the computing system (100) can obtain an image pyramid corresponding to an original pathology slide image (S110). The computing system (100) may obtain the image pyramid previously stored in a storage device (104), or may obtain the image pyramid from a predetermined external device (not shown) connected to the computing system (100) via a network.

[0100] The image pyramid may include L level-specific images, i.e., a 0th level image to an (L-1)th level image (L is a predefined integer greater than or equal to 2). At this time, the 0th level image of the image pyramid is the original full slide image, and for each integer m satisfying 1<=m<=L-1, the mth level image of the image pyramid is an image obtained by reducing the (m-1)th level image of the image pyramid by a predetermined magnification. For example, the mth level image of the image pyramid may be an image obtained by reducing the (m-1)th level image of the image pyramid by a magnification of 1 / 2.

[0101] The computing system (100) can determine a predetermined compression criterion level K (S120). The compression criterion level K may be an integer satisfying 0 < K < L, which is determined in advance through experimentation or other methods.

[0102] In one embodiment, the computing system (100) may perform the image pyramid compression method according to one embodiment of the present invention for each integer satisfying 0 < K < L to select K having the highest compression ratio, i.e., K that makes the size of the compressed image pyramid the smallest. Alternatively, the above method may be performed on each of the test image pyramids prepared in advance and K having the highest average compression ratio may be selected in advance.

[0103] The above computing system (100) is 0<=i <K인 임의의 정수 i에 대하여, 제i레벨 차이 이미지 I'(i)를 생성할 수 있다(S130, S140). 바람직하게는 상기 컴퓨팅 시스템(100)은 정수 i에 K-1부터 0까지 순차적으로 할당하면서 상기 제i레벨 차이 이미지를 생성할 수 있다(S130, S140).

[0104] Figure 6 is a drawing illustrating step S140 of Figure 5 in more detail.

[0105] Referring to FIG. 6, the computing system (100) can generate an i-th level approximate image P(i) having the same size as the i-th level image I(i) of the image pyramid based on the K-th level image I(K) of the image pyramid (S141).

[0106] For example, the computing system (100) can generate an i-th level approximate image P(i) by enlarging the K-th level image I(K) of the image pyramid by the size of the i-th level image I(i) of the image pyramid, which can be expressed by the following operation.

[0107] * P(i) = SHRINK -1 (I(K), D(K) / D(i))

[0108] In one embodiment, the computing system (100) can apply bilinear interpolation or bicubic interpolation to the K-th level image I(K) of the image pyramid to enlarge it to the size of the i-th level image of the image pyramid.

[0109] In another embodiment, the computing system (100) may also enlarge the K-th level image I(K) of the image pyramid using a pre-trained super-resolution neural network (103). The super-resolution neural network (103) is an artificial neural network for converting an input original image into a high-resolution image, and is a neural network that is pre-trained before the image pyramid compression method according to the technical idea of ​​the present invention is performed.

[0110] In this specification, a neural network includes a multilayer perceptron model and may mean a set of information representing a series of design specifications defining an artificial neural network.

[0111] In one embodiment, the super-resolution neural network may be a convolutional neural network. As is well known, a convolutional neural network may include an input layer, multiple hidden layers, and an output layer.

[0112] Each of the multiple hidden layers may include a convolutional layer and a pooling layer (or subsampling layer). A convolutional neural network may be defined by functions, filters, strides, weight factors, etc. that define each of these layers. Furthermore, the output layer may be defined as a fully connected feedforward layer.

[0113] The design specifications for each layer that constitutes a convolutional neural network are widely known. For example, the number of layers to be included in a plurality of layers, the convolution function for defining the plurality of layers, the pooling function, and the activation function, each of which may be known functions, or separately defined functions may be used to implement the technical idea of ​​the present invention. An example of a convolutional function is a discrete convolution sum. An example of a pooling function may be max pooling, average pooling, etc. An example of an activation function may be a sigmoid, a tangent hyperbolic (tanh), a rectified linear unit (ReLU), etc. Once the design specifications of such a convolutional neural network are defined, the convolutional neural network with the defined design specifications may be stored in a storage device.

[0114] And when the above convolutional neural network is trained, the weight factors corresponding to each layer can be specified. In other words, training the convolutional neural network can mean the process of determining the weight factors of each layer. And when the convolutional neural network is trained, the trained convolutional neural network can receive input data in the input layer and output output data through the predefined output layer.

[0115] A neural network according to an embodiment of the present invention may be defined by selecting one or more of the widely known design features as described above, or an independent design feature may be defined for the neural network.

[0116] Meanwhile, pathology slide images are images in which pixel clusters with the same or very similar colors or brightnesses are combined and contrasted with each other, rather than images in which the colors or brightnesses change gradually. In other words, they have a characteristic of having sharper boundaries compared to other images. Considering this characteristic, a deep learning model that performs super-resolution for pathology slide images (i.e., the super-resolution neural network) must be trained in a direction that benefits the sharpness of the image, and therefore, a deep learning model that performs super-resolution (i.e., the super-resolution neural network) is trained in advance by utilizing a specific loss that benefits the sharpness.

[0117] In one embodiment, the computing system (100) may input the K-th level image I(K) to be enlarged as it is into the super-resolution neural network, but in another embodiment, the computing system (100) may input an expanded region, which is an expanded region that is expanded by p pixels (where, p is ) above, below, left, and right of the boundary of the K-th level image I(K) to be enlarged, into the super-resolution neural network, so as to obtain an approximate image corresponding to the K-th level image I(K). At this time, p is an integer greater than or equal to 1, and is a value determined by reflecting the characteristics of the super-resolution neural network. The computing system (100) may obtain the expanded region from an image of a lower level (i.e., from level 0 to level K-1) than the K-th level image I(K), and may include information about the expanded region in a compressed image pyramid to be generated later.

[0118] Meanwhile, the parameter p may be a value determined based on the structural characteristics of the convolutional layer included in the super-resolution neural network. In particular, p may be a Z value determined by the following [Formula 1].

[0119] [Formula 1]

[0120]

[0121] Here, C is a set containing all convolutional layers included in the super-resolution neural network, and f i is the filter size of the ith convolutional layer included in the above super-resolution neural network.

[0122] The above p may be a value less than or equal to Z determined by the above [Formula 1].

[0123] The computing system (100) can generate the K-level image of the image pyramid enlarged by the interpolation method described above or a method using a super-resolution neural network as an i-level approximate image, but according to an embodiment, the computing system (100) can also generate the i-level approximate image by inputting the K-level image of the image pyramid enlarged by the size of the i-level image of the image pyramid by the interpolation method described above or a separate artificial neural network for image noise removal and sharpening that has been learned in advance.

[0124] Alternatively, in another embodiment, the computing system (100) may use a sequential enlargement method that sequentially enlarges the K-th level image I(K) of the image pyramid by one level until it becomes the size of the i-th level image I(i).

[0125] An example of a sequential enlargement method is a method of enlarging the approximate image of the previous step by one level to create another approximate image. For example, when the compression reference level K is 4, the computing system (100) creates a third-level approximate image from a fourth-level image, creates a second-level approximate image from the third-level approximate image, creates a first-level approximate image from the second-level approximate image, and creates a zero-level approximate image from the first-level approximate image. A specific process of step S141 of FIG. 6, which applies this method, is illustrated in FIG. 7A.

[0126] Referring to FIG. 7a, the computing system (100) can determine whether i is less than K-1 or not (i.e., whether i is equal to K-1) (S1411), and if i is equal to K-1, the computing system (100) can generate an i-th level approximate image having the same size as the i-th level image of the image pyramid based on the K-th level image of the image pyramid (S1412).

[0127] On the other hand, if i is less than K-1, the computing system (100) can generate an i-th level approximate image based on the (i+1)-th level approximate image (S1413).

[0128] In the process of generating an i-th level approximation image based on the K-th level image or the (i+1)-th level approximation image, the interpolation method described above or a method using a super-resolution neural network can be used.

[0129] Another example of a sequential magnification method is a method of restoring an original image from an approximate image of a previous level and then magnifying the restored image by one level to create an approximate image. For example, when the compression criterion level K is 3, the computing system can create a second-level approximate image from a third-level image. Thereafter, the second-level difference image can be reflected on the second-level approximate image to restore the second-level image, and then a first-level approximate image can be created from the restored second-level image. Thereafter, the first-level difference image can be reflected on the first-level approximate image to restore the first-level image, and then a zero-level approximate image can be created from the restored first-level image.

[0130] The specific process of step S141 of FIG. 6 applying this method is illustrated in FIG. 7b.

[0131] Referring to FIG. 7b, the computing system (100) can determine whether i is less than K-1 or not (i.e., whether i is equal to K-1) (S1414), and if i is equal to K-1, the computing system (100) can enlarge the K-th level image of the image pyramid to generate an i-th level approximate image of the same size as the i-th level image of the image pyramid (S1415).

[0132] On the other hand, if i is less than K-1, the computing system (100) can generate an i-th level approximation image by reflecting the (i+1)-th level difference image on the (i+1)-th level approximation image (S1416). At this time, the (i+1)-th level difference image may have been generated in advance through step S150 of FIG. 5.

[0133] Thereafter, the computing system (100) can enlarge the restored (i+1) level image to generate an i-th level approximate image (S1417).

[0134] Referring again to FIG. 6, the computing system (100) can calculate each pixel-wise difference value D(i) between the i-th level image I(i) and the i-th level approximate image P(i) (S142), and can generate the i-th level difference image I'(i) based on each pixel-wise difference value D(i) between the i-th level image I(i) and the i-th level approximate image P(i) (S143).

[0135] This can be expressed by the following operation:

[0136] * I'(i) = DIFF(I(i), P(i))

[0137] In one embodiment, the computing system (100) can generate a difference image by directly assigning each pixel-wise difference value D(i) between the i-th level image I(i) and the i-th level approximate image P(i) to each pixel.

[0138] However, when creating a difference image in the above manner, each pixel of the difference image is expressed as a value in the range of -255 to 255, which is not a value suitable for saving in a general image compression format, so a process of converting the value range to 0 to 255 may be necessary. Hereinafter, this will be called the range conversion process, and the process of returning the converted values ​​through the range conversion process to the original range will be called the range restoration process.

[0139] As an example of a range conversion process, a process of adding 255 to each pixel difference value of the i-th level image and the i-th level approximate image and then dividing it by 2 to create an i-th level difference image corresponding to the i-th level image can be exemplified.

[0140] Figure 8a is a drawing illustrating in more detail an example of step 143 of Figure 6 to which the above range conversion process is applied.

[0141] Referring to FIG. 8a, the computing system (100) is configured to store all integers w that satisfy 0 <= w <= (the number of horizontal pixels of I'(i) - 1) and 0 <= <h<=(I'(i)의 세로 픽셀 수 - 1)를 만족하는 모든 정수 h에 대하여, S1431 단계를 수행할 수 있다(S1430).

[0142] In step S1431, the computing system (100) can add 255 to the difference value (D(i)[w][h]) between the (w, h)th pixel (I(i)[w][h]) of the i-th level image and the (w, h)th pixel (P(i)[w][h]) of the i-th level approximation image, and then divide the difference value by 2 to assign it to the (w, h)th pixel (I'(i)[w][h]) of the i-th level difference image. This can be expressed by the following operation.

[0143] * I'(i)[w][h] = (D(i)[w][h] + 255) / 2

[0144] Another example of a range conversion process is a method of applying a parity image, and an example of step 143 of FIG. 6 to which this range conversion process is applied is illustrated in FIG. 8b.

[0145] In the method illustrated in Fig. 8b, not only the i-th level difference image I'(i) but also the i-th level parity image R(i) is generated, and the i-th level parity image R(i) is used in a later range restoration process.

[0146] Referring to FIG. 8b, the computing system (100) is configured to store all integers w that satisfy 0 <= w <= (the number of horizontal pixels of I'(i) - 1) and 0 <= <h<=(I'(i)의 세로 픽셀 수 - 1)를 만족하는 모든 정수 h에 대하여, S1433 단계 내지 S1437 단계를 수행할 수 있다(S1432).

[0147] In step S1433, the computing system (100) can add 255 to the difference value (D(i)[w][h]) between the (w, h)th pixel (I(i)[w][h]) of the i-th level image and the (w, h)th pixel (P(i)[w][h]) of the i-th level approximate image and assign it to the (w, h)th pixel (U(i)[w][h]) of the first temporary image corresponding to the i-th level image, which can be expressed by the following operation.

[0148] * U(i)[w][h] = D(i)[w][h] + 255

[0149] In step S1434, the computing system (100) can determine whether the (w, h)th pixel (U(i)[w][h]) of the first temporary image is even, and if it is even, assign a value 0 indicating even parity to the (w, h)th pixel (R(i)[w][h]) of the i-th level parity image (S1435), and if it is odd, assign a value 1 indicating odd parity to the (w, h)th pixel (R(i)[w][h]) of the i-th level parity image (S1436).

[0150] In step S1437, the computing system (100) can divide the difference value between the (w, h)th pixel (U(i)[w][h]) of the i-th level temporary image and the (w, h)th pixel (R(i)[w][h]) of the i-th level parity image by 2 and assign it to the (w, h)th pixel (I'(i)[w][h]) of the i-th level difference image, which can be expressed by the following operation.

[0151] * I'(i)[w][h] = (U(i)[w][h] - R(i)[w][h]) / 2

[0152] Meanwhile, the computing system (100) may store the i-th level parity image R(i) so that it can be used in a later range restoration process (S1438). Depending on the embodiment, the computing system (100) may store the i-th level parity image R(i) within a compressed image pyramid, but depending on the embodiment, it may also store it separately from the compressed image pyramid.

[0153] Referring again to FIG. 5, the computing system (100) can generate a compressed image pyramid including a 0th level difference image to a (K-1)th level difference image and a Kth level image to a (L-1)th level image of the image pyramid (S150).

[0154] In one embodiment, the computing system (100) can store the compressed image pyramid in an image format such as JPEG or PNG.

[0155] Figure 9 is a diagram showing the structure of an image pyramid before and after compression.

[0156] Referring to FIG. 9, the image pyramid before compression includes the 0th level image to the (L-1)th level image, but the image pyramid after compression includes the actual images, i.e., the Kth level image I(K) to the (L-1)th level image I(L-1), for the compression reference level to the largest level, and the difference image (i.e., the difference in each pixel between the Kth level image I(K), which is the compression reference level, and the actual image, or a processed value thereof) for the levels before the compression reference level.

[0157] FIG. 10 is a flowchart illustrating an image pyramid restoration method (S200) according to one embodiment of the present invention.

[0158] Referring to Fig. 10, the computing system (100) 0<=j <K인 임의의 정수 j에 대하여, 압축된 이미지 피라미드에 포함된 제j레벨 차이 이미지로부터 제j레벨 이미지를 복원할 수 있다. 바람직하게는 상기 컴퓨팅 시스템(100)은 정수 j에 (K-1)부터 0까지 순차적으로 할당하면서 차례로 제j레벨 이미지를 복원할 수 있다(S210).

[0159] More specifically, the computing system (100) can generate a j-th level approximate image P(j) of the same size as the j-th level image I(j) based on the K-th level image I(K) of the original image pyramid (S211).

[0160] For example, the computing system (100) can generate a j-level approximate image P(j) by enlarging the K-level image I(K) of the original image pyramid by the size of the j-level image I(j), which can be expressed by the following operation.

[0161] * P(j) = SHRINK -1 (I(K), D(K) / D(j))

[0162] Alternatively, the computing system (100) may generate the j-level approximate image P(j) using the sequential magnification method described above with reference to FIG. 7a or FIG. 7b.

[0163] Thereafter, the computing system (100) can calculate the difference value D(j) for each pixel between the j-th level image I(j) and the j-th level approximation image P(j) from the j-th level difference image I'(j) included in the compressed image pyramid (S212). Then, the computing system (100) can restore the j-th level image I''(j) by adding D(j) and the enlarged j-th level approximation image P(j) on a pixel basis (S2132), which can be expressed by the following operation.

[0164] * I''(j) = ADD(I'(j), P(j))

[0165] If a range conversion process as shown in FIG. 8a or FIG. 8b is performed during the image pyramid compression process, the computing system (100) must perform a corresponding range restoration process.

[0166] For example, if a range conversion process as illustrated in FIG. 8a is performed, a range restoration process of multiplying each pixel of the j-level difference image by 2 and then subtracting 255 must be performed, and if a range conversion process as illustrated in FIG. 8b is performed, a range restoration process applying a parity image must be performed.

[0167] FIG. 11a is a drawing showing an example of a specific process of step S212 in which a range restoration process corresponding to the range conversion process illustrated in FIG. 8a is performed, and FIG. 11b is a drawing showing an example of a specific process of step S212 in which a range restoration process corresponding to the range conversion process illustrated in FIG. 8b is performed.

[0168] First, referring to FIG. 11a, the computing system (100) satisfies 0 <= w <= (the number of horizontal pixels of I'(i) - 1) and all integers w and 0 <= <h<=(I'(i)의 세로 픽셀 수 - 1)를 만족하는 모든 정수 h에 대하여, S2121 단계를 수행할 수 있다(S2120).

[0169] In step S2121, the computing system (100) can multiply the (w, h)th pixel (I'(i)[w][h]) of the i-th level difference image by 2 and then subtract 255 to calculate the difference value (D(i)[w][h]) between the (w, h)th pixel (I(i)[w][h]) of the i-th level image and the (w, h)th pixel (P(i)[w][h]) of the i-th level approximation image. This can be expressed by the following operation.

[0170] * D(j)[w][h] = I'(i)[w][h] × 2 - 255

[0171] Meanwhile, referring to FIG. 11b, the computing system (100) is configured to store all integers w that satisfy 0 <= w <= (the number of horizontal pixels of I'(i) - 1) and 0 <= <h<=(I'(i)의 세로 픽셀 수 - 1)를 만족하는 모든 정수 h에 대하여, S2123 단계 내지 S2124 단계를 수행할 수 있다(S2122).

[0172] In step S2123, the computing system (100) can multiply the (w, h)th pixel (I'(i)[w][h]) of the j-th level difference image by 2 and assign it to the (w, h)th pixel V(j)[w][h] of the second temporary image corresponding to the j-th level difference image, which can be expressed by the following operation.

[0173] * V(j)[w][h] = I'(i)[w][h] × 2

[0174] In step S2124, the computing system (100) can calculate a difference value (D(j)[w][h]) between the (w, h)th pixel (I(j)[w][h]) of the j-th level image and the (w, h)th pixel (P(j)[w][h]) of the j-th level approximation image by subtracting 255 from the sum of the (w, h)th pixel (V(j)[w][h]) of the second temporary image and the (w, h)th pixel (R(j)[w][h]) of the j-th level parity image, which can be expressed by the following operation.

[0175] * D(j)[w][h] = (V(j)[w][h] + R(i)[w][h]) - 255

[0176] Meanwhile, the computing system (100) is K<=j <L-1인 j의 경우, 압축된 이미지 피라미드에는 원본 그대로의 이미지가 포함되어 있으므로 별도의 복원 작업 없이 그대로 이미지를 이용하거나 저장할 수 있다.

[0177] The computing system (100) above may be implemented with a single physical device, but an average expert in the technical field of the present invention will be able to easily infer that a plurality of physical devices may be organically combined to implement the computing system (100) according to the technical idea of ​​the present invention, as needed.

[0178] The computing system (100) above may refer to a logical configuration equipped with hardware resources and / or software necessary to implement the technical idea of ​​the present invention, and does not necessarily refer to a single physical component or a single device. That is, the computing system (100) may refer to a logical combination of hardware and / or software provided to implement the technical idea of ​​the present invention, and, if necessary, may be implemented as a set of logical configurations for implementing the technical idea of ​​the present invention by being installed in devices spaced apart from each other and performing each function. In addition, the computing system (100) may refer to a set of configurations that are separately implemented for each function or role for implementing the technical idea of ​​the present invention. The combination of software and / or hardware constituting the computing system (100) may also be located in different physical devices, and the configurations located in different physical devices may be organically combined with each other to implement each of the modules.

[0179] Additionally, the term "module" in this specification may refer to a functional and structural combination of hardware for implementing the technical concepts of the present invention and software for operating the hardware. For example, the module may refer to a logical unit of a given code and hardware resources for executing the given code, and it is readily apparent to an average expert in the technical field of the present invention that it does not necessarily refer to physically connected code or a single type of hardware.

[0180] Meanwhile, the method according to an embodiment of the present invention may be implemented in the form of computer-readable program commands and stored on a computer-readable recording medium, and the control program and target program according to an embodiment of the present invention may also be stored on a computer-readable recording medium. A computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system.

[0181] The program commands recorded on the recording medium may be specially designed and configured for the present invention or may be known and available to those skilled in the software field.

[0182] Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Additionally, the computer-readable storage media can be distributed over network-connected computer systems so that the computer-readable code can be stored and executed in a distributed manner.

[0183] Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a device that processes information electronically, such as a computer, using an interpreter.

[0184] The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

[0185] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0186] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

[0187]

[0188] The present invention can be used in a method for compressing a high-resolution pathology slide image pyramid, a method for restoring a compressed pathology slide image pyramid, and a computing system for performing the same.

Claims

1. A step of a computing system obtaining an image pyramid corresponding to an original pathology slide image, wherein the image pyramid includes a 0th level image to an (L-1)th level image (L is a predefined integer greater than or equal to 2), the 0th level image of the image pyramid is the original pathology slide image, and for each integer m satisfying 1<=m<=L-1, the mth level image of the image pyramid is a reduced image of the (m-1)th level image of the image pyramid; The above computing system, 0<=i <K인 임의의 정수 i에 대하여, 제i레벨 차이 이미지를 생성하는 단계-여기서, 압축 기준 레벨 K는, 0<K<L을 만족하는 소정의 정수임; 및 The computing system comprises a step of generating a compressed image pyramid including a 0th level difference image to a (K-1)th level difference image and a Kth level image to a (L-1)th level image of the image pyramid, The step of generating the above i-level difference image is: A step of generating an i-th level approximate image having the same size as the i-th level image of the image pyramid based on the K-th level image of the image pyramid; and A method comprising the step of generating the i-th level difference image based on each pixel unit difference value between the i-th level image and the i-th level approximate image.

2. In paragraph 1, The step of generating an i-th level approximate image of the same size as the i-th level image of the image pyramid based on the K-th level image of the image pyramid is as follows. A method comprising the step of applying bilinear interpolation or bicubic interpolation to the K-th level image of the image pyramid to enlarge the image by the size of the i-th level image of the image pyramid.

3. In paragraph 2, The step of generating an i-th level approximate image of the same size as the i-th level image of the image pyramid based on the K-th level image of the image pyramid is as follows. A method further comprising the step of generating the i-th level approximate image by inputting the K-th level image of the image pyramid, which is enlarged by the size of the i-th level image of the image pyramid, into a pre-learned artificial neural network for image noise removal and sharpening.

4. In paragraph 1, The step of generating an i-th level approximate image of the same size as the i-th level image of the image pyramid based on the K-th level image of the image pyramid is as follows. A step of inputting the K-th level image, or an extended image that extends the K-th level image by p pixels (wherein p is an integer greater than or equal to 1) upward, downward, left, and right, into a pre-trained super-resolution neural network to generate the i-th level approximate image, A method characterized in that the above p is a value less than or equal to Z determined by the following [Formula 1]. [Formula 1] (Here, C is a set containing all convolutional layers included in the super-resolution neural network, and f i is the filter size of the i-th convolutional layer included in the above super-resolution neural network.

5. In paragraph 1, The above computing system, 0<=i <K인 임의의 정수 i에 대하여, 제i레벨 차이 이미지를 생성하는 단계는, Including a step of generating the i-th level difference image by sequentially assigning from K-1 to 0 to the integer i, The step of generating an i-th level approximate image of the same size as the i-th level image of the image pyramid based on the K-th level image of the image pyramid is as follows. When i = K-1, a step of generating an i-th level approximate image based on the K-th level image of the image pyramid; and A method comprising the step of generating an i-th level approximate image based on an (i+1)-th level approximate image, when i < K-1.

6. In paragraph 1, The above computing system, 0<=i <K인 임의의 정수 i에 대하여, 제i레벨 차이 이미지를 생성하는 단계는, Including a step of generating the i-th level difference image by sequentially assigning from K-1 to 0 to the integer i, The step of generating an i-th level approximate image of the same size as the i-th level image of the image pyramid based on the K-th level image of the image pyramid is as follows. When i = K-1, a step of generating an i-th level approximate image based on the K-th level image of the image pyramid; and A method comprising the steps of restoring the (i+1)-th level image of the image pyramid by reflecting the (i+1)-th level difference image to the (i+1)-th level approximate image when i < K-1, and generating the i-th level approximate image based on the restored (i+1)-th level image.

7. In paragraph 1, The above computing system, 0<=j <K인 임의의 정수 j에 대하여, 제j레벨 차이 이미지로부터 제j레벨 이미지를 복원하는 단계를 더 포함하되, The step of restoring the j-level image from the j-level difference image is as follows. A step of generating a j-th level approximate image having the same size as the j-th level image of the original image pyramid based on the K-th level image of the original image pyramid; A step of calculating each pixel unit difference value between the j-th level image and the j-th level approximate image based on the j-th level difference image; and A method comprising the step of restoring the j-level image by adding the pixel-wise difference values ​​of each pixel of the j-level image and the j-level approximate image and the j-level approximate image.

8. In paragraph 7, The step of generating the i-th level difference image based on the difference value of each pixel between the i-th level image and the i-th level approximate image is as follows. A step of adding 255 to each pixel difference value of the i-th level image and the i-th level approximate image and dividing it by 2 to generate an i-th level difference image corresponding to the i-th level image, The step of calculating the pixel-wise difference value between the j-th level image and the j-th level approximate image based on the j-th level difference image is as follows. A method comprising the step of calculating a pixel-wise difference value between the j-th level image and the j-th level approximate image by multiplying each pixel of the j-th level difference image by 2 and then subtracting 255 from the difference.

9. In paragraph 7, The step of generating the i-th level difference image based on the difference value of each pixel between the i-th level image and the i-th level approximate image is as follows. A step of generating a first temporary image corresponding to the i-th level image by adding 255 to each pixel difference value between the i-th level image and the i-th level approximate image; A step of generating an i-th level parity image having the same size as the first temporary image, wherein each pixel of the i-th level parity image is assigned 0 if the number of pixels corresponding to it in the i-th level temporary image is even, and 1 otherwise; a step of storing the i-level parity image; and A step of generating an i-level difference image corresponding to the i-level image by dividing the pixel-wise difference value between the i-level temporary image and the i-level parity image by 2, The step of calculating the pixel-wise difference value between the j-th level image and the j-th level approximate image based on the j-th level difference image is as follows. A step of generating a second temporary image corresponding to the j-th level difference image by multiplying each pixel of the j-th level difference image by 2; and A method comprising the step of calculating a pixel-wise difference value between the j-th level image and the j-th level approximate image by subtracting 255 from each pixel-wise sum value of the second temporary image and the j-th level parity.

10. A method for restoring an original image pyramid from a compressed image pyramid generated by the method described in paragraph 1, A step of obtaining a compressed image pyramid including a 0th level difference image to a (K-1)th level difference image and a Kth level image to a (L-1)th level image of the original image pyramid; and The above computing system, 0<=j <K인 임의의 정수 j에 대하여, 제j레벨 차이 이미지로부터 제j레벨 이미지를 복원하는 단계를 포함하되, The step of restoring the j-level image from the j-level difference image is as follows. A step of generating a j-th level approximate image having the same size as the j-th level image of the original image pyramid based on the K-th level image of the original image pyramid; A step of calculating each pixel unit difference value between the j-th level image and the j-th level approximate image based on the j-th level difference image; and A method comprising the step of restoring the j-level image by adding the pixel-wise difference values ​​of each pixel of the j-level image and the j-level approximate image and the j-level approximate image.

11. A computer program recorded on a recording medium for installing in a data processing device and performing the method described in any one of claims 1 to 10.

12. A computer-readable recording medium having recorded thereon a computer program for performing the method described in any one of claims 1 to 10.

13. As a computing system, Including the processor and memory, The above memory, when executed by the processor, causes the computing system to perform a method of compressing a pathology slide image pyramid, The method of compressing the above pathology slide image pyramid is as follows: A step in which the computing system obtains an image pyramid corresponding to an original pathology slide image, wherein the image pyramid includes a 0th level image to an (L-1)th level image (L is a predefined integer greater than or equal to 2), the 0th level image of the image pyramid is the original pathology slide image, and for each integer m satisfying 1<=m<=L-1, the mth level image of the image pyramid is a reduced image of the (m-1)th level image of the image pyramid; The above computing system, 0<=i <K인 임의의 정수 i에 대하여, 제i레벨 차이 이미지를 생성하는 단계-여기서, 압축 기준 레벨 K는, 0<K<L을 만족하는 소정의 정수임; 및 The computing system comprises a step of generating a compressed image pyramid including a 0th level difference image to a (K-1)th level difference image and a Kth level image to a (L-1)th level image of the image pyramid, The step of generating the above i-level difference image is: A step of generating an i-th level approximate image having the same size as the i-th level image of the image pyramid based on the K-th level image of the image pyramid; and A computing system comprising a step of generating the i-th level difference image based on each pixel unit difference value between the i-th level image and the i-th level approximate image.

14. As a computing system, Including the processor and memory, The above memory, when executed by the processor, causes the computing system to perform a method for restoring an original image pyramid from a compressed image pyramid generated by the method described in claim 1. The method for restoring the above original image pyramid is: The computing system obtains a compressed image pyramid including a 0th level difference image to a (K-1)th level difference image and a Kth level image to a (L-1)th level image of the image pyramid; The above computing system, 0<=j <K인 임의의 정수 j에 대하여, 제j레벨 차이 이미지로부터 제j레벨 이미지를 복원하는 단계를 포함하되, The step of restoring the j-level image from the j-level difference image is as follows. A step of generating a j-th level approximate image having the same size as the j-th level image of the original image pyramid based on the K-th level image of the original image pyramid; A step of calculating each pixel unit difference value between the j-th level image and the j-th level approximate image based on the j-th level difference image; A computing system including a step of restoring a j-level image by adding the pixel-wise difference values ​​of each pixel of the j-level image and the j-level approximate image and the j-level approximate image.

Images