Region-of-interest and magnification-based histopathology slide retrieval method and apparatus

Abstract

A region-of-interest and magnification-based histopathology slide retrieval method includes steps in which a retrieval device: receives query histopathology slides; divides, into patches of predetermined sizes, each of histopathology slide images having different magnifications of the query histopathology slides; selects query patches while changing the location and magnification level of patches from among the patches; calculates the similarity between the query patches and each of previously stored candidate histopathology slides; and selects, as a target histopathology slide, a histopathology slide with the highest similarity to the query patches from among the candidate histopathology slides.

Description

Method and device for retrieving pathology slides based on region of interest and magnification

[0001] The technique described below relates to a technique for retrieving pathology slides. In particular, the technique described below relates to a technique for retrieving the most similar pathology slice from a database based on a specific pathology slide.

[0002] Tissue biopsies are a common diagnostic technique for observing disease signs or lesions. Tissue sections are placed on glass slides and observed under a microscope or other device. Slides prepared for tissue biopsies are called histopathology slides. Recently, medical institutions have been scanning pathology slides and managing them as digital images. Consequently, a large number of digital pathology slides are being collected in databases.

[0003] Large volumes of pathology slides stored in databases can be used to diagnose related diseases or other conditions. In this case, image-to-image search techniques are necessary. Conventional image search techniques have limitations in operating accurately on normalized image sets, such as those with the same magnification or size.

[0004] The technique described below is intended to provide a technique for retrieving from a database the pathology slide most similar to a specific pathology slide that is desired to be searched.

[0005] A method for searching pathology slides based on a region of interest and magnification includes a step of a search device receiving a query pathology slide, a step of the search device dividing each of pathology slide images of different magnifications of the query pathology slide into patches of a predetermined size, a step of the search device selecting query patches while changing the location and magnification level of the patches, a step of the search device calculating a similarity between the query patches and each of pre-stored candidate pathology slides, and a step of the search device selecting a pathology slide having the highest similarity to the query patches among the candidate pathology slides as a target pathology slide.

[0006] A search device for searching a pathology slide includes an interface device for receiving a query pathology slide, a storage device for storing a plurality of candidate pathology slides and clinical information for each of the candidate pathology slides, and a calculation device for dividing each of pathology slide images of different magnifications of the query pathology slide into patches of a predetermined size, calculating the similarity between query patches selected while changing the position and magnification level of the patches and each of the candidate pathology slides stored in advance, and selecting a pathology slide having the highest similarity to the query patches among the candidate pathology slides as a target pathology slide.

[0007] The technique described below dynamically searches for pathology slides most similar to a specific subject's pathology slide by changing the region of interest and magnification.

[0008] Figure 1 is an example of a pathology slide navigation system.

[0009] Figure 2 is an example of the pathology slide exploration process.

[0010] Figure 3 is an example of similarity evaluation using multiple patches.

[0011] Figure 4 is an example of a space for navigating a pathology slide while changing the region of interest and magnification.

[0012] Figure 5 is an example of a process for calculating the similarity between a query patch and patches in a database.

[0013] Figure 6 is an example of a process for calculating the similarity between query patches and database patches using Windows.

[0014] Figure 7 is an example of the penalty for the process of calculating the similarity between query patches and database patches using Windows.

[0015] Figure 8 is an example of a search device for exploring pathology slides.

[0016] The technology described below is susceptible to various modifications and embodiments. Specific embodiments are illustrated and described in detail in the drawings. However, this does not limit the technology described below to specific embodiments, and it should be understood that all modifications, equivalents, and alternatives fall within the spirit and scope of the technology described below.

[0017] Terms such as first, second, A, and B may be used to describe various components, but these components are not limited by these terms and are used solely to distinguish one component from another. For example, without departing from the scope of the technology described below, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0018] As used herein, the singular expressions should be understood to include the plural expressions unless the context clearly dictates otherwise, and the term "comprises" and the like should be understood to mean the presence of a described feature, number, step, operation, component, part, or combination thereof, but not to exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0019] Before going into a detailed description of the drawings, it should be made clear that the division of components in this specification is merely a division based on the main function of each component. In other words, two or more components described below may be combined into a single component, or a single component may be further subdivided into two or more components with more detailed functions. In addition to its own main function, each component described below may additionally perform some or all of the functions of other components, and of course, some of the main functions of each component may be exclusively performed by other components.

[0020] Additionally, in performing a method or method of operation, each process constituting the method may occur in a different order than the stated order, unless the context clearly indicates a specific order. That is, each process may occur in the same order as the stated order, may be performed substantially simultaneously, or may be performed in the opposite order.

[0021] The technique described below relates to a technique for navigating pathology slides in an image-to-image manner.

[0022] Pathology slides are slides of tissue from a specific patient. Pathology slides can be stained with specific staining patterns, depending on the tissue type, lesion type, etc. The techniques described below can be applied to various types of pathology slides.

[0023] The pathology slide exploration technique described below can utilize various image exploration techniques. For example, the image exploration technique may be a deep learning-based technique that extracts features at the patch level and explores the image. Alternatively, the image exploration technique may utilize traditional image processing techniques.

[0024] The following describes how a search device searches a database for the most similar pathology slide based on the input pathology slide. The search device refers to a computing device capable of digital image preprocessing, image feature extraction, and deep learning. The search device can be implemented using a variety of data processing devices. For example, the search device can be implemented using a PC, a networked server, a smart device, or a chipset embedded with a dedicated program.

[0025] The pathology slide entered at this time is the input (reference) image that serves as the basis for slide exploration. Hereinafter, the entered pathology slide is referred to as the query pathology slide. Furthermore, among the pathology slides stored in the database, the pathology slide most similar to the query pathology slide is referred to as the target pathology slide.

[0026] Fig. 1 is an example of a pathology slide search system (100). In Fig. 1, the search device is illustrated as an example of a computer terminal (110) and a server (130).

[0027] The database (DB, 140) can store pathology slides collected from a plurality of subjects (patients) in advance. The database (140) can store pathology slides for various tissues. The database (140) can store pathology slices with different magnifications for the same tissue. The database (140) can store pathology slides by dividing them into patches of a certain size. In some cases, the database (140) can also store feature values ​​for pathology slides or each patch. In this case, the feature values ​​can be extracted through a certain algorithm or deep learning model (e.g., an encoder including a convolutional layer).

[0028] The database (140) stores clinical information corresponding to each pathology slide. The clinical information may include at least one of the following: findings regarding the tissue (e.g., presence of a lesion, degree of disease progression, etc.), the prognosis of the patient, and medical treatment for the patient.

[0029] The computer terminal (110) receives a query pathology slide of a specific subject from the user.

[0030] The computer terminal (110) can store the query pathology slides at different magnifications. The different magnifications can be various values ​​such as the same magnification, 10x, 40x, 100x, 200x, 400x, and 1000x.

[0031] The computer terminal (110) can divide query pathology slides of different magnifications into patches of a certain size. The computer terminal (110) can also distinguish and extract patches of different magnifications from each of the candidate pathology slides in the database (140). The computer terminal (110) compares the similarity between the patches of different magnifications and the patches of the candidate pathology slides in the database (140).

[0032] The computer terminal (110) can select a target pathology slide from among candidate pathology slides based on the similarity of all patches to be compared. The computer terminal (110) can receive the target pathology slide and / or clinical information of the corresponding pathology slide from the database (140).

[0033] The user terminal (120) receives a query pathology slide of a specific subject from the user. The server (130) can receive the query pathology slide from the user terminal (120).

[0034] The server (130) can store query pathology slides at different magnifications. The different magnifications can be various values ​​such as the same magnification, 10x, 40x, 100x, 200x, 400x, and 1000x.

[0035] The server (130) can divide query pathology slides of different magnifications into patches of a certain size. The server (130) can also distinguish and extract patches of different magnifications from each of the candidate pathology slides in the database (140). The server (130) compares the similarity between the patches of different magnifications and the patches of the candidate pathology slides in the database (140).

[0036] The server (130) can select a target pathology slide from among candidate pathology slides based on the similarity of all patches to be compared. The server (130) can receive the target pathology slide and / or clinical information of the corresponding pathology slide from the database (140).

[0037] The computer terminal (110) and / or server (130) can store the target pathology slide and / or clinical information of the pathology slide in EMR, etc.

[0038] Figure 2 is an example of a pathology slide exploration process (200). It is assumed that the database stores candidate pathology slides and clinical information in advance.

[0039] The retrieval device receives a query pathology slide (210). The retrieval device divides the query pathology slide into patches of a certain size (220). At this time, the retrieval device can arrange the query pathology slide into different arrangements and divide the query pathology slides of different magnifications into patches of a certain size.

[0040] The search device can extract features from each patch identified in the query pathology slide (230). At this time, the search device can extract features from the patches using a specific image processing technique. Alternatively, the search device can extract feature vectors from the patches using a neural network that outputs features.

[0041] The retrieval device can extract features for each of the patches identified from a single pathology slide. The retrieval device can extract features for each of the patches identified from each of the pathology slides at various magnifications. In some cases, the retrieval device can extract features from each of the patches at different magnifications for the same region of interest. In this case, the patches for the same region of interest may be patches from pathology slides at different magnifications that have the same center coordinates. In this case, the coordinates can define the location of the patch in the corresponding tissue.

[0042] The search device can evaluate the similarity between each candidate pathology slide in the database and the features extracted from the patch (240). The detailed similarity evaluation process is described below.

[0043] The search device can evaluate the similarity between the query pathology slide and each of the candidate pathology slides, and select the pathology slide with the highest similarity among the candidate pathology slides as the target pathology slide (250). At this time, the search device can extract the target pathology slide and clinical information about the corresponding slide from the database.

[0044] Figure 3 is an example of similarity evaluation using multiple patches. Figure 3 is an example of a process for evaluating image similarity based on multiple patches. Figure 3 is an example of evaluating image similarity based on a total of m patches. In Figure 3, L represents a magnification level. L1, L2, ..., L m Each represents a different magnification. The magnification is L1< L2< ... < L m It is a relationship. A patch belonging to a query can be named a query patch.

[0045] L1, L2, ..., L in query patches m refers to patches of the same region of interest. The query patch is a patch identified from the query pathology slide. In the candidate patches, L1, L2, ..., L m Also, it refers to patches of the same region of interest. A candidate patch can be a patch identified from any of the candidate pathology slides stored in the database. In this case, patches of the same region of interest can have the same center coordinates in the corresponding tissue.

[0046] The search device is an example of evaluating similarity based on the distance between patches of the same magnification. In this case, the distance can be the cosine distance. The search device measures the distances d1, d2, ..., d between patches of the same magnification. m can be calculated. The search device calculates the distance d between patches of the same magnification as in the mathematical expression 1 below. i The mean distance D can be calculated as the similarity between query patches and candidate patches.

[0047]

[0048] In Figure 3, the search device selects patches centered around a single point. Therefore, the similarity results produced by the search device may vary depending on the location of the center point.

[0049] To address this limitation, the search device can compute the similarity of patches while varying the region of interest and / or magnification. That is, the search device can compute the similarity of patches while varying the center point of the region of interest. The search device can compute the similarity of patches while arbitrarily varying the region of interest and magnification.

[0050] Fig. 4 is an example of a space for exploring a pathology slide while changing the region of interest and magnification. In Fig. 4, the XY plane is a coordinate system representing the center of a patch. In Fig. 4, the Z-axis represents magnification. The search device can calculate the similarity between the corresponding query patch and the candidate patch while changing the center position and magnification of the patch in the X, Y, and Z coordinate systems. The values ​​of each axis in Fig. 4 may be a constant normalized value. The curve displayed in the space in Fig. 4 represents a path or criterion for selecting a patch while changing the region of interest and magnification. At this time, a specific patch can be defined as in the following mathematical expression 2.

[0051]

[0052] A particular patch P can be defined by a center point (x,y) and a scale (z). Time t corresponds to the order of the patches in the patch sequence.

[0053] The database stores candidate pathology slides for various tissues. The database can categorize and store candidate pathology slides by tissue. The database can also categorize and store pathology slides based on tissue and image characteristics. Table 1 below provides examples of pathology slides stored in the database. Table 1 below shows an example of a pathology slide for lung tissue for lung adenocarcinoma.

[0054] SetSubsetPanel(ROI) Lung Adenocarcinoma Malepidic pattern, flat shape Panel 1 Panel 2 Acinar pattern, linear shape Panel 3 Panel 4 Panel 5 Papillary pattern, papillary shape Panel 6 Panel 7

[0055] First, a panel corresponds to a region of interest (ROI) that contains representative histological characteristics for a given tissue. The search engine can evaluate the similarity between query patches and patches within the panel. Panels can be manually selected by a pathologist or automatically selected through a similarity search. A subset is a collection of panels with similar patterns, representing a single type of histological characteristic. A set can consist of multiple subsets.

[0056] The database stores multiple pathology slides and their corresponding clinical information. The database can store pathology slides for the same tissue separately. Clinical information can also store tissue type. The database can further store information on image patterns or subsets. Clinical information can also store information on image patterns or subsets.

[0057] As described above, the database can store a set of panels for the same tissue or subset. A panel consists of patches for regions of interest with histological characteristics in that tissue.

[0058] The search device selects a patch to be compared by changing the center point and magnification of the patch, as shown in Figure 4. However, for convenience of explanation, the description focuses on an example in which the similarity is evaluated using patches with different magnifications in the same region of interest. Figures 5 to 7 illustrate examples of evaluating the similarity between one panel (Panel 1) and a query patch in the same region of interest.

[0059] Figure 5 illustrates an example of a process for calculating the similarity between query patches and database patches. In Figure 5, a panel consists of consecutive patches with different magnifications. Patches within a panel can be referred to as panel patches. In Figure 5, the magnification of patches within a panel increases as they move to the right. Figure 5 illustrates panel 1 (ROI 1) and query 1 as examples. Query 1 consists of query patches with different magnifications.

[0060] Panel 1 consists of n patches, where n is the number of different magnifications in the panel (Level panel ) means.

[0061] The search device can select a certain number of query patches from the query pathology slide. The search device can select query patches to be evaluated for similarity at regular intervals or randomly among patches with different magnifications. Query 1 consists of N patches. N is the number of different magnifications (Level query ) means. It is a relationship where n ≥ N. Figure 5 shows a case where the number of query patches is smaller than the number of patches belonging to the panel. In Figure 5, query 1 is an example consisting of 5 patches.

[0062] Figure 5(A) is an example in which a search device calculates the similarity between the first patch (p1) of query 1 and the patches of panel 1. The similarity may be the distance between the features of p1 and the features of the panel patches. In this case, the distance may be a cosine distance.

[0063] Figure 5(A) is an example in which a search device calculates the similarity between the fourth patch (p4) of query 1 and the patches of panel 1. In this case, the similarity may be the average distance between p4 and the patches of panel 1. In this case, the distance may be a cosine distance.

[0064] The search device can repeat the similarity calculation for each of the patches belonging to query 1. The final similarity can be the average distance of all patches belonging to query 1.

[0065] Figure 6 is an example of a process for calculating the similarity between query patches and database patches using a window. In Figure 6, panel 1 represents n(=Level panel ) consists of patches. In Fig. 6, query 1 is N(=Level query ) consists of patches.

[0066] A window is an interval in which a panel patch, which is the target of a distance (similarity) calculation for a specific query patch, is expected to be located. In other words, a window corresponds to an interval in which a panel patch valid for similarity calculation is located. A window may be composed of panel patches with scales adjacent to a scale of a specific query patch. For example, panel patches may be L1 to L n It can be composed of patches of magnification. That is, panel patches can be identified by a series of numbers (1 to n). If the magnification of a specific query patch is i, the window can be composed of patches of magnifications iw to i+w. Here, w is a value that sets the window size. For example, if the magnification of a specific query patch is L4 and w is 1, the panel patches belonging to the window can be L3 ≤ magnification of the panel patch ≤ L5.

[0067] The window interval may vary depending on the number of patches, the number of different scaling factors, or the performance of the search device.

[0068] Figure 6(A) is an example of a window and similarity calculation process for query patch p4. The search device calculates the similarity between query patch p4 and each of the panel patches belonging to panel 1.

[0069] A global match is a patch (global matching patch) that has the highest similarity to the query patch among all panel patches, regardless of the window interval. Figure 6(B) is an example of the global matching process for query patch p4. In Figure 6(B), the arrow indicates the patch with the highest similarity to the query patch.

[0070] A local match is a patch (local matching patch) that has the highest similarity to the query patch among the panel patches within the window. Figure 6(C) is an example of the local matching process for query patch p4. In Figure 6(C), the arrow indicates the patch with the highest similarity to the query patch.

[0071] Among the panel patches in a panel, the patch most similar to a specific query patch is likely to be a query patch within the window. Conversely, if the patch most similar to a specific query patch is outside the window, the result is likely to be incorrect.

[0072] Figure 7 illustrates another example of a process for calculating the similarity between query patches and database patches using windows. Figure 7 illustrates examples of global and local matches. Figure 7(A) illustrates a case where a global match exists within a window. In other words, Figure 7(A) illustrates a case where a global match = a local match.

[0073] Figure 7(B) illustrates a case where a global match exists outside the window. In Figure 7(B), x represents the distance between the global match and the local match, as in Equation 3 below. In other words, x is the difference between the magnification of the global match and the magnification of the local match. In the case of Figure 7(A), since the global match = local match, x = 0.

[0074]

[0075] If no global match exists within the window, x > 0. A larger value of x indicates lower search performance. Therefore, the value of x can be considered a penalty for the search results.

[0076] Fig. 8 is an example of a process for calculating patch similarity while changing the region of interest and magnification. Fig. 8 is an example of evaluating similarity by selecting query patches while changing the location of the region of interest along with the magnification. The search device can select query patches for similarity evaluation while changing the location of the region of interest (coordinates of the patch center) and magnification in a space such as Fig. 4. The search device can select a smaller number of query patches than the number of panel patches in the panel. In Fig. 8, points selected at regular intervals while changing the location of the region of interest (coordinates of the patch center) and magnification are a1 to a6, and the query patches corresponding to these are p1 to p6. Panels in the database include panel patch sequences for various regions of interest. Fig. 8 illustrates panels for k regions of interest 1 (ROI 1) to regions of interest k (ROI k). For each selected query patch, the search device calculates the similarity between the corresponding region of interest (x, y) and the panel patch of the corresponding magnification (z) in the region of interest.

[0077] The search device can calculate the similarity between the selected query patches and the corresponding panel patches as shown in the mathematical expression 4 below.

[0078]

[0079] In Equation 4, S is the similarity between the query patches selected from the query pathology slide and the panels in the database. m is the number of query patches. d i is a query patch p i is the distance between the corresponding panel patches x i is a query patch p iThis is the difference in scale between the global match and the local match selected in the panel as the standard.

[0080] The search device can calculate the similarity between the query pathology slide and all candidate pathology slides in the database and ultimately determine the pathology slide with the highest similarity as the target pathology slide.

[0081] Meanwhile, the search device can narrow the range of candidate pathology slides subject to similarity evaluation based on information about the query pathology slide. For example, the search device can calculate similarity among candidate pathology slides belonging to the same tissue based on the tissue information of the query pathology slide. Furthermore, the search device can calculate similarity among candidate pathology slides belonging to a subset of the same tissue with the same pattern based on the image pattern of the query pathology slide.

[0082] Figure 9 illustrates an example of a search device (300) for searching pathology slides. The search device (300) corresponds to the aforementioned search devices (110 and 130 of Figure 1). The search device (300) may be physically implemented in various forms. For example, the search device (300) may take the form of a computer device such as a PC, a network server, a data processing chipset, or the like.

[0083] The search device (300) may include a storage device (310), a memory (320), a computing device (330), an interface device (340), a communication device (350), and an output device (360).

[0084] The storage device (310) can store the aforementioned database.

[0085] The storage device (310) can store candidate pathology slides and clinical information about the corresponding pathology slides. The clinical information may include findings about the corresponding tissue (e.g., presence of a lesion, degree of disease progression, etc.), the patient's prognosis, and medical treatment for the patient. Furthermore, the clinical information may further store tissue type, image pattern information, etc.

[0086] The storage device (310) can store the input query pathology slide.

[0087] The storage device (310) can store a program or code for searching a target pathology slide in a database based on a query pathology slide.

[0088] The storage device (310) can store target pathology slides and clinical information of the target pathology slides.

[0089] The storage device (310) can store a program or neural network model that extracts features from a pathology slide or a patch of a pathology slide.

[0090] The memory (320) can store data and information generated during the process of the search device (300) searching for a target pathology slide in a database based on a query pathology slide.

[0091] The interface device (340) is a device that receives certain commands and data from the outside.

[0092] The interface device (340) can receive a query pathology slide from a physically connected input device or an external storage device.

[0093] The interface device (340) can receive additional information about the query pathology slide. The additional information may include at least one of tissue type, image pattern information, and patient information from which the specimen was collected.

[0094] The interface device (340) can also transmit the searched target pathology slide and clinical information of the target pathology slide to an external object.

[0095] The interface device (340) also includes a configuration for receiving data or packets from the communication device (350) into the search device (300).

[0096] A communication device (350) refers to a configuration that receives and transmits certain information through a wired or wireless network.

[0097] The communication device (350) can receive a query pathology slide from an external object.

[0098] The communication device (350) can receive additional information about the query pathology slide.

[0099] The communication device (350) can also transmit the searched target pathology slide and clinical information of the target pathology slide to an external entity.

[0100] The output device (360) is a device that outputs certain information. The output device (360) can output interfaces, search results, etc. required for the target pathology slide search process.

[0101] The operating device (330) can adjust the query pathology slides to different magnifications.

[0102] The computational device (330) can divide each of the query pathology slides of different magnifications into patches of a certain size.

[0103] The storage device (310) can store pathology slide images of various magnifications for the same candidate pathology slide in advance. Alternatively, the computing device (330) can adjust a single candidate pathology slide to different magnifications and store them in the storage device (310). In addition, the computing device (330) can divide each of the candidate pathology slides of different magnifications into patches of a certain size.

[0104] The computational device (330) selects query patches while changing the location and magnification of the region of interest in a space defined by the location (center coordinates of the patch) and magnification of the region of interest. The computational device (330) can select query patches while arbitrarily changing the location and magnification of the region of interest. At this time, the computational device (330) can select a certain number of query patches.

[0105] The computational device (330) may limit the scope of candidate pathology slides based on additional information about the query pathology slide. For example, the computational device (330) may select candidate pathology slides from the database that have the same tissue type and / or image pattern as the query pathology slide based on the tissue type and / or image pattern.

[0106] The computing device (330) can compute the similarity between each of the candidate pathology slides and the query pathology slide.

[0107] The computational device (330) can compute the similarity between the query patches of the query pathology slide and the panel patches of the candidate pathology slide. At this time, the computational device (330) can also compute a similarity average that weights the difference between the global matching and the local matching as a penalty, as in Equation 4.

[0108] The computational device (330) can select the pathology slide with the highest similarity to the query pathology slide (query patches) among the candidate pathology slides as the final target pathology slide.

[0109] The computational device (330) can extract a target pathology slide and clinical information of the pathology slide.

[0110] The computing device (330) may be a device such as a processor, AP, or chip embedded with a program that processes data and performs certain operations.

[0111] Additionally, the pathology slide search method or pathology slide-based clinical information extraction method described above may be implemented as a program (or application) including an executable algorithm that can be executed on a computer. The program may be stored and provided on a temporary or non-transitory computer-readable medium.

[0112] A non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on a non-transitory readable medium, such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), EPROM (Erasable PROM, EPROM), EEPROM (Electrically EPROM), or flash memory.

[0113] Temporarily readable media refers to various types of RAM, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous DRAM (Synclink DRAM, SLDRAM), and Direct Rambus RAM (DRRAM).

[0114] The present embodiment and the drawings attached to the present specification only clearly illustrate a part of the technical idea included in the above-described technology, and it will be obvious that all modified examples and specific embodiments that can be easily inferred by a person skilled in the art within the scope of the technical idea included in the specification and drawings of the above-described technology are included in the scope of the rights of the above-described technology.

Claims

1. A step in which a search device receives a query pathology slide; A step in which the above search device divides each of the different magnification pathology slide images of the above query pathology slide into patches of a certain size; A step of selecting query patches by changing the location and magnification level of the patches among the above-mentioned patches; A step in which the search device calculates the similarity between the query patches and the candidate pathology slides stored in advance; and A method for searching pathology slides based on a region of interest and magnification, comprising a step of selecting a pathology slide having the highest similarity to the query patches among the candidate pathology slides as a target pathology slide.

2. In paragraph 1, A method for searching a pathology slide based on a region of interest and magnification, wherein the search device selects a certain number of query patches while randomly changing at least one of the location and magnification level of the patches among the patches of the query pathology slide.

3. In paragraph 1, The above search device calculates the similarity between the query patches and each of the candidate pathology slides, The above search device calculates the similarity between the candidate patches corresponding to the patch positions and magnification levels of the query patches among the patches of the candidate pathology slide and the query patches, A method for searching pathology slides based on a region of interest and magnification, wherein the patches of the above candidate pathology slides are each divided into patches of a certain size from pathology slide images of different magnifications of the above candidate pathology slides.

4. In paragraph 1, The above search device is a region of interest and magnification-based pathology slide search method that calculates the similarity based on the distance between the features of each of the query patches and the features of each of the candidate patches of the candidate pathology slide.

5. In paragraph 1, The above search device A method for searching pathology slides based on a region of interest and magnification, which calculates the similarity of the query patches by penalizing the difference between (i) the magnification of the global matching patch with the highest similarity to the query patch among the candidate patches of the candidate pathology slide and (ii) the magnification of the local matching patch with the highest similarity to the query patch among patches belonging to a similar magnification section within a certain threshold range based on the magnification of the query patch among the candidate patches.

6. In paragraph 1, The above search device further comprising a step of selecting the candidate pathology slides based on additional information of the query pathology slide, The above additional information is a region of interest and magnification-based pathology slide retrieval method including at least one of tissue type, image pattern information, and patient information from which the specimen was collected.

7. In paragraph 1, The above search device further includes a step of extracting clinical information of the target pathology slide, A method for searching pathology slides based on a region of interest and magnification, wherein the clinical information includes at least one of clinical findings for the target pathology slide, prognosis of the patient, and medical treatment for the patient.

8. Interface device for receiving query pathology slides; A storage device storing a plurality of candidate pathology slides and clinical information for each of the candidate pathology slides; and A search device for a pathology slide, comprising a calculation device that divides each of the different magnification pathology slide images of the query pathology slide into patches of a certain size, calculates the similarity between the query patches selected by changing the location and magnification level of the patches and each of the candidate pathology slides stored in advance, and selects the pathology slide having the highest similarity to the query patches among the candidate pathology slides as a target pathology slide.

9. In paragraph 8, The above-mentioned computing device calculates the similarity between the query patches and each of the candidate pathology slides, and searches for a pathology slide in which the candidate patches corresponding to the patch positions and magnification levels of the query patches among the patches of the candidate pathology slides calculate the similarity between the query patches and the candidate patches.

10. In paragraph 8, The above-mentioned calculation device calculates the similarity of the query patches by imposing a penalty as a difference between (i) the magnification of the global matching patch with the highest similarity to the query patch among the candidate patches of the candidate pathology slide, and (ii) the magnification of the local matching patch with the highest similarity to the query patch among the patches belonging to a similar magnification section within a certain threshold range based on the magnification of the query patch among the candidate patches, for each of the query patches, on the basis of the magnification of the query patch, and searching for a pathology slide.

11. In paragraph 8, The above-mentioned computational device selects the candidate pathology slides based on additional information of the above-mentioned query pathology slide, A retrieval device for retrieving a pathology slide, wherein the additional information includes at least one of tissue type, image pattern information, and patient information from which the specimen was collected.

12. In paragraph 8, The above-mentioned computing device selects clinical information corresponding to the target pathology slide, A retrieval device for retrieving a pathology slide, wherein the clinical information includes at least one of clinical findings for the target pathology slide, prognosis of the patient, and medical treatment for the patient.

Images