Hierarchical retrieval method and device for whole slide pathological images, electronic equipment and medium

Abstract

This invention relates to the field of whole-section pathological image retrieval technology, and more specifically, to a hierarchical retrieval method, apparatus, electronic device, and medium for whole-section pathological images. The method includes constructing a graph index structure for the whole-section pathological image; the graph index structure of the whole-section pathological image establishes connections at the same resolution level through semantic links, and establishes cross-resolution connections through spatial relationships at different resolution levels; obtaining a query region; and obtaining candidate regions with the highest similarity to the query region based on the semantic information of the query region and the graph index structure of the whole-section pathological image. This invention employs a hierarchical vector indexing strategy to maintain spatial and semantic connections between multi-level patches, ensuring that features at different magnification levels can be hierarchically aggregated while maintaining spatial consistency and improving retrieval accuracy.

Description

Technical Field

[0001] This invention relates to the field of whole-section pathological image retrieval technology, and more specifically, to a hierarchical retrieval method, apparatus, electronic device, and medium for whole-section pathological images. Background Technology

[0002] With the development of deep learning and large-scale models, the demand for processing high-resolution whole-slice images in pathology is increasing. However, most existing methods are limited to fixed-size patch-level or whole-slice-level granularity, leading to fragmentation of semantic context and insufficient workflow flexibility. Furthermore, these methods fail to adequately consider multi-scale spatial relationships, failing to meet the needs of zooming and different-sized regions of interest (ROIs) in practical diagnostic processes. How to adapt to region retrieval of different sizes and reduce the time consumed by large-scale retrieval is a key focus of whole-slice image region retrieval.

[0003] Existing WSI retrieval frameworks primarily operate at the slice-level or fixed-patch level. Slice-level methods typically aggregate features across the entire WSI, providing a global perspective but losing crucial fine morphological details essential for local diagnosis. For example, studies by Kalra et al. and Li et al. employed this strategy, but their neglect of subtle differences in local regions led to poor performance in real-world clinical applications. On the other hand, fixed-size patch-based methods divide the WSI into a uniform grid, ignoring hierarchical spatial relationships between different patches and forcing pathologists to adhere to fixed patch boundaries. This approach not only easily leads to mismatches between biological structures (such as tumor margins and gland formation) and fixed patches, thus fragmenting the semantic context and reducing retrieval accuracy, but also lacks support for variable magnification and variable-size regions of interest queries, significantly limiting its applicability in real-world diagnostic workflows. Summary of the Invention

[0004] To overcome the technical problem that the existing WSI retrieval framework focuses on operating at the slice level or fixed patch level, which limits its applicability in real-world diagnostic processes, this invention provides a hierarchical retrieval method, device, electronic device, and medium for whole-slice pathological images.

[0005] The present invention aims to solve the above-mentioned technical problems to at least some extent.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: The first aspect of the present invention provides a hierarchical retrieval method for whole-section pathological images, comprising the following steps: A graph index structure for whole-slice pathological images is constructed. The graph index structure of the whole-slice pathological images constructs connections at the same resolution level through semantic connections, and constructs cross-resolution connections through spatial relationships at different resolution levels. Get the area to be queried; Based on the semantic information of the region to be queried and the graph index structure of the whole-slice pathological image, the candidate region with the highest similarity to the region to be queried is obtained.

[0007] Furthermore, the construction of the graph index structure for whole-slice pathological images includes: All whole-slice pathological images are divided into non-overlapping patches using a sliding window; Semantic extraction is performed on each patch to obtain the feature vector of each patch; Based on the feature vectors of patches at the same resolution level, a connection graph of patches at the same resolution level is constructed; based on the spatial relationship of patches at different resolution levels, patch connections across resolution levels are constructed to obtain the graph index structure of the whole-slice pathological image.

[0008] Furthermore, based on the feature vectors of patches at the same resolution level, a connection graph of patches at the same resolution level is constructed, including: Based on the cosine similarity of the feature vectors of patches at the same resolution level, a connection graph of patches at the same resolution level is constructed.

[0009] Furthermore, the graph index structure of the whole-slice pathological image is pruned, including: For each patch, patch connections across resolution levels are retained, and based on the feature vectors of patches at the same resolution level, similarity scores with other patches are calculated. Patch connections with the highest similarity scores are retained until the sum of the number of patch connections across resolution levels and patch connections at the same resolution level reaches a preset number, thus obtaining the graph index structure of the pruned whole-slice pathological image.

[0010] Furthermore, the semantic information of the region to be queried includes: The query area is divided into multiple sub-regions of standard size by a sliding window, where the standard size is the size of a patch in the graph index structure of the whole-slice pathological image; Semantic extraction is performed on each sub-region to obtain the semantic information of each sub-region, which is then used as the semantic information of the region to be queried.

[0011] Furthermore, based on the semantic information of the region to be queried and the graph index structure of the whole-slice pathological image, candidate regions with the highest similarity to the region to be queried are obtained, including: For the semantic information of each sub-region, an approximate nearest neighbor search is performed in the graph index structure of the whole-slice pathological image to obtain one or more patches similar to the sub-region; Using the spatial adjacency of one or more patches retrieved from each sub-region in the whole-slice pathological image, spatially adjacent patches are aggregated into one or more candidate regions. The similarity between all candidate regions and the query region is calculated, and the candidate region with the highest similarity to the query region is output.

[0012] Furthermore, using a GPU-accelerated parallel search engine, an approximate nearest neighbor search is performed on the semantic information of each sub-region in the graph index structure of the whole-slice pathological image to obtain one or more patches similar to the sub-region.

[0013] A second aspect of the present invention provides a hierarchical retrieval device for whole-section pathological images, comprising: The graph index module constructs a graph index structure for whole-slice pathological images. The graph index structure of the whole-slice pathological images constructs connections at the same resolution level through semantic connections, and constructs cross-resolution connections through spatial relationships at different resolution levels. The acquisition module acquires the region to be queried; The query module obtains the candidate region with the highest similarity to the region to be queried based on the semantic information of the region to be queried and the graph index structure of the whole slice pathological image.

[0014] A third aspect of the present invention provides an electronic device including a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the steps of the hierarchical retrieval method for whole-slice pathological images.

[0015] A fourth aspect of the present invention provides a computer storage medium storing a computer program, wherein when the computer program is executed by the processing, it implements the steps of the hierarchical retrieval method for whole-slice pathological images.

[0016] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention discloses a hierarchical retrieval method, apparatus, electronic device, and medium for whole-slice pathological images. It employs a hierarchical vector indexing strategy to maintain spatial and semantic connectivity between multi-level patches, ensuring that features at different magnification levels can be hierarchically aggregated while maintaining spatial consistency. Furthermore, it significantly improves retrieval efficiency by utilizing a GPU-optimized computational pipeline, and reduces memory latency and achieves parallel processing through techniques such as unified out-degree optimization, warp-centric computation, and forgettable hash tables. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a hierarchical retrieval method for whole-section pathological images provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the process for constructing a graph index structure for whole-slice pathological images according to an embodiment of the present invention; Figure 3 A flowchart illustrating the WSI region retrieval process provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a hierarchical retrieval device for whole-slice pathological images provided in an embodiment of the present invention. Detailed Implementation

[0018] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0019] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] The first embodiment of the present invention provides a hierarchical retrieval method for whole-section pathological images, such as... Figure 1 As shown, it includes the following steps: A graph index structure for whole-slice pathological images is constructed. The graph index structure of the whole-slice pathological images constructs connections at the same resolution level through semantic connections, and constructs cross-resolution connections through spatial relationships at different resolution levels. Get the area to be queried; Based on the semantic information of the region to be queried and the graph index structure of the whole-slice pathological image, the candidate region with the highest similarity to the region to be queried is obtained.

[0021] In a further embodiment, whole-slice pathological images (WSIs) are characterized by their multi-level pyramid structure, with patches at different magnifications representing different levels of detail. The hierarchical vector indexing strategy constructed in this invention preserves this pyramid layout, aligning with the inherent structure of WSIs to achieve efficient and scalable searching. The graph index structure for constructing whole-slice pathological images is as follows: Figure 2 As shown, it includes: All whole-slice pathological images are divided into non-overlapping patches using a sliding window; In this embodiment, the step size of the sliding window is equal to the patch size, which is 224×224, to match the input requirements of the encoder; Semantic extraction is performed on each patch to obtain the feature vector of each patch; In this embodiment, a state-of-the-art UNI encoder is used to extract semantic representations from patches at all resolution levels. Each patch is encoded as a 1024-dimensional feature vector. This feature extraction process is performed offline to ensure the scalability and flexibility of large-scale datasets. Based on the feature vectors of patches at the same resolution level, a connection graph of patches at the same resolution level is constructed; based on the spatial relationship of patches at different resolution levels, patch connections across resolution levels are constructed to obtain the graph index structure of the whole-slice pathological image. In this embodiment, to preserve hierarchical information across different resolution levels, "supported connections" are introduced. Patches at different resolution levels are connected based on spatial correspondence. Specifically, a patch at a higher magnification (e.g., 40x) is spatially contained within a parent patch at a lower magnification (e.g., 20x). This parent-child relationship is precisely calculated based on the coordinate system of the WSI image at different resolutions, thus establishing cross-level connections in the graph. Simultaneously, within each resolution level, an initial K-Nearest Neighbor (K-NN) graph is constructed to establish "semantic connections" between patches. The K-NN graph can be constructed using the NN-Descent algorithm (K=64). NN-Descent is an efficient approximate nearest neighbor search algorithm that approximates the true nearest neighbor by iteratively optimizing the neighbor list. It is particularly suitable for processing massive amounts of data and can efficiently identify the nearest neighbor set of each node in large-scale data.

[0022] In a further embodiment, based on the feature vectors of patches at the same resolution level, a connection graph of patches at the same resolution level is constructed, including: Based on the cosine similarity of the feature vectors of patches at the same resolution level, a connection graph of patches at the same resolution level is constructed.

[0023] In a further embodiment, the graph index structure of the whole-slice pathological image is also pruned, including: For each patch, patch connections across resolution levels are retained, and based on the feature vectors of patches at the same resolution level, similarity scores with other patches are calculated. Patch connections with the highest similarity scores are retained until the sum of the number of patch connections across resolution levels and patch connections at the same resolution level reaches a preset number, thus obtaining the graph index structure of the pruned whole-slice pathological image.

[0024] In this embodiment, after constructing the initial graph, further optimization is needed to adapt to the requirements of GPU acceleration, ensuring that each node has a uniform out-degree *d*. Assuming the initial K-NN graph provides sufficient connectivity, the optimization objective is to limit the degree of each node while maintaining connectivity. Specifically, for each node, all its supporting connections are unconditionally retained because they are crucial for maintaining the hierarchical structure of WSI, and its semantic connections are selectively pruned. For example, in... Figure 2 In the example, for node A, its semantic connections are sorted according to similarity, and the least similar connections that can be reached by other paths are pruned until the total degree of the node (supporting connections + retained semantic connections) reaches a preset out-degree d. Once pruning is complete, no further adjustments are made to the connections of the node. This method preserves the information of the WSI hierarchy and optimizes the graph structure by unifying node degrees, enabling efficient parallel processing on GPUs.

[0025] In a further embodiment, the semantic information of the region to be queried includes: The query area is divided into multiple sub-regions of standard size by a sliding window, where the standard size is the size of a patch in the graph index structure of the whole-slice pathological image; In this embodiment, when performing region retrieval, the user-selected query region is first split into multiple scales using a sliding window method. The window length is set to 224x224 to ensure consistency with the optimal input dimension of the Uni encoder while maintaining spatial relationships, resulting in multiple query nodes. Semantic extraction is performed on each sub-region to obtain the semantic information of each sub-region, which is then used as the semantic information of the region to be queried.

[0026] In a further embodiment, based on the semantic information of the region to be queried and the graph index structure of the whole-slice pathological image, a candidate region with the highest similarity to the region to be queried is obtained, including: For the semantic information of each sub-region, an approximate nearest neighbor search is performed in the graph index structure of the whole-slice pathological image to obtain one or more patches similar to the sub-region; Using the spatial adjacency of one or more patches retrieved from each sub-region in the whole-slice pathological image, spatially adjacent patches are aggregated into one or more candidate regions. The similarity between all candidate regions and the query region is calculated, and the candidate region with the highest similarity to the query region is output.

[0027] In a further embodiment, a GPU-accelerated parallel search engine is used to perform an approximate nearest neighbor search on the semantic information of each sub-region in the graph index structure of the whole-slice pathological image to obtain one or more patches similar to the sub-region.

[0028] In a specific implementation, three lists are maintained to manage the search status: The result list R (stores the top k most similar nodes, with a length of k): This list is used to store the top k graph nodes in the index graph after graph optimization and pruning that are most similar to the feature vector of the query patch. Candidate list C (length p×d, where p is the number of nodes explored in each iteration and d is the degree of the graph): This list contains candidate graph nodes that may become nearest neighbors in the next iteration. Here, p represents the number of top nodes selected from the result list R in each iteration, and d is the uniform out-degree set for each node during the graph optimization and pruning phases. Visited list V (stores the indices of visited nodes): This list is used to record the indices of all graph nodes that have been visited during the search process to avoid duplicate calculations and infinite loops.

[0029] The following process is used to output the candidate region with the highest similarity to the query region: In step 1, the result list R is updated by finding the top k minimum distances for a given query from the union of itself and the current candidate list C; Step 2 involves traversing the neighbors of the p nearest nodes in R, marking them as visited in V, and collecting their unvisited neighbors to expand C; Step 3: Calculate the distances of these new candidates, compare them with the previous C, and keep only the nearest p×d nodes; The above process is repeated until R stops updating, ensuring efficient convergence to the top k most similar nodes.

[0030] After obtaining the query results for all query nodes, all results need to be aggregated into spatially coherent regions with diagnostic significance using a topology reorganization algorithm. Specifically, patches are clustered into spatially relevant regions through connected set analysis. The minimum bounding box covering patches with a coverage of 95% or more is determined and weighted by the region aggregation similarity score. The final output can preserve diagnostically relevant structures while filtering out false positives from fragmentation.

[0031] like Figure 3 As shown, WSI region retrieval is an end-to-end process consisting of three main stages, designed to efficiently find the pathological regions most similar to the user-specified query region from a massive WSI database. Specifically: Phase 1: Query Region Decomposition Figure 3 Left side (a) Input: The process begins with the user selecting a "QueryRegion" of any size and shape on WSI.

[0032] Processing: The "query region" is first divided into one or more standard-sized image patches (e.g., 224x224 pixels) using a sliding window. This step normalizes the irregular query region for subsequent processing.

[0033] Encoding: These segmented patches are then fed into a pre-trained deep learning "encoder" (and...). Figure 1 The same encoder is used in the index building phase to convert the visual information of each patch into a high-dimensional feature vector.

[0034] Output: The output of this stage is a "Batch Query", which consists of a set of feature vectors representing the semantic information of the entire query region.

[0035] Phase Two: Parallel GPU Accelerated Search Figure 2 Left side (b) Input: The "Batch Query" feature vector generated in the previous stage.

[0036] Processing: These query vectors are fed into a parallel search engine that utilizes GPU acceleration. For each vector in the batch query, the engine performs an efficient approximate nearest neighbor search within a pre-built graph-based index. The parallel computing power of the GPU allows the system to process search requests for all query patches simultaneously, significantly reducing retrieval time.

[0037] Output: The output of this stage is a series of independent image patches from one or more WSIs in the database that are most similar to the query patch.

[0038] Phase 3: Topology Recomposition Figure 2 Left side (c) part) Input: Scattered, independent best-match patches obtained from the search phase.

[0039] Processing: Since search results are based on individual patches, they may be spatially fragmented. To provide a diagnostically meaningful, coherent region, this invention performs "recomposition." This step analyzes the spatial adjacency of the retrieved patches in their original WSI, aggregating spatially adjacent patches into one or more candidate regions. The system calculates a comprehensive similarity score for these recombined regions, which considers not only the similarity of individual patches but also the integrity and spatial continuity of the region.

[0040] Output: The final "Region Retrieval Results". This consists of one or more spatially coherent pathological regions that are morphologically most similar to the user's original query region, effectively filtering out scattered and meaningless matching patches.

[0041] Figure 3 This fully demonstrates the entire process from user input of a query region, through three steps: decomposition, parallel search, and reorganization, to finally obtaining accurate and coherent pathological region matching results. This workflow design fully leverages the parallel computing power of GPUs and an optimized graph index structure, ensuring the efficiency and accuracy of the retrieval process.

[0042] A second embodiment of the present invention provides a hierarchical retrieval device for whole-section pathological images, such as... Figure 4 As shown, it includes: The graph index module constructs a graph index structure for whole-slice pathological images. The graph index structure of the whole-slice pathological images constructs connections at the same resolution level through semantic connections, and constructs cross-resolution connections through spatial relationships at different resolution levels. The acquisition module acquires the region to be queried; The query module obtains the candidate region with the highest similarity to the region to be queried based on the semantic information of the region to be queried and the graph index structure of the whole slice pathological image.

[0043] A third embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the hierarchical retrieval method for whole-slice pathological images described in the first embodiment of the present invention.

[0044] The fourth embodiment of the present invention provides a computer storage medium storing a computer program. When the computer program is executed by the processing, it implements the steps of the layered retrieval method for whole-slice pathological images described in the first embodiment of the present invention.

[0045] The same or similar labels correspond to the same or similar parts; The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A hierarchical retrieval method for whole-section pathological images, characterized in that, Includes the following steps: A graph index structure for whole-slice pathological images is constructed. The graph index structure of the whole-slice pathological images constructs connections at the same resolution level through semantic connections, and constructs cross-resolution connections through spatial relationships at different resolution levels. Get the area to be queried; Based on the semantic information of the region to be queried and the graph index structure of the whole-slice pathological image, the candidate region with the highest similarity to the region to be queried is obtained.

2. The hierarchical retrieval method for whole-section pathological images according to claim 1, characterized in that, The graph index structure for constructing whole-slice pathological images includes: All whole-slice pathological images are divided into non-overlapping patches using a sliding window; Semantic extraction is performed on each patch to obtain the feature vector of each patch; Based on the feature vectors of patches at the same resolution level, a connection graph of patches at the same resolution level is constructed; based on the spatial relationship of patches at different resolution levels, patch connections across resolution levels are constructed to obtain the graph index structure of the whole-slice pathological image.

3. The hierarchical retrieval method for whole-section pathological images according to claim 2, characterized in that, Based on the feature vectors of patches at the same resolution level, a connection graph of patches at the same resolution level is constructed, including: Based on the cosine similarity of the feature vectors of patches at the same resolution level, a connection graph of patches at the same resolution level is constructed.

4. The hierarchical retrieval method for whole-section pathological images according to claim 2 or 3, characterized in that, The graph index structure of the whole-slice pathological image is also pruned, including: For each patch, patch connections across resolution levels are retained, and based on the feature vectors of patches at the same resolution level, similarity scores with other patches are calculated. Patch connections with the highest similarity scores are retained until the sum of the number of patch connections across resolution levels and patch connections at the same resolution level reaches a preset number, thus obtaining the graph index structure of the pruned whole-slice pathological image.

5. The hierarchical retrieval method for whole-section pathological images according to claim 4, characterized in that, The semantic information of the region to be queried includes: The query area is divided into multiple sub-regions of standard size by a sliding window, where the standard size is the size of a patch in the graph index structure of the whole-slice pathological image; Semantic extraction is performed on each sub-region to obtain the semantic information of each sub-region, which is then used as the semantic information of the region to be queried.

6. The hierarchical retrieval method for whole-section pathological images according to claim 5, characterized in that, Based on the semantic information of the region to be queried and the graph index structure of the whole-slice pathological image, candidate regions with the highest similarity to the region to be queried are obtained, including: For the semantic information of each sub-region, an approximate nearest neighbor search is performed in the graph index structure of the whole-slice pathological image to obtain one or more patches similar to the sub-region; Using the spatial adjacency of one or more patches retrieved from each sub-region in the whole-slice pathological image, spatially adjacent patches are aggregated into one or more candidate regions. The similarity between all candidate regions and the query region is calculated, and the candidate region with the highest similarity to the query region is output.

7. The hierarchical retrieval method for whole-section pathological images according to claim 6, characterized in that, Using a GPU-accelerated parallel search engine, an approximate nearest neighbor search is performed on the semantic information of each sub-region in the graph index structure of the whole-slice pathological image to obtain one or more patches similar to the sub-region.

8. A hierarchical retrieval device for whole-section pathological images, characterized in that, include: The graph index module constructs a graph index structure for whole-slice pathological images. The graph index structure of the whole-slice pathological images constructs connections at the same resolution level through semantic connections, and constructs cross-resolution connections through spatial relationships at different resolution levels. The acquisition module acquires the region to be queried; The query module obtains the candidate region with the highest similarity to the region to be queried based on the semantic information of the region to be queried and the graph index structure of the whole slice pathological image.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, which, when executed by the processor, implements the steps of the hierarchical retrieval method for whole-slice pathological images according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by the processing, implements the steps of the hierarchical retrieval method for whole-slice pathological images according to any one of claims 1 to 7.

Images