Method and system for super large image hash index and topology retrieval

Abstract

The application provides a super large image hash index and topology retrieval method and system, aiming at the contradiction between high-dimensional floating point calculation power demand and long time consumption in mass pathological image retrieval, and adopts multi-scale feature extraction and cascade retrieval architecture.In offline database construction, the image block features are decoupled by a deep network and quantified into segmented structured hash codes;in online retrieval, a dynamic mask is generated according to the retrieval intention, coarse screening is carried out through the bottom mask XOR logic, then high-dimensional floating point feature accurate rearrangement is called, and the spatial microenvironment topology subgraph is constructed combining with the absolute physical coordinates of the block to carry out topology verification rearrangement.The application is mainly used for similar block retrieval of mass pathological sections in digital pathology, effectively reduces the memory and calculation power consumption, and improves the retrieval speed and accuracy of super large images.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of digital pathology and medical image information retrieval, and in particular to a method and system for ultra-large image hash indexing and topological retrieval. Background Technology

[0002] With the development of digital pathology, whole-field digital slices (WSI) have been widely used in clinical diagnosis and medical research. These ultra-large pathological images typically reach the billion-pixel level. In actual data processing and storage, a complete pathological image needs to be divided into tens of thousands or even hundreds of thousands of local patches and stored in a database.

[0003] In actual clinical image reading scenarios, when doctors select a rare cell or a specific pathological area and hope to search for similar cases in the hospital's database of tens of millions or even hundreds of millions of pathological images, the system faces significant technical bottlenecks.

[0004] On the one hand, existing deep learning retrieval systems typically extract high-dimensional floating-point feature vectors, such as 512-dimensional or 1024-dimensional ones. If the high-dimensional vector of the query image is directly used to calculate Euclidean distance or cosine similarity with hundreds of millions of vectors in the database, it will cause the "curse of dimensionality" and the computational load will be extremely large.

[0005] On the other hand, traditional linear brute-force scanning or conventional tree indexing takes a very long time when dealing with hundreds of millions of high-dimensional dense vectors, often requiring several minutes or even longer. This is completely unable to meet the "real-time / second-level" interaction requirements of doctors in clinical diagnosis.

[0006] Therefore, there is an urgent need for a method and system for ultra-large image hash indexing and topological retrieval to solve the problems existing in the current technology. Summary of the Invention

[0007] This invention provides a method and system for ultra-large image hash indexing and topological retrieval. It addresses the problem that existing technologies face an irreconcilable contradiction between the "huge demand for high-dimensional floating-point computing power" and the "low latency requirements of clinical practice" in the retrieval of massive medical image data, resulting in extremely slow retrieval speeds at the scale of hundreds of millions of image tiles and easy limitations due to hardware computing power bottlenecks.

[0008] The core technology of this invention is to completely abandon the brute-force comparison of global floating-point vectors and innovatively adopt the architecture of "multi-scale feature extraction combined with cascaded retrieval". It uses the structured segmented hash code generated by forced feature decoupling to perform ultra-fast coarse screening of the underlying mask XOR, and combines high-dimensional floating-point fine sorting with graph neural network spatial topology verification and rearrangement to achieve sub-second high-precision retrieval for massive image tiles of billions of pixels.

[0009] In a first aspect, the present invention provides a method for ultra-large image hash indexing and topological retrieval, the method comprising the following steps:

[0010] The process involves acquiring ultra-large images and segmenting them into image patches. Multidimensional floating-point features of the image patches are then extracted using a pre-defined deep neural network, and these features are forcibly decoupled into multiple sub-features with different semantic dimensions.

[0011] The sub-features are quantized into segmented binary hash codes, which are then concatenated to form a complete structured hash code. An in-memory hash index library containing the structured hash codes of all tiles is built in memory, and a feature database containing the multi-dimensional floating-point features and absolute physical coordinates of the corresponding tiles is built in the storage medium.

[0012] Receive query tiles and search intent, extract multi-dimensional floating-point features of the query tiles and generate query structured hash codes simultaneously, and generate corresponding dynamic mask vectors according to the search intent;

[0013] Perform the first-level search: Based on the dynamic mask vector, calculate the mask Hamming distance between the query structured hash code and each structured hash code in the memory hash index in the underlying logical operation, and roughly screen out the first candidate map tile set;

[0014] Perform the second-level search: retrieve the multi-dimensional floating-point features of each tile in the first candidate tile set from the feature database, calculate the feature similarity between the multi-dimensional floating-point features and the query multi-dimensional floating-point features, and refine the second candidate tile set.

[0015] Perform the third-level search: Based on the absolute physical coordinates, extract the spatial neighbor tiles of the query tile and the second candidate tile set to construct a micro-environment topology subgraph. Calculate the topological structure similarity based on the micro-environment topology subgraph, and then perform a weighted fusion of the feature similarity and topological structure similarity to output the final search results.

[0016] Furthermore, multi-dimensional floating-point features of image patches are extracted using a pre-defined deep neural network, and these multi-dimensional floating-point features are forcibly decoupled into multiple sub-features with different semantic dimensions, including:

[0017] The basic feature map is extracted using a shared backbone network. Then, by taking independent multi-task branches from different depth levels of the shared backbone network, the first sub-feature focusing on the macro-organizational structure, the second sub-feature focusing on the micro-local texture, and the third sub-feature focusing on the background base color are extracted respectively.

[0018] During the network model training phase, the deep neural network is optimized through a joint loss function, which includes a task loss to ensure feature discriminativeness, a hash quantization loss to reduce the discretization error of continuous features, and an orthogonal constraint loss to achieve forced decoupling at the mathematical level.

[0019] Furthermore, the orthogonal constraint loss is calculated as follows:

[0020] Calculate the inner product or cosine similarity between each pair of continuous feature vectors output by different independent multi-task branches, and construct a penalty term with the goal of making the inner product or cosine similarity between feature vectors of each branch approach zero, so as to force the features extracted by different branches to be orthogonal to each other in the feature space.

[0021] Furthermore, the sub-features are quantized into segmented binary hash codes, and a hard quantization function is used as the hash activation layer;

[0022] In the backpropagation process of network model training, in order to solve the problem of the non-differentiability of the hard quantized function, a pass-through estimator mechanism is used to approximate the gradient of the hard quantized function using a differentiable surrogate function; or, a smooth differentiable function with an inverse temperature scaling factor is used to approximate the hard quantized function, and the inverse temperature scaling factor is dynamically and monotonically increased during training for annealing.

[0023] Furthermore, a corresponding dynamic mask vector is generated based on the search intent, including:

[0024] Analyze the feature emphasis instructions contained in the search intent to determine the target feature segments that need to be focused on and the ignored feature segments that need to be blocked in the target search;

[0025] In the generated dynamic mask vector, the bits corresponding to the target feature segments are set to reserved logical values, and the bits corresponding to the ignored feature segments are set to mask logical values. The total bit length of the dynamic mask vector is consistent with the total bit length of the structured hash code.

[0026] Furthermore, in the underlying logical operations, the masked Hamming distance between the query structured hash code and each structured hash code in the in-memory hash index is calculated, including:

[0027] Perform a bitwise XOR operation between the query structured hash code and the structured hash code in the library to obtain the initial sequence of differences.

[0028] Perform a bitwise AND operation between the initial difference bit sequence and the dynamic mask vector, and use the mask logic value to clear and mask specific bits;

[0029] Perform bitwise statistical operations on the sequence result after the bitwise logical AND operation to obtain the masked Hamming distance.

[0030] Furthermore, spatial neighbor tiles of the queried tile and the second candidate tile set are extracted respectively to construct a micro-environment topology subgraph, including:

[0031] Using the absolute physical coordinates of the central target tile as the center, tiles within the neighborhood are delineated based on a preset physical pixel radius; if the number of valid tiles within the delineated area is lower than a preset completion threshold, the neighborhood is expanded outward using a nearest neighbor algorithm to complete the area, resulting in a preset number of spatial neighbor tiles.

[0032] The central target tile and its corresponding spatial neighbor tiles are abstracted as topological nodes, and the physical spatial connectivity between nodes is abstracted as topological edges to construct a micro-environment topological subgraph. In the construction process, the node features directly reuse the multi-dimensional floating-point features already existing in the feature database, and the edge weights are calculated by attenuation based on the asymmetric relative distance of the differences between the coordinates of the node centers in each dimension.

[0033] Furthermore, the edge weights are specifically obtained by mapping and decaying the asymmetric relative distance using an exponential decay kernel function that includes a truncation-to-zero mechanism;

[0034] Calculating topological similarity based on microenvironment topological subgraphs includes: using graph convolutional neural networks to perform message passing and fusion on node features in the microenvironment topological subgraphs, and generating graph-level embedding vectors representing local microenvironments through global pooling; calculating the feature distance between the graph-level embedding vectors corresponding to the query plots and the graph-level embedding vectors corresponding to the candidate plots as the topological similarity.

[0035] Furthermore, the feature similarity and topological similarity are weighted and fused to output the final retrieval results, including:

[0036] A first fusion weight is assigned to the feature similarity to characterize the matching degree of the cell's own morphology, and a second fusion weight is assigned to the topological similarity to characterize the matching degree of the surrounding microenvironment.

[0037] The comprehensive search score is calculated using a linear weighting formula. The second candidate image set is then globally reordered according to the comprehensive search score from high to low. The pathological images with the highest number of target images and related case information are selected as the final search results.

[0038] Secondly, the present invention provides a structured semantic hash index construction and multi-scale topological retrieval device for ultra-large images, comprising:

[0039] The offline database module is used to acquire ultra-large images and perform multi-scale slicing to obtain image patches. It extracts multi-dimensional floating-point features of the image patches through deep neural networks and decouples them into sub-features with different semantic dimensions. The sub-features are quantized and concatenated into segmented structured hash codes. At the same time, the structured hash codes are fully loaded into the memory hash index library, and the multi-dimensional floating-point features and absolute physical coordinates are stored in the feature database.

[0040] The online query parsing module is used to receive query tiles and search intents input by doctors, extract multi-dimensional floating-point features and structured hash codes of the query, and generate dynamic mask vectors based on the emphasis instructions.

[0041] A three-level cascaded retrieval engine module is used to execute the following steps in a cascaded manner: rapid coarse screening based on dynamic mask vectors and low-level bitwise operations using masked Hamming distance to obtain a first candidate tile set; precise rearrangement based on similarity calculation using high-dimensional multi-dimensional floating-point features to obtain a second candidate tile set; and...

[0042] Based on absolute physical coordinates, spatial neighbors are extracted to construct a micro-environment topology subgraph. Then, the topological similarity is calculated and the similarity is rearranged using an asymmetric exponential decay kernel function with a truncation and zeroing mechanism. Finally, the retrieval results are output by combining features and topological similarity.

[0043] The main contributions and innovations of this invention are as follows:

[0044] 1. Breaking through computing power bottlenecks and achieving ultra-fast retrieval of hundreds of millions of data points: This invention reduces the time-consuming process of traditional retrieval methods. High-dimensional floating-point computation is cleverly transformed into Low-level ultra-fast bit operations and High-dimensional floating-point computation (of which) This cascaded architecture reduces the retrieval time for tens of millions of image databases from minutes to milliseconds. At the scale of hundreds of millions of image tiles, the total retrieval time of this invention is only about 48 milliseconds, which is more than 260 times faster than traditional floating-point brute-force scanning, perfectly meeting the needs of clinical second-level interaction.

[0045] 2. Significantly reduces system memory usage and lowers hardware requirements: During the offline database construction phase, this invention transforms high-dimensional floating-point features into short binary hash codes using Locality Sensitive Hashing (LSH) or Deep Hashing networks. For example, 100 million 512-dimensional floating-point vectors would normally require approximately 200GB of memory, but after being converted to 128-bit hash codes, they only require about 1.5GB. This allows a typical single-machine server to fully load and complete retrieval operations on a scale of hundreds of millions of elements in memory.

[0046] 3. Compensating for Hash Quantization Errors, Recall Approaching Physical Limits: Traditional single-level hash retrieval suffers from severe feature information loss due to binary compression, resulting in low recall. This invention ensures a high recall baseline through a first-level rapid coarse screening based on Hamming distance, and introduces a second-level fine ranking step that retrieves high-dimensional vectors to calculate cosine similarity, successfully eliminating the precision truncation error caused by hash quantization. Experimental data shows that its recall rate reaches 91.5% in a database of hundreds of millions of images, almost perfectly approaching the theoretical limit of 92.2% for uncompressed floating-point scanning.

[0047] 4. Innovative Spatial Topology Verification: Surpassing Traditional Limits in Accuracy: Addressing the shortcomings of traditional retrieval schemes that rely solely on comparing cell morphology, leading to false positives, this invention extracts spatial neighbors based on tile physical coordinates in the third-level retrieval stage. It then uses a Graph Neural Network (GCN) to construct a pathological microenvironment topology subgraph for verification. This mechanism not only examines the similarity of the target cells themselves but also mandates that the topological microenvironments of the glands or tissue structures in which they reside must not differ too significantly. Experiments demonstrate that by eliminating similar cells in incorrect tissue environments, the retrieval accuracy and overall F1 score of this invention far exceed those of single-level hashing schemes and even surpass the accuracy limit of traditional floating-point brute-force scanning, thus aligning more closely with the clinical diagnostic thinking of pathologists.

[0048] Details of one or more embodiments of the present invention are set forth in the following drawings and description, so that other features, objects and advantages of the invention will be more readily understood. Attached Figure Description

[0049] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0050] Figure 1 This is a structural diagram of a large-scale image hash index and topology retrieval system according to an embodiment of the present invention;

[0051] Figure 2 This is a flowchart of the cascaded multi-scale retrieval process according to an embodiment of the present invention. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0053] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0054] The core idea of ​​this invention is to abandon direct global floating-point vector comparison and adopt a "multi-scale feature extraction combined with cascaded retrieval architecture". By combining low-level bitwise coarse screening with high-dimensional floating-point and spatial topology fine sorting, it solves the contradiction between high-dimensional computing power bottleneck and low latency requirements faced by billion-pixel ultra-large images when searching massive image tiles.

[0055] Example 1: A method for hash indexing and topological retrieval of ultra-large images

[0056] like Figure 1 As shown, this embodiment provides a specific retrieval method, which is divided into an offline database construction stage and an online retrieval stage. The specific steps are as follows:

[0057] S1, Offline Database Building Stage (Index Construction)

[0058] S11. Image slicing and multi-scale feature extraction and decoupling:

[0059] Acquire a one-billion-pixel full-field-of-view (WSI) pathological image and segment it into image patches of uniform pixel size (e.g., uniformly set to 256×256 pixels) at preset magnifications (e.g., 20x and 40x magnification). The 40x patch is used to extract microscopic features such as cell nucleus details, while the 20x patch is used to extract macroscopic features such as glandular tissue structures.

[0060] The aforementioned image patches are input into a pre-trained deep neural network. Regarding feature dimensions and mapping relationships, taking the preferred ResNet-50 as an example, after the image is input into the network, the initial output after the pooling layer is a high-dimensional feature of 2048 dimensions. Subsequently, this initial feature is dimensionality-reduced by a fully connected layer (FC), outputting a 512-dimensional Float32 floating-point feature vector for subsequent fine-tuning (this floating-point vector will be stored on disk for the second-level precise rearrangement). Simultaneously, this embodiment employs a physical layer decoupling architecture combining a shared backbone feature pyramid with independent multi-task branch heads, drawing branches from different depth levels of the network to forcibly decouple three types of features:

[0061] 1. Global macro-branch F1: It is derived from the deep layers of the network (such as Stage 4 of ResNet) and outputs a continuous feature vector that reflects the organizational structure.

[0062] 2. Local micro-branch F2: It is derived from the middle layer of the network (such as Stage 3 of ResNet) and outputs a continuous feature vector that is sensitive to the cell nucleus boundary and morphology.

[0063] 3. Coloring base branch F3: It is drawn from the shallow layers of the network (such as Stage 1 or 2 of ResNet) and outputs a continuous feature vector that preserves pixel-level color and basic texture information.

[0064] To achieve forced decoupling at the mathematical level of backpropagation, the network's total loss function... It consists of three weighted parts:

[0065]

[0066] in To ensure the task loss for feature discrimination, Add a hash quantization loss (to smoothly binarize floating-point numbers into 0 and 1 (or -1 and 1), and reduce the loss of precision during the discretization of continuous features). To ensure that the semantics of the three hash segments do not overlap, an orthogonal constraint loss is applied. - These are preset weighting coefficients or hyperparameters used to adjust the proportion of each loss term in the total loss. Let... , , The orthogonal constraint loss formula is as follows: (The feature vectors output from the three branches are given above.)

[0067]

[0068] This mechanism forces the features of each branch to be orthogonal to each other in space, preventing feature overlap.

[0069] S12. Segmented hash quantization and binarization:

[0070] During the hash quantization stage, for a single scale, the 512-dimensional vector after dimensionality reduction can be quantized into a 128-bit binary hash code through an activation layer (stored in memory for the first-stage rapid coarse screening). In the preferred embodiment of the multi-scale fusion of the present invention, the system concatenates the 512-dimensional floating-point feature vectors extracted from the 40x and 20x blocks respectively, and fuses them into a 1024-dimensional floating-point feature vector; then, the 1024-dimensional vector is quantized into a structured binary hash code with a total length of 256 bits.

[0071] Specifically, the 256-bit hash code is generated by binarizing the decoupled features (F1-F3) through a hash activation layer (such as the Sign function), and includes a macro hash segment of the first 128 bits (Hash_Macro(128bit)), a micro hash segment of the middle 96 bits (Hash_Micro(96bit)), and a color hash segment of the last 32 bits (Hash_Color(32bit)).

[0072] To address the gradient blocking problem caused by the standard Sign function having a derivative of 0, a pass-through estimator (STE) mechanism is employed during the training phase: hard quantization is used during forward propagation. Backpropagation uses a differentiable surrogate function (such as Hardtanh) to approximate the gradient (the gradient is set to 0 when the absolute value of the feature is greater than 1, and otherwise to 1).

[0073] Preferably, a "soft quantization approximation" strategy can also be adopted to design the hash activation layer function as follows: ,in This is the inverse temperature scaling factor. During the initial training phase... When the value is small, the gradient is smoothly backpropagated; as the training cycle increases, the system dynamically and monotonically increases the gradient through the annealing algorithm. The value of . When hour, The curve will infinitely approximate the step Sign(V) function, achieving a smooth transition to hard quantization.

[0074] To further support the above scheme and quantify the loss item This is done by penalizing the mean square error between the absolute value of the feature vector V and the all-1 vector, forcing the continuous feature values ​​of the output to move closer to the two extremes (+1 and -1).

[0075] S13, Constructing a Cascaded Database:

[0076] The generated 256-bit structured hash code is fully stored in an in-memory hash index library; the 512-dimensional Float32 high-dimensional floating-point features after dimensionality reduction from the fully connected layer of the network and the absolute physical coordinates of the source image are stored in a disk feature database. Taking 100 million image tiles as an example, the 512-dimensional Float32 vector requires approximately 200GB of disk space, while after being converted into hash code, it only requires approximately 1.5GB of memory to be fully loaded, greatly reducing memory usage.

[0077] S2, Online Search Stage (Cascading Search Architecture)

[0078] S21. Query Feature Processing and Dynamic Mask Generation:

[0079] like Figure 2 As shown, when a doctor selects a rare cell patch as the query input and specifies the focus of the search, the system extracts the multi-dimensional floating-point features of the query and simultaneously generates a 256-bit query hash code. The front end generates a corresponding 256-bit dynamic mask vector based on the search intent.

[0080] Specific mask mapping rules are illustrated below:

[0081] 1. Default full-view search: All 256 bits of the mask are set to 1.

[0082] 2. Focus only on microscopic cells: Set the first 128 bits (macroscopic) of the mask to 0, the middle 96 bits (microscopic) to 1, and the last 32 bits (staining) to 0.

[0083] 3. Cross-hospital anti-color difference retrieval: The first 128 bits and the middle 96 bits of the mask are all set to 1, and only the last 32 bits are set to 0, in order to mask the staining differences of slides from different hospitals.

[0084] S22, First-stage rapid coarse screening based on Hamming distance:

[0085] In hash libraries with tens of millions to hundreds of millions of hashes, the Hamming distance of the mask is calculated through underlying logical gate operations, using the following formula:

[0086]

[0087] This instantly filters out the first set of candidate tiles with the smallest Hamming distance (e.g., Top-5000).

[0088] S23, Second-level precise rearrangement based on high-dimensional floating-point features:

[0089] Using the IDs selected through coarse screening, retrieve 512-dimensional floating-point feature vectors from the disk feature database, calculate the cosine similarity between them and the queried multi-dimensional floating-point features, and filter them into the second candidate tile set (such as Top-100).

[0090] S24. Construction and rearrangement of the third-level spatial topology graph:

[0091] 1. Extract spatial neighbors: using absolute physical coordinates of the tiles Using a circle as the center, a physical pixel radius R (e.g., 1024 pixels) is set to define the neighborhood. If the number of effective patches is insufficient, the KNN algorithm is used to expand outwards to fill the gap, extracting N spatial neighbor patches. Preferably, N=8 represents extracting the 8 patches (Moore neighbors) surrounding the immediate "nine-square grid" to capture extremely local intercellular matrix interactions; N=24 represents extracting neighbors extending outwards by two layers to capture more macroscopic glandular or cancer nest edge topology.

[0092] 2. Constructing a microenvironment topology subgraph The graph tiles are treated as nodes V (edges E represent their physical spatial connectivity in the original graph), and the node features directly reuse existing 512-dimensional floating-point features from disk. Edge weights between nodes... Based on the center coordinates (x) of the two plots i ,y i ) and (x j ,y j The Euclidean distance between them is calculated using the Gaussian radial basis function (RBF kernel) for mapping attenuation:

[0093]

[0094] Among them, bandwidth parameter The decay rate for controlling spatial influence is preferably set to twice the side length of the tile. When the preset connectivity conditions between nodes are met, the weight decay is calculated using the aforementioned exponential function. When the conditions are not met or the effective influence range is exceeded, the weight is directly truncated and output as 0. The preset connectivity conditions refer to: node j being located within a circular physical region centered on node i with a preset pixel radius R; or, node j belonging to the K nearest neighbors of node i in spatial coordinates. In a specific embodiment, the preset pixel radius R is preferably 1024 pixels, and the number of nearest neighbors K is preferably 8 or 24.

[0095] 3. Topology Verification and Weighted Output: A two-layer Graph Convolutional Neural Network (GCN) is used for message passing and global mean pooling to generate a graph-level embedding vector representing the microenvironment. Since the initial node features (512 dimensions) already contain rich deep semantics, the role of GCN is simply to perform message passing and integrate information from surrounding neighbors. The first layer maps the dimensions from 512 to 256, and the second layer maps them to 128. After GCN processing, global mean pooling is performed on the new features of all nodes in the subgraph to obtain a 128-dimensional "graph embedding vector" representing the entire microenvironment.

[0096] Calculate topological similarity Then, the similarity with the second-level floating-point features. Perform linear weighted fusion calculation to obtain the overall score:

[0097]

[0098] in, The floating-point cosine similarity of the central patch itself obtained at the second level; This represents the spatial topological similarity obtained at the third level.

[0099] In a preferred embodiment, the weighting parameter The value range is [0.5, 0.7] (preferred). This indicates that the system prioritizes morphological matching of the target cell (60% weight) while also forcibly assessing the similarity of the microenvironment topology (40% weight).

[0100] S3. Finally, output the top-10 search results with the highest scores.

[0101] Example 2: A Large-Scale Image Hash Index and Topology Retrieval System

[0102] This embodiment provides a system corresponding to the method described in Embodiment 1. This system is deployed on a computing device equipped with a CPU and GPU accelerator (such as a single server equipped with an Intel Xeon CPU and an NVIDIA RTX 3090 GPU), and includes:

[0103] Offline database construction module: Configured to input billion-pixel pathological WSI slices into a multi-branch deep network for multi-scale feature extraction and orthogonal decoupling. This module contains an in-memory hash index library and a disk feature database. The former resides entirely in the system's running memory and is used to store segmented structured hash codes with a very small footprint (only about 1.5GB for billions of data points); the latter relies on a large-capacity disk to store high-dimensional floating-point features and absolute physical coordinates.

[0104] Online query parsing module: configured to receive rare cell patches and retrieval operation instructions initiated by user terminals, extract query features and map them to generate dynamic mask vectors.

[0105] The three-level cascaded retrieval engine module is the core computing hub of the system. This engine first calls the CPU's low-level assembly instructions to perform Masked XOR and POPCOUNT operations to complete the first level of massive coarse screening; then it calls the GPU's computing power to retrieve high-dimensional floating-point vectors from the disk to perform the second level of cosine similarity fine ranking; finally, it establishes a micro-environment topology subgraph based on the absolute physical coordinates on the disk, calculates topological similarity through an embedded two-layer graph convolutional neural network module to complete the third level of rearrangement, and returns high-precision, spatially interpretable case results to the front end.

[0106] Example 3

[0107] To objectively verify the effectiveness of the proposed "cascaded binary hash index and spatial topology rearrangement" method in the retrieval of ultra-large pathological images, this embodiment conducted comparative experiments on the same hardware platform (a single server equipped with an Intel Xeon CPU and an NVIDIA RTX 3090 GPU) for pathological tile libraries of two scales: 10 million and 100 million.

[0108] 1. Baseline Explanation:

[0109] Method A (Traditional Linear Brute-Force Scan): Directly uses a 512-dimensional Float32 floating-point vector to perform global cosine similarity calculation on the entire database, and selects the Top-10 based on the score. (Industry-standard practice, with the highest accuracy, but extremely slow).

[0110] Method B (Traditional Single-Level LSH Hash Retrieval): High-dimensional features are mapped to 128-bit hash codes using conventional LSH, and the Top-10 are directly output based solely on Hamming distance calculation. (Fast, but extremely low accuracy due to lack of spatial verification and floating-point compensation).

[0111] The method of this invention (three-level cascaded topology search): first level 256-bit segmented hash mask coarse screening (Top-1000) → second level 512-dimensional floating-point fine sorting (Top-100) → third level WSI spatial topology graph (GCN) verification and rearrangement (final Top-10).

[0112] 2. Performance Comparison Experiment Results Table:

[0113] Test task: Initiate a single query for "rare signet ring cells" in image libraries of different sizes and retrieve the top-10 most similar results.

[0114] Evaluation metrics: total retrieval time (Time), recall (Recall@10), precision (Precision@10), and overall F1 score (F1-Score). Results are shown in Table 1.

[0115] Table 1

[0116]

[0117] As can be seen, this invention has achieved groundbreaking experimental results in both speed and accuracy:

[0118] 1. Break through the computing power bottleneck and achieve "real-time / second-level" retrieval at the scale of hundreds of millions.

[0119] Data Performance: With hundreds of millions of data points, Method A's processing time increased linearly and explosively to 12800ms (approximately 13 seconds), completely unsuitable for real-time clinical interactions by doctors. In contrast, this invention, due to its first-level low-level bitwise operations filtering out 99.99% of invalid comparisons, requires only 48ms in total processing time. This demonstrates that this invention is more than 260 times faster than traditional floating-point brute-force scanning, truly achieving millisecond-level retrieval of massive image patches containing billions of pixels.

[0120] 2. It mitigates hash quantization errors, bringing recall rate close to physical limits.

[0121] Data Performance: Method B (traditional hashing) sacrifices accuracy for speed, with a recall rate of only 65.1%, making it highly susceptible to missing rare cases. In contrast, this invention achieves a recall rate of 91.5% in a database of hundreds of millions. This demonstrates that this invention successfully compensates for the hash quantization error caused by coarse screening through "second-level high-dimensional floating-point fine sorting," making its recall rate almost perfectly approach the theoretical limit of 92.2% for uncompressed floating-point scanning (Method A).

[0122] Advantage 3: Introducing spatial topology verification, accuracy surpasses traditional upper limits.

[0123] Data Performance: Traditional method A easily identifies false positives—cells that look similar but are located in non-cancerous areas—resulting in an accuracy rate of only 87.5%. This invention achieves an accuracy rate of 95.2% on datasets with hundreds of millions of records, with an overall F1 score of 93.3%. This demonstrates that this invention utilizes a third-level "GCN spatial topology verification," where the system not only compares the target cell itself but also rigorously examines the topological structure of its surrounding microenvironment. Experimental data proves that the retrieval results of this invention not only surpass traditional hashing schemes but also exceed the accuracy limit of traditional floating-point brute-force scanning, and its retrieval logic is more aligned with the actual diagnostic thinking of human pathologists.

[0124] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The above embodiments are merely illustrative of several implementations of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the appended claims.

Claims

1. A method for hash indexing and topological retrieval of ultra-large images, characterized in that, Includes the following steps: A very large image is acquired and sliced ​​to obtain image patches. Multidimensional floating-point features of the image patches are extracted through a preset deep neural network, and the multidimensional floating-point features are forcibly decoupled into multiple sub-features with different semantic dimensions. The sub-features are quantized into segmented binary hash codes, which are then concatenated to form a complete structured hash code. A memory hash index library containing the structured hash codes of all the maps is constructed in memory, and a feature database containing the multi-dimensional floating-point features and absolute physical coordinates of the corresponding maps is constructed in the storage medium. Receive query plots and retrieval intent, extract the query multidimensional floating-point features of the query plots and synchronously generate query structured hash codes, and generate corresponding dynamic mask vectors according to the retrieval intent; Perform the first-level search: Based on the dynamic mask vector, calculate the mask Hamming distance between the query structured hash code and each structured hash code in the memory hash index in the underlying logical operation, and roughly screen out the first candidate map tile set; Perform the second-level search: retrieve the multi-dimensional floating-point features of each tile in the first candidate tile set from the feature database, calculate the feature similarity between the feature features and the queried multi-dimensional floating-point features, and refine the second candidate tile set; Perform the third-level search: Based on the absolute physical coordinates, extract the spatial neighbor tiles of the query tile and the tile in the second candidate tile set to construct a micro-environment topology subgraph. Calculate the topological structure similarity based on the micro-environment topology subgraph, and perform a weighted fusion of the feature similarity and the topological structure similarity to output the final search result. Specifically, the spatial neighbor tiles of the query tile and the second candidate tile set are extracted to construct a micro-environment topology subgraph, including: Using the absolute physical coordinates of the central target tile as the center, tiles within the neighborhood are delineated based on a preset physical pixel radius; if the number of valid tiles within the delineated area is lower than a preset completion threshold, the neighborhood is expanded outward using a nearest neighbor algorithm to complete the area, thereby obtaining the preset number of spatial neighbor tiles. The central target tile and its corresponding spatial neighbor tiles are abstracted as topological nodes, and the physical spatial connectivity between nodes is abstracted as topological edges to construct the micro-environment topological subgraph. In the construction process, the node features directly reuse the multi-dimensional floating-point features already existing in the feature database, and the edge weights are calculated by attenuation based on the asymmetric relative distance of the differences between the coordinates of the center of the nodes in each dimension.

2. The ultra-large image hash indexing and topological retrieval method as described in claim 1, characterized in that, The image patches are extracted using a pre-defined deep neural network, and these multidimensional floating-point features are forcibly decoupled into multiple sub-features with different semantic dimensions, including: The basic feature map is extracted using a shared backbone network. Then, by taking independent multi-task branches from different depth levels of the shared backbone network, the first sub-feature focusing on the macro-organizational structure, the second sub-feature focusing on the micro-local texture, and the third sub-feature focusing on the background base color are extracted respectively. During the network model training phase, the deep neural network is optimized using a joint loss function, which includes a task loss to ensure feature discriminativeness, a hash quantization loss to reduce continuous feature discretization error, and an orthogonal constraint loss to achieve forced decoupling at the mathematical level.

3. The ultra-large image hash indexing and topological retrieval method as described in claim 2, characterized in that, The orthogonal constraint loss is calculated as follows: Calculate the inner product or cosine similarity between each pair of continuous feature vectors output by different independent multi-task branches, and construct a penalty term with the goal of making the inner product or cosine similarity between the feature vectors of each branch approach zero, so as to force the features extracted by different branches to be orthogonal to each other in the feature space.

4. The ultra-large image hash indexing and topological retrieval method as described in claim 2, characterized in that, The sub-features are quantized into segmented binary hash codes, and a hard quantization function is used as the hash activation layer. In the backpropagation process of training the network model, to solve the problem of the non-differentiability of the hard quantized function, a pass-through estimator mechanism is used to approximate the gradient of the hard quantized function using a differentiable surrogate function; or, a smooth differentiable function with an inverse temperature scaling factor is used to approximate the hard quantized function, and the inverse temperature scaling factor is dynamically and monotonically increased during training for annealing.

5. The ultra-large image hash indexing and topological retrieval method as described in claim 1, characterized in that, Generate a corresponding dynamic mask vector based on the search intent, including: The feature emphasis instructions contained in the search intent are analyzed to determine the target feature segments that need to be focused on and the ignored feature segments that need to be blocked in the target search. In the generated dynamic mask vector, the bits corresponding to the target feature segment are set to reserved logical values, and the bits corresponding to the ignored feature segment are set to mask logical values. The total bit length of the dynamic mask vector is consistent with the total bit length of the structured hash code.

6. The ultra-large image hash indexing and topological retrieval method as described in claim 5, characterized in that, The underlying logical operations calculate the masked Hamming distance between the query structured hash code and each structured hash code in the memory hash index, including: Perform a bitwise XOR operation between the query structured hash code and the structured hash code in the library to obtain the initial difference bit sequence; Perform a bitwise AND operation between the initial difference bit sequence and the dynamic mask vector, and use the masking logic value to clear and mask specific bits; The mask Hamming distance is obtained by performing bitwise statistical operations on the sequence result after the bitwise logical AND operation.

7. The ultra-large image hash indexing and topological retrieval method as described in claim 1, characterized in that, The edge weights are specifically obtained by mapping and decaying the asymmetric relative distance using an exponential decay kernel function that includes a truncation-to-zero mechanism. Calculating topological similarity based on the microenvironment topological subgraph includes: using a graph convolutional neural network to perform message passing and fusion on the node features in the microenvironment topological subgraph, and generating a graph-level embedding vector representing the local microenvironment after global pooling; calculating the feature distance between the graph-level embedding vector corresponding to the query graph patch and the graph-level embedding vector corresponding to the candidate graph patch as the topological similarity.

8. The ultra-large image hash indexing and topological retrieval method as described in claim 1, characterized in that, The feature similarity and the topological structure similarity are weighted and fused to output the final search results, including: Assign a first fusion weight to the feature similarity to characterize the matching degree of the cell's own morphology, and assign a second fusion weight to the topological similarity to characterize the matching degree of the surrounding microenvironment; A comprehensive search score is calculated using a linear weighting formula. The second candidate image set is then globally rearranged according to the comprehensive search score from high to low. The pathological images with the highest number of target images and related case information are selected as the final search results.

9. A system for ultra-large image hash indexing and topological retrieval, characterized in that, The method for implementing the ultra-large image hash indexing and topological retrieval method according to any one of claims 1-8 includes: The offline database construction module is used to acquire ultra-large images and perform multi-scale slicing to obtain image patches. It extracts multi-dimensional floating-point features of the image patches through a deep neural network and decouples them into sub-features with different semantic dimensions. The sub-features are quantized and concatenated into segmented structured hash codes. At the same time, the structured hash codes are fully loaded into the memory hash index library. The multi-dimensional floating-point features and absolute physical coordinates are stored in the feature database. The online query parsing module is used to receive query tiles and search intents input by doctors, extract multi-dimensional floating-point features and structured hash codes of the query, and generate dynamic mask vectors based on the emphasis instructions. A three-level cascaded retrieval engine module is used to sequentially execute: rapid coarse screening based on the dynamic mask vector and the mask Hamming distance of the underlying bit operations to obtain a first candidate tile set; precise rearrangement based on the similarity calculated by retrieving the high-dimensional multi-dimensional floating-point features to obtain a second candidate tile set; and Based on absolute physical coordinates, spatial neighbors are extracted to construct a micro-environment topology subgraph. Then, the topological similarity is calculated and the similarity is rearranged using an asymmetric exponential decay kernel function with a truncation and zeroing mechanism. Finally, the retrieval results are output by combining features and topological similarity.

Images