Method and system for extracting region of interest from pathological whole slide image of lung disease

Abstract

The application discloses a lung disease pathological whole-section image interest region extraction method and system, relates to the extraction of an interest region, and aims to solve the technical problem of low extraction accuracy of the interest region in the prior art. The method comprises the following steps: acquiring lung image data and calculating the semantic similarity score and the visual feature vector of each block; obtaining an optimal threshold value according to the semantic similarity score and screening out an index set; performing clustering processing on the visual feature vectors of the blocks in the index set, selecting the block with the highest semantic similarity score in each cluster as a representative block, and obtaining the index set; based on the index set, extracting the feature matrix of each representative block, and backtracking the index to the spatial coordinate positions of the representative blocks in the corresponding lung image data to obtain the interest region. The interest region is filtered and identified through adaptive thresholding according to the semantic similarity score, and the selected interest region retains the heterogeneous distribution of different infiltrative features in the section.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology and relates to the extraction of regions of interest, particularly to a method and system for extracting regions of interest from whole-section images of pathological pulmonary diseases. Background Technology

[0002] Digital pathology is an emerging discipline that digitizes and informs the examination of tissue sections under a traditional optical microscope. It is considered a key supporting technology for the transformation of pathology from experience-based medicine to precision medicine. Its core data carrier—whole-slice images—is digitized using high-resolution scanners to create ultra-large-scale medical images at the gigapi scale. Whether in traditional manual slide reading or emerging computer-aided diagnostic processes, the identification and localization of regions of interest (ROIs) are indispensable core steps: pathologists need to quickly locate diagnostically valuable areas (such as tumor infiltration foci, special differentiation patterns, and vascular invasion) within massive amounts of tissue structures; while in artificial intelligence processing, deep learning models are often limited by computational resources and cannot directly process gigapipixel-level images, requiring the ROI extraction step to narrow the analysis scope to a computable scale.

[0003] For example, a typical lung adenocarcinoma pathology slide, when magnified 40x, can generate an image of approximately 100,000 x 100,000 pixels, containing 5,000 to 50,000 blocks. However, a large number of these blocks are non-diagnostic areas (such as normal alveolar tissue, fibrous interstitium, blood vessels, necrotic areas, and technical artifacts). Pathologists need to repeatedly browse, locate, and label these areas at different magnifications, which consumes a significant amount of time. Furthermore, the selection of regions of interest (ROIs) is subjectively heterogeneous due to the physician's personal experience, professional background, and cognitive preferences. Therefore, this labor-intensive labeling method faces serious challenges to sustainability in the context of big data dataset construction and downstream AI-driven diagnostic tasks. Thus, providing a deep learning-based method for automatic RIO extraction is particularly important.

[0004] Patent application number 202111255965.1 discloses a method, electronic device, and storage medium for extracting the region of interest (ROI) of the aorta. The method involves first segmenting the acquired medical image to obtain a lung mask image; then, based on the lung mask image, obtaining the lower boundary position information of the lung region and the coordinate information of the minimum horizontal bounding rectangle of the lung region; next, based on the lower boundary position information of the lung region, obtaining the physical distance information from the starting and ending image layers of the medical image to the lower boundary of the lung region; then, based on the physical distance information from the starting and ending image layers of the medical image to the lower boundary of the lung region, obtaining the upper and lower boundary position information of the corresponding aortic ROI; and finally, extracting the aortic ROI based on the coordinate information of the minimum horizontal bounding rectangle and the upper and lower boundary position information of the aortic ROI. This method can automatically and accurately extract the aortic ROI with low cost and high efficiency.

[0005] The aforementioned patent application focuses on identifying and extracting regions of interest (ROIs) from the pulmonary aorta. However, the extraction of ROIs for lung adenocarcinoma (especially invasive lung adenocarcinoma) differs. Invasive lung adenocarcinoma can present five main histological growth patterns: lepidic, acinar, papillary, micropapillary, and solid. A key pathological fact is that approximately 70%-80% of invasive lung adenocarcinomas simultaneously contain two or more histological patterns. Pathology reports must record the area percentage of each pattern in 5% increments. For example, a report might state "predominantly acinar (60%), with lepidic (20%) and micropapillary (20%) components." Different patterns have significantly different prognostic significance (e.g., micropapillary and solid patterns indicate greater invasiveness and a worse prognosis). Therefore, in addition to the dominant pattern that occupies the most significant visual position (usually the largest and most morphologically typical region), minor components with smaller areas but significant prognostic value should not be ignored when labeling regions of interest (ROIs). ROI extraction should be able to capture this intratumoral heterogeneity proportionally. Furthermore, the distribution of similarity scores across different slices also exhibits significant cross-sample heterogeneity; that is, some slices have approximately uniform score distributions, while others show severely skewed distributions (long-tailed or bimodal distributions). This instability in distribution characteristics also makes fixed threshold strategies (such as uniformly setting "similarity ≥ 0.5" as a screening criterion) lack robustness in practical applications—the same threshold may lead to drastically different screening results on different slices, making it difficult to guarantee cross-sample consistency and comparability. Summary of the Invention

[0006] The purpose of this invention is to address the problems of low accuracy in extracting regions of interest (ROIs) from lung adenocarcinoma slice images, such as the tendency to overlook minor components with small areas but significant prognostic value, and the inconsistent screening results on different slices due to fixed thresholds. This invention provides a method and system for extracting ROIs from whole-slice pathological images of lung diseases.

[0007] To achieve the above objectives, the present invention specifically adopts the following technical solution:

[0008] A method for extracting the region of interest from a whole-section pathological image of a lung disease includes the following steps:

[0009] Step 1: Acquisition of lung image data;

[0010] Obtain lung image data, and generate a semantic similarity score based on the i-th block in the lung image data. Visual representation vectors ;

[0011] Step 2: Filter the index set;

[0012] Based on semantic similarity score The optimal threshold is obtained by employing adaptive threshold filtering, percentile thresholding, bimodal detection, or entropy-based threshold selection, and an index set is then selected based on the optimal threshold. ;

[0013] Step 3: Feature space clustering;

[0014] For index set Visual representation vector of the middle block Perform clustering to obtain the cluster affiliation label for each block. and the set of member indexes for each cluster , ;

[0015] Step 4: Extract the region of interest;

[0016] The block with the highest semantic similarity score in each cluster is selected as the representative block to obtain the index set. Based on index set Extract the feature matrix of each representative block The region of interest is obtained by tracing back to the spatial coordinates of representative blocks in the corresponding lung image data through indexing.

[0017] in, This represents the semantic similarity score of the i-th block. Let represent the visual representation vector of the i-th block. This represents the cluster number to which the i-th block belongs. Indicates the first Cluster number, Indicates the number of cluster numbers. Indicates belonging to the index set And the cluster number it belongs to is A set of blocks.

[0018] Furthermore, in step 2, when obtaining the optimal threshold using the adaptive threshold filtering method, the specific steps of the adaptive threshold filtering method are as follows:

[0019] Step 2-1: Histogram construction;

[0020] semantic similarity score The score range was divided into 256 intervals, and the frequency of each interval was counted to form a normalized histogram. , ,and ;

[0021] Step 2-2: Calculation of cumulative amount;

[0022] For each possible threshold t, calculate the cumulative weight of the classes below the threshold. The cumulative mean of classes below the threshold and all mean ;

[0023] Steps 2-3: Calculate the inter-class variance;

[0024] Based on the cumulative weight of classes below the threshold The cumulative mean of classes below the threshold and all mean Calculate the inter-class variance ;

[0025] Steps 2-4: Determining the optimal threshold;

[0026] Based on inter-class variance Determine the optimal threshold ;

[0027] Steps 2-5: Add constraints;

[0028] For the optimal threshold Add a lower bound constraint based on the mean and standard deviation to obtain the final threshold. ;

[0029] Steps 2-6: Filter the index set;

[0030] Select block i that meets the following criteria: All blocks i that satisfy the conditions constitute a set. ;

[0031] in, This represents the semantic similarity score of the i-th block.

[0032] Furthermore, in step 2-2, the cumulative weight of classes below the threshold... The calculation formula is:

[0033] ;

[0034] Cumulative mean of classes below the threshold The calculation formula is:

[0035] ;

[0036] All Means The calculation formula is:

[0037] ;

[0038] In steps 2-3, the variance between classes The calculation formula is:

[0039] ;

[0040] In steps 2-4, the optimal threshold The calculation formula is:

[0041] ;

[0042] In steps 2-5, the final threshold The calculation formula is:

[0043] ;

[0044] in, This represents the normalized histogram of the nth interval. Indicates the threshold. This represents the mean similarity score of all M blocks. This represents the standard deviation of the similarity scores for all M blocks.

[0045] Furthermore, in step 3, the specific steps of feature space clustering are as follows:

[0046] Step 3-1: Normalize the feature matrix;

[0047] index set Visual representation vector of the corresponding block Combine them into a feature matrix, perform row-by-row L2 normalization on the feature matrix, and project the normalized feature vectors onto the unit hypersphere.

[0048] Step 3-2: Configure clustering parameters;

[0049] Configure clustering parameters, including the number of blocks in a cluster. Initialization strategy, distance metric, batch size, maximum number of iterations, convergence tolerance;

[0050] Step 3-3: Iterative optimization;

[0051] The MiniBatch K-Means algorithm is used to perform an iterative process until convergence or the maximum number of iterations is reached, to obtain the final centroid. ;

[0052] Steps 3-4: Global cluster allocation;

[0053] Using the final center of mass Perform cluster assignment on all M samples in the candidate set and output the index set. Cluster affiliation label for each block and the member index set of each cluster ;

[0054] in, Indicates the first The centroid vector of each cluster, This represents the coordinate indices of all members contained in the k-th cluster.

[0055] Furthermore, in step 3-1, the calculation formula for L2 normalization is:

[0056] ;

[0057] Projecting the normalized feature vectors onto the unit hypersphere, the squared Euclidean distance between any two normalized feature vectors has a monotonic mapping relationship with their cosine similarity, i.e.:

[0058] ;

[0059] In step 3-2, the clustering parameters are configured as follows:

[0060] Cluster number : Set to the specified target number of samples N;

[0061] Initialization strategy: K-Means++;

[0062] Distance metric: Euclidean distance;

[0063] Batch size: min(1024, M);

[0064] Maximum number of iterations: 300;

[0065] Convergence tolerance: 10 -4 ;

[0066] In step 3-3, the specific steps of iterative optimization are as follows:

[0067] Step 3-3-1: Initialization;

[0068] Selecting K initial centroids using the K-Means++ strategy ;

[0069] Step 3-3-2: Random batch sampling;

[0070] From the index set A subset of size q is randomly selected from the set of corresponding feature vectors. ;

[0071] Step 3-3-3: Cluster allocation;

[0072] For each sample in the batch , , sample Assign to the cluster corresponding to the nearest centroid:

[0073] ;

[0074] Step 3-3-4: Centroid incremental update;

[0075] For each cluster The centroid is updated based on the samples assigned to this cluster in the current batch;

[0076] ;

[0077] Step 3-3-5: Convergence determination;

[0078] If there exists a centroid whose displacement norm is not less than the convergence tolerance, and the maximum number of iterations has not been reached, then return to step 3-3-1 and reselect K initial centroids; if the displacement norm of all centroids is less than the convergence tolerance, or the maximum number of iterations has been reached, then terminate the iteration and obtain the final centroid. ;

[0079] In steps 3-4, the specific method for global cluster allocation is as follows:

[0080] Using the final center of mass Cluster assignment is performed on all M samples in the candidate set to obtain cluster labels. The formula for calculating cluster allocation is:

[0081] ;

[0082] After cluster allocation is complete, output the index set. Cluster affiliation label for each block and the member index set of each cluster ;

[0083] in, Let represent the visual representation vector of the i-th block. This represents the normalized visual feature vector of the i-th block. Let represent the normalized visual feature vector of the j-th block. This represents the initial centroid of the Kth cluster. This represents the centroid of the k-th cluster. This represents the set of samples assigned to cluster k in the current batch. This represents the learning rate related to the number of historical samples in the cluster. This represents the centroid of the k-th cluster at the q-th iteration. It represents the centroid of the k-th cluster at the (q+1)-th iteration.

[0084] Furthermore, in step 4, during the extraction of the region of interest, for each cluster number... The specific steps are as follows:

[0085] Step 4-1: Obtain the cluster member index set;

[0086] Get the set of member indices for this cluster: ;

[0087] Step 4-2: Determine if a cluster is empty;

[0088] like If the cluster is skipped, no representative is generated; if the set If the cluster is non-empty, proceed to step 4-3;

[0089] Step 4-3: Extreme value retrieval;

[0090] Perform extreme value search within a non-empty cluster: ;

[0091] Step 4-4: Generate the region of interest;

[0092] Will Add to the final representative index set: Based on index set Extract the feature matrix of representative blocks By indexing and tracing back the spatial coordinates of representative blocks in the original lung slices, the region of interest can be obtained.

[0093] in, This represents the cluster number to which the i-th block belongs. Indicates the first Cluster number, Let represent the semantic similarity score of the i-th block. This represents the index of the optimal block selected in the k-th cluster.

[0094] A system for extracting regions of interest from whole-section pathological images of lung diseases, comprising:

[0095] The lung image data acquisition module is used to acquire lung image data and generate a semantic similarity score based on the i-th block in the lung image data. Visual representation vectors ;

[0096] The index set filtering module is used to score semantic similarity. The optimal threshold is obtained by employing adaptive threshold filtering, percentile thresholding, bimodal detection, or entropy-based threshold selection, and an index set is then selected based on the optimal threshold. ;

[0097] The feature space clustering module is used for clustering the index set. Visual representation vector of the middle block Perform clustering to obtain the cluster affiliation label for each block. and the set of member indexes for each cluster , ;

[0098] The region of interest extraction module is used to select the block with the highest semantic similarity score in each cluster as the representative block, thus obtaining the index set. Based on index set Extract the feature matrix of each representative block The region of interest is obtained by tracing back to the spatial coordinates of representative blocks in the corresponding lung image data through indexing.

[0099] in, Let represent the semantic similarity score of the i-th block. Let represent the visual representation vector of the i-th block. This represents the cluster number to which the i-th block belongs. Indicates the first Cluster number, Indicates the number of cluster numbers. Indicates belonging to the index set And the cluster number it belongs to is A set of blocks.

[0100] The beneficial effects of this invention are as follows:

[0101] 1. In this invention, the image is divided into several blocks, and based on the semantic similarity score and visual representation vector of each block, and combined with a determined optimal threshold, an index set is selected and clustered. In each cluster, the block with the highest semantic similarity score is selected as the index set, its feature matrix is ​​extracted, and then the spatial coordinates of the original image are traced back. In this process, minor components with small area but significant prognostic value in the image are still considered, and adaptive threshold filtering is performed based on the semantic similarity score to adaptively identify regions of interest with diagnostic value in pathological sections. The selected regions of interest retain the heterogeneous distribution of different invasive features within the section, improving the accuracy of extracting regions of interest in lung section images. This solves the problems of low accuracy in extracting regions of interest in lung adenocarcinoma section images, such as easily ignoring minor components with small area but significant prognostic value, and the different screening results caused by fixed thresholds on different sections.

[0102] 2. In this invention, during adaptive threshold filtering, the inter-class variance is maximized in the one-dimensional semantic similarity score space to achieve adaptive and unsupervised dynamic threshold determination, which helps to initially screen blocks related to diagnosis. The screening accuracy of these blocks is higher, thereby improving the extraction accuracy of regions of interest in lung slice images.

[0103] 3. In this invention, a heterogeneity sampling mechanism is designed based on the K-Means clustering characteristics. The number N of blocks expected to be contained in the region of interest is set as the number of clusters. Since the centroids in the clustering mechanism are automatically distributed according to the data density, the heterogeneity of pathological slides is captured proportionally while strictly controlling the number of samples, thereby improving the accuracy of extracting regions of interest in lung slide images. Detailed Implementation

[0104] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0105] Therefore, all other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0106] Example 1

[0107] This embodiment provides a method for extracting regions of interest from whole-section pathological images of lung diseases. The method includes the following steps:

[0108] Step 1: Acquisition of lung image data;

[0109] Obtain lung image data, and generate a semantic similarity score based on the i-th block in the lung image data. Visual representation vectors .

[0110] In generating semantic similarity scores Visual representation vectors It utilizes pathology vision base models or other deep vision models. For example, it uses pathology vision base models, including but not limited to CONCH, PLIP, and UNI, to generate a semantic similarity score for each block in a lung slice. Visual representation vectors Among them, semantic similarity score This represents whether the area has diagnostic value, i.e., whether it is a foreground region of interest for the tumor; visual representation vector. This indicates that the visual features of the block are encoded in the algebraic space. It can be used to evaluate whether there is visual similarity between blocks, and can also be used for downstream tasks after extracting the region of interest, such as diagnostic classification and prognostic prediction.

[0111] This embodiment uses the CONCH model as an example. For a pathological slide with B blocks, its visual representation vector can be obtained by comprehensive analysis. ,in The model's feature dimensions (the CONCH model has 512 features) and its semantic similarity score with input prompts (such as "lung adenocarcinoma") can be obtained. The value ranges from [0,1]. The closer it is to 1, the more likely it is to be a prospective region of interest for lung adenocarcinoma.

[0112] in, Let represent the semantic similarity score of the i-th block. This represents the visual representation vector of the i-th block.

[0113] Step 2: Filter the index set;

[0114] The optimal threshold is obtained by employing adaptive threshold filtering, percentile thresholding, bimodal detection, or entropy-based threshold selection, and an index set is then selected based on the optimal threshold. .

[0115] The percentile threshold method refers to taking the Pth percentile of the similarity score as the threshold (for example, P=50 means taking the blocks above the median).

[0116] The bimodal detection method refers to detecting whether the score histogram shows a bimodal distribution; if so, the valley between the two peaks is taken as the threshold.

[0117] Entropy-based threshold selection refers to selecting a threshold that maximizes the sum of the foreground and background entropies.

[0118] The threshold screening methods described above are all existing technologies that can be directly applied by those skilled in the art without requiring any creative effort.

[0119] However, this embodiment uses an innovative adaptive threshold filtering method, which is based on semantic similarity scores. The optimal threshold is calculated using the maximum inter-class variance method, and the index set is then selected based on the optimal threshold. This step aims to initially screen a subset of all pathological slides that are relevant to the diagnostic target (e.g., "lung adenocarcinoma"). The core idea is to find a threshold T that maximizes the inter-class variance between the two classes separated by T (foreground and background; foreground refers to blocks with high similarity, i.e., regions relevant to the diagnostic target; background refers to blocks with low similarity, i.e., non-diagnostic regions). A larger inter-class variance indicates a higher degree of discriminative power between the two classes. Let the semantic similarity score vector be... The value range is [0,1].

[0120] The specific steps of this adaptive threshold filtering method are as follows:

[0121] Step 2-1: Histogram construction;

[0122] semantic similarity score The score range (e.g., [0,1]) is divided into 256 intervals. The frequency of each interval is counted to form a normalized histogram. , ,and .

[0123] Step 2-2: Calculation of cumulative amount;

[0124] For each possible threshold t (i.e., the upper boundary of the t-th bin), calculate the cumulative weight of the classes below the threshold. The cumulative mean of classes below the threshold and all mean .

[0125] Among them, the cumulative weight of classes (i.e., classes) below the threshold. The calculation formula is:

[0126] ;

[0127] Cumulative mean of classes below the threshold The calculation formula is:

[0128] ;

[0129] All Means The calculation formula is:

[0130] .

[0131] Steps 2-3: Calculate the inter-class variance;

[0132] Based on the cumulative weight of classes below the threshold The cumulative mean of classes below the threshold and all mean Calculate the inter-class variance .

[0133] Among them, inter-class variance The calculation formula is:

[0134] .

[0135] Steps 2-4: Determining the optimal threshold;

[0136] Based on inter-class variance Determine the optimal threshold .

[0137] Among them, the optimal threshold The calculation formula is:

[0138] .

[0139] Steps 2-5: Add constraints;

[0140] For the optimal threshold Add a lower bound constraint based on the mean and standard deviation to obtain the final threshold. .

[0141] To prevent extreme distributions from causing the threshold to be too low or too high, the final threshold... The lower bound constraint based on the mean and standard deviation is introduced; the final threshold is... The calculation formula is:

[0142] .

[0143] Steps 2-6: Filter the index set;

[0144] Select block i that meets the following criteria: That is, filtering out those that meet the requirements. Let i be a block i, and let the set of all blocks i that satisfy the condition be denoted as . .

[0145] in, Let represent the semantic similarity score of the i-th block. This represents the normalized histogram of the nth interval. Indicates the threshold t, This represents the mean similarity score of all M blocks. This represents the standard deviation of the similarity scores for all M blocks.

[0146] Step 3: Feature space clustering;

[0147] For the index set Visual representation vector of the middle block Perform clustering to obtain the cluster affiliation label for each block. and the set of member indexes for each cluster , .

[0148] in, This represents the cluster number to which the i-th block belongs. Indicates the first Cluster number, Indicates the number of cluster numbers. Indicates belonging to the index set And the cluster number it belongs to is A set of blocks.

[0149] This step aims to identify different heterogeneous visual feature clusters from the screened foreground regions, thereby classifying various histological patterns in pathological sections into different clusters. Hierarchical clustering, spectral clustering, Gaussian mixture models, and K-Means clustering methods can be used for clustering.

[0150] Hierarchical clustering refers to the generation of tree-like cluster structures, allowing users to select the number of samples at different granularities.

[0151] Spectral clustering refers to the ability of spectral clustering to obtain more reasonable cluster divisions when the feature distribution is non-convex.

[0152] Gaussian mixture models provide soft clustering results, where the probability of each block belonging to each cluster can be used as additional information.

[0153] The clustering methods described above are all existing technologies. The K-Means clustering method used in this embodiment is also an existing method and can be directly applied. However, the innovation of this embodiment lies in using a clustering strategy from the index set... A subset of the region of interest is obtained from the image, and the mathematical properties of K-Means are used to adaptively aggregate (cluster) different cell morphologies and differentiation degrees (pathological subtypes) (which can be roughly regarded as clusters) corresponding to each image patch, as well as to sample in roughly equal proportions. The comparison method is to obtain the data directly using top K similarity, and random sampling. By innovatively using the K-Means clustering method to adaptively capture the heterogeneous patterns within the sample while preserving semantic representativeness, the neglect of heterogeneous regions in the top K similarity method and the neglect of sample reliability (semantic similarity) in the random sampling strategy are avoided. The specific steps of this feature space clustering are as follows:

[0154] Step 3-1: Normalize the feature matrix;

[0155] index set Visual representation vector of the corresponding block Combine them into a feature matrix, perform row-by-row L2 normalization on the feature matrix, and project the normalized feature vectors onto the unit hypersphere.

[0156] The formula for calculating L2 normalization is as follows:

[0157] ;

[0158] Projecting the normalized feature vectors onto the unit hypersphere, the squared Euclidean distance between any two normalized feature vectors has a monotonic mapping relationship with their cosine similarity, i.e.:

[0159] ;

[0160] This transformation makes K-Means clustering based on Euclidean distance equivalent to clustering based on cosine similarity, which is more in line with the similarity measurement habits of semantic embedding vectors in deep learning.

[0161] Step 3-2: Configure clustering parameters;

[0162] Configure clustering parameters, including the number of blocks in a cluster. Initialization strategy, distance metric, batch size, maximum number of iterations, and convergence tolerance.

[0163] The specific parameter configuration is as follows:

[0164] Cluster number : Set to the specified target number of samples;

[0165] Initialization strategy: K-Means++;

[0166] Distance metric: Euclidean distance;

[0167] Batch size (MiniBatch): min(1024, M);

[0168] Maximum number of iterations: 300;

[0169] Convergence tolerance: 10 -4 (Maximum norm of centroid displacement between successive iterations).

[0170] Step 3-3: Iterative optimization;

[0171] The MiniBatch K-Means algorithm is used to perform an iterative process until convergence or the maximum number of iterations is reached, to obtain the final centroid. .

[0172] The specific steps of iterative optimization are as follows:

[0173] Step 3-3-1: Initialization;

[0174] For the normalized feature vector obtained in step 3-1, K initial centroids are selected using the K-Means++ strategy. ;

[0175] Step 3-3-2: Random batch sampling;

[0176] From the index set A subset of size q is randomly selected from the set of corresponding feature vectors. .

[0177] Step 3-3-3: Cluster allocation;

[0178] When assigning clusters, methods such as centroid nearest method, multiple representative selection method, and weighted fusion method can be used.

[0179] The multiple representative selection method refers to the method that when the size of a cluster is significantly larger than that of other clusters, multiple representatives can be selected from the cluster proportionally to further improve the sampling accuracy.

[0180] The weighted fusion method refers to not selecting a single representative, but instead weighting and averaging the features of all blocks within a cluster according to their similarity scores to generate a "virtual" representative vector.

[0181] This application employs the centroid nearest method, selecting the block closest to the cluster centroid, as this sample best represents the "average" shape of the cluster. Specifically:

[0182] For each sample in the batch , , sample Assign to the cluster corresponding to the nearest centroid:

[0183] .

[0184] Step 3-3-4: Centroid incremental update;

[0185] For each cluster The centroid is updated based on the samples assigned to this cluster in the current batch;

[0186] .

[0187] Step 3-3-5: Convergence determination;

[0188] If there exists a centroid whose displacement norm is not less than the convergence tolerance, and the maximum number of iterations has not been reached, then return to step 3-3-1 and reselect K initial centroids; if the displacement norm of all centroids is less than the convergence tolerance, or the maximum number of iterations has been reached, then terminate the iteration and obtain the final centroid. .

[0189] Steps 3-4: Global cluster allocation;

[0190] Using the final center of mass Perform cluster assignment on all M samples in the candidate set and output the index set. Cluster affiliation label for each block and the member index set of each cluster ;

[0191] Using the final center of mass Cluster assignment is performed on all M samples in the candidate set to obtain cluster labels. The formula for calculating cluster allocation is:

[0192] ;

[0193] After cluster allocation is complete, output the index set. Cluster affiliation label for each block and the member index set of each cluster .

[0194] in, The cluster number is represented as The set of member indices, This represents the cluster number to which the i-th block belongs. Indicates the first Cluster number, Indicates the number of cluster numbers. Indicates belonging to the index set And the cluster number it belongs to is A set consisting of the indexes of the blocks; Let represent the visual representation vector of the i-th block. This represents the normalized visual feature vector of the i-th block. Let represent the normalized visual feature vector of the j-th block. Indicates the first The centroid vector of each cluster, Indicates the first The initial centroid of each cluster, Indicates the first The centroid of a cluster, Indicates the cluster assigned in the current batch The sample set, This represents the learning rate related to the number of historical samples in the cluster. This represents the centroid of the k-th cluster at the q-th iteration. It represents the centroid of the k-th cluster at the (q+1)-th iteration.

[0195] Step 4: Extract the region of interest;

[0196] The block with the highest semantic similarity score in each cluster is selected as the representative block to obtain the index set. Based on index set Extract the feature matrix of each representative block The region of interest is obtained by tracing back to the spatial coordinates of representative blocks in the corresponding lung image data through indexing.

[0197] After the aforementioned feature space partitioning, a high-quality candidate set (i.e., index set) is obtained. The feature space is subdivided into K clusters, each corresponding to a relatively compact region in the feature space, which can be interpreted as a specific morphological pattern or its sub-variant. The task of this step is to select a single optimal representative from each cluster, employing an intra-cluster extremum selection strategy: selecting the sample with the highest semantic similarity score within the cluster as the representative. Given that the diversity constraint has been guaranteed by clustering, this further maximizes the diagnostic value (highest score) of each representative. While the centroid nearest point may be "typical," it may not be optimal in terms of semantic similarity; the highest-scoring sample within the cluster ensures that this representative is the instance most closely associated with the diagnostic target within this morphological pattern.

[0198] For each cluster number The specific process is as follows:

[0199] Step 4-1: Obtain the cluster member index set;

[0200] Get the set of member indices for this cluster: ;

[0201] Step 4-2: Determine if a cluster is empty;

[0202] like (This indicates that the cluster has no members after partitioning, i.e., it is an empty cluster), then skip the cluster and do not generate a representative; if the set If the cluster is non-empty, proceed to step 4-3;

[0203] Step 4-3: Extreme value retrieval;

[0204] Perform extreme value search within a non-empty cluster: ;

[0205] Step 4-4: Generate the region of interest;

[0206] Will Add to the final representative index set: Based on index set Extract the feature matrix of representative blocks By indexing and tracing back the spatial coordinates of representative blocks in the original lung slices, the region of interest can be obtained.

[0207] The final set of indexes of the representative blocks selected Its scale ,in Equal to the number of empty clusters (usually) ≈ Based on index set The feature matrix of representative blocks can be directly extracted. And by indexing back the spatial coordinates of each representative block in the original WSI, it can be used for subsequent visualization, pathologist review, or downstream intelligent analysis tasks.

[0208] in, This represents the cluster number to which the i-th block belongs. Indicates the first Cluster number, Indicates the number of cluster numbers. Indicates belonging to the index set And the cluster number it belongs to is A set of blocks, This represents the semantic similarity score of the i-th block. This represents the index of the optimal block selected in the k-th cluster.

[0209] Example 2

[0210] This embodiment provides a system for extracting regions of interest from whole-section pathological images of lung diseases, including:

[0211] The lung image data acquisition module is used to acquire lung image data and generate a semantic similarity score based on the i-th block in the lung image data. Visual representation vectors ;

[0212] The index set filtering module is used to score semantic similarity. The optimal threshold is obtained by employing adaptive threshold filtering, percentile thresholding, bimodal detection, or entropy-based threshold selection, and an index set is then selected based on the optimal threshold. ;

[0213] The feature space clustering module is used for clustering the index set. Visual representation vector of the middle block Perform clustering to obtain the cluster affiliation label for each block. and the member index set of each cluster , ;

[0214] The region of interest extraction module is used to select the block with the highest semantic similarity score in each cluster as the representative block, thus obtaining the index set. Based on index set Extract the feature matrix of each representative block The region of interest is obtained by tracing back to the spatial coordinates of representative blocks in the corresponding lung image data through indexing.

[0215] in, This represents the cluster number to which the i-th block belongs. Indicates the first Cluster number, Indicates the number of cluster numbers. Indicates belonging to the index set And the cluster number it belongs to is A set of blocks.

[0216] Example 3

[0217] A computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of a method for extracting regions of interest from whole-slice images of pathological pulmonary diseases.

[0218] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0219] The memory includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or D-interface display memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device. Of course, the memory may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is often used to store the operating system and various application software installed on the computer device, such as the program code for the method of extracting the region of interest from the whole slide image of the pathology of lung disease. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0220] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is used to run program code stored in the memory or process data, for example, to run program code for a method of extracting regions of interest from whole-slice images of lung disease pathology.

[0221] Example 4

[0222] A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of a method for extracting regions of interest from whole-slice images of pathological pulmonary diseases.

[0223] The computer-readable storage medium stores an interface display program that can be executed by at least one processor to perform the steps of the method for extracting the region of interest from a whole-section pathological image of a lung disease as described above.

[0224] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the method for extracting the region of interest from the whole slide image of the pathology of lung disease described in the embodiments of this application.

Claims

1. A method for extracting regions of interest from whole-section pathological images of lung diseases, characterized in that, Includes the following steps: Step 1: Acquisition of lung image data; Obtain lung image data, and generate a semantic similarity score based on the i-th block in the lung image data. Visual representation vectors ; Generate semantic similarity scores At that time, the CONCH, PLIP, or UNI pathology visual pedestal model is used to generate a semantic similarity score between each block in a lung slice and the input prompt word. ; Step 2: Filter the index set; Based on semantic similarity score The optimal threshold is obtained by employing adaptive threshold filtering, percentile thresholding, bimodal detection, or entropy-based threshold selection, and an index set is then selected based on the optimal threshold. ; Step 3: Feature space clustering; For index set Visual representation vector of the middle block Perform clustering to obtain the cluster affiliation label for each block. and the set of member indexes for each cluster , ; Step 4: Extract the region of interest; The block with the highest semantic similarity score in each cluster is selected as the representative block to obtain the index set. Based on index set Extract the feature matrix of each representative block The region of interest is obtained by tracing back to the spatial coordinates of representative blocks in the corresponding lung image data through indexing. in, Let represent the semantic similarity score of the i-th block. Let represent the visual representation vector of the i-th block. This represents the cluster number to which the i-th block belongs. Indicates the first Cluster number, Indicates the number of cluster numbers. Indicates belonging to the index set And the cluster number it belongs to is A set of blocks.

2. The method for extracting the region of interest from a whole-section pathological image of a lung disease as described in claim 1, characterized in that, In step 2, when obtaining the optimal threshold using the adaptive threshold filtering method, the specific steps of the adaptive threshold filtering method are as follows: Step 2-1: Histogram construction; semantic similarity score The score range was divided into 256 intervals, and the frequency of each interval was counted to form a normalized histogram. , ,and ; Step 2-2: Calculation of cumulative amount; For each possible threshold t, calculate the cumulative weight of the classes below the threshold. The cumulative mean of classes below the threshold and all mean ; Steps 2-3: Calculate the inter-class variance; Based on the cumulative weight of classes below the threshold The cumulative mean of classes below the threshold and all mean Calculate the inter-class variance ; Steps 2-4: Determining the optimal threshold; Based on inter-class variance Determine the optimal threshold ; Steps 2-5: Add constraints; For the optimal threshold Add a lower bound constraint based on the mean and standard deviation to obtain the final threshold. ; Steps 2-6: Filter the index set; Select block i that meets the following criteria: All indexes that meet the conditions constitute a set. ; in, This represents the semantic similarity score of the i-th block.

3. The method for extracting the region of interest from a whole-section pathological image of a lung disease as described in claim 2, characterized in that: In step 2-2, the cumulative weight of classes below the threshold The calculation formula is: ； Cumulative mean of classes below the threshold The calculation formula is: ； All Means The calculation formula is: ； In steps 2-3, the variance between classes The calculation formula is: ； In steps 2-4, the optimal threshold The calculation formula is: ； In steps 2-5, the final threshold The calculation formula is: ； in, This represents the normalized histogram of the nth interval. Indicates the threshold. This represents the mean similarity score of all M blocks. This represents the standard deviation of the similarity scores for all M blocks.

4. The method for extracting the region of interest from a whole-section pathological image of a lung disease as described in claim 1, characterized in that: In step 3, the specific steps of feature space clustering are as follows: Step 3-1: Normalize the feature matrix; index set Visual representation vector of the corresponding block Combine them into a feature matrix, perform row-by-row L2 normalization on the feature matrix, and project the normalized feature vectors onto the unit hypersphere. Step 3-2: Configure clustering parameters; Configure clustering parameters, including the number of blocks in a cluster. Initialization strategy, distance metric, batch size, maximum number of iterations, convergence tolerance; Step 3-3: Iterative optimization; The MiniBatch K-Means algorithm is used to perform an iterative process until convergence or the maximum number of iterations is reached, to obtain the final centroid. ; Steps 3-4: Global cluster allocation; Using the final center of mass Perform cluster assignment on all M samples in the candidate set and output the index set. Cluster affiliation label for each block and the member index set of each cluster ; in, Indicates the first The centroid vector of each cluster, This represents the coordinate indices of all members contained in the k-th cluster.

5. The method for extracting the region of interest from a whole-section pathological image of a lung disease as described in claim 4, characterized in that: In step 3-1, the calculation formula for L2 normalization is: ； Projecting the normalized feature vectors onto the unit hypersphere, the squared Euclidean distance between any two normalized feature vectors has a monotonic mapping relationship with their cosine similarity, i.e.: ； In step 3-2, the clustering parameters are configured as follows: Cluster number : Set to the specified target number of samples N; Initialization strategy: K-Means++; Distance metric: Euclidean distance; Batch size: min(1024, M); Maximum number of iterations: 300; Convergence tolerance: 10 -4 ; In step 3-3, the specific steps of iterative optimization are as follows: Step 3-3-1: Initialization; Selecting K initial centroids using the K-Means++ strategy ; Step 3-3-2: Random batch sampling; From the index set A subset of size q is randomly selected from the set of corresponding feature vectors. ; Step 3-3-3: Cluster allocation; For each sample in the batch , , sample Assign to the cluster corresponding to the nearest centroid: ； Step 3-3-4: Centroid Increment Update; For each cluster The centroid is updated based on the samples assigned to this cluster in the current batch; ； Step 3-3-5: Convergence determination; If there exists a centroid whose displacement norm is not less than the convergence tolerance, and the maximum number of iterations has not been reached, then return to step 3-3-1 and reselect K initial centroids; if the displacement norm of all centroids is less than the convergence tolerance, or the maximum number of iterations has been reached, then terminate the iteration and obtain the final centroid. ; In steps 3-4, the specific method for global cluster allocation is as follows: Using the final center of mass Cluster assignment is performed on all M samples in the candidate set to obtain cluster labels. The formula for calculating cluster allocation is: ； After cluster allocation is complete, output the index set. Cluster affiliation label for each block and the member index set of each cluster ; in, Let represent the visual representation vector of the i-th block. This represents the normalized visual feature vector of the i-th block. Let represent the normalized visual feature vector of the j-th block. This represents the initial centroid of the Kth cluster. This represents the centroid of the k-th cluster. This represents the set of samples assigned to cluster k in the current batch. This represents the learning rate related to the number of historical samples in the cluster. This represents the centroid of the k-th cluster at the q-th iteration. It represents the centroid of the k-th cluster at the (q+1)-th iteration.

6. The method for extracting the region of interest from a whole-section pathological image of a lung disease as described in claim 1, characterized in that, Step 4, during region of interest extraction, for each cluster number... The specific steps are as follows: Step 4-1: Obtain the cluster member index set; Get the set of member indices for this cluster: ; Step 4-2: Determine if a cluster is empty; like If the cluster is skipped, no representative is generated; if the set If the cluster is non-empty, proceed to step 4-3; Step 4-3: Extreme value retrieval; Perform extreme value search within a non-empty cluster: ; Step 4-4: Generate the region of interest; Will Add to the final representative index set: Based on index set Extract the feature matrix of representative blocks By indexing and tracing back the spatial coordinates of representative blocks in the original lung slices, the region of interest can be obtained. in, This represents the cluster number to which the i-th block belongs. Indicates the first Cluster number, This represents the semantic similarity score of the i-th block. This represents the index of the optimal block selected in the k-th cluster.

7. A system for extracting regions of interest from whole-section images of pathological slides of lung diseases, characterized in that, include: The lung image data acquisition module is used to acquire lung image data and generate a semantic similarity score based on the i-th block in the lung image data. Visual representation vectors ; Generate semantic similarity scores At that time, the CONCH, PLIP, or UNI pathology visual pedestal model is used to generate a semantic similarity score between each block in a lung slice and the input prompt word. ; The index set filtering module is used to score semantic similarity. The optimal threshold is obtained by employing adaptive threshold filtering, percentile thresholding, bimodal detection, or entropy-based threshold selection, and an index set is then selected based on the optimal threshold. ; The feature space clustering module is used for clustering the index set. Visual representation vector of the middle block Perform clustering to obtain the cluster affiliation label for each block. and the set of member indexes for each cluster , ; The region of interest extraction module is used to select the block with the highest semantic similarity score in each cluster as the representative block, thus obtaining the index set. Based on index set Extract the feature matrix of each representative block The region of interest is obtained by tracing back to the spatial coordinates of representative blocks in the corresponding lung image data through indexing. in, This represents the semantic similarity score of the i-th block. Let represent the visual representation vector of the i-th block. This represents the cluster number to which the i-th block belongs. Indicates the first Cluster number, Indicates the number of cluster numbers. Indicates belonging to the index set And the cluster number it belongs to is A set of blocks.

Images