Multi-scale feature integrated lymph node identification system and method

Abstract

The invention provides a lymph node identification system and method integrating multi-scale features, and the method comprises the steps: carrying out the grid processing of a pre-obtained original pathological section image, and obtaining a grid data set; based on a preset local feature extraction algorithm and a preset cell detection model, performing local texture information extraction and cell information extraction on each piece of grid data in the grid data set to obtain a plurality of grid data local features; obtaining a candidate organization set based on the plurality of grid data local features; on the basis of a preset global feature analysis algorithm, capturing overall cell distribution features in the candidate tissue set to obtain a target tissue set; obtaining a lymph node distribution diagram based on the candidate tissue set, the target tissue set and an original pathological section image; and recognizing the lymph node of the original pathological section image based on the lymph node distribution diagram. According to the method, comprehensive analysis combining local and global information is realized, and the lymph node identification accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis, and in particular to a lymph node identification system and method that integrates multi-scale features. Background Technology

[0002] The automated processing and analysis of pathological images, especially in the identification of lymph nodes, has presented a significant challenge in ensuring the accuracy of lymph node identification, regardless of whether it relies on traditional manual feature extraction and rule-based classification models or the successful application of convolutional neural networks (CNNs) in image classification tasks.

[0003] Currently, existing technologies typically employ texture analysis techniques, such as gray-level co-occurrence matrices, to describe the distribution characteristics of cells. This local texture feature analysis can acquire detailed information about the image, but local texture features cannot well reflect the spatial distribution of cells or the overall structure of the tissue. Alternatively, a step-by-step approach is used, processing coarse lymph node structure identification and cell identification independently. This approach cannot fully utilize the correlation between the overall tissue and local texture features. In pathological images, the spatial distribution pattern of cells is crucial for tissue identification. Therefore, while existing technologies can extract local texture features, they lack consideration for the overall tissue structure, leading to poor accuracy in lymph node identification. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a lymph node identification system and method that integrates multi-scale features, enabling comprehensive analysis combining local and global information to improve the accuracy of lymph node identification.

[0005] To achieve the above objectives, embodiments of the present invention provide a lymph node identification system integrating multi-scale features, comprising: a grid data acquisition module, a local feature extraction module, a candidate tissue acquisition module, a target tissue acquisition module, a lymph node distribution map generation module, and a lymph node identification module; the grid data acquisition module is used to perform gridding processing on the pre-acquired original pathological slide image to obtain a grid dataset; the local feature extraction module is used to extract local texture information and cell information from each grid data in the grid dataset based on a preset local feature extraction algorithm and a preset cell detection model to obtain several local features of the grid data; the candidate tissue acquisition module is used to obtain a set of candidate tissues based on several local features of the grid data; the target tissue acquisition module is used to capture the overall cell distribution features in the candidate tissue set based on a preset global feature analysis algorithm to obtain a set of target tissues; the lymph node distribution map generation module is used to obtain a lymph node distribution map based on the candidate tissue set, the target tissue set, and the original pathological slide image; and the lymph node identification module is used to identify lymph nodes in the original pathological slide image based on the lymph node distribution map.

[0006] This invention proposes a lymph node identification system integrating multi-scale features. A grid data acquisition module processes the pre-acquired original pathological slide image into a grid, facilitating subsequent feature extraction and analysis by focusing on the effective region without compressing pixels. A local feature extraction module then uses a preset local feature extraction algorithm and a preset cell detection model to extract local texture and cell information from each grid, obtaining the local features of each grid. A candidate tissue acquisition module then constructs a candidate tissue set containing microscopic features. A target tissue acquisition module then uses a preset global feature analysis algorithm to capture the overall cell distribution features in the candidate tissue set, constructing a target tissue set containing macroscopic tissue structures. A lymph node distribution map generation module then combines the candidate tissue set containing microscopic features with the target tissue set containing macroscopic tissue structures to obtain a lymph node distribution map. A lymph node identification module then identifies the lymph nodes in the original pathological slide image. This effectively integrates microscopic and macroscopic features, comprehensively analyzing local and global information to improve the accuracy of lymph node identification.

[0007] Furthermore, the grid data acquisition module is used to perform gridding processing on the pre-acquired original pathological slide images to obtain a grid dataset, including: a region segmentation unit, an image enhancement unit, a grid splitting unit, and a grid filtering unit; the region segmentation unit is used to acquire the full-view pathological slide image and cut the full-view pathological slide image into several sub-regions of a preset pixel size; the image enhancement unit is used to randomly flip, rotate, and color-perturb each sub-region to obtain the original pathological slide image; the grid splitting unit is used to split the original pathological slide image into several grids based on a preset window and a preset step size; the grid filtering unit is used to acquire the grayscale value of each grid and remove grids whose grayscale values ​​do not meet the preset grayscale threshold to obtain the grid dataset.

[0008] In the above scheme, the region segmentation unit cuts the full-view pathological slide into sub-regions of a preset size, and the image enhancement unit enhances the pathological slide image by randomly flipping, rotating and color perturbing to obtain the original pathological slide image, thus expanding the diversity of data. The grid segmentation unit then divides the original pathological slide image into several grids through a preset window and step size. The grid filtering unit then identifies invalid regions by comparing grayscale thresholds and retains valid grids so that subsequent feature extraction and analysis can focus on the valid regions. This also provides an accurate data foundation for the subsequent comprehensive analysis of local and global information and improves the accuracy of lymph node identification.

[0009] Furthermore, the local feature extraction module is used to extract local texture information and cell information from each grid data in the grid dataset based on a preset local feature extraction algorithm and a preset cell detection model, obtaining several local features of the grid data. This includes: a grid sample acquisition unit, a local texture feature extraction unit, a cell feature extraction unit, and a grid data local feature acquisition unit. The grid sample acquisition unit sets corresponding labels for each grid data in the grid dataset and randomly selects several grid data with different labels to obtain a grid data sample set. The local texture feature extraction unit extracts local texture information from each grid data in the grid data sample set based on a preset local feature extraction algorithm, obtaining several local texture features. The cell feature extraction unit extracts cell information from each grid data in the grid data sample set based on a preset cell detection model, obtaining several cell features. The grid data local feature acquisition unit combines several local texture features and several cell features based on a preset local feature extraction algorithm and performs several rounds of feature extraction to obtain several grid data local features.

[0010] In the above scheme, the grid sample acquisition unit sets a label for each grid data and randomly selects grids with different labels to construct a grid data sample set to ensure the balance of the sample set. Then, the local texture feature extraction unit uses a preset local feature extraction algorithm to extract the local texture information of the grid data, and the cell feature extraction unit uses a preset cell detection model to extract the cell information of the grid data. The extracted local texture features and cell features are combined, and the grid data local feature acquisition unit performs several rounds of feature extraction of local features of the grid data. By introducing cell information into the local texture information, the representation ability of micro-texture features is enhanced, providing an accurate data foundation for the subsequent comprehensive analysis of local and global information and improving the accuracy of lymph node identification.

[0011] Furthermore, the candidate organization acquisition module is used to obtain a candidate organization set based on several local features of grid data, including: a feature labeling unit, a feature splicing unit, and an organization set acquisition unit; the feature labeling unit is used to set prediction labels for several local features of grid data; the feature splicing unit is used to splice several local features of grid data based on the prediction labels to obtain several candidate organizations; the organization set acquisition unit is used to integrate several candidate organizations to obtain a candidate organization set.

[0012] In the above scheme, the feature labeling unit sets a prediction label for the local features of each grid data, and the feature splicing unit splices the entire slice network according to the prediction label to obtain candidate tissues. By extracting the local features of each independent grid, the tissue set acquisition unit splices them into the overall slice network to construct a candidate tissue set containing micro-features. This maximizes the preservation of the texture information of the pathological slice image, enhances the representation ability of micro-texture features, provides an accurate data foundation for subsequent comprehensive analysis of local and global information, and improves the accuracy of lymph node identification.

[0013] Furthermore, the target tissue acquisition module is used to capture the overall cell distribution features in the candidate tissue set based on a preset global feature analysis algorithm to obtain the target tissue set. This module includes: a cell distribution feature extraction unit, a feature distribution sequence construction unit, a distribution probability acquisition unit, and a tissue screening unit. The cell distribution feature extraction unit extracts cell information from the candidate tissue set using a gray-level co-occurrence matrix based on the preset global feature analysis algorithm to obtain several cell distribution features. The feature distribution sequence construction unit constructs a feature distribution sequence based on these cell distribution features. The distribution probability acquisition unit normalizes and classifies the feature distribution sequence to obtain the tissue type distribution probability. The tissue screening unit selects candidate tissues whose tissue type distribution probabilities meet the preset target task requirements based on the candidate tissue set to obtain the target tissue set.

[0014] In the above scheme, in order to compensate for the shortcomings of local feature extraction in extracting global tissue information, the cell distribution feature extraction unit uses a preset global feature analysis algorithm to extract cell information from the candidate tissue set, thereby obtaining cell distribution features. The feature distribution sequence construction unit then constructs a feature distribution sequence to reflect the overall tissue structure. The distribution probability acquisition unit further normalizes and classifies the feature distribution sequence to eliminate scale differences and maps it to tissue type distribution probability. Finally, the tissue screening unit selects candidate tissues whose tissue type distribution probability meets the preset target task requirements, constructs a target tissue set containing macroscopic tissue structure, and constructs macroscopic cell distribution by combining microscopic cell information, providing an accurate data foundation for subsequent comprehensive analysis of local and global information and improving the accuracy of lymph node identification.

[0015] Furthermore, the lymph node distribution map generation module is used to obtain a lymph node distribution map based on the candidate tissue set, the target tissue set, and the original pathological slide image. It includes: a tissue combination splicing unit and an inference prediction unit; the tissue combination splicing unit is used to splice the candidate tissue set and the target tissue set to obtain several grid data fusion features; the inference prediction unit is used to infer and predict the lymph node distribution in the original pathological slide image based on several grid data fusion features to obtain the lymph node distribution map.

[0016] In the above scheme, the tissue combination and splicing unit combines the candidate tissue set containing microscopic features with the target tissue set containing macroscopic tissue structure to obtain grid data fusion features. The inference and prediction unit then infers and predicts the lymph node distribution in the original pathological slide image based on the grid data fusion features to obtain a lymph node distribution map. The local and global information are comprehensively analyzed to improve the accuracy of lymph node identification.

[0017] Furthermore, the lymph node identification module is used to identify lymph nodes in the original pathological slide image based on the lymph node distribution map, including: a lymph node region acquisition unit, a lymph node confidence region acquisition unit, and a lymph node identification unit; the lymph node region acquisition unit is used to obtain several lymph node regions based on the lymph node distribution map; the lymph node confidence region acquisition unit is used to assign corresponding probability values ​​to the several lymph node regions, remove lymph node regions whose probability values ​​do not meet a preset threshold, and obtain lymph node confidence regions; the lymph node identification unit is used to identify lymph nodes in the original pathological slide image based on the lymph node confidence regions.

[0018] In the above scheme, in order to remove false positive areas, the lymph node region acquisition unit assigns a corresponding probability value to each lymph node region in the lymph node distribution map, and the lymph node confidence region acquisition unit removes lymph node regions whose probability values ​​do not meet the preset threshold as noise, thereby obtaining the lymph node confidence region. The lymph node identification unit then accurately identifies the lymph nodes in the original pathological slide image.

[0019] Furthermore, the lymph node region acquisition unit is used to obtain several lymph node regions based on the lymph node distribution map, including: a lymph node distribution optimization subunit and a feature annotation subunit; the lymph node distribution optimization subunit is used to remove the edge regions of the lymph node distribution map and fill the grid gaps based on a preset morphological algorithm to obtain an optimized lymph node distribution map; the feature annotation subunit is used to perform feature annotation on the optimized lymph node distribution map to obtain several lymph node regions.

[0020] In the above scheme, in order to remove false positive areas, the lymph node distribution optimization subunit uses morphological algorithm erosion operation to remove the edge area of ​​the lymph node distribution map, and uses morphological algorithm dilation operation to fill the gaps that may be caused by cutting errors. The feature annotation subunit then performs feature annotation on the optimized lymph node distribution map to annotate the lymph node areas in the image, thereby removing false positive areas and improving the accuracy of lymph node identification.

[0021] This invention also provides a method for lymph node identification integrating multi-scale features, comprising: performing gridding processing on a pre-acquired original pathological slide image to obtain a grid dataset; extracting local texture information and cell information from each grid data in the grid dataset based on a preset local feature extraction algorithm and a preset cell detection model to obtain several local features of the grid data; obtaining a candidate tissue set based on the several local features of the grid data; capturing the overall cell distribution features in the candidate tissue set based on a preset global feature analysis algorithm to obtain a target tissue set; obtaining a lymph node distribution map based on the candidate tissue set, the target tissue set, and the original pathological slide image; and identifying lymph nodes in the original pathological slide image based on the lymph node distribution map.

[0022] This invention proposes a lymph node identification method integrating multi-scale features. The method involves gridding the pre-acquired original pathological slide image, which avoids pixel compression and facilitates subsequent feature extraction and analysis focusing on effective regions. Then, a preset local feature extraction algorithm and a preset cell detection model are used to extract local texture and cell information from each grid, obtaining the local features of each grid. This constructs a candidate tissue set containing microscopic features. A preset global feature analysis algorithm then captures the overall cell distribution features of the candidate tissue set, constructing a target tissue set containing macroscopic tissue structures. Finally, the candidate tissue set containing microscopic features is combined with the target tissue set containing macroscopic tissue structures to obtain a lymph node distribution map, which is then used to identify lymph nodes in the original pathological slide image. This effectively integrates microscopic and macroscopic features, comprehensively analyzing local and global information to improve the accuracy of lymph node identification.

[0023] Furthermore, the pre-acquired original pathological slide images are processed into a grid to obtain a grid dataset, including: acquiring a full-view pathological slide image; cutting the full-view pathological slide image into several sub-regions of a preset pixel size; randomly flipping, rotating, and color perturbing each sub-region to obtain the original pathological slide image; dividing the original pathological slide image into several grids based on a preset window and a preset step size; acquiring the grayscale value of each grid and removing grids whose grayscale values ​​do not meet a preset grayscale threshold to obtain a grid dataset.

[0024] In the above scheme, the full-view pathological slide is cut into sub-regions of a preset size, and the pathological slide image is enhanced by random flipping, rotation and color perturbation to obtain the original pathological slide image, thus expanding the diversity of data. Then, the original pathological slide image is divided into several grids by a preset window and step size. Invalid regions are identified by grayscale threshold comparison, and valid grids are retained so that subsequent feature extraction and analysis can focus on the valid regions. This also provides an accurate data foundation for the comprehensive analysis of local and global information and improves the accuracy of lymph node identification. Attached Figure Description

[0025] Figure 1 A schematic diagram of the module structure of a lymph node identification system integrating multi-scale features provided in a certain embodiment of the present invention;

[0026] Figure 2 This is a flowchart illustrating the steps of a lymph node identification method integrating multi-scale features according to a certain embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Example 1

[0029] See Figure 1 , Figure 1 This is a schematic diagram of the module structure of a lymph node identification system integrating multi-scale features, provided in one embodiment of the present invention. Figure 1 As shown, this embodiment of the invention provides a lymph node identification system integrating multi-scale features, including: a grid data acquisition module 201, a local feature extraction module 202, a candidate tissue acquisition module 203, a target tissue acquisition module 204, a lymph node distribution map generation module 205, and a lymph node identification module 206; the grid data acquisition module 201 is used to perform gridding processing on the pre-acquired original pathological slide images to obtain a grid dataset; the local feature extraction module 202 is used to perform local feature extraction on each grid data in the grid dataset based on a preset local feature extraction algorithm and a preset cell detection model. Texture information extraction and cell information extraction are used to obtain several local features of grid data; the candidate tissue acquisition module 203 is used to obtain a set of candidate tissues based on several local features of grid data; the target tissue acquisition module 204 is used to capture the overall cell distribution features in the candidate tissue set based on a preset global feature analysis algorithm to obtain a set of target tissues; the lymph node distribution map generation module 205 is used to obtain a lymph node distribution map based on the candidate tissue set, the target tissue set and the original pathological slide image; the lymph node identification module 206 is used to identify lymph nodes in the original pathological slide image based on the lymph node distribution map.

[0030] One possible implementation involves acquiring high-resolution whole-slide images (WSI) of pathological slides (HE-stained), and then dividing the pathological slide images into several grids using a fixed window to obtain grid data x. uThen, a local feature extraction algorithm is used to process each grid data x. u In this embodiment, local texture information extraction is performed using a Local Tissue Pixel Network (LTPN). The LTPN can also perform gridding processing on pathological slide images. The ResNet-based image encoder G within the LTPN is used to process each grid data x. u Local texture information is extracted, and then the ResNet residual is adjusted while introducing cell features. The existing cell detection model P is then used to extract the grid data x for each grid cell. u Cell information x cell The local texture information, along with the local texture information, is input into the image encoder G to perform multiple rounds of feature extraction, thereby obtaining several local features of the grid data. Then, the local texture pixel network is used to process each grid data x. u Prediction and labeling are performed to complete the splicing of the entire slice network, resulting in a candidate tissue set Y = {y1, y2, ..., y}. n To compensate for the shortcomings of local texture pixel networks in global information extraction, a Tissue Cell Feature Network (TCFN) is employed to further analyze the overall tissue. Specifically, in each candidate tissue region y of the candidate tissue set Y... n Extracting cell information x cell The gray-level co-occurrence matrix (GLCM) is used to capture cell distribution features and generate feature sequences m. i By further analyzing the feature sequence m i Normalization and classification are performed to construct the target organization set. Finally, the candidate organization set Y = {y1, y2, ..., y} extracted from the local texture pixel network is used to construct the target organization set. n The system performs feature fusion with the target tissue set, extracts local texture information from the image through mesh slicing, and combines cellular information to enhance the learning of local details. From these detailed features, it infers subtle lesions and structural changes in pathological images, especially in high-resolution images. This is further enhanced by combining cellular information. cell By utilizing the spatial relationships and density of cells to analyze the overall characteristics of tissues, the complete tissue structure and cell distribution patterns can be deduced. Finally, by integrating the candidate tissue set Y = {y1, y2, ..., y...}, the overall characteristics of the tissue can be analyzed. n The system generates a lymph node distribution map of the pathological slide images and a set of target tissues. It then performs feature annotation on the lymph node distribution map of each pathological slide image to identify the lymph nodes in the original pathological slide images.

[0031] This invention proposes a lymph node identification system integrating multi-scale features. A grid data acquisition module 201 processes the pre-acquired original pathological slide image into a grid, facilitating subsequent feature extraction and analysis by focusing on the effective region without compressing pixels. A local feature extraction module 202 then uses a preset local feature extraction algorithm and a preset cell detection model to extract local texture and cell information from each grid, obtaining the local features of each grid. A candidate tissue acquisition module 203 then constructs a candidate tissue set containing microscopic features. A target tissue acquisition module 204 then uses a preset global feature analysis algorithm to capture the overall cell distribution features in the candidate tissue set, constructing a target tissue set containing macroscopic tissue structures. A lymph node distribution map generation module 205 combines the candidate tissue set containing microscopic features with the target tissue set containing macroscopic tissue structures to obtain a lymph node distribution map. A lymph node identification module 206 then identifies the lymph nodes in the original pathological slide image. This effectively integrates microscopic and macroscopic features, comprehensively analyzing local and global information to improve the accuracy of lymph node identification.

[0032] In a preferred embodiment, the grid data acquisition module 201 is used to perform gridding processing on the pre-acquired original pathological slide image to obtain a grid dataset, including: a region segmentation unit, an image enhancement unit, a grid splitting unit, and a grid filtering unit; the region segmentation unit is used to acquire the full-view pathological slide image and cut the full-view pathological slide image into several sub-regions of a preset pixel size; the image enhancement unit is used to randomly flip, rotate, and color-perturb each sub-region to obtain the original pathological slide image; the grid splitting unit is used to split the original pathological slide image into several grids based on a preset window and a preset step size; the grid filtering unit is used to acquire the gray value of each grid and remove grids whose gray values ​​do not meet a preset gray value threshold to obtain the grid dataset.

[0033] One preferred implementation involves acquiring high-resolution whole-slide images (WSI) of pathological slides (HE-stained), dividing the slides into several sub-regions of 256×256 pixels (equivalent to a preset pixel size), enhancing the images using methods such as random flipping, rotation, and color perturbation to obtain the original pathological slide images, and then cutting the pathological slide images into several grids using a fixed window. In this embodiment, the fixed window size is a×a and the step size is s (equivalent to a preset window and a preset step size). By comparing the grayscale value of each grid with a set grayscale threshold, white areas in the grids are removed to obtain a grid dataset. In this embodiment, the preset grayscale threshold is used to exclude white areas, and the specific value can be set according to the actual situation.

[0034] In the above scheme, the region segmentation unit cuts the full-view pathological slide into sub-regions of a preset size, and the image enhancement unit enhances the pathological slide image by randomly flipping, rotating and color perturbing to obtain the original pathological slide image, thus expanding the diversity of data. The grid segmentation unit then divides the original pathological slide image into several grids through a preset window and step size. The grid filtering unit then identifies invalid regions by comparing grayscale thresholds and retains valid grids so that subsequent feature extraction and analysis can focus on the valid regions. This also provides an accurate data foundation for the subsequent comprehensive analysis of local and global information and improves the accuracy of lymph node identification.

[0035] In a preferred embodiment, the local feature extraction module 202 is used to extract local texture information and cell information from each grid data in the grid dataset based on a preset local feature extraction algorithm and a preset cell detection model, thereby obtaining several local features of the grid data. This module includes: a grid sample acquisition unit, a local texture feature extraction unit, a cell feature extraction unit, and a grid data local feature acquisition unit. The grid sample acquisition unit sets corresponding labels for each grid data in the grid dataset and randomly selects several grid data with different labels to obtain a grid data sample set. The local texture feature extraction unit extracts local texture information from each grid data in the grid data sample set based on the preset local feature extraction algorithm, thereby obtaining several local texture features. The cell feature extraction unit extracts cell information from each grid data in the grid data sample set based on the preset cell detection model, thereby obtaining several cell features. The grid data local feature acquisition unit combines several local texture features and several cell features based on the preset local feature extraction algorithm and performs several rounds of feature extraction to obtain several local features of the grid data.

[0036] One preferred implementation method is for each grid data x u Set the corresponding label. u And randomly sample grid data x with different labels u To construct a gridded data sample set, the local feature extraction algorithm employs a Local Tissue Pixel Network (LTPN). The LTPN can also perform gridding processing on pathological slide images. The ResNet-based image encoder G within the LTPN is used to process each grid data x. u Local texture information is extracted to obtain local texture features t. l Then adjust the ResNet residuals while introducing cell features c. l Data x from each grid is extracted using the existing cell detection model P. u Cell information x cellThe local texture information, along with the local texture information, is input into the image encoder G to perform multiple rounds of feature extraction. The specific process of multiple rounds of feature extraction is as follows:

[0037] Local texture features t l The extraction process is as follows:

[0038]

[0039] Among them, F l For the texture feature t of the lth round l Extract, W i Here, x represents the weights for the convolution operation, k is the number of convolutional layers, and x represents the weights for the convolution operation. l-1 For the local texture features t in the l-th round l Extracted input;

[0040] Cellular characteristics c l The extraction process is as follows:

[0041]

[0042] Among them, O l For the cell characteristics of the lth round c l Extract, V i Here, m represents the weights for the convolution operation, and c represents the number of convolutional layers. l-1 Cellular characteristics c in round l l Extracted input;

[0043] Through AS c Mechanism, combined with local texture features t l With cell characteristics c l Input to the next round of feature extraction:

[0044] x l =σ(F l (x l-1 )+O l (c l-1 ));

[0045] Finally, vectorized feature x is generated. L Through the fully connected layer f u Classification yields grid data x u probability distribution p x :

[0046] p x =softmax(σ(f) u (x L )));

[0047] During feature extraction, to guide the LTPN in learning discriminative feature representations, we optimize the LTPN using the cross-entropy loss function. This enables the network to learn features that effectively distinguish different categories, thereby improving classification performance. Specifically, we optimize the network parameters using the cross-entropy loss function:

[0048] LOSS=H(p x ,l u );

[0049] In the formula, l u For each grid data x u The corresponding label set;

[0050] Furthermore, several local features of grid data.

[0051] In the above scheme, the grid sample acquisition unit sets a label for each grid data and randomly selects grids with different labels to construct a grid data sample set to ensure the balance of the sample set. Then, the local texture feature extraction unit uses a preset local feature extraction algorithm to extract the local texture information of the grid data, and the cell feature extraction unit uses a preset cell detection model to extract the cell information of the grid data. The extracted local texture features and cell features are combined, and the grid data local feature acquisition unit performs several rounds of feature extraction of local features of the grid data. By introducing cell information into the local texture information, the representation ability of micro-texture features is enhanced, providing an accurate data foundation for the subsequent comprehensive analysis of local and global information and improving the accuracy of lymph node identification.

[0052] In a preferred embodiment, the candidate organization acquisition module 203 is used to obtain a candidate organization set based on several local features of grid data, including: a feature marking unit, a feature splicing unit, and an organization set acquisition unit; the feature marking unit is used to set prediction marks on several local features of grid data; the feature splicing unit is used to splice several local features of grid data based on the prediction marks to obtain several candidate organizations; and the organization set acquisition unit is used to integrate several candidate organizations to obtain a candidate organization set.

[0053] One preferred implementation involves using each grid region x u Input into LTPN to achieve processing of each grid region x u Predicting the category and for each grid region x u After marking the prediction results and completing the stitching of the entire sliced ​​mesh, the candidate tissue set Y = {y1, y2, ..., y} is obtained. n}

[0054] In the above scheme, the feature labeling unit sets a prediction label for the local features of each grid data, and the feature splicing unit splices the entire slice network according to the prediction label to obtain candidate tissues. By extracting the local features of each independent grid, the tissue set acquisition unit splices them into the overall slice network to construct a candidate tissue set containing micro-features. This maximizes the preservation of the texture information of the pathological slice image, enhances the representation ability of micro-texture features, provides an accurate data foundation for subsequent comprehensive analysis of local and global information, and improves the accuracy of lymph node identification.

[0055] In a preferred embodiment, the target tissue acquisition module 204 is used to capture the overall cell distribution features of a candidate tissue set based on a preset global feature analysis algorithm to obtain a target tissue set. The module includes: a cell distribution feature extraction unit, a feature distribution sequence construction unit, a distribution probability acquisition unit, and a tissue screening unit. The cell distribution feature extraction unit is used to extract cell information from the candidate tissue set using a gray-level co-occurrence matrix based on the preset global feature analysis algorithm to obtain several cell distribution features. The feature distribution sequence construction unit is used to construct a feature distribution sequence based on several cell distribution features. The distribution probability acquisition unit is used to normalize and classify the feature distribution sequence to obtain the tissue type distribution probability. The tissue screening unit is used to screen candidate tissues whose tissue type distribution probabilities meet the preset target task requirements based on the candidate tissue set to obtain the target tissue set.

[0056] One preferred implementation method, in order to compensate for the shortcomings of local texture pixel networks in global information extraction, employs a Tissue Cell Feature Network (TCFN) to further analyze the overall tissue. Specifically, in each candidate tissue region y of the candidate tissue set Y... n Extracting cell information x cell The gray-level co-occurrence matrix (GLCM) is used to capture cell distribution features and generate feature sequences m. i By further analyzing the feature sequence m i Normalization and classification are performed to output the probability distribution p of the organization type. i :

[0057] p i =softmax(σ(fc(m)) i ));

[0058] Based on the preset task requirements, the probability distribution of each candidate organization type is screened, and the candidate organizations whose organization type distribution probability meets the preset target task requirements are selected to obtain the target organization set. The preset target task requirements can be set according to the actual situation.

[0059] In the above scheme, in order to compensate for the shortcomings of local feature extraction in extracting global tissue information, the cell distribution feature extraction unit uses a preset global feature analysis algorithm to extract cell information from the candidate tissue set, thereby obtaining cell distribution features. The feature distribution sequence construction unit then constructs a feature distribution sequence to reflect the overall tissue structure. The distribution probability acquisition unit further normalizes and classifies the feature distribution sequence to eliminate scale differences and maps it to tissue type distribution probability. Finally, the tissue screening unit selects candidate tissues whose tissue type distribution probability meets the preset target task requirements, constructs a target tissue set containing macroscopic tissue structure, and constructs macroscopic cell distribution by combining microscopic cell information, providing an accurate data foundation for subsequent comprehensive analysis of local and global information and improving the accuracy of lymph node identification.

[0060] In a preferred embodiment, the lymph node distribution map generation module 205 is used to obtain a lymph node distribution map based on a candidate tissue set, a target tissue set, and an original pathological slide image. The module includes: a tissue-binding splicing unit and an inference prediction unit. The tissue-binding splicing unit splices the candidate tissue set and the target tissue set to obtain several grid data fusion features. The inference prediction unit performs inference prediction on the lymph node distribution in the original pathological slide image based on these grid data fusion features to obtain the lymph node distribution map.

[0061] One preferred implementation involves fusing the candidate tissue set and the target tissue set. This is typically achieved by combining two sets of features, such as (Feature_LTPN, Feature_TCFN). After feature integration, a prediction result for each pathological slide image is generated, namely a lymph node distribution map. These lymph node distribution maps annotate all lymph node regions in the image and assign a probability value to each lymph node region, representing the confidence that the region is a lymph node. This allows for effective localization and annotation of lymph node regions in the image during subsequent lymph node identification, leading to better lymph node recognition.

[0062] In the above scheme, the tissue combination and splicing unit combines the candidate tissue set containing microscopic features with the target tissue set containing macroscopic tissue structure to obtain grid data fusion features. The inference and prediction unit then infers and predicts the lymph node distribution in the original pathological slide image based on the grid data fusion features to obtain a lymph node distribution map. The local and global information are comprehensively analyzed to improve the accuracy of lymph node identification.

[0063] In a preferred embodiment, the lymph node identification module 206 is used to identify lymph nodes in the original pathological slide image based on a lymph node distribution map, and includes: a lymph node region acquisition unit, a lymph node confidence region acquisition unit, and a lymph node identification unit; the lymph node region acquisition unit is used to obtain several lymph node regions based on the lymph node distribution map; the lymph node confidence region acquisition unit is used to assign corresponding probability values ​​to the several lymph node regions, remove lymph node regions whose probability values ​​do not meet a preset threshold, and obtain lymph node confidence regions; the lymph node identification unit is used to identify lymph nodes in the original pathological slide image based on the lymph node confidence regions.

[0064] In one preferred implementation, a lymph node distribution map labels all lymph node regions in an image. False positive regions, i.e., regions incorrectly labeled as positive, often appear during the generation of the lymph node distribution map. These regions are typically considered noise regions. To reduce the impact of noise, a probability threshold filtering method is used to select valid regions. By setting an appropriate probability threshold, only predicted regions with a confidence level higher than the threshold are retained. For example, if the confidence level of some regions in the lymph node distribution map is lower than a certain threshold (e.g., 50%), they are considered noise and removed, resulting in confident lymph node regions. This allows for the identification of the characteristics of each lymph node region.

[0065] In the above scheme, in order to remove false positive areas, the lymph node region acquisition unit assigns a corresponding probability value to each lymph node region in the lymph node distribution map, and the lymph node confidence region acquisition unit removes lymph node regions whose probability values ​​do not meet the preset threshold as noise, thereby obtaining the lymph node confidence region. The lymph node identification unit then accurately identifies the lymph nodes in the original pathological slide image.

[0066] In a preferred embodiment, the lymph node region acquisition unit is used to obtain several lymph node regions based on the lymph node distribution map, including: a lymph node distribution optimization subunit and a feature annotation subunit; the lymph node distribution optimization subunit is used to remove the edge regions of the lymph node distribution map and fill the grid gaps based on a preset morphological algorithm to obtain an optimized lymph node distribution map; the feature annotation subunit is used to perform feature annotation on the optimized lymph node distribution map to obtain several lymph node regions.

[0067] In one preferred embodiment, a lymph node distribution map annotates all lymph node regions in an image. False positive regions, i.e., regions incorrectly marked as positive, often appear during the generation of the lymph node distribution map and are generally considered noise areas. To reduce the impact of noise, the lymph node distribution map is further processed using erosion and dilation operations. The erosion operation removes edge regions of the lymph node distribution map, which typically refer to small areas located at the edges of other regions. The dilation operation fills gaps that may be caused by cutting errors, thereby obtaining an optimized lymph node distribution map. Further feature annotation is performed on the optimized lymph node distribution map, including cell density, tissue type, and distribution pattern, to obtain the lymph node regions. This allows for effective localization and annotation of lymph node regions in the image during subsequent lymph node identification, leading to better lymph node recognition.

[0068] In the above scheme, in order to remove false positive areas, the lymph node distribution optimization subunit uses morphological algorithm erosion operation to remove the edge area of ​​the lymph node distribution map, and uses morphological algorithm dilation operation to fill the gaps that may be caused by cutting errors. The feature annotation subunit then performs feature annotation on the optimized lymph node distribution map to annotate the lymph node areas in the image, thereby removing false positive areas and improving the accuracy of lymph node identification.

[0069] Example 2

[0070] See Figure 2 , Figure 2 This is a flowchart illustrating the steps of a lymph node identification method integrating multi-scale features according to a certain embodiment of the present invention. Figure 2 As shown, this embodiment of the invention also provides a method for lymph node identification that integrates multi-scale features, including steps 101 to 106, each step of which is as follows:

[0071] Step 101: Perform gridding processing on the pre-acquired original pathological slide images to obtain a grid dataset;

[0072] Step 102: Based on the preset local feature extraction algorithm and preset cell detection model, local texture information and cell information are extracted from each grid data in the grid dataset to obtain several local features of the grid data;

[0073] Step 103: Based on several local features of grid data, obtain a set of candidate tissues;

[0074] Step 104: Based on a preset global feature analysis algorithm, capture the overall cell distribution features in the candidate tissue set to obtain the target tissue set;

[0075] Step 105: Based on the candidate tissue set, target tissue set, and original pathological slide images, obtain the lymph node distribution map;

[0076] Step 106: Identify lymph nodes in the original pathological slide image based on the lymph node distribution map.

[0077] One specific implementation involves acquiring high-resolution whole-slide images (WSI) of pathological slides (HE-stained), and then dividing the pathological slide images into several grids using a fixed window to obtain grid data x. u Then, a local feature extraction algorithm is used to process each grid data x. u In this embodiment, local texture information extraction is performed using a Local Tissue Pixel Network (LTPN). The LTPN can also perform gridding processing on pathological slide images. The ResNet-based image encoder G within the LTPN is used to process each grid data x. u Local texture information is extracted, and then the ResNet residual is adjusted while introducing cell features. The existing cell detection model P is then used to extract the grid data x for each grid cell. u Cell information x cell The local texture information, along with the local texture information, is input into the image encoder G to perform multiple rounds of feature extraction, thereby obtaining several local features of the grid data. Then, the local texture pixel network is used to process each grid data x. u Prediction and labeling are performed to complete the splicing of the entire slice network, resulting in a candidate tissue set Y = {y1, y2, ..., y}. n To compensate for the shortcomings of local texture pixel networks in global information extraction, a Tissue Cell Feature Network (TCFN) is employed to further analyze the overall tissue. Specifically, in each candidate tissue region y of the candidate tissue set Y... n Extracting cell information x cell The gray-level co-occurrence matrix (GLCM) is used to capture cell distribution features and generate feature sequences m. i By further analyzing the feature sequence m i Normalization and classification are performed to construct the target organization set. Finally, the candidate organization set Y = {y1, y2, ..., y} extracted from the local texture pixel network is used to construct the target organization set. nThe system performs feature fusion with the target tissue set, extracts local texture information from the image through mesh slicing, and combines cellular information to enhance the learning of local details. From these detailed features, it infers subtle lesions and structural changes in pathological images, especially in high-resolution images. This is further enhanced by combining cellular information. cell By utilizing the spatial relationships and density of cells to analyze the overall characteristics of tissues, the complete tissue structure and cell distribution patterns can be deduced. Finally, by integrating the candidate tissue set Y = {y1, y2, ..., y...}, the overall characteristics of the tissue can be analyzed. n The system generates a lymph node distribution map of the pathological slide images and a set of target tissues. It then performs feature annotation on the lymph node distribution map of each pathological slide image to identify the lymph nodes in the original pathological slide images.

[0078] A preferred approach involves performing gridding processing on pre-acquired original pathological slide images to obtain a grid dataset, including: acquiring a full-field pathological slide image; cutting the full-field pathological slide image into several sub-regions of a preset pixel size; randomly flipping, rotating, and color-perturbing each sub-region to obtain the original pathological slide image; dividing the original pathological slide image into several grids based on a preset window and a preset step size; acquiring the grayscale value of each grid and removing grids whose grayscale values ​​do not meet a preset grayscale threshold to obtain the grid dataset.

[0079] One preferred implementation involves acquiring high-resolution whole-slide images (WSI) of pathological slides (HE-stained), dividing the slides into several sub-regions of 256×256 pixels (equivalent to a preset pixel size), enhancing the images using methods such as random flipping, rotation, and color perturbation to obtain the original pathological slide images, and then cutting the pathological slide images into several grids using a fixed window. In this embodiment, the fixed window size is a×a and the step size is s (equivalent to a preset window and a preset step size). By comparing the grayscale value of each grid with a set grayscale threshold, white areas in the grids are removed to obtain a grid dataset. In this embodiment, the preset grayscale threshold is used to exclude white areas, and the specific value can be set according to the actual situation.

[0080] This invention proposes a lymph node identification method integrating multi-scale features. The method involves gridding the pre-acquired original pathological slide image, which avoids pixel compression and facilitates subsequent feature extraction and analysis focusing on effective regions. Then, a preset local feature extraction algorithm and a preset cell detection model are used to extract local texture and cell information from each grid, obtaining the local features of each grid. This constructs a candidate tissue set containing microscopic features. A preset global feature analysis algorithm then captures the overall cell distribution features of the candidate tissue set, constructing a target tissue set containing macroscopic tissue structures. Finally, the candidate tissue set containing microscopic features is combined with the target tissue set containing macroscopic tissue structures to obtain a lymph node distribution map, which is then used to identify lymph nodes in the original pathological slide image. This effectively integrates microscopic and macroscopic features, comprehensively analyzing local and global information to improve the accuracy of lymph node identification.

[0081] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0082] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the described specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0083] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

Claims

1. A lymph node identification system integrating multi-scale features, characterized in that, include: The module includes a grid data acquisition module, a local feature extraction module, a candidate tissue acquisition module, a target tissue acquisition module, a lymph node distribution map generation module, and a lymph node identification module. The grid data acquisition module is used to perform gridding processing on the pre-acquired original pathological slide images to obtain a grid dataset; The local feature extraction module is used to extract local texture information and cell information from each grid data in the grid dataset based on a preset local feature extraction algorithm and a preset cell detection model, so as to obtain several local features of the grid data. The candidate tissue acquisition module is used to obtain a set of candidate tissues based on several local features of the grid data; The target tissue acquisition module is used to capture the overall cell distribution characteristics in the candidate tissue set based on a preset global feature analysis algorithm, and obtain the target tissue set. The lymph node distribution map generation module is used to obtain a lymph node distribution map based on the candidate tissue set, the target tissue set, and the original pathological slide image; The lymph node identification module is used to identify lymph nodes in the original pathological slide image based on the lymph node distribution map.

2. The lymph node identification system integrating multi-scale features as described in claim 1, characterized in that, The grid data acquisition module is used to perform gridding processing on the pre-acquired original pathological slide images to obtain a grid dataset, including: Region segmentation unit, image enhancement unit, grid segmentation unit, and grid filtering unit; The region segmentation unit is used to acquire a full-view pathological slide image and cut the full-view pathological slide image into several sub-regions of a preset pixel size. The image enhancement unit is used to randomly flip, rotate, and color-perturb each of the sub-regions to obtain the original pathological slide image; The grid segmentation unit is used to segment the original pathological slide image into several grids based on a preset window and a preset step size; The grid filtering unit is used to obtain the gray value of each grid and remove grids whose gray values ​​do not meet the preset gray value threshold to obtain a grid dataset.

3. The lymph node identification system integrating multi-scale features as described in claim 1, characterized in that, The local feature extraction module is used to extract local texture information and cell information from each grid data in the grid dataset based on a preset local feature extraction algorithm and a preset cell detection model, to obtain several local features of the grid data, including: Mesh sample acquisition unit, local texture feature extraction unit, cell feature extraction unit, and mesh data local feature acquisition unit; The grid sample acquisition unit is used to set corresponding labels for each grid data in the grid dataset, and randomly extract several grid data with different labels to obtain a grid data sample set; The local texture feature extraction unit is used to extract local texture information of each grid data in the grid data sample set based on the preset local feature extraction algorithm, and obtain several local texture features. The cell feature extraction unit is used to extract cell information of each grid data in the grid data sample set based on the preset cell detection model, and obtain several cell features. The local feature acquisition unit of the grid data is used to perform several rounds of feature extraction based on the preset local feature extraction algorithm, combining several local texture features and several cell features to obtain several local features of the grid data.

4. The lymph node identification system integrating multi-scale features as described in claim 1, characterized in that, The candidate tissue acquisition module is used to obtain a set of candidate tissues based on several local features of the grid data, including: Feature labeling unit, feature splicing unit, and organization set acquisition unit; The feature labeling unit is used to set prediction labels for several local features of the grid data; The feature splicing unit is used to splice several local features of the grid data based on the prediction label to obtain several candidate tissues; The organization set acquisition unit is used to integrate several candidate organizations to obtain a candidate organization set.

5. The lymph node identification system integrating multi-scale features as described in claim 4, characterized in that, The target tissue acquisition module is used to capture the overall cell distribution characteristics of the candidate tissue set based on a preset global feature analysis algorithm, thereby obtaining a target tissue set, including: Cell distribution feature extraction unit, feature distribution sequence construction unit, distribution probability acquisition unit, and tissue screening unit; The cell distribution feature extraction unit is used to extract cell information from the candidate tissue set based on a preset global feature analysis algorithm and a gray-level co-occurrence matrix to obtain several cell distribution features. The feature distribution sequence construction unit is used to construct a feature distribution sequence based on several of the cell distribution features; The probability distribution acquisition unit is used to normalize and classify the feature distribution sequence to obtain the tissue type probability distribution. The organization screening unit is used to screen candidate organizations whose organization type distribution probability meets the preset target task requirements based on the candidate organization set, thereby obtaining the target organization set.

6. The lymph node identification system integrating multi-scale features as described in claim 1, characterized in that, The lymph node distribution map generation module is used to obtain a lymph node distribution map based on the candidate tissue set, the target tissue set, and the original pathological slide image, including: The organization combines splicing units and inference / prediction units; The tissue combination splicing unit is used to splice the candidate tissue set and the target tissue set to obtain several grid data fusion features; The inference and prediction unit is used to infer and predict the distribution of lymph nodes in the original pathological slice image based on several grid data fusion features, so as to obtain a lymph node distribution map.

7. A lymph node identification system integrating multi-scale features as described in claim 6, characterized in that, The lymph node identification module is used to identify lymph nodes in the original pathological slide image based on the lymph node distribution map, including: Lymph node region acquisition unit, lymph node confidence region acquisition unit, and lymph node identification unit; The lymph node region acquisition unit is used to obtain several lymph node regions based on the lymph node distribution map; The lymph node confidence region acquisition unit is used to assign corresponding probability values ​​to several lymph node regions, remove lymph node regions whose probability values ​​do not meet the preset threshold, and obtain lymph node confidence regions. The lymph node identification unit is used to identify lymph nodes in the original pathological slide image based on the lymph node confidence region.

8. The lymph node identification system integrating multi-scale features as described in claim 7, characterized in that, The lymph node region acquisition unit is used to obtain several lymph node regions based on the lymph node distribution map, including: Lymph node distribution optimization subunit and feature annotation subunit; The lymph node distribution optimization subunit is used to remove the edge region of the lymph node distribution map and fill the grid gaps based on a preset morphological algorithm to obtain an optimized lymph node distribution map. The feature annotation subunit is used to annotate the optimized lymph node distribution map to obtain several lymph node regions.

9. A method for lymph node identification integrating multi-scale features, characterized in that, A lymph node identification system integrating multi-scale features as described in any one of claims 1 to 8, comprising: The pre-acquired raw pathological slide images are processed into a grid to obtain a grid dataset; Based on a preset local feature extraction algorithm and a preset cell detection model, local texture information and cell information are extracted from each grid data in the grid dataset to obtain several local features of the grid data. Based on several local features of the grid data, a candidate tissue set is obtained; Based on a preset global feature analysis algorithm, the overall cell distribution characteristics in the candidate tissue set are captured to obtain the target tissue set; Based on the candidate tissue set, the target tissue set, and the original pathological slide images, a lymph node distribution map is obtained; The lymph nodes in the original pathological slide image are identified based on the lymph node distribution map.

10. The lymph node identification method integrating multi-scale features as described in claim 9, characterized in that, The process of gridding the pre-acquired original pathological slide images to obtain a grid dataset includes: Obtain a full-view pathological slide image, and cut the full-view pathological slide image into several sub-regions of a preset pixel size; Each of the sub-regions is randomly flipped, rotated, and color-perturbed to obtain the original pathological slide image; Based on a preset window and a preset step size, the original pathological slide image is divided into several grids; Obtain the grayscale value of each grid and remove grids whose grayscale values ​​do not meet the preset grayscale threshold to obtain the grid dataset.

Images