Pathological image segmentation method and system based on multi-scale classification

Abstract

The invention discloses a pathological image segmentation method and system based on multi-scale classification, and relates to the technical field of image processing. Comprises: acquiring a to-be-classified pathological image; preprocessing the to-be-classified pathological image; and inputting the preprocessed to-be-classified pathological image into the multi-scale classification model, extracting multi-scale features of the to-be-classified pathological image, and outputting a category prediction result of the to-be-classified pathological image. According to the invention, efficient, accurate and practical pathological image detection is realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a pathological image segmentation method and system based on multi-scale classification. Background Art

[0002] The examination of pathological images is crucial to help doctors make early treatment decisions and assess patient prognosis. However, tissue sections are still usually examined manually by pathologists, which is a tedious and time-consuming process. Therefore, in the past few years, the use of computers to analyze pathological images to assist doctors in diagnosis has attracted widespread attention.

[0003] Currently, many studies have adopted patch-based methods, using convolutional neural networks combined with multiple instance learning methods to classify WSI. Although the existing technology has made a lot of efforts, there are still some challenges for the detection of pathological images.

[0004] 1. First, most deep learning methods use the method of directly detecting patches at the highest resolution. Although this method can maintain a high detection accuracy, it is computationally intensive, consumes a lot of computing resources and greatly increases processing time. Selecting the most representative patch detection in the WSI patch is a very effective solution to the above problem, but how to select representative patches is still a problem to be studied in depth.

[0005] 2. In actual examinations, pathologists usually determine the location of cancer by carefully examining information at different magnifications. However, most current WSI detection frameworks still cannot imitate the actual detection methods of pathologists. They can only detect at a single scale and cannot make a comprehensive judgment based on the detection results at multiple scales.

[0006] 3. Currently, with the development of Transformer architecture in the field of image detection, many existing technologies have begun to replace the original convolutional neural network with the neural network of Transformer architecture and apply it to the pathological image detection system, achieving good results. However, the most advanced frameworks currently use networks with large parameters such as Vision Transformer, Swin Transformer, and ResNet. These networks themselves require a lot of computing resources, which slows down the overall detection speed of the system.

[0007] Therefore, it is an urgent problem for technical personnel in this field to propose a pathological image segmentation method and system based on multi-scale classification to achieve efficient, accurate and practical pathological image detection. Summary of the invention

[0008] In view of this, the present invention provides a pathological image segmentation method and system based on multi-scale classification, so as to solve the technical problems existing in the above-mentioned prior art and realize efficient, accurate and practical pathological image detection.

[0009] In order to achieve the above object, the present invention adopts the following technical solution: A pathological image segmentation method based on multi-scale classification, comprising: Acquire a pathological image to be classified; Preprocess the pathological images to be classified; The preprocessed pathological image to be classified is input into the multi-scale classification model, the multi-scale features of the pathological image to be classified are extracted, and the category prediction result of the pathological image to be classified is output.

[0010] In the above method, optionally, the multi-scale classification model adopts a Pyramid-Net architecture, and introduces a PSM position selection module and a DNU directed node update module, wherein: The Pyramid-Net architecture includes a low-resolution detection network and a high-resolution detection network, which are used to detect suspicious areas in pathological images to be classified and locate cancerous tissues; The PSM position selection module is used to receive the suspicious area position information output by the low-resolution detection network, and feed the position information back to the high-resolution detection network to achieve sparse detection of pathological images; The DNU directed node update module comprehensively analyzes the detection results at different resolutions at the same location by simulating the information transmission mechanism in the graph neural network; Finally, the position information of cancerous tissue and attention images corresponding to different scales are output.

[0011] In the above method, optionally, the PSM position selection module is used to receive the suspicious area position information output by the low-resolution detection network, and feed the position information back to the high-resolution detection network to achieve sparse detection of pathological images, including: Traverse and detect all patches at low resolution at the top of the pyramid in the Pyramid-Net architecture to obtain a preliminary category activation map; For each location The patch of which predicts the Softmax output of category c Calculated by the following formula: In the formula, Predict the category for the corresponding patch The confidence level, is the predicted score of the patch at position (x, y) in category c; The PSM position selection module selects the area that needs to be focused on in the next layer of the network based on the prediction results and spatial information of each patch. The PSM position selection module selects the area as follows: in, Represents the coordinates corresponding to the patch, Represents the number of layers of the WSI pyramid, , Represents the threshold for filtering patch confidence; In the next layer of the WSI pyramid, the patches corresponding to these key areas are found by comparing the position information and input into a higher resolution network for further detection. The selection of the next layer of patches is as follows: in, Represents the patch in the specified area. Represents the coordinates corresponding to the second layer patch, represents the nth key area of ​​the first layer, represents the kth box of the second layer of the WSI pyramid, and n represents the nth key area of ​​the first layer. If the kth box of the second layer is in the key area, it will be marked as a box to be detected and the next step of detection will be carried out. Through this layer-by-layer approach, sparse detection of pathological images can be completed.

[0012] In the above method, optionally, the DNU directed node update module comprehensively analyzes the detection results at different resolutions at the same position by simulating the information transmission mechanism in the graph neural network, including: The DNU directed node update module regards the prediction result of each patch as a node in the pathological image, and the edges between nodes represent the neighbor relationship between patches in the same layer or the inclusion relationship between patches in different layers; A high-resolution patch is contained by multiple low-resolution patches, and the relationship between nodes is expressed as: In the formula, Represents the coordinates corresponding to the second layer patch, represents the updated confidence, Represents the case where two key regions contain the same patch. Represents the situation where the patch of the second layer is only included in one key area of ​​the first layer. , Represents the different weights given to different inclusion situations.

[0013] In the above method, optionally, the final output of the position information of the cancerous tissue and the attention images corresponding to different scales includes: An attention score is generated for each patch based on the final result of the DNU directed node update module, where each attention score reflects the relative importance of each region in lesion recognition at different scales, showing the focus of the Pyramid-Net architecture at each scale.

[0014] A pathological image segmentation system based on multi-scale classification, executing any one of the pathological image segmentation methods based on multi-scale classification described above, comprising: A data acquisition module, used for acquiring pathological images to be classified; A data preprocessing module, used for preprocessing the pathological images to be classified; The classification detection module is used to input the preprocessed pathological image to be classified into the multi-scale classification model, extract the multi-scale features of the pathological image to be classified, and output the category prediction result of the pathological image to be classified.

[0015] Through the above technical solutions, it can be seen that compared with the prior art, the present invention provides a pathological image segmentation method and system based on multi-scale classification. First, in terms of how to select representative patches, the present invention proposes an efficient patch selection scheme, that is, using the PSM position selection module to imitate the actual operation of pathologists when detecting pathological images. At low magnification, the suspicious area is first found in combination with the attention heat map, and then the suspicious area is checked intensively at high magnification. Compared with the existing methods, this method has better robustness and higher detection accuracy, and effectively reduces the detection time. Secondly, the present invention proposes a WSI message passing mechanism between patches at different magnifications, each patch is regarded as a node, and the position relationship between patches is regarded as an edge. The patch neighbor information at the same position at different magnifications is aggregated through message passing, and the feature representation of the node is updated. By introducing the message passing mechanism at different levels, the accuracy of the system is significantly improved. Finally, the present invention uses a more lightweight feature extraction network MobileViT to realize the detection of pathological images. The lightweight network enables the system to have fewer parameters, reduces the occupation of computing resources, and speeds up the overall detection speed of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0017] Figure 1 A flow chart of a pathological image segmentation method based on multi-scale classification disclosed in the present invention; Figure 2 The multi-scale method disclosed in the present invention; Figure 3 It is a schematic diagram of the Pyramid-Net framework structure disclosed in the present invention; Figure 4 A comparison of the way information is accumulated from surrounding nodes in GNNs and the way higher resolution patch information is updated; Figure 5 A block diagram of a pathological image segmentation system based on multi-scale classification disclosed by the present invention; Figure 6 It is a pyramid structure in the Pyramid-Net architecture disclosed in the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Embodiment 1:

[0020] Reference Figure 1 As shown, the present invention discloses a pathological image segmentation method based on multi-scale classification, comprising: Acquire a pathological image to be classified; Preprocess the pathological images to be classified; The preprocessed pathological image to be classified is input into the multi-scale classification model, the multi-scale features of the pathological image to be classified are extracted, and the category prediction result of the pathological image to be classified is output.

[0021] In a specific embodiment, the multi-scale classification model adopts the Pyramid-Net architecture, and introduces the PSM position selection module and the DNU directed node update module, wherein: The Pyramid-Net architecture includes a low-resolution detection network and a high-resolution detection network, which are used to detect suspicious areas in pathological images to be classified and locate cancerous tissues; The PSM position selection module is used to receive the suspicious area position information output by the low-resolution detection network and feed the position information back to the high-resolution detection network to achieve sparse detection of pathological images; The DNU directed node update module comprehensively analyzes the detection results at different resolutions at the same location by simulating the information transmission mechanism in the graph neural network; Finally, the position information of cancerous tissue and attention images corresponding to different scales are output.

[0022] Specifically, in the field of pathological image detection, WSI images usually have ultra-high resolution and rich detail information. Limited by the computing power and memory limitations of the hardware, it is computationally infeasible to put the entire WSI image directly into the neural network for prediction. To address this problem, the prior art proposes a patch-based processing method, which divides the WSI image into several patches and inputs them into the neural network one by one for analysis. At present, patch-based processing methods have become the mainstream in pathological image analysis, and are particularly suitable for classification and detection tasks based on deep learning. Although such methods can reduce computational costs to a certain extent, since they only process local areas, they may lead to the loss of contextual information, thereby affecting the model's understanding of the overall image.

[0023] Furthermore, the present invention proposes a PSM position selection module, which guides the network through the heat map generated by the module and selects representative PSM position selection modules in WSI for higher resolution detection without the need to train additional controllers.

[0024] Furthermore, the framework of the present invention retains the spatial relationship at different scales, introduces the DNU directed node update module, realizes the information exchange and fusion between cross-scale prediction results, and obtains the final prediction result by comprehensively considering and integrating the prediction outputs at each scale.

[0025] Furthermore, in the actual pathological diagnosis process, pathologists often follow a hierarchical inspection strategy from low-power to high-power: first use a low-power microscope to quickly browse the whole-field image (WSI) to identify suspicious areas, and then switch to a high-power microscope to observe and analyze these areas in detail. Drawing on this step-by-step inspection process, the present invention proposes a Pyramid-Net model. Given a WSI image, Pyramid-Net constructs a WSI pyramid structure with different magnifications (×5, ×10, ×20). The structure arranged by resolution size is shown in Figure 6 .

[0026] Specifically, Figure 2 As shown in the figure, suspicious areas are detected layer by layer from the top of the pyramid (low resolution) to the bottom (high resolution). Under the premise of ensuring accuracy, the image is detected as sparsely as possible, thus avoiding the situation where a large amount of computing resources are consumed by detailed inspection of the entire WSI image.

[0027] Network specific details such as Figure 3 The Pyramid-Net framework shown in the figure, wherein a is the multi-scale detection method proposed by the present invention: by integrating the network detection heat map under different magnifications, the area to be focused on in the next layer is identified to achieve accurate and efficient detection. b is the information transmission part, which uses a directed node update module to combine local information of different scales with neighbor information of the same scale, comprehensively generate the final prediction result, and form an attention map under different magnifications to provide a reference for the interpretability of the final detection result. c is a DNU directed node update module, which takes the prediction result of each patch as a node, and updates the node through a directed information transmission mechanism, and integrates the information of each layer to obtain the final prediction result. Specifically, the present invention introduces a patch selection module (PSM), and Pyramid-Net first detects the pathological map under a low-power microscope to locate all suspicious areas. The detected suspicious area position information is fed back to the high-resolution detection network of the next layer through PSM, so that it only focuses on the patches near the suspicious area to further extract the detailed features in the area. In the detection process, not only is there a neighbor relationship between patches under the same magnification, but there is also a clear neighbor or inclusion relationship between patches at different magnifications. Based on the relationship between these multi-scale patches, we introduced the idea of ​​information transfer in graph neural networks and constructed a directed node update module (DNU). This module regards the prediction result of each patch as a node, combines the prediction results of three different scales, and uses the inclusion or neighbor relationship of patches at different scales to comprehensively analyze the detection results of different resolutions at the same position in the pathological image. Finally, the localization information of the cancerous tissue and the attention images at different scales are output.

[0028] In a specific embodiment, the overall architecture of Pyramid-Net, the PSM module and the DNU module are described in detail as follows: Cross-scale learning structures such as Figure 3As shown in part a of . Given a pathological image, a WSI pyramid is constructed by using different magnifications (×5, ×10, ×20) to detect tissue morphological information at multiple scales. Pyramid-Net explores the region of interest layer by layer from the top of the pyramid (low resolution). Since lesions are usually sparsely distributed in tissues, there is no need to perform pathological detection on all patches. To this end, the present invention proposes a patch selection module (Patch SelectionModule, PSM) to achieve sparse detection of pathological images.

[0029] Specifically, the category Class activation map of In , the calculation formula for each spatial element is: in Represents the spatial position of channel k in the last convolutional layer The activation value at Represents the weight of channel k corresponding to category c. The category activation map can indicate which areas are more valuable for diagnosis, which will guide the network to further detect specific areas at higher resolution.

[0030] In the network of the present invention, all patches are first traversed and detected at low resolution at the top of the pyramid to obtain a preliminary category activation map. The patch of which predicts the Softmax output of category c Calculated by the following formula: That is, the patch is predicted to be a category The confidence level, is the prediction score of the patch at position (x, y) in category c.

[0031] The PSM module selects the area that needs to be focused on in the next layer of the network based on the prediction results and spatial information of each patch. The PSM region selection process can be derived as follows: in, Represents the coordinates corresponding to the patch, Represents the number of layers of the WSI pyramid, , Represents the threshold for filtering patch confidence. In this way, the positions saved in PSM contain more task-related information, and these positions need to be further checked with higher resolution.

[0032] In the next layer of the WSI pyramid, the patches corresponding to these key areas are found by comparing the position information and input into a higher resolution network for further detection. The selection of the next layer of patches is as follows: in, Represents the patch in the specified area. Represents the coordinates corresponding to the second layer patch, represents the nth key area of ​​the first layer, represents the kth box in the second layer of the WSI pyramid, and n represents the nth key area in the first layer. If the kth box in the second layer is in the key area, it is marked as a box to be detected and the next step of detection is performed. Through this layer-by-layer approach, sparse detection of WSI images is completed. This structural design effectively reduces unnecessary computational overhead, improves detection efficiency, and ensures accurate identification of key lesion areas.

[0033] In order to help pathologists better understand the detection preferences of the model at different magnifications, the present invention visualizes the detection areas of each layer. This not only can intuitively present the detection areas of the model at different resolutions, but also enhances the interpretability of the model prediction results, which helps doctors interpret the test results more efficiently in clinical practice.

[0034] In a specific embodiment, in a graph neural network (GNN), the node update mechanism is its core process, which usually updates the state or feature representation of the current node through the information of neighbor nodes, such as Figure 4 As shown in Figure 2, a is the way to accumulate information from surrounding nodes in GNNs, and b is the way to update higher resolution patch information in DNU. This paper designs a directed node update module (Directed Node Update, DNU) to integrate multi-scale prediction results in cross-scale detection tasks and improve the accuracy of pathological image diagnosis, such as Figure 3 As shown in part c.

[0035] Specifically, the DNU directed node update module regards the prediction result of each patch as a node in the graph, and the edges between nodes represent the neighbor relationship between patches in the same layer or the inclusion relationship between patches in different layers. In addition, a high-resolution patch may be included in multiple low-resolution patches, such as Figure 4 As shown in . Therefore, the patches that are commonly included will get a larger weight than the patches on the edge. The relationship between nodes can be expressed as follows: In the formula, Represents the coordinates corresponding to the second layer patch, represents the updated confidence, Represents the case where two key regions contain the same patch. Represents the situation where the patch of the second layer is only included in one key area of ​​the first layer. , Represents the different weights given to different inclusion situations.

[0036] In addition, in order to generate more accurate multi-scale attention maps, the DNU directed node update module introduces a bottom-up reverse information propagation mechanism. Based on the final high-resolution prediction results, the information is gradually transmitted back to the lower resolution layers through back propagation. In this way, the information of the upper nodes will be updated based on the final prediction results of the bottom layer, thereby improving the prediction accuracy of each layer in the entire pyramid structure and optimizing the final generated heat map.

[0037] In a specific embodiment, the multi-scale attention Figure 3 As shown in Figure 2b, an attention score is generated for each patch based on the final result of the DNU directed node update module. Each attention score reflects the relative importance of each region at different scales in lesion identification, showing the focus of Pyramid-Net at each scale. The generation of multi-scale attention maps can not only clearly show the impact of prediction results at different scales on the final diagnosis, but also provide an intuitive diagnostic basis for pathologists.

[0038] Specifically, this attention mechanism based on multi-scale information not only improves the diagnostic accuracy of the model, but also greatly enhances its interpretability. The multi-scale attention map intuitively shows the key areas that the network focuses on at each scale, making pathological analysis more interpretable, thereby providing doctors with reliable auxiliary information in actual diagnosis.

[0039] On the other hand, see Figure 5 As shown, a pathological image segmentation system based on multi-scale classification executes any one of the pathological image segmentation methods based on multi-scale classification described above, including A data acquisition module, used for acquiring pathological images to be classified; A data preprocessing module, used for preprocessing the pathological images to be classified; The classification detection module is used to input the preprocessed pathological image to be classified into the multi-scale classification model, extract the multi-scale features of the pathological image to be classified, and output the category prediction result of the pathological image to be classified.

[0040] Embodiment 2:

[0041] In order to verify the pathological image segmentation method and system based on multi-scale classification proposed in the present invention, the present invention conducted classification experiments and cancer location verification on the H&E staining dataset of thyroid papillary carcinoma lymph node metastasis and the public breast cancer lymph node metastasis dataset Camelyon16. The experimental results show that the structure is superior to the current advanced technology in performance.

[0042] Specifically, this experiment is implemented using the PyTorch framework adopted by the present invention, and our model is trained on a single NVIDIA 3090 GPU for a total of 200 epochs. The hyperparameters used in this experiment are shown in Table 1.

[0043] Table 1 Network hyperparameters Hyperparameters choose Resize the size of images 512*512 Epochs 200 Batch size 32 Optimizer Adam Begin Learning rate 0.001 Last Learning rate 0.00001 scheduler The learning rate is reduced by a factor of 10 every 40 rounds Specifically, this embodiment conducts experiments on two WSI datasets: the CAMELYON16 dataset and the papillary thyroid carcinoma lymph node metastasis dataset of this embodiment.

[0044] The data set of this embodiment is a collection of lymph node biopsy pathological specimens of PTC patients in Yantaishan Hospital, Yantai City from January 2022 to December 2023. The biopsy specimens are all larger than 4 mm. 2 And according to the standard histopathology procedures, the quality-controlled pathological sections were prepared, totaling 343 cases, and the lesions on the lymph node pathological sections were marked by the attending physician of the pathology department, and confirmed and cross-checked by 2 deputy chief physicians of the pathology department who have been engaged in diagnosis for more than 5 years. Specifically, the present embodiment obtained the informed consent of the subjects and was reviewed and approved by the ethics committee.

[0045] The CAMELYON16 dataset contains 400 breast cancer lymph node WSI images, including 270 training images and 129 test images.

[0046] In order to compare the fairness, the dataset uses a unified preprocessing method. First, the OSTU algorithm is used to extract the tissue to be detected and remove useless background information. Then these tissues are divided into non-overlapping patches of 512×512 pixels. Table 2 lists the detailed information of the dataset, including the specifications of the dataset and the number of training and test sets used in this study.

[0047] Table 2 Dataset information table Datasets Patch training set Patch Test Set WSI training set WSI Test Set PTC 6952 1386 268 75 CAMELYON16 22831 5708 270 129 Specifically, this embodiment uses four indicators to evaluate the performance of the proposed method: Accuracy, Macro F1 Score, Kappa, and Matthews Correlation Coefficient (MCC). Accuracy reflects the correctness of the overall prediction; Macro F1 score is suitable for unbalanced data sets because it balances the precision and recall of each category, ensuring that the performance of each category is considered; Kappa score measures the consistency between model predictions and true labels, effectively reducing the impact of accidental factors; MCC score evaluates the relationship between prediction results and true labels through correlation analysis, which is particularly suitable for unbalanced data sets. Through these four indicators, this embodiment can comprehensively evaluate the classification ability and reliability of the model from different angles.

[0048] Specifically, the PTC experimental results are as follows: In the Pyramid-Net method proposed in the present invention, the classification and quantitative analysis of the PTC data set is shown in Table 3. Compared with the existing models, Pyramid-Net has achieved significant improvements in indicators such as accuracy (ACC), Kappa coefficient (Kappa) and Matthews correlation coefficient (MCC). In addition, the inference speed of Pyramid-Net reaches 21.49 seconds / WSI, showing a high processing efficiency. The performance improvement is attributed to the introduction of the PSM module, which effectively focuses on the key areas in the WSI through a sparse detection mechanism, ensuring that the computational overhead is greatly reduced without a significant loss of accuracy. The DNU module further integrates the spatial relationship between multiple layers of patches, and through cross-scale neighborhood information transmission, it realizes seamless communication of information between cross-layer patches, thereby improving the detection accuracy of details at different levels in the image. These structural designs enable Pyramid-Net to have accurate and efficient classification capabilities, showing its potential value in practical pathology detection applications.

[0049] Table 3 PTC WSI comparison results Methods ACC(%) F1-Score(%) Kappa(%) MCC(%) AB_MIL 85.33 80.26 60.6 62 TRANS_MIL 93.33 91.03 82.2 83.9 DS_MIL 72.22 64.64 31.4 34.7 CLAM_MB_MIL 83.33 80.36 61.35 62.5 CLAM_SB_MIL 84 78.07 56.5 58 DTFT_MIL 85.18 82.82 65.88 21.16 RRT_MIL 77.33 61.6 28 36.2 FR_MIL 81.33 74.41 40.5 14.11 Pyramid-Net(ours) 94.67 89.47 85.92 86.13 Specifically, based on the positive localization effect diagram of Pyramid-Net in WSI images, it can be seen that the positive areas annotated by the model are highly consistent with the annotations of pathologists, and can accurately capture the distribution of lesions. This shows that Pyramid-Net's positive localization in WSI images is also very advantageous, showing good clinical application potential.

[0050] Furthermore, based on the comparison of heat map detection between Pyramid-Net and other models, it can be seen that compared with traditional models, Pyramid-Net focuses more accurately on the positive areas marked by doctors, shows stronger attention to positive tissues and higher positioning accuracy, and significantly improves the recognition and positioning capabilities of positive areas. The structural design based on multi-scale detection and spatial information transmission enables Pyramid-Net to show excellent reliability and broad application prospects in the task of positive detection of pathological images.

[0051] Specifically, in order to verify the performance of the two key modules of Pyramid-Net, PSM and DNU, several experiments (ablation experiments) were conducted, and the experimental results are shown in Table 4. The addition of the PSM module enables sparse detection of images and significantly improves the reasoning speed of the network. The addition of the DNU module enables information transfer between different layers, further improving the detection accuracy. The addition of the two modules significantly improves the performance of the model.

[0052] Table 4 Ablation experiment results Methods ACC(%) F1-Score(%) Kappa(%) MCC(%) Backbone 81.33 74.5 57 50 Backbone+PSM 92 86.96 81.19 81.19 Backbone+PSM+DNU 94.67 89.47 85.92 86.13 In addition, the PSM module generates a selection diagram for the detection area at different levels, which is visualized in the present invention. From the visualization, the distribution of the detection areas selected by the network in each layer can be intuitively observed, reflecting its sparse detection mechanism in the feature extraction process. This visualization not only helps pathologists understand the feature extraction preferences of the model at each layer, but also provides stronger interpretability in the case of correct and incorrect predictions, which facilitates in-depth evaluation of the network's performance on complex pathological images.

[0053] In a specific embodiment, in order to further evaluate the generalization ability of Pyramid-Net, this embodiment conducted experiments on the public dataset Camelyon16, and compared the results with other excellent models. The specific results are shown in Table 5. The experiment shows that the Pyramid-Net proposed in the present invention not only performs well on the lymph node dataset, but also shows good adaptability in other cancer tissue detection tasks, indicating that the model of the present invention has excellent generalization ability. The robust performance of Pyramid-Net on the new dataset further verifies its potential for cross-dataset migration and its application prospects in different pathological analysis tasks.

[0054] Table 5 Camelyon16 comparison results Methods year ACC(%) F1-Score(%) AB_MIL 2018 84.5 78 TRANS_MIL 2021 85.8 79.7 DS_MIL 2021 85.6 81.5 CLAM_MB_MIL 2021 82.3 77.4 CLAM_SB_MIL 2021 83.7 77.5 DTFT_MIL 2022 90.8 88.2 CIMIL 2024 89.75 - PAMIL 2024 91.6 91.3 Pyramid-Net(ours) - 92.31 90.7 Specifically, it should be noted that the Pyramid-Net mentioned in this embodiment 2 is a multi-scale classification model proposed by the present invention.

[0055] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0056] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined in the present invention may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in the present invention, but will conform to the widest scope consistent with the principles and novel features disclosed in the present invention.

Claims

1. A pathological image segmentation method based on multi-scale classification, characterized in that: include: Acquire a pathological image to be classified; Preprocess the pathological images to be classified; The preprocessed pathological image to be classified is input into the multi-scale classification model, the multi-scale features of the pathological image to be classified are extracted, and the category prediction result of the pathological image to be classified is output.

2. The pathological image segmentation method based on multi-scale classification according to claim 1, characterized in that: The multi-scale classification model adopts the Pyramid-Net architecture and introduces the PSM position selection module and the DNU directed node update module, where: The Pyramid-Net architecture includes a low-resolution detection network and a high-resolution detection network, which are used to detect suspicious areas in pathological images to be classified and locate cancerous tissues; The PSM position selection module is used to receive the suspicious area position information output by the low-resolution detection network, and feed the position information back to the high-resolution detection network to achieve sparse detection of pathological images; The DNU directed node update module comprehensively analyzes the detection results at different resolutions at the same location by simulating the information transmission mechanism in the graph neural network; Finally, the position information of cancerous tissue and attention images corresponding to different scales are output.

3. The pathological image segmentation method based on multi-scale classification according to claim 2, characterized in that: The PSM position selection module is used to receive the suspicious area position information output by the low-resolution detection network, and feed the position information back to the high-resolution detection network to achieve sparse detection of pathological images, including: Traverse and detect all patches at low resolution at the top of the pyramid in the Pyramid-Net architecture to obtain a preliminary category activation map; For each location The patch of which predicts the Softmax output of category c Calculated by the following formula: In the formula, Predict the category for the corresponding patch The confidence level, is the predicted score of the patch at position (x, y) in category c; The PSM position selection module selects the area that needs to be focused on in the next layer of the network based on the prediction results and spatial information of each patch. The PSM position selection module selects the area as follows: in, Represents the coordinates corresponding to the patch, Represents the number of layers of the WSI pyramid, , Represents the threshold for filtering patch confidence; In the next layer of the WSI pyramid, the patches corresponding to these key areas are found by comparing the position information and input into a higher resolution network for further detection. The selection of the next layer of patches is as follows: in, Represents the patch in the specified area. Represents the coordinates corresponding to the second layer patch, represents the nth key area of ​​the first layer, represents the kth box of the second layer of the WSI pyramid, and n represents the nth key area of ​​the first layer. If the kth box of the second layer is in the key area, it will be marked as a box to be detected and the next step of detection will be carried out. Through this layer-by-layer approach, sparse detection of pathological images can be completed.

4. The pathological image segmentation method based on multi-scale classification according to claim 2, characterized in that: The DNU directed node update module simulates the information transmission mechanism in the graph neural network and comprehensively analyzes the detection results at different resolutions at the same location, including: The DNU directed node update module regards the prediction result of each patch as a node in the pathological image, and the edges between nodes represent the neighbor relationship between patches in the same layer or the inclusion relationship between patches in different layers; A high-resolution patch is contained by multiple low-resolution patches, and the relationship between nodes is expressed as: In the formula, Represents the coordinates corresponding to the second layer patch, represents the updated confidence, Represents the case where two key regions contain the same patch. Represents the situation where the patch of the second layer is only included in one key area of ​​the first layer. , Represents the different weights given to different inclusion situations.

5. The pathological image segmentation method based on multi-scale classification according to claim 2, characterized in that: The final output of the position information of the cancerous tissue and the attention images corresponding to different scales include: An attention score is generated for each patch based on the final result of the DNU directed node update module, where each attention score reflects the relative importance of each region in lesion recognition at different scales, showing the focus of the Pyramid-Net architecture at each scale.

6. A pathological image segmentation system based on multi-scale classification, characterized in that: Executing a pathological image segmentation method based on multi-scale classification as described in any one of claims 1 to 5, comprising: A data acquisition module, used for acquiring pathological images to be classified; A data preprocessing module, used for preprocessing the pathological images to be classified; The classification detection module is used to input the preprocessed pathological image to be classified into the multi-scale classification model, extract the multi-scale features of the pathological image to be classified, and output the category prediction result of the pathological image to be classified.

Images