Pathological image segmentation method and device based on deep learning and readable storage medium thereof

Abstract

The invention provides a pathological image segmentation method and device based on deep learning and a readable storage medium. According to the invention, efficient and accurate segmentation is realized through a two-stage cascade processing strategy. The method comprises the following steps: firstly, carrying out downsampling on an original high-resolution pathological full-slice image to generate a low-resolution thumbnail; rapidly positioning the region of interest by combining Otsu method threshold segmentation with morphological processing, and generating an accurate bounding box through a two-stage rectangular frame merging algorithm; the method comprises the steps that firstly, boundary frame coordinates are mapped back to an original WSI, high-resolution sub-images are cut and extracted, a Mask2Former model is input for fine pixel-level segmentation, and a global organization structure and local cell details are considered through a mask attention mechanism and multi-level feature fusion. In addition, through a cross-scale coordinate mapping technology, a high-resolution training label is quickly generated based on low-resolution labeling, and the problem of scarcity of pathological image labeling data is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing and computer vision, and in particular to a pathological image segmentation method and device based on deep learning and a readable storage medium thereof, which are used for automatic recognition and accurate segmentation of whole-slice pathological images (WSI). Background Art

[0002] Automatic segmentation of digital pathology sections (WSI) is a key technology in the field of pathology AI. It can significantly improve diagnostic efficiency and standardization by identifying lesion areas and microstructures. Existing pathology image segmentation technology mainly faces the following challenges: Traditional methods use sliding window technology to divide high-resolution WSI into small blocks for processing. Although it can retain local details, it has low computational efficiency and is prone to boundary discontinuity and inconsistent segmentation between image blocks. Another method directly processes low-resolution images by downsampling, which improves the speed but seriously loses detail information, resulting in a decrease in the accuracy of detecting small lesions (such as lymph node micrometastasis); In addition, the single-stage segmentation model has difficulty in taking into account both global context and local fine features, and requires a large amount of computing resources when processing large WSIs, which cannot meet clinical real-time processing requirements.

[0003] The above problems limit the practicality of pathological image-assisted diagnosis systems, and an efficient and accurate segmentation solution is urgently needed. Summary of the invention

[0004] The embodiments of the present invention provide a pathological image segmentation method, device and readable storage medium based on deep learning, which address the problems of low efficiency, loss of details, discontinuous boundaries and high consumption of computing resources in current technologies, and the inability to simultaneously meet the requirements of high precision and real-time processing.

[0005] The core technology of this invention is to achieve efficient and accurate segmentation of pathological images through a two-stage cascade strategy of coarse segmentation (threshold + morphological processing) and fine segmentation (Mask2Former model), taking into account both global efficiency and local details.

[0006] In a first aspect, the present invention provides a pathological image segmentation method based on deep learning, the method comprising the following steps: Downsampling the original high-resolution pathology full-slice image to obtain a low-resolution pathology thumbnail; Perform coarse segmentation on low-resolution pathology thumbnails, locate the region of interest and generate the corresponding bounding box coordinates; Map the bounding box coordinates to the original high-resolution pathology full-slice image, and cut and extract the high-resolution sub-image of the corresponding area; The Mask2Former model is used to perform fine pixel-level segmentation on the high-resolution sub-image and output the segmentation result containing the outline of the target tissue structure.

[0007] Furthermore, the coarse segmentation process includes: The low-resolution pathological thumbnail was grayed and binarized by Otsu's method to obtain a binary image. Perform morphological closing and opening operations on the binary image to fill holes in the tissue area and remove isolated noise points; The bounding rectangle of each tissue region in the processed image is extracted, and the final bounding box coordinates of the region of interest are generated through a two-stage merging algorithm.

[0008] Furthermore, the two-stage merging algorithm includes: The first stage of merging: when the ratio of the overlapping area of ​​two external rectangular boxes to the area of ​​the smaller rectangular box exceeds a preset threshold, they are merged into one rectangular box; The second stage of merging: if one rectangular box contains the tissue area in another rectangular box, they are merged into one rectangular box; Repeat the above merging process until the number of rectangular boxes no longer changes, and use the union-find algorithm to achieve efficient merging.

[0009] Furthermore, the training data of the Mask2Former model is obtained in the following way: Manually annotate low-resolution pathology thumbnails to generate annotated data; Map the coordinates of the annotated data to the original high-resolution pathology full-slice image, and cut and generate labels corresponding to the high-resolution sub-images; The labels are manually fine-tuned to obtain accurate segmentation masks as training labels for the Mask2Former model.

[0010] Furthermore, the Mask2Former model achieves fine segmentation of pathological tissue through mask attention mechanism and multi-scale feature fusion.

[0011] Furthermore, the resolution of the low-resolution pathology thumbnail is 1024×1024.

[0012] Furthermore, it is suitable for pathological image segmentation of lymph nodes, tumor tissues or cell structures.

[0013] In a second aspect, the present invention provides a pathological image segmentation device based on deep learning, comprising: A downsampling module is used to downsample the original high-resolution pathology full-slice image to obtain a low-resolution pathology thumbnail; A coarse segmentation module is used to perform coarse segmentation processing on low-resolution pathology thumbnails, locate the region of interest and generate the corresponding bounding box coordinates; The coordinate mapping and image cutting module is used to map the bounding box coordinates to the original high-resolution pathology full-slice image and cut and extract the high-resolution sub-image of the corresponding area; The fine segmentation module is used to perform fine pixel-level segmentation on high-resolution sub-images using the Mask2Former model and output segmentation results containing the contours of the target tissue structure.

[0014] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned deep learning-based pathological image segmentation method.

[0015] In a fourth aspect, the present invention provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process, and the process includes the pathological image segmentation method based on deep learning described above.

[0016] The main contributions and innovations of the present invention are as follows: 1. Significantly improve segmentation efficiency Through the two-stage cascade strategy of coarse segmentation (low-resolution screening ROI) and fine segmentation (high-resolution local processing), the amount of calculation is greatly reduced. Compared with traditional sliding window or full-image high-resolution processing, the speed is significantly improved, which is especially suitable for real-time or near real-time analysis of large-size pathological sections (WSI).

[0017] 2. Take into account both global and local accuracy In the rough segmentation stage, threshold segmentation (such as Otsu's method) and morphological operations (closing operation, opening operation) are used to quickly locate the ROI, avoiding computational redundancy in the whole image processing; In the fine segmentation stage, Mask2Former's mask attention mechanism and multi-scale feature fusion are used to accurately identify tiny lesions (such as lymph node micrometastasis) and fuzzy boundaries, solving the problem of detail loss caused by downsampling methods.

[0018] 3. Solve the boundary discontinuity problem In the coarse segmentation, the union-find algorithm is used to merge the overlapping or included ROI boundary boxes, eliminating the fragmentation effect caused by traditional block processing and ensuring the regional continuity of the segmentation results.

[0019] 4. Data efficiency and versatility By expanding the data set through coordinate mapping, multi-scale training data can be automatically generated by simply annotating low-resolution thumbnails, significantly reducing annotation costs and solving the industry problem of scarce annotations for pathological images; The algorithm parameters (such as morphological kernel size and threshold) are adjustable to adapt to different staining conditions or tissue types (such as tumors and lymph nodes), and it has strong scalability.

[0020] Through the two-stage design of "rough screening + fine classification", the present invention comprehensively surpasses the existing technology in terms of efficiency, accuracy and robustness, and provides an efficient, accurate and feasible solution for intelligent auxiliary diagnosis of pathological images.

[0021] The details of one or more embodiments of the invention are set forth in the following drawings and description so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of a pathological image segmentation method based on deep learning according to an embodiment of the present invention; Figure 2 is a diagram of the Mask2Former model training process according to an embodiment of the present invention; Figure 3 is a flow chart of intelligently merging overlapping areas of the circumscribed rectangular frames of each area according to an embodiment of the present invention; Figure 4 is a flow chart of a rough area detection stage according to an embodiment of the present invention; Figure 5 is another flow chart of a pathological image segmentation method based on deep learning according to an embodiment of the present invention; Figure 6 is a flow chart of expanding data according to an embodiment of the present invention; Figure 7 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0024] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0025] The existing pathological image segmentation has problems such as low computational efficiency and insufficient segmentation accuracy (such as discontinuous boundaries and loss of details of tiny lesions).

[0026] Based on this, the present invention realizes efficient and accurate segmentation based on a two-stage cascade processing strategy to solve the problems existing in the prior art.

[0027] Embodiment 1 The present invention aims to propose a pathological image segmentation method based on deep learning, specifically, referring to Figure 1 and Figure 5 As shown, the method includes: S1, downsampling the original high-resolution pathology full-slice image to obtain a low-resolution pathology thumbnail; The purpose of this step is to downsample the original high-resolution WSI to a 1024×1024 thumbnail, compressing the processing volume of billions of pixels to millions, greatly reducing the computational complexity.

[0028] S2, performing rough segmentation processing on the low-resolution pathology thumbnail, locating the region of interest and generating the corresponding bounding box coordinates; This step uses Otsu's method for adaptive binarization, combining closing operations (filling holes) and opening operations (removing noise) to enhance the integrity of tissue regions; image inversion is used to highlight tissue regions to facilitate subsequent contour extraction.

[0029] A rectangular frame merging algorithm is also used: a two-step cyclic merging mechanism is designed (overlap ratio threshold merging + inclusion relationship detection), and a union-find algorithm is used to efficiently merge external rectangular frames to avoid incorrect segmentation of lymph node tissue and output accurate ROI coordinates.

[0030] S3, mapping the bounding box coordinates to the original high-resolution pathology full-slice image, cutting and extracting the high-resolution sub-image of the corresponding area; In this step, the ROI coordinates obtained by rough segmentation are mapped back to the original WSI, and a 1024×1024 high-resolution region image is cut out. For example, the coordinate mapping is achieved by a scaling factor k, where k is the ratio of the original WSI resolution to the low-resolution thumbnail resolution (e.g., k=WSI width / 1024), and the coordinates (x, y) of the bounding box correspond to (kx, ky) in the original image, and the corresponding region is cut out according to the sub-image resolution (1024×1024).

[0031] S4. Use the Mask2Former model to perform fine pixel-level segmentation on the high-resolution sub-image and output the segmentation result containing the outline of the target tissue structure.

[0032] Among them, according to the coordinate position of the bounding box in the original high-resolution pathological full-slice image, the segmentation results of each high-resolution sub-image are mapped back to the original image according to the pixel coordinates to obtain the final segmentation result. For example, the mapping step of the segmentation result includes: according to the original coordinate position of the bounding box, the mask in each sub-image is translated according to the absolute position, so that the segmentation mask of each high-resolution sub-image is mapped back to the original WSI according to the pixel coordinates, thereby generating a complete full-slice segmentation result.

[0033] This step uses the mask attention mechanism of Mask2Former to focus on the target area and filter out background interference; through multi-level feature extraction and fusion, it takes into account both the global context (tissue structure) and local details (cell features) to achieve pixel-level precise segmentation.

[0034] In this embodiment, if Figure 2 As shown, the specific training steps of the Mask2Former model are as follows: 1.1 Training Process This system takes the lymph node WSI image dataset as an example. The training process uses a coarse and fine two-stage cascade method to process pathological images. First, the original WSI is downsampled to obtain a low-resolution thumbnail with a resolution of 1024×1024. In the first stage, a coarse segmentation method is used to divide the thumbnail into two processing paths: one is mapped to a high-magnification image through the detection box coordinates and a higher-resolution image is cut out; the other is mapped to a high-magnification image through manually labeled coordinates to obtain the label corresponding to the larger magnification image. The images after these two processing paths are processed at a resolution of 1024×1024 (the cut images are also uniformly scaled to 1024*1024 for fine segmentation, and 1024*1024 is the input size of all segmentation models), and the labeled images are manually fine-tuned to obtain accurate segmentation masks. Finally, the high-resolution images obtained by cutting and their corresponding manually fine-tuned segmentation masks are used as training data for the training of the Mask2Former model, so that the model can accurately identify lymph node structures.

[0035] Among them, manual fine-tuning includes manually correcting the boundary contour of the segmentation mask to ensure that the error between the mask edge and the actual boundary of the target tissue does not exceed 2 pixels, or filling and repairing the discontinuous area of ​​the mask through image editing tools. Figure 6 As shown in the figure, by annotating low-resolution thumbnails and generating multi-scale annotation data through coordinate mapping, each thumbnail annotation can correspond to multiple high-resolution image labels, solving the problem of scarce pathological image annotation data and improving the efficiency of dataset construction.

[0036] 1.1.1 Stage 1: Rough Region Detection The core task of the first stage is to quickly and accurately lock the potential lesion area in WSI and build an efficient spatial screening mechanism to lay the foundation for subsequent fine segmentation. The rough area detection stage uses a series of image processing technologies and innovative algorithms to significantly reduce the computational complexity while ensuring detection accuracy.

[0037] The initial processing of pathological WSI begins with the analysis of low-resolution thumbnails. This strategy compresses the processing of billions of pixels to the megapixel level, achieving optimal allocation of computing resources. The thumbnails are first adaptively binarized using the Otsu's method, which determines the optimal threshold by maximizing the inter-class variance, effectively addressing the problems of illumination changes and uneven staining of different tissue sections. To enhance the recognition of the target area, the algorithm also performs an image inversion operation to highlight the tissue area, facilitating subsequent contour extraction.

[0038] Morphological operations play a key role in region detection. Adjustable kernel parameters are designed in the code implementation, which directly affect the scale characteristics of morphological transformation. The introduction of closing operation (dilation followed by erosion) solves the continuity problem of tissue boundaries, fills small holes, and enhances the integrity of the region; while opening operation (erosion followed by dilation) effectively eliminates scattered isolated noise points and improves the robustness of region recognition. This dual morphological processing strategy significantly improves the accuracy of boundary positioning while retaining the regional morphological characteristics.

[0039] like Figure 3 As shown, the overlapping areas of the bounding rectangles of each area will then be intelligently merged. The algorithm designs a two-step loop merging mechanism: In the first step, the ratio threshold controlled by the threshold parameter is used to merge the two rectangular boxes. When the ratio of the overlapping area of ​​the two rectangular boxes to the smaller area exceeds the threshold, the algorithm merges them into one rectangular box. The second step is to detect whether the area contained in the rectangular box contains the tissue of another lymph node. If so, the two rectangular boxes are merged. At the same time, the union-find algorithm is used to achieve efficient merging.

[0040] The above process is repeated until the number of rectangular frames no longer changes. This two-step cyclic merging algorithm can effectively avoid the situation where some lymph node tissues are incorrectly cut and separated.

[0041] like Figure 4 As shown in the figure, the final rough area detection stage outputs a set of merged bounding boxes, which accurately define the coordinates of the area worthy of attention, which is roughly as follows: The low-resolution pathological thumbnail was grayed and binarized by Otsu's method to obtain a binary image. Perform morphological closing and opening operations on the binary image to fill holes in the tissue area and remove isolated noise points; The bounding rectangle of each tissue region in the processed image is extracted, and the final bounding box coordinates of the region of interest are generated through a two-stage merging algorithm.

[0042] Among them, the morphological closing and opening operations are processed using a kernel with adjustable size, and the size of the kernel is adaptively set according to the average size of the tissue area in the pathological image. The preset threshold is 0.95~0.99, preferably 0.99, which is used to determine whether the overlap ratio of two rectangular boxes triggers merging.

[0043] These regional information will serve as spatial guidance for the second stage of fine segmentation, enabling the focused allocation of computing resources in high-value areas, thereby significantly improving the execution efficiency of the overall algorithm while ensuring the quality of analysis.

[0044] 1.1.2 The second stage: fine instance segmentation based on Mask2Former algorithm The fine instance segmentation stage constitutes the core of this algorithm, and its main goal is to perform high-precision pixel-level segmentation processing on the tissue candidate regions identified in the first stage. Lymph nodes in pathological images show large differences in volume and morphology, and blurred boundaries with adjacent tissues. They are often hidden in complex tissue backgrounds, making it difficult for traditional segmentation techniques to effectively deal with them, often resulting in insufficient feature extraction and false detection problems.

[0045] With the advancement of computer vision technology, Mask2Former, as a new generation of general image segmentation framework, provides a technical breakthrough for fine lymph node segmentation through its innovative mask attention mechanism and hierarchical feature integration method. The model can accurately capture the complex boundaries and internal structures of lymph nodes while maintaining efficient calculation. Mask2Former's outstanding performance in the field of pathological image segmentation is mainly due to three technical advantages: first, its uniquely designed mask attention mechanism can adaptively focus on the lymph node area and effectively filter out background interference information; second, the multi-level feature extraction and fusion strategy enables the model to grasp both macroscopic tissue structure and microscopic cell features; finally, the training strategy optimized for the characteristics of pathological images significantly improves the model's adaptability to lymph node morphological variations. These technical innovations enable Mask2Former to achieve a dual improvement in the accuracy and robustness of lymph node segmentation while maintaining a reasonable consumption of computing resources.

[0046] In this system, the Mask2Former model receives the high-resolution ROI region extracted in the first stage as input, and after deep learning network analysis and processing, it outputs accurate lymph node contours and internal structure segmentation results, providing reliable data support for subsequent pathological diagnosis and analysis. For example, the input features of the Mask2Former model include three-channel RGB images that have been preprocessed with pathological images (such as color normalization), and the model's multi-level feature fusion module contains at least 3 layers of feature maps with different resolutions (resolutions are 1 / 4, 1 / 8, and 1 / 16 of the original image size, respectively), and the fusion of global context and local details is achieved through cross-layer connections.

[0047] 1.2 Algorithm Effect Evaluation This evaluation uses the lymph node WSI image dataset as an example, and collects a total of 2108 full-slice images stored in SDPC format. The experimental platform uses a high-performance server equipped with 8 NVIDIA RTX 4090 cards to ensure efficient and stable deep learning model training. The experimental evaluation is carried out from multiple dimensions to fully verify the performance of the algorithm in pathological image segmentation tasks.

[0048] 1.2.1 First-stage segmentation effectiveness verification The effectiveness of the first-stage rough segmentation, as the basic link of the entire algorithm process, directly affects the quality of the subsequent fine segmentation. Missing any part will have a significant impact on the segmentation result. The experimental results show that the rough segmentation method based on threshold and morphological processing can accurately locate the potential lymph node area. Compared with the manually marked area, the detection rate has reached 100%. At the same time, the proportion of the processed area in the original WSI is greatly reduced, and the consumption of computing resources is significantly reduced. In addition, the average processing time of the rough segmentation stage is only 0.1 seconds per image, and the time consumption can be ignored, which fully reflects the efficiency advantage of the two-stage processing strategy.

[0049] 1.2.2 Segmentation Accuracy Evaluation of Two-stage Pathology Image Segmentation Algorithm In order to comprehensively evaluate the performance of the two-stage pathological image segmentation algorithm proposed in this paper, this paper compares it with the solution of using the Mask2Former model alone. The evaluation uses the performance indicators recognized in the field of computer vision: mAP (mean average precision) and mAR (mean average recall), which comprehensively reflect the segmentation performance of the model at different target scales and thresholds.

[0050] In the experiment, the present invention pays special attention to two high-demand indicators, mAP@75 (mAP with an IoU threshold of 0.75) and mAR@100 (mAR when retaining up to 100 predictions), to measure the performance of the algorithm in fine segmentation tasks.

[0051] The experimental results show that, from the overall performance point of view, the coarse-fine two-stage segmentation algorithm based on Mask2Former is significantly better than the solution of using Mask2Former alone in multiple key evaluation indicators: the overall mAP is improved from 0.9019 to 0.9179); mAP@75 is improved by 0.0111, indicating that this method has stronger segmentation ability under high precision threshold; it is particularly noteworthy that the mAR@1 index is significantly improved by 0.1399, which fully proves the excellent performance of the two-stage algorithm in single target recall ability. As shown in Table 1: Table 1

[0052] In addition, further analysis of the segmentation results of lymph nodes of different sizes revealed that the algorithm has a particularly significant improvement in the detection ability of small lymph nodes, which is of great significance for the early detection of small metastases in clinical practice. At the same time, in terms of processing efficiency, compared with the single model method, the two-stage algorithm maintains higher segmentation accuracy while reducing the overall processing time, providing an efficient and feasible solution for large-scale pathological image analysis in clinical practice.

[0053] Embodiment 2 Based on the same concept, the present invention also proposes a pathological image segmentation device based on deep learning, comprising: A downsampling module is used to downsample the original high-resolution pathology full-slice image to obtain a low-resolution pathology thumbnail; A coarse segmentation module is used to perform coarse segmentation processing on low-resolution pathology thumbnails, locate the region of interest and generate the corresponding bounding box coordinates; The coordinate mapping and image cutting module is used to map the bounding box coordinates to the original high-resolution pathology full-slice image and cut and extract the high-resolution sub-image of the corresponding area; The fine segmentation module is used to perform fine pixel-level segmentation on high-resolution sub-images using the Mask2Former model and output segmentation results containing the contours of the target tissue structure.

[0054] Using the device of the present invention, Figure 5 As shown in the figure, the segmentation process first receives the input pathological slice thumbnail, performs preliminary processing on the image through the first stage of the coarse segmentation algorithm, and automatically marks all areas of interest. Subsequently, the device maps these segmentation box coordinates to the high-resolution image and accurately cuts out the high-definition images of these areas from the original WSI. In the second stage, all the high-resolution regional images obtained by cutting are sent to the pre-trained Mask2Former model for fine segmentation. With its powerful mask attention mechanism and multi-scale feature fusion capabilities, the model can accurately identify the boundaries and internal structures of the segmented objects. Finally, the device splices and integrates the segmentation results of each area, outputs the segmentation results of the complete segmented object, and presents it as a complete image with precise contour markings. The entire process realizes an efficient conversion from low-resolution coarse screening to high-resolution fine segmentation, which not only ensures computational efficiency but also ensures segmentation accuracy. It is particularly suitable for processing large-scale pathological full-slice images. Accurate recognition and segmentation tasks.

[0055] In this embodiment, during the stitching process, the absolute position of the sub-image in the original WSI is first calculated based on the bounding box coordinates (x, y, width, height) recorded in the coarse segmentation stage, combined with the downsampling scaling factor (such as k = original resolution / thumbnail resolution). For example, if the thumbnail resolution is 1024×1024 and the original WSI is 10240×10240, the scaling factor k=10, and the coordinates in the thumbnail (x=200, y=300) correspond to the original image (2000, 3000). The resolution of the cut sub-image is 1024×1024, so it occupies the area of ​​2000-3023×3000-4023 in the original image.

[0056] After the segmentation is completed, the mask results of each sub-image are embedded in the corresponding area of ​​the original image according to the coordinate position to obtain the final segmentation result.

[0057] Embodiment 3 This embodiment also provides an electronic device, referring to Figure 7 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0058] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0059] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0060] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0061] The processor 402 implements any one of the deep learning-based pathological image segmentation methods in the above-mentioned embodiments by reading and executing computer program instructions stored in the memory 404 .

[0062] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0063] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0064] Input / output devices 408 are used to input or output information.

[0065] Embodiment 4 This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process. The process includes the pathological image segmentation method based on deep learning according to the first embodiment.

[0066] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0067] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0068] Embodiments of the present invention can be implemented by computer software, which is executable by a data processor of a mobile device, such as in a processor entity, or implemented by hardware, or implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros can be stored in any device readable data storage medium, and they include program instructions for performing specific tasks. Computer program products can include one or more computer executable components configured to perform embodiments when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow in the figure can represent program steps, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored in physical media such as memory chips or storage blocks implemented in processors, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs. Physical media are non-transient media.

[0069] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0070] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A pathological image segmentation method based on deep learning, characterized in that: The following steps are involved: Downsampling the original high-resolution pathology full-slice image to obtain a low-resolution pathology thumbnail; Performing a coarse segmentation process on the low-resolution pathology thumbnail, locating the region of interest and generating corresponding bounding box coordinates; Mapping the bounding box coordinates to the original high-resolution pathology full-slice image, cutting and extracting a high-resolution sub-image of the corresponding area; The Mask2Former model is used to perform fine pixel-level segmentation on the high-resolution sub-image, and a segmentation result including the outline of the target tissue structure is output.

2. A pathological image segmentation method based on deep learning as claimed in claim 1, characterized in that: The coarse segmentation process comprises: graying and performing Otsu method binarization processing on the low-resolution pathological thumbnail to obtain a binary image; Performing morphological closing and opening operations on the binary image to fill holes in the tissue area and remove isolated noise points; The bounding rectangle of each tissue region in the processed image is extracted, and the final bounding box coordinates of the region of interest are generated through a two-stage merging algorithm.

3. A pathological image segmentation method based on deep learning as claimed in claim 2, characterized in that: The two-stage merging algorithm includes: The first stage of merging: when the ratio of the overlapping area of ​​two external rectangular boxes to the area of ​​the smaller rectangular box exceeds a preset threshold, they are merged into one rectangular box; The second stage of merging: if one rectangular box contains the tissue area in another rectangular box, they are merged into one rectangular box; Repeat the above merging process until the number of rectangular boxes no longer changes, and use the union-find algorithm to achieve efficient merging.

4. A pathological image segmentation method based on deep learning as claimed in claim 1, characterized in that: The training data of the Mask2Former model is obtained in the following way: Manually annotating the low-resolution pathology thumbnails to generate annotated data; Mapping the coordinates of the annotated data to the original high-resolution pathology full-slice image, and cutting and generating labels corresponding to the high-resolution sub-images; The labels are manually fine-tuned to obtain accurate segmentation masks as training labels for the Mask2Former model.

5. A pathological image segmentation method based on deep learning as claimed in claim 1, characterized in that: The Mask2Former model achieves fine segmentation of pathological tissue through mask attention mechanism and multi-scale feature fusion.

6. A pathological image segmentation method based on deep learning as claimed in claim 1, characterized in that: The resolution of the low-resolution pathology thumbnail is 1024×1024.

7. A pathological image segmentation method based on deep learning as described in any one of claims 1 to 6, characterized in that: Suitable for pathological image segmentation of lymph nodes, tumor tissue or cell structures.

8. A pathological image segmentation device based on deep learning, characterized in that: include: A downsampling module is used to downsample the original high-resolution pathology full-slice image to obtain a low-resolution pathology thumbnail; A coarse segmentation module is used to perform coarse segmentation processing on low-resolution pathology thumbnails, locate the region of interest and generate the corresponding bounding box coordinates; The coordinate mapping and image cutting module is used to map the bounding box coordinates to the original high-resolution pathology full-slice image and cut and extract the high-resolution sub-image of the corresponding area; The fine segmentation module is used to perform fine pixel-level segmentation on high-resolution sub-images using the Mask2Former model and output segmentation results containing the contours of the target tissue structure.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the deep learning-based pathological image segmentation method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, wherein the process includes the deep learning-based pathological image segmentation method according to any one of claims 1 to 7.

Images