Tumor tissue cell image segmentation method, device, equipment, medium and product
Through the combination of CE-Net dual-branch network and ASPP/CBAM attention, the edge blur and adhesion problems in cell image segmentation are solved, segmentation accuracy and efficiency are improved, and automated cell segmentation is realized to assist pathological diagnosis.
Patent Information
- Application Number
- CN202410755124.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-12
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-06-12
AI Technical Summary
In traditional pathological diagnosis, cell image segmentation has time-consuming, highly dependent on personnel experience and possible human errors, and the cell edges are blurred, the texture is not obvious, and there are many adhesions and overlaps between cells, resulting in low segmentation accuracy.
Using a dual-branch network method based on CE-Net, combining ASPP module and CBAM attention, the boundary map and foreground map of cells are obtained, and multi-scale feature extraction and attention weighting are performed through the jump connection between feature encoder and context extractor. Fusion boundary map and foreground map are used to realize instance segmentation of cells using the watershed algorithm.
It effectively improves the accuracy and segmentation accuracy of tumor cell image segmentation, solves the problems of adhesion and overlap between cells, realizes automated cell segmentation, assists doctors in conducting quantitative analysis, and improves the efficiency of clinical pathological diagnosis.
Smart Images

Figure CN118447041B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment, medium and product for segmenting tumor tissue cell images. Background Art
[0002] In pathology, physicians diagnose cancer and assess prognosis by analyzing factors such as cell morphology, structural changes, quantity, and size in pathological images. However, traditional screening methods are time-consuming, highly dependent on personnel experience, and have the potential for human error. In addition, the shortage of pathologists leads to heavy workloads and delayed diagnosis.
[0003] As an important part of pathological diagnosis, cell image segmentation quality directly affects the diagnosis results of the disease. Histopathological images are affected by factors such as light and staining, and features such as cell edges and textures are not obvious. Cell adhesion and overlap often occur. These problems require in-depth research on cell segmentation methods to achieve automated and accurate cell segmentation.
[0004] In recent years, the application of computer-aided diagnosis methods in digital pathology has been increasing, effectively reducing the workload of pathologists and improving the accuracy of diagnosis. In particular, combining deep learning technology with pathology image analysis has shown good results.
[0005] In the past few years, many cell segmentation methods have emerged, however, these methods do not directly address the difficulties faced in the task of cell segmentation in pathological images. In tissue pathology cell images, cell nuclei are often mixed with other structures and noise, and cell boundary contours are blurred. Summary of the invention
[0006] The present invention provides a tumor tissue cell image segmentation method, device, equipment, medium and product to solve the problem of how to improve the accuracy of tumor tissue cell image segmentation.
[0007] The first aspect of the present invention provides a method for segmenting a tumor tissue cell image, comprising the following steps: acquiring a tumor tissue cell image; inputting the tumor tissue cell image into a segmentation model, the segmentation model outputting a boundary map and a foreground map of the tumor tissue cells, wherein the segmentation model comprises a feature encoder, a context extractor and a feature decoder, the jump connection between the feature encoder and the feature decoder adds an ASPP module and / or CBAM attention, the feature decoder splices and upsamples the feature maps extracted by the feature encoder and the context extractor, and the attention-weighted feature maps calculated by the CBAM attention to obtain a boundary map and a foreground map of the tumor tissue cells; and the boundary map and the foreground map are fused to obtain a segmentation result of the tumor tissue cell image.
[0008] Optionally, the ASPP module is used to splice the multi-scale feature maps to obtain a spliced feature map, and the CBAM attention calculates channel and spatial attention on the spliced feature map to obtain an attention-weighted feature map.
[0009] Optionally, the feature encoder includes multiple feature extraction blocks, wherein each feature extraction block includes a pooling layer and an SD block, the pooling layer downsamples the image, and the SD block extracts local features.
[0010] Optionally, the multiple feature extraction blocks include first to fifth feature extraction blocks, and the feature decoder includes first to fourth convolutions, wherein the second to fourth feature extraction blocks are jump-connected to the second to fourth convolutions to add an ASPP module and CBAM attention, and the fifth feature extraction block is jump-connected to the first four convolutions to add CBAM attention.
[0011] Optionally, before the tumor tissue cell image is input into the segmentation model, it also includes: obtaining a training sample image of the tumor tissue cell; adjusting the size of the training sample image according to the input size of the network structure of the segmentation model, and using data enhancement to expand the training sample image; dividing the expanded training sample image into a training set, a validation set and a test set, training the segmentation model based on the training set, validating the trained segmentation model based on the validation set, and testing the trained segmentation model based on the test set; during the training process, back propagation is performed based on the loss function of the segmentation model, wherein the first branch of the segmentation model is used to segment the boundary of the cell and the second branch is used to segment the foreground image of the cell, the loss function of the first branch is a binary cross entropy loss function, and the loss function of the second branch is a Dice loss function.
[0012] Optionally, the boundary map and the foreground map are fused to obtain the segmentation result of the tumor tissue cell image, including: calculating the Hadamard product of the boundary map and the foreground map to obtain a first map; performing a distance transformation on the foreground map to obtain a seed point, and instantiating the seed point to obtain a second map; adding the first map and the second map, using a watershed algorithm to start water injection from the seed point, and obtaining the segmentation result of the tumor tissue cell image when the water injection stops.
[0013] The second aspect of the present invention provides a tumor tissue cell image segmentation device, including: an acquisition module, used to acquire tumor tissue cell images; a segmentation module, used to input the tumor tissue cell images into a segmentation model, and the segmentation model outputs a boundary map and a foreground map of the tumor tissue cells, wherein the segmentation model includes a feature encoder, a context extractor and a feature decoder, and the jump connection between the feature encoder and the feature decoder is added with an ASPP module and / or CBAM attention, and the feature decoder splices and upsamples the feature maps extracted by the feature encoder and the context extractor, and the attention-weighted feature maps calculated by the CBAM attention to obtain a boundary map and a foreground map of the tumor tissue cells; a fusion module, used to fuse the boundary map and the foreground map to obtain the segmentation result of the tumor tissue cell image.
[0014] A third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the tumor tissue cell image segmentation method as described in the above embodiment.
[0015] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program or instructions stored thereon, and the computer program or instructions are executed by a processor to implement the tumor tissue cell image segmentation method as described in the above embodiment.
[0016] A fifth aspect of the present invention provides a computer program product, including a computer program or instructions, which, when executed, are used to implement the tumor tissue cell image segmentation method as described in the above embodiment.
[0017] Thus, the present invention includes the following beneficial effects:
[0018] The embodiment of the present invention obtains the foreground image and boundary image of cells by building a dual-branch supervision network to accurately segment cells, effectively solving the problem of cell adhesion and overlap in pathological images; selects the CE-Net model as the segmentation framework, introduces the CBAM attention and ASPP modules, and enhances the network's response to important areas while capturing multi-scale information, thereby effectively improving the accuracy and segmentation precision of tumor cell image segmentation; finally, the foreground image and boundary image of cells are fused, and the watershed algorithm is used to realize the instance segmentation of cells. Therefore, the embodiment of the present invention can automatically segment cells in images through computer vision technology, obtain the number, size, and distribution of cells, assist doctors in quantitative analysis of pathological images, and improve the efficiency of clinical pathological diagnosis.
[0019] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0021] Figure 1 is a flow chart of a tumor tissue cell image segmentation method according to an embodiment of the present invention;
[0022] Figure 2 is a structural diagram of an SD block according to an embodiment of the present invention;
[0023] Figure 3 is a structural diagram of a segmentation model according to an embodiment of the present invention;
[0024] Figure 4 is a flow chart of a tumor tissue cell image segmentation method according to an embodiment of the present invention;
[0025] Figure 5 is an image fusion flow chart according to an embodiment of the present invention;
[0026] Figure 6 is an exemplary diagram of a tumor tissue cell imaging device according to an embodiment of the present invention;
[0027] Figure 7 FIG. 4 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0028] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0029] In the related art, insufficient attention is paid to the boundary area of the cell nucleus, resulting in errors in the segmentation of overlapping cell nuclei. Therefore, the embodiment of the present invention proposes a tumor tissue cell image segmentation method, device, equipment, medium and product based on a dual-branch network of CE-Net, which obtains the boundary information of the cell while obtaining the cell information, improves the accuracy of segmentation, and effectively solves the above problems. The tumor tissue cell image segmentation method based on the dual-branch network of CE-Net will be described in conjunction with the accompanying drawings, as follows:
[0030] Figure 1 The present invention provides a flow chart of a tumor tissue cell image segmentation method according to an embodiment of the present invention.
[0031] like Figure 1 As shown, the tumor tissue cell image segmentation method includes the following steps:
[0032] In step S101, a tumor tissue cell image is acquired.
[0033] It is understandable that tumor tissue cell images are important tools in medical research and clinical diagnosis, and they can help doctors and researchers identify and understand the characteristics of tumors. The embodiments of the present invention can obtain tumor tissue cell images by medical imaging technology and other means.
[0034] In step S102, the tumor tissue cell image is input into the segmentation model, and the segmentation model outputs the boundary map and foreground map of the tumor tissue cells, wherein the segmentation model includes a feature encoder, a context extractor and a feature decoder, and the jump connection between the feature encoder and the feature decoder is added with an ASPP module and / or CBAM attention, and the feature decoder splices and upsamples the feature maps extracted by the feature encoder and the context extractor, and the attention-weighted feature maps calculated by the CBAM attention to obtain the boundary map and foreground map of the tumor tissue cells.
[0035] It can be understood that the embodiment of the present invention improves the CE-Net network model, adds the ASPP module and CBAM attention in the jump connection between the encoder and the decoder, and implements dual branches in the last module of the decoder to obtain the boundary map and foreground map of the cell.
[0036] Therefore, the embodiment of the present invention obtains the foreground image and boundary image of cells by building a dual-branch supervision network to accurately segment cells, effectively solving the problem of cell adhesion and overlap in pathological images. The CE-Net model is selected as the segmentation framework, and the CBAM attention and ASPP modules are introduced to capture multi-scale information while enhancing the network's response to important areas, thereby effectively improving the accuracy and precision of tumor cell image segmentation.
[0037] In an embodiment of the present invention, the feature encoder includes multiple feature extraction blocks, wherein each feature extraction block includes a pooling layer and an SD block, the pooling layer downsamples the image, and the SD block extracts local features; the ASPP module is used to splice multi-scale feature maps to obtain a spliced feature map, and the CBAM attention calculates channel and spatial attention on the spliced feature map to obtain an attention-weighted feature map.
[0038] It is understandable that the SD block directly splices the output of the previous layer to the input of the current layer, which can effectively avoid the vanishing gradient and accelerate the convergence of the network. Using CBAM attention can suppress irrelevant background area features and focus on cell boundaries. In the design of the ASPP module, three 3×3 convolutional layers can be used with different expansion rates of 1, 3, and 5. Since cells are small in the image, a smaller expansion rate is used to capture smaller features. In actual use, for parts with smaller cross-jump long connections, the convolutional layers in the ASPP module are reduced; since the lower the feature map, the larger its receptive field for the original image, which is equivalent to a larger scale itself, it only needs to be subjected to less multi-scale joint enhancement to achieve a balance of scale fusion at each layer.
[0039] Specifically, an SD block is added after each downsampling and upsampling stage, as Figure 2 As shown in Figure 1, the SD block can directly splice the output of the previous layer to the input of the current layer, which can avoid gradient disappearance and accelerate network convergence.
[0040] The specific process of adding the ASPP module and CBAM attention to the CE-Net network model is as follows: use the maximum pooling and SD blocks for downsampling in the encoder, obtain the multi-scale features {A1, A2, A3, A4, A5} through the ASPP module and concatenate them, and then use CBAM attention to calculate the channel and spatial attention of the concatenated feature map, and finally obtain the weighted initial feature map.
[0041] The definition of CBAM attention is as follows:
[0042] MC(F) = σ(MLP(AvgPool(F)) + MLP(MaxPool(F)));
[0043] Where, F: input feature map, a three-dimensional tensor, whose dimensions are usually C×H×W, C is the number of channels, H and W are the height and width of the feature map respectively; AvgPool ( F ) and MaxPool(F) MaxPool ( F ): respectively represent the input feature map F Apply average pooling and maximum pooling operations, which are usually used to extract global information in feature maps; MLP: Multilayer Perceptron, a fully connected neural network, is used to learn the nonlinear transformation of pooled features. In CBAM, MLP is used to map the globally pooled features from low dimensions back to the original feature dimensions; σ: activation function, usually the sigmoid function, used to convert the output of MLP into weights between 0 and 1, which will be used to weight the channels of the input features; MC(F): represents the output of CBAM attention, which is a weight map with the same dimension as the input F, used to weight the channels of the input feature map.
[0044] The feature map obtained after the encoder and context extractor is concatenated with the attention-weighted feature map and upsampled, and finally the dual branches are used to segment the cell information.
[0045] Furthermore, in an embodiment of the present invention, if Figure 3 As shown, multiple feature extraction blocks include first to fifth feature extraction blocks, and the feature decoder includes first to fourth convolutions, wherein the second to fourth feature extraction blocks are jump-connected with the second to fourth convolutions to add the ASPP module and CBAM attention, and the fifth feature extraction block is jump-connected with the first four convolutions to add the CBAM attention.
[0046] In an embodiment of the present invention, before the tumor tissue cell image is input into the segmentation model, it also includes: obtaining a training sample image of the tumor tissue cell; adjusting the size of the training sample image according to the input size of the network structure of the segmentation model, and using data enhancement to expand the training sample image; dividing the expanded training sample image into a training set, a verification set and a test set, training the segmentation model based on the training set, verifying the trained segmentation model based on the verification set, and testing the trained segmentation model based on the test set; during the training process, back propagation is performed based on the loss function of the segmentation model, wherein the first branch of the segmentation model is used to segment the boundary of the cell and the second branch is used to segment the foreground image of the cell, the loss function of the first branch is a binary cross entropy loss function, and the loss function of the second branch is a Dice loss function.
[0047] It is understandable that if Figure 4 As shown, the embodiment of the present invention can preprocess the data set, divide the training set, validation set and test set in the pathological image cell segmentation task; input the divided data set into the improved network model for feature extraction, train and test the model, and obtain the segmentation result. Among them, the samples are enlarged and reduced, randomly cropped, randomly rotated 90 degrees, and randomly mirrored to achieve the effect of data enhancement, improve the robustness and generalization ability of the model; the data set is pixel-level labeled, and each image, especially the corresponding mask image, has a background pixel value of 0 and a target pixel value of 1, and the training set, validation set and test set are divided according to the ratio of 8:1:1.
[0048] Specifically, (1) the size of the sample image is adjusted to the input size of the network structure, and the training samples are expanded using data augmentation; for example, the original image size is 500×500, and in order to adapt to the input size of the network, the image is compressed to 448×448. The samples and their labels in the training set are preprocessed and data augmented, including operations such as enlargement and reduction, random cropping, random 90° rotation, and random mirror flipping, to improve the robustness and generalization ability of the model.
[0049] (2) All datasets used are public datasets and are pixel-level annotated. For each sample, especially the corresponding mask image, the background pixel value is 0 and the target pixel value is 1. The training set, validation set, and test set are divided in a ratio of 8:1:1. The mask image is the foreground image.
[0050] (3) Use the improved model for training and testing, use the SD block in the encoder, add the ASPP module and CBAM attention in the jump connection, and output the foreground map and boundary map of the cell. Specifically: Use the improved model for training, calculate the loss function with the true label, perform back propagation, and use two branches to output the foreground map and boundary map of the cell. The first branch is used to segment the boundary of the cell, and the second branch is used to segment the mask of the cell.
[0051] The first branch uses a binary cross entropy loss function, which can be expressed as:
[0052]
[0053] Where N is the total number of pixels in the image, is the true value of the ith pixel, is the expected value of the i-th pixel. The second branch uses the Dice loss function, which can be expressed as:
[0054]
[0055] Among them, X represents the label value, Y represents the predicted value, and weighted cross entropy loss and Dice loss are used for supervised learning:
[0056] .
[0057] In step S103, the boundary image and the foreground image are fused to obtain the segmentation result of the tumor tissue cell image.
[0058] In an embodiment of the present invention, a boundary map and a foreground map are fused to obtain a segmentation result of a tumor tissue cell image, including: calculating the Hadamard product of the boundary map and the foreground map to obtain a first map; performing a distance transformation on the foreground map to obtain a seed point, and instantiating the seed point to obtain a second map; adding the first map and the second map, using a watershed algorithm to start water injection from the seed point, and obtaining a segmentation result of the tumor tissue cell image when the water injection stops.
[0059] It can be understood that the embodiment of the present invention obtains the foreground image and the boundary image of the cell by fusing the model, and then obtains the final segmentation result through some operations and the watershed algorithm. Specifically: Figure 4 As shown, the embodiment of the present invention utilizes distance transformation, Hadamard product and other operations to fuse the foreground image and the boundary image of cells, and then uses the watershed algorithm to implement instance segmentation of cells, thereby obtaining the number, size and distribution of cells.
[0060] Specifically, if Figure 5 As shown, the embodiment of the present invention uses the Hadamard product on the cell mask map and the boundary map to obtain a map H; then the cell mask map is distance transformed to obtain seed points, and the seed points are instantiated to obtain a map I, and the map I and the map H are added at the pixel level; finally, the watershed algorithm is used to obtain the result map of the cell instance segmentation.
[0061] According to the tumor tissue cell image segmentation method proposed in the embodiment of the present invention, a dual-branch supervision network is built to obtain the foreground image and boundary image of the cell to accurately segment the cell, effectively solving the problem of cell adhesion and overlap in pathological images; the CE-Net model is selected as the segmentation framework, and the CBAM attention and ASPP modules are introduced to enhance the network's response to important areas while capturing multi-scale information, thereby effectively improving the accuracy and segmentation precision of tumor cell image segmentation; finally, the foreground image and boundary image of the cell are fused, and the watershed algorithm is used to realize the instance segmentation of the cell. Therefore, the embodiment of the present invention can automatically segment the cells in the image through computer vision technology, obtain the number, size, and distribution of cells, assist doctors in quantitative analysis of pathological images, and improve the efficiency of clinical pathological diagnosis.
[0062] Next, the tumor tissue cell image segmentation device proposed in the embodiment of the present invention is described with reference to the accompanying drawings.
[0063] Figure 6 4 is a block diagram of a tumor tissue cell image segmentation device according to an embodiment of the present invention.
[0064] like Figure 6 As shown, the tumor tissue cell image segmentation device 10 includes: an acquisition module 100 , a segmentation module 200 and a fusion module 300 .
[0065] Among them, the acquisition module 100 is used to acquire the tumor tissue cell image; the segmentation module 200 is used to input the tumor tissue cell image into the segmentation model, and the segmentation model outputs the boundary map and foreground map of the tumor tissue cells, wherein the segmentation model includes a feature encoder, a context extractor and a feature decoder, and the jump connection between the feature encoder and the feature decoder is added with the ASPP module and / or CBAM attention, and the feature decoder splices and upsamples the feature map extracted by the feature encoder and the context extractor, and the attention-weighted feature map calculated by the CBAM attention to obtain the boundary map and foreground map of the tumor tissue cells; the fusion module 300 is used to fuse the boundary map and the foreground map to obtain the segmentation result of the tumor tissue cell image.
[0066] It should be noted that the above explanation of the embodiment of the tumor tissue cell image segmentation method is also applicable to the tumor tissue cell image segmentation device of this embodiment, and will not be repeated here.
[0067] According to the tumor tissue cell image segmentation device proposed in the embodiment of the present invention, a dual-branch supervision network is built to obtain the foreground image and boundary image of the cell to accurately segment the cell, effectively solving the problem of cell adhesion and overlap in pathological images; the CE-Net model is selected as the segmentation framework, and the CBAM attention and ASPP modules are introduced to enhance the network's response to important areas while capturing multi-scale information, thereby effectively improving the accuracy and segmentation precision of tumor cell image segmentation; finally, the foreground image and boundary image of the cell are fused, and the watershed algorithm is used to realize the instance segmentation of the cell. Therefore, the embodiment of the present invention can automatically segment the cells in the image through computer vision technology, obtain the number, size, and distribution of cells, assist doctors in quantitative analysis of pathological images, and improve the efficiency of clinical pathological diagnosis.
[0068] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:
[0069] A memory 701 , a processor 702 , and a computer program stored in the memory 701 and executable on the processor 702 .
[0070] When the processor 702 executes the program, the tumor tissue cell image segmentation method provided in the above embodiment is implemented.
[0071] Furthermore, the electronic device further comprises:
[0072] The communication interface 703 is used for communication between the memory 701 and the processor 702 .
[0073] The memory 701 is used to store computer programs that can be executed on the processor 702 .
[0074] The memory 701 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.
[0075] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0076] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.
[0077] The processor 702 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.
[0078] An embodiment of the present invention provides a computer-readable storage medium on which a computer program or instruction is stored. The computer program or instruction is executed by a processor to implement the tumor tissue cell image segmentation method as described in the above embodiment.
[0079] An embodiment of the present invention provides a computer program product, including a computer program or instructions. When the computer program or instructions are executed, they are used to implement the tumor tissue cell image segmentation method as described in the above embodiment.
[0080] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0081] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0082] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.
[0083] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, the steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.
[0084] A person of ordinary skill in the art may understand that all or part of the steps carried by the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.
[0085] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A tumor tissue cell image segmentation method, It is characterized in that The following steps are involved: Acquire tumor tissue cell images; The tumor tissue cell image is input into a segmentation model, and the segmentation model outputs a boundary map and a foreground map of the tumor tissue cells, wherein the segmentation model includes a feature encoder, a context extractor, and a feature decoder, and the jump connection between the feature encoder and the feature decoder is added with an ASPP module and / or CBAM attention. The feature decoder splices and upsamples the feature map extracted by the feature encoder and the context extractor, and the attention-weighted feature map calculated by the CBAM attention to obtain the boundary map and foreground map of the tumor tissue cells, and the ASPP module is used to splice the multi-scale feature map to obtain the spliced feature map. The feature map of the image is obtained by performing channel and spatial attention calculation on the concatenated feature map by the CBAM attention to obtain an attention-weighted feature map; the feature encoder includes a plurality of feature extraction blocks, wherein each feature extraction block includes a pooling layer and an SD block, the pooling layer downsamples the image, and the SD block extracts local features; the plurality of feature extraction blocks include first to fifth feature extraction blocks, the feature decoder includes first to fourth convolutions, wherein the second to fourth feature extraction blocks are connected to the second to fourth convolutions by jump connection to add the ASPP module and the CBAM attention, and the fifth feature extraction block is connected to the first four convolutions by jump connection to add the CBAM attention; The boundary map and the foreground map are fused to obtain a segmentation result of the tumor tissue cell image.
2. The tumor tissue cell image segmentation method according to claim 1, It is characterized in that Before the tumor tissue cell image is input into the segmentation model, the method further includes: Acquire training sample images of tumor tissue cells; Adjust the size of the training sample image according to the input size of the network structure of the segmentation model, and expand the training sample image using data enhancement; Dividing the expanded training sample images into a training set, a validation set and a test set, training the segmentation model based on the training set, validating the trained segmentation model based on the validation set, and testing the trained segmentation model based on the test set; During the training process, back propagation is performed based on the loss function of the segmentation model, wherein the first branch of the segmentation model is used to segment the boundary of the cell and the second branch is used to segment the foreground image of the cell, the loss function of the first branch is a binary cross entropy loss function, and the loss function of the second branch is a Dice loss function.
3. The tumor tissue cell image segmentation method according to claim 1, It is characterized in that The fusing of the boundary map and the foreground map to obtain the segmentation result of the tumor tissue cell image includes: Calculating the Hadamard product of the boundary image and the foreground image to obtain a first image; Performing distance transformation on the foreground image to obtain seed points, and instantiating the seed points to obtain a second image; The first image and the second image are added together, water injection is started from the seed point using a watershed algorithm, and a segmentation result of the tumor tissue cell image is obtained when the water injection stops.
4. A tumor tissue cell image segmentation device, It is characterized in that include: An acquisition module, used for acquiring tumor tissue cell images; A segmentation module is used to input the tumor tissue cell image into a segmentation model, and the segmentation model outputs a boundary map and a foreground map of the tumor tissue cells, wherein the segmentation model includes a feature encoder, a context extractor and a feature decoder, and the jump connection between the feature encoder and the feature decoder is added with an ASPP module and / or CBAM attention. The feature decoder splices and upsamples the feature map extracted by the feature encoder and the context extractor, and the attention-weighted feature map calculated by the CBAM attention to obtain the boundary map and foreground map of the tumor tissue cells, and the ASPP module is used to splice the multi-scale feature map to obtain The CBAM attention performs channel and spatial attention calculations on the spliced feature map to obtain an attention-weighted feature map; the feature encoder includes a plurality of feature extraction blocks, wherein each feature extraction block includes a pooling layer and an SD block, the pooling layer downsamples the image, and the SD block extracts local features; the plurality of feature extraction blocks include first to fifth feature extraction blocks, the feature decoder includes first to fourth convolutions, wherein the second to fourth feature extraction blocks and the second to fourth convolutions are jump-connected to add the ASPP module and the CBAM attention, and the fifth feature extraction block and the first four convolutions are jump-connected to add the CBAM attention; A fusion module is used to fuse the boundary image and the foreground image to obtain a segmentation result of the tumor tissue cell image.
5. An electronic device, It is characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the tumor tissue cell image segmentation method according to any one of claims 1 to 3.
6. A computer-readable storage medium having a computer program or instructions stored thereon, It is characterized in that When the computer program or instruction is executed, the tumor tissue cell image segmentation method according to any one of claims 1 to 3 is implemented.
7. A computer program product comprising a computer program or instructions, It is characterized in that When the computer program or instruction is executed, the tumor tissue cell image segmentation method according to any one of claims 1 to 3 is implemented.
