Cell segmentation method and device combining residual connection and multi-task multi-weight loss

Abstract

A cell segmentation method and device combining residual connection and multi-task multi-weight loss, the device comprising: (1) opening a medical cell image, intercepting different styles and characteristics of cells to establish a cell image dataset; (3) using a label making program to make cell labels; (3) splitting the cell image dataset into a training set, a test set and a validation set according to a ratio of 8:1:1; (4) using the cell data training set to train the cell segmentation network with multi-weight and residual connection; (5) using the test set to perform loss verification evaluation on the training model of each round; (6) using the trained model to verify the validation set to finally evaluate the model precision; (7) judging the result of model evaluation, when the model evaluation precision >= 90% or meets the expectation, step (8) is executed, otherwise step (1) is executed to increase the sample quantity of the dataset; (8) using the trained model to predict target images. The present application has significant advantages in dense medical cell segmentation and can effectively improve the cell segmentation precision.

Description

Technical Field

[0001] This invention relates to the field of cell segmentation technology, and more specifically, to a cell segmentation method and apparatus that combines residual connections and multi-task multi-weight loss. Background Technology

[0002] Currently, in the medical field, research methods for various diseases are gradually shifting from macroscopic to microscopic, and from passive treatment to proactive observation and prevention. This has rendered the traditional medical model, where doctors rely solely on examination reports for diagnosis, increasingly inadequate. Consequently, research into the relationship between cell morphology, quantity, and various diseases is rapidly increasing. Various cell segmentation and sampling methods are the first and crucial step in this research. A cell segmentation neural network that can efficiently and flexibly balance various factors is of great significance for medical professionals in significantly improving the feasibility of subsequent research and reducing the time cost of initial data collection.

[0003] Early research on medical image segmentation largely relied on traditional network frameworks, utilizing features from labeled data to optimize segmentation performance. However, as is well known, medical images are diverse, acquired through various methods, and their quality is limited by equipment precision. Therefore, traditional segmentation neural networks are insufficient for specific types of medical images, resulting in extremely low efficiency. With the application of deep learning, researchers have trained networks on various image datasets, using convolutional neural networks (CNNs) to learn and extract image features. CNNs learn models from massive amounts of data, adjusting the model's fitting ability through parameter settings of convolutional, pooling, and fully connected layers, as well as attention mechanisms.

[0004] For cell image segmentation, traditional segmentation neural networks are currently widely used. However, the diversity of medical cell images makes it difficult for traditional segmentation neural networks to accurately segment cells of different shapes and densities, thus failing to adapt to segmentation tasks that balance various data categories and multiple targets. Against this backdrop, this invention proposes a cell segmentation network that combines residual connections and multi-task, multi-weight loss. Summary of the Invention

[0005] To overcome the aforementioned shortcomings of existing technologies and improve the accuracy of cell image segmentation, this invention proposes a cell segmentation method and apparatus that combines residual connectivity and multi-task multi-weight loss.

[0006] This invention proposes a method combining residual connections with multi-task outputs to construct a multi-output deep network model that integrates skip connections and residual connections for medical cell image segmentation. Residual connection blocks effectively promote information transfer and reduce the gradient vanishing problem, significantly improving the convergence speed of deep network training. The two task outputs from the deep and shallow neural networks can be combined with different loss functions to synthesize a weighted loss function, enhancing the model's generalization ability and neural network flexibility. Using residual connection blocks and multi-output weighted functions greatly improves the training efficiency and network generalization ability of the medical cell image segmentation model.

[0007] The technical solution to achieve the objective of this invention is:

[0008] A cell segmentation method combining residual connectivity and multi-task multi-weight loss includes the following steps:

[0009] (1) Open medical cell images and extract cells with different styles and features to create a cell image dataset;

[0010] (2) Use a label-making program to create cell labels;

[0011] (3) Split the cell image dataset into training set, test set and validation set in a ratio of 8:1:1;

[0012] (4) Train the multi-weighted and residual-connected cell segmentation network using the cell data training set;

[0013] (5) Use the test set to perform loss validation evaluation on the trained model in each round;

[0014] (6) Validate the trained model using the validation set and finally evaluate the model accuracy.

[0015] (7) Judge the model evaluation results. If the model evaluation accuracy is >= 90% or meets the expectations, proceed to step (8); otherwise, proceed to step (1) to increase the number of dataset samples.

[0016] (8) Use the trained model to predict the target image.

[0017] The label making process described in step (2) is characterized by binarizing the captured image, filling the cell portion with black, filling the irrelevant portion with white, and finally scaling the image to a size of 256*256 and storing the label file.

[0018] The multi-weighted cell segmentation network described in step (4) and using residual connections is shown below:

[0019] (4.1) In the downsampling phase, the DOWN downsampling module, which includes maxpool, convolution, RELU function, SEBlock and Shortcut, and the ResidualBlock sampling module, which includes residual connections, are used alternately. In the upsampling phase, the UP upsampling module, which includes maxpool, convolution and RELU function, is used.

[0020] (4.2) During the downsampling phase, the DOWN module and the ResidualBlock module are used alternately. The use of one DOWN sampling module and one ResidualBlock sampling module is called performing one downsampling. When performing three downsampling and four downsampling, an output is generated respectively. The two outputs are connected to the three-stage and four-stage upsampling modules respectively to generate the final outputs OUT1 and OUT2 of this network.

[0021] Furthermore, the outputs OUT1 and OUT2 of the two tasks finally generated in step (4.2) will be synthesized into a final weighted loss function using Diceloss and BCEloss respectively. The weights of Diceloss and BCEloss are 2:3, and the overall loss function of this network structure is:

[0022] L=λ*L D +β*L B

[0023] L D Let L be the Diceloss loss function. B Let be the BCEloss loss function, and λ and β be the weights of each loss function. Here, λ is 0.4 and β is 0.6.

[0024] Furthermore, the outputs OUT1 and OUT2 of the two tasks generated in step (4.2) are synthesized into a final weighted loss function using Diceloss and BCEloss, respectively. The final result of this network depends on the output OUT2 of the deep network (matching the BCEloss function), while the output OUT1 of the shallow network (matching the Diceloss function) serves as an incentive during learning to promote pixel-level true label matching.

[0025] The verification evaluation feature described in step (6) is to compare the pixel-level predicted image with the original label and generate a similarity percentage.

[0026] The prediction feature of the target image described in step (8) is to use prediction code to give the prediction result of the image, wherein the part predicted as a cell is displayed in black and the other parts are displayed in white.

[0027] A second aspect of the invention relates to a cell segmentation apparatus combining residual connectivity and multi-task multi-weight loss, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a cell segmentation method combining residual connectivity and multi-task multi-weight loss according to the present invention.

[0028] A third aspect of the invention relates to a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements a cell segmentation method of the invention that combines residual connections and multi-task multi-weight loss.

[0029] This invention presents a cell segmentation network combining residual connections and multi-task, multi-weight loss. It is a deep network structure with multiple outputs using residual connections, aiming to address the problems of overfitting in deep networks and the inflexibility of single loss function weights, which limit cell segmentation accuracy. In this invention, an intermediate block based on residual connections is added to the skip connections used in the upsampling stage, making the network more robust while mitigating the overfitting problem of deep networks. Multi-task output is employed, using deep and shallow networks in combination with training outputs, and different loss functions are combined with different weights to synthesize a novel weight loss function, making the proposed network more flexible in learning cell segmentation images. The final model output depends on the results of the deep network, while the output of the shallow network is more inclined to incentivize model learning. This invention has significant advantages in intensive medical cell segmentation and can effectively improve cell segmentation accuracy.

[0030] The beneficial effects of this invention are:

[0031] This approach addresses the limitations of traditional cell image segmentation methods, which rely on a single loss function, have limited network depth, and suffer from poor generalization. By incorporating residual connection blocks, deeper networks can converge faster, facilitate information transfer, and minimize overfitting. Furthermore, by combining the multi-task outputs of deep and shallow networks to match different loss functions and assigning varying weights, a new weighted loss function is synthesized. This allows the network to focus on more information within the image, thereby improving the efficiency and accuracy of cell segmentation. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the method for model training using this network according to the present invention.

[0033] Figure 2 This is a specific process for labeling cell segmentation datasets according to the present invention.

[0034] Figure 3 This is a schematic diagram of the specific model architecture of the present invention.

[0035] Figures 4(a)-4(c)Figure 4(a) shows the first set of comparisons of the effects of the present invention on a common UET network; Figure 4(b) shows the original image, Figure 4(c) shows the typical UET prediction image, and Figure 4(c) shows the prediction results using the network of this implementation.

[0036] Figures 5(a)-5(c) Figure 5(a) shows the second set of comparisons of the effects of the present invention on a common UET network; Figure 5(a) is the original image, Figure 5(b) is a typical UET prediction image, and Figure 5(c) is the prediction result image using the network of this implementation.

[0037] Figures 6(a)-6(c) Figure 6(a) shows the third set of comparisons of the effects of the present invention on a common UET network; Figure 6(b) shows the original image, Figure 6(c) shows the typical UET prediction image, and Figure 6(c) shows the prediction results using the network of this implementation. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] Example 1

[0040] like Figure 1 As shown, a cell segmentation method using multi-weighting is presented. Figure 1 A flowchart illustrating the cell image segmentation process is provided. (Refer to...) Figure 1 The steps of this invention are as follows:

[0041] (1) Open medical cell images and extract cells with different styles and features to create a cell image dataset;

[0042] The specific annotation process is as follows: Figure 2 As shown, the specific process of cell dataset annotation mainly includes three stages:

[0043] a) Batch annotation stage. The cell patterns used for segmentation are first-round simple annotations performed using binarization tool code;

[0044] b) Manual inspection and supplementation stage. After the data has been labeled by the program, it is then manually processed to remove noise points and manually delineate areas with severe adhesion.

[0045] c) Data augmentation stage. Data augmentation is performed during the dataset construction process using methods such as rotation, scaling, and cropping.

[0046] (2) The cell image dataset was split into training set, test set and validation set in a ratio of 8:1:1;

[0047] (3) The cell segmentation network with multi-output multi-weight loss and residual connection is trained using the cell data training set;

[0048] Step 1: Connect the output of the deep network to the Dice loss function with a weight of 0.4, and connect the output of the shallow network to the BCE loss function with a weight of 0.6.

[0049] Step 2: Use the Adam trainer with a learning rate of 0.04 and 200 training epochs.

[0050] The loss function of the overall objective function of this network structure can be expressed as:

[0051] L=λ*L D +β*L B ,

[0052] L D Let L be the Diceloss loss function. B Let be the BCEloss loss function, and λ and β be the weights of each loss function. Here, λ is 0.4 and β is 0.6.

[0053] (4) During training, the test set data is used to perform validation loss detection in each round until the training is completed and the final model is output.

[0054] Step 1, the method to verify the loss is to calculate the matching of pixels. First, calculate the total number N of all pixels of the label.

[0055] Step 2: Compare each pixel in the predicted image, and count the number of identical pixels as M.

[0056] Step 3: Calculate the final similarity ratio λ = M / N * 100.

[0057] (5) Validate the trained model using a cell segmentation image validation set to evaluate the model’s retrieval accuracy.

[0058] Step 1: Perform image prediction on the data in the prediction set.

[0059] Step 2: Compare each predicted image with the original label image at the pixel level and output the accuracy.

[0060] Step 3: If the accuracy meets expectations, proceed to step (6); otherwise, return to step (1) to expand and enhance the data.

[0061] (6) Perform final cell prediction.

[0062] Step 1: Prepare the cell images that need to be predicted.

[0063] Step 2: Use the trained model to make predictions.

[0064] Example 2

[0065] A second aspect of the invention relates to a cell segmentation apparatus combining residual connectivity and multi-task multi-weight loss, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a cell segmentation method combining residual connectivity and multi-task multi-weight loss according to the present invention.

[0066] Example 3

[0067] A third aspect of the invention relates to a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements a cell segmentation method of the invention that combines residual connections and multi-task multi-weight loss.

[0068] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A cell segmentation method combining residual connectivity and multi-task multi-weight loss, comprising the following steps: (1) Open medical cell images and extract cells with different styles and features to create a cell image dataset; (2) Use a label-making program to create cell labels; (3) Split the cell image dataset into training set, test set and validation set in a ratio of 8:1:1; (4) Train the multi-weighted and residual-connected cell segmentation network using the cell data training set; The multi-weighted cell segmentation network using residual connections includes: (4.1) In the downsampling phase, the DOWN downsampling module, which includes maxpool, convolution, RELU function, SEBlock and Shortcut, and the ResidualBlock sampling module, which includes residual connections, are used alternately. In the upsampling phase, the UP upsampling module, which includes maxpool, convolution and RELU function, is used. (4.2) During the downsampling phase, the DOWN module and the ResidualBlock module are used alternately; the use of one DOWN sampling module and one ResidualBlock sampling module is called performing one downsampling. When performing three downsampling and four downsampling, an output will be generated respectively, and the two outputs will be connected to the three-time and four-time upsampling modules to generate the final outputs OUT1 and OUT2 of this network. The outputs OUT1 and OUT2 of the two tasks generated in step (4.2) will be synthesized using Diceloss and BCEloss respectively to form a final weighted loss function. The overall loss function of the cell segmentation network is: L = λ*L D +β*L B L D Let L be the Diceloss loss function. B Let BCEloss be the loss function, and λ and β be the weights of each loss function, respectively. The final result of the cell segmentation network depends on the output OUT2 of the deep network, while the output OUT1 of the shallow network serves as an incentive during learning to promote pixel-level true label matching. (5) Use the test set to perform loss validation evaluation on the trained model in each round; (6) Validate the trained model using the validation set and finally evaluate the model accuracy; (7) Judge the model evaluation results. If the model evaluation accuracy is >= 90% or meets the expectations, execute step (8); otherwise, execute step (1) to increase the number of dataset samples. (8) Use the trained model to predict the target image.

2. The cell segmentation method combining residual connectivity and multi-task multi-weight loss as described in claim 1, characterized in that, The label creation described in step (2) includes: binarizing the captured image, filling the cell parts with black, filling the irrelevant parts with white, and finally scaling the image to a size of 256*256 and storing the label file.

3. The cell segmentation method combining residual connectivity and multi-task multi-weight loss as described in claim 1, characterized in that, In step (4.2), the weights of Diceloss and BCEloss are 2:3; λ is 0.4 and β is 0.

6.

4. The cell segmentation method combining residual connectivity and multi-task multi-weight loss as described in claim 1, characterized in that, The verification evaluation feature described in step (6) is to compare the pixel-level predicted image with the original label and generate a similarity percentage.

5. The cell segmentation method combining residual connectivity and multi-task multi-weight loss as described in claim 1, characterized in that, The prediction feature of the target image described in step (8) is to use prediction code to give the prediction result of the image, wherein the part predicted as a cell is displayed in black and the other part is displayed in white.

6. A cell segmentation device combining residual connectivity and multi-task multi-weight loss, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a cell segmentation method combining residual connectivity and multi-task multi-weight loss as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements a cell segmentation method combining residual connections and multi-task multi-weight loss as described in any one of claims 1-5.

Images