A combined global regularization and local adversarial learning method for cell nucleus detection in pathological images
By combining global regularization and local adversarial learning, the problems of false detection and high computational cost in semi-supervised cell nucleus detection are solved, and efficient and accurate cell nucleus detection in pathological images is achieved.
Patent Information
- Application Number
- CN202310052538.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-02
- Publication Date
- 2025-12-02
- Estimated Expiration
- 2043-02-02
AI Technical Summary
Existing semi-supervised deep learning cell nucleus detection methods are prone to producing erroneous detection results in pathological images, and methods based on consistency regularization have high computational costs and lack contextual information when detecting the entire image, which affects the detection effect.
By employing a method combining global regularization and local adversarial learning, a cell nucleus detection network and a discriminator network are constructed. Global consistency regularization and local adversarial learning are combined, and the loss function is established using equations (1)-(8) for training, thereby achieving semi-supervised detection of pathological images.
It improves the accuracy and efficiency of cell nucleus detection in pathological images, mitigates the impact of background noise, makes full use of image-level contextual information, and enhances the local higher-order consistency of cell nucleus detection.
Smart Images
Figure CN116309310B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of semi-supervised cell nucleus detection technology in pathological images, specifically a method for cell nucleus detection in pathological images that combines global regularization and local adversarial learning. Background Technology
[0002] The goal of cell nucleus detection in pathological images is to accurately locate cell nuclei within these images using artificial intelligence (AI) technology. This involves providing the coordinates of the cell nucleus's center position or determining its classification through further calculations. The morphology and spatial distribution of cell nuclei in pathological images can accurately reflect cancer type and clinical grade; therefore, researching precise automated cell nucleus detection methods is crucial for pathological image analysis. However, labeling cell nuclei in pathological images is time-consuming and tedious. Researchers have proposed using semi-supervised deep learning based on pseudo-labels or consistency regularization to improve cell nucleus detection methods, achieving significant progress.
[0003] However, existing semi-supervised deep learning methods for cell nucleus detection still have certain problems. On the one hand, due to the large amount of background noise in pathological images, the model is prone to generating incorrect detection results or false labels during prediction, leading to convergence difficulties or biased prediction results in existing pseudo-label-based semi-supervised cell nucleus detection methods. On the other hand, while existing semi-supervised methods based on consistency regularization can avoid the problem of biased prediction results caused by false labels, they are only applicable to the detection of cell nuclei in pathological image blocks. When detecting the entire image, this method needs to extract image blocks one by one using the sliding window method and perform prediction, which greatly increases the computational cost. Furthermore, since each pathological image block is detected independently during the detection process, contextual information at the pathological image level is lost, thus affecting the overall performance of the method in cell nucleus detection. Summary of the Invention
[0004] This invention addresses the shortcomings of existing technologies by proposing a combined global regularization and local adversarial learning method for cell nucleus detection in pathological images. This method aims to fully utilize the contextual information of cell nuclei at the pathological image level and enhance the local high-order consistency of cell nucleus detection results, thereby improving the accuracy of semi-supervised cell nucleus detection.
[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:
[0006] The present invention provides a method for detecting cell nuclei in pathological images that combines global regularization and local adversarial learning, characterized by the following steps:
[0007] Step 1: Obtain cell nucleus detection data and perform preprocessing;
[0008] Several pathological images were acquired and normalized and augmented to obtain a sample set; the pathological image samples in the sample set were then randomly divided into labeled sample sets. and unlabeled sample set in, G represents the nth labeled pathological image sample. n Representing an image The corresponding actual distribution map of cell nuclei. Let m represent the m-th unlabeled pathological image sample, where n = 1, 2, ..., N, and m = 1, 2, ..., M; N represents the number of labeled samples, and M represents the number of unlabeled samples.
[0009] The pathological image sample The coordinates of the location of any cell nucleus in the array are denoted as in, Indicates that the i-th cell nucleus is in The x-coordinate in the middle, Indicates that the i-th cell nucleus is in The x-coordinate in the diagram, K n Representing pathological image samples The total number of cell nuclei;
[0010] Step 2: Construct a cell nucleus detection model consisting of a cell nucleus detection network and a discriminator network for local pathological images;
[0011] Step 2.1: Construct a cell nucleus detection network consisting of an encoder and a decoder;
[0012] The encoder contains C1 convolutional layers and Q residual coding modules, and the decoder contains C2 deconvolutional layers; each residual coding module consists of W concatenated residual blocks, and each residual block consists of C3 convolutional layers and their skip connections.
[0013] Set the stride of all convolutional layers to c, and the stride of the deconvolutional layers to 2c; set the activation function of the first C1-1 convolutional layers and C2 deconvolutional layers to ReLU; set the activation function of the C1th convolutional layer to Softmax.
[0014] Step 2.2: Construct a discriminator network for local pathological images;
[0015] The discriminator network comprises C4 convolutional layers, R max-pooling layers, and F fully connected layers. The stride of all convolutional layers is set to c. The activation functions of all convolutional layers and the first F-1 fully connected layers are ReLU functions. The activation function of the Fth fully connected layer is the Softmax function.
[0016] Step 3: Supervised learning of labeled pathological image samples is performed by combining supervised cell nucleus detection and local adversarial learning;
[0017] Step 3.1: Take the nth labeled pathological image sample The input to the cell nucleus detection network first passes through The processing of each convolutional layer is then sequentially processed by Q residual coding modules before being input into the remaining... The processing is carried out in several convolutional layers. The feature map output from the second residual coding module is concatenated with the feature map output from the C1th convolutional layer after a convolution, according to their dimensions, to output the final encoded feature map.
[0018] The encoded feature map After processing by C2 deconvolution layers in the decoder, the output is... Corresponding cell nucleus location probability map
[0019] Step 3.2: Establish the detection loss function L using equation (1). det , used for supervised learning of cell nuclear detection networks;
[0020]
[0021] In equation (1), l bce (·) denotes the binary cross-entropy loss function;
[0022] Step 3.3: From the labeled pathological images Probability map of cell nucleus location and the actual distribution map of cell nuclei G n Extract pathological local images for adversarial training.
[0023] According to the local region coordinates of the i-th cell nucleus From respectively G n Extracting local pathological image patches from the corresponding source Local location probability map Local distribution map of cell nuclei Among them, h and w are the height and width of the local region;
[0024] Define tags when When, it indicates that the probability map is derived from the local location. and local pathological image blocks The stitching together yields a local pathological image; when At that time, it indicates the distribution of local cell nuclei. and local pathological image blocks The images were stitched together to obtain local pathological images;
[0025] Step 3.5: Transfer the local pathological image block and local location probability map In the discriminator network, the data first passes through the first three convolutional layers, then through R max-pooling layers, and finally through the remaining C4-3 convolutional layers and F fully connected layers. The Fth fully connected layer outputs the local prediction probability of the cell nucleus.
[0026] The local pathological image patch and local distribution diagram of cell nuclei The input is processed in the discriminator network, resulting in the output of the Fth fully connected layer showing the local prediction probability of the true cell nucleus.
[0027] Step 3.6: Use equation (2) to establish the identification loss L of labeled pathological images. sl_D This is used for adversarial learning of the discriminator network;
[0028]
[0029] Step 3.7: Use equation (3) to establish the labeled pathological image generation loss L sl_G , used for adversarial learning of cell nucleus detection networks;
[0030]
[0031] Step 3.8: Use equation (4) to establish the local adversarial loss L of the labeled pathological image. sl_adv :
[0032] L sl_adv =L sl_D +L sl_G (4)
[0033] Step 4: Perform unsupervised learning on unlabeled pathological images by combining global consistency regularization and local adversarial learning;
[0034] Step 4.1: Take the m-th unlabeled pathological image sample Perform a flipping operation to generate the corresponding perturbation image. Will and After being input into the cell nucleus detection network for appropriate processing, the results are obtained. Corresponding cell nucleus location probability map and Corresponding cell nucleus location probability map
[0035] Step 4.2: Use equation (5) to establish the global regularization loss L G_cons This is used for unsupervised learning of the cell nucleus detection network;
[0036]
[0037] In equation (5), l mse (·) represents the mean squared error loss function; Represents the probability diagram of cell nucleus location Perform a flip operation;
[0038] Step 4.2, from the unlabeled pathological images Perturbation image Probability map of cell nucleus location and the probability map of cell nucleus locations generated from the perturbation image Extract pathological local images for adversarial training.
[0039] The probability map of the cell nucleus location Perform nonmaximum suppression to obtain the m-th unlabeled pathological image sample. and its perturbation image Pseudo-position coordinates of the cell nucleus in, This indicates that the j-th cell nucleus is in the image. The x-coordinate in the middle, This indicates that the j-th cell nucleus is in the image. The x-coordinate in the diagram, K m Representing an image The total number of cell nuclei;
[0040] According to the local region coordinates of the j-th cell nucleus From respectively as well as Extract the corresponding local pathological image blocks and Local location probability map and
[0041] Define tags when When, it indicates that the probability map is derived from the local location. and local pathological image patches of perturbed images The stitching together yields a local pathological image; when When, it indicates that the probability map is derived from the local location. and local pathological image blocks The images were stitched together to obtain local pathological images;
[0042] Step 4.3: Transfer the local pathological image block and The input is processed in the discriminator network, resulting in the output of the Fth fully connected layer: the local prediction probability of the cell nucleus in the original image.
[0043] Local location probability map and The input discriminator network performs corresponding processing, resulting in the output of the Fth fully connected layer showing the local prediction probability of the augmented image cell nucleus.
[0044] Step 4.4: Use equation (6) to establish the identification loss L for unlabeled pathological images. ul_D This is used for adversarial learning of the discriminator network;
[0045]
[0046] Step 4.5: Use equation (7) to establish the labelless pathological image generation loss L. ul_G , used for adversarial learning of cell nucleus detection networks;
[0047]
[0048] Step 4.6: Use equation (8) to establish the local adversarial loss L of the final unlabeled pathological image. ul_adv :
[0049] L ul_adv =L ul_D +L ul_G (8)
[0050] Step 5: Construct the overall loss function and use an alternating iterative optimization method to semi-supervised train the cell nucleus detection model to obtain the trained cell nucleus detection model, which is used to detect cell nuclei in pathological image data to obtain the corresponding detection results.
[0051] The method for detecting cell nuclei in pathological images by combining global regularization and local adversarial learning described in this invention is also characterized in that step 5 includes the following steps:
[0052] Step 5.1: Construct the overall loss function L of the cell nucleus detection model using equation (9). total :
[0053] L total =L det +αL G_cons +βL L_adv (9)
[0054] In equation (9), L L_adv Indicates localized combat losses, and L L_adv =L sl_adv +L ul_adv α and β represent the global regularization loss L, respectively. G_cons and local combat losses L L_adv The corresponding weighting coefficients;
[0055] Step 5.2: Set the total number of training steps to T; set the initial learning rate to l1; set the decay steps of the learning rate at the three internal locations to t1, t2 and t3 respectively, and the corresponding learning rates after decay to l2, l3 and l4 respectively; set the number of labeled pathological images and unlabeled pathological images sampled in each training session to B1 and B2 respectively.
[0056] Step 5.3: Following the initial learning rate l1, perform t4 supervised learning iterations on the cell nucleus detection network using labeled pathological image samples, and calculate the detection loss function L. det To update the network parameters, unlabeled pathological image samples were added, and the cell nucleus detection model was trained alternately with supervised and unsupervised learning according to learning rates l2, l3, and l4, and decay steps t1, t2, and t3, and the overall loss function L was calculated. total The model parameters are updated until the total number of training steps T is reached, thereby obtaining a trained cell nucleus detection model.
[0057] Step 5.4: Use the trained cell nucleus detection model to perform cell nucleus detection on the pathological image data to obtain the corresponding detection results.
[0058] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program that supports the processor in executing the pathological image cell nucleus detection method, and the processor is configured to execute the program stored in the memory.
[0059] The present invention provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, performs the steps of the pathological image cell nucleus detection method.
[0060] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0061] 1. The method of the present invention first utilizes a global regularization loss function, enabling the detection network to directly extract features and perform consistency calculations on the entire unlabeled pathological image, thereby avoiding the use of the sliding window method to extract image blocks one by one and make predictions. Furthermore, it fully utilizes the contextual information at the pathological image level, improving the inference efficiency of the detection network and its ability to represent cell nucleus features.
[0062] 2. In addition to global computation, this invention proposes a local adversarial learning technique to enhance the local spatial continuity and higher-order consistency of the cell nucleus location probability map, thereby alleviating the problem of deviation in detection results caused by a large amount of background noise in the image and significantly improving the performance of cell nucleus detection in semi-supervised pathological images. Attached Figure Description
[0063] Figure 1 This is a schematic diagram of the process of the present invention;
[0064] Figure 2 This is a diagram of the detection network structure of the present invention;
[0065] Figure 3 This is a diagram of the discriminator network structure of the present invention;
[0066] Figure 4 This is a schematic diagram of the global consistency regularization and local consistency adversarial learning technique of the present invention;
[0067] Figure 5 This is a diagram showing the detection results of the present invention. Detailed Implementation
[0068] In this embodiment, a method for detecting cell nuclei in pathological images that combines global regularization and local adversarial learning techniques is described, such as... Figure 1 As shown, the procedure is as follows:
[0069] Step 1: Obtain cell nucleus detection data and perform preprocessing;
[0070] Several pathological images were acquired and normalized and augmented to obtain a sample set; the pathological image samples in the sample set were then randomly divided into labeled sample sets. and unlabeled sample set in, G represents the nth labeled pathological image sample. n Representing an image The corresponding actual distribution map of cell nuclei. Let m represent the m-th unlabeled pathological image sample, where n = 1, 2, ..., N, and m = 1, 2, ..., M; N represents the number of labeled samples, and M represents the number of unlabeled samples.
[0071] Pathological image samples The coordinates of the location of any cell nucleus in the array are denoted as in, Indicates that the i-th cell nucleus is in The x-coordinate in the middle, Indicates that the i-th cell nucleus is in The x-coordinate in the diagram, K n Representing pathological image samples The total number of cell nuclei;
[0072] In this embodiment, 50 pathological images were selected from the SemiBCN dataset as labeled samples and 100 pathological images as unlabeled samples; the size of each pathological image was normalized to 512×512.
[0073] Step 2: Construct a detection model consisting of a cell nucleus detection network and a discriminator network for local pathological images;
[0074] Step 2.1: Construct a cell nucleus detection network consisting of an encoder and a decoder;
[0075] Cell nucleus detection network such as Figure 2 As shown, the encoder contains C1 convolutional layers and Q residual coding modules, and the decoder contains C2 deconvolutional layers; each residual coding module consists of W concatenated residual blocks, and each residual block consists of C3 convolutional layers and their skip connections; in specific implementations, C1 and Q are both 3, C2 and C3 are 2, and W is 9.
[0076] Set the stride of all convolutional layers to c, and the stride of the deconvolutional layers to 2c; set the activation function of the first C1-1 convolutional layers and C2 deconvolutional layers to the ReLU function; set the activation function of the C1th convolutional layer to the Softmax function; in the specific implementation, c is set to 3.
[0077] Step 2.2: Construct a discriminator network for local pathological images;
[0078] Discriminator networks such as Figure 3 As shown, the discriminator network contains C4 convolutional layers, R max pooling layers, and F fully connected layers. The stride of all convolutional layers is set to c. The activation functions of all convolutional layers and the first F-1 fully connected layers are ReLU functions. The activation function of the Fth fully connected layer is the Softmax function. In a specific embodiment, C4 is 5, R is 1, and F is 2.
[0079] Step 3: Supervised learning of labeled pathological image samples is performed by combining supervised cell nucleus detection and local adversarial learning;
[0080] Step 3.1, supervised nuclear detection process as follows: Figure 4 As shown in the upper left section, it represents the nth labeled pathological image sample. The input to the cell nucleus detection network first passes through The processing of each convolutional layer is then sequentially processed by Q residual coding modules before being input into the remaining... The processing is carried out in several convolutional layers. The feature map output from the second residual coding module is concatenated with the feature map output from the C1th convolutional layer after a convolution, according to their dimensions, to output the final encoded feature map.
[0081] Encoded feature map After processing by C2 deconvolution layers in the decoder, the output is... Corresponding cell nucleus location probability map
[0082] Step 3.2: Establish the detection loss function L using equation (1). det , used for supervised learning of cell nuclear detection networks;
[0083]
[0084] In equation (1), l bce (·) denotes the binary cross-entropy loss function;
[0085] Step 3.3: To alleviate the problem of biased detection results caused by a large amount of background noise in pathological images, it is considered to use labeled pathological images... Probability map of cell nucleus location and the actual distribution map of cell nuclei G n Local adversarial learning is performed on extracted local images to enhance the local spatial continuity and higher-order consistency of the cell nucleus location probability map, thereby improving detection accuracy. The process of performing local adversarial learning on labeled pathological image samples is as follows: Figure 4 As shown in the upper right section, it first appears from labeled pathological images. Probability map of cell nucleus location and the actual distribution map of cell nuclei G n Extract pathological local images for adversarial training.
[0086] According to the local region coordinates of the i-th cell nucleus From respectively G n Extracting local pathological image patches from the corresponding source Local location probability map Local distribution map of cell nuclei Where h and w represent the height and width of the local region, respectively; in the specific embodiment, both h and w are 32.
[0087] Define tags when When, it indicates that the probability map is derived from the local location. and local pathological image blocks The stitching together yields a local pathological image; when At that time, it indicates the distribution of local cell nuclei. and local pathological image blocks The images were stitched together to obtain local pathological images;
[0088] Step 3.5: Extract local pathological image blocks and local location probability map In the discriminator network, the data first passes through the first three convolutional layers, then through R max-pooling layers, and finally through the remaining C4-3 convolutional layers and F fully connected layers. The Fth fully connected layer outputs the local prediction probability of the cell nucleus.
[0089] Local pathological image patches and local distribution diagram of cell nuclei The input is processed in the discriminator network, resulting in the output of the Fth fully connected layer showing the local prediction probability of the true cell nucleus.
[0090] Step 3.6: Use equation (2) to establish the identification loss L of labeled pathological images. sl_D This is used for adversarial learning of the discriminator network;
[0091]
[0092] Step 3.7: Use equation (3) to establish the labeled pathological image generation loss L sl_G , used for adversarial learning of cell nucleus detection networks;
[0093]
[0094] Step 3.8: Use equation (4) to establish the local adversarial loss L of the labeled pathological image. sl_adv :
[0095] L sl_adv =L sl_D +L sl_G (4)
[0096] Step 4: Perform unsupervised learning on unlabeled pathological images by combining global consistency regularization and local adversarial learning;
[0097] Step 4.1: Utilize global consistency regularization technology to enable the detection network to directly extract features and perform consistency calculations on the entire unlabeled pathological image. This allows for full utilization of contextual information at the pathological image level, alleviating computational pressure and improving the inference efficiency and characterization ability of the detection network for cell nucleus features. The global consistency regularization cell nucleus detection process is as follows: Figure 4 As shown in the lower left section, it first samples the m-th unlabeled pathological image. Perform a flipping operation to generate the corresponding perturbation image. Will and After being input into the cell nucleus detection network for appropriate processing, the results are obtained. Corresponding cell nucleus location probability map and Corresponding cell nucleus location probability map
[0098] Step 4.2: Use equation (5) to establish the global regularization loss L G_cons This is used for unsupervised learning of the cell nucleus detection network;
[0099]
[0100] In equation (5), l mse (·) represents the mean squared error loss function; Represents the probability diagram of cell nucleus location Perform a flip operation;
[0101] Step 4.2: Similar to labeled data, to mitigate the impact of significant background noise on performance, data is extracted from unlabeled pathological images. Perturbation image Probability map of cell nucleus location and the probability map of cell nucleus locations generated from the perturbation image Local adversarial learning is performed on unlabeled pathological image samples to enhance the local spatial continuity and higher-order consistency of the cell nucleus location probability map, thereby improving detection accuracy. The process of performing local adversarial learning on unlabeled pathological image samples is as follows: Figure 4 As shown in the lower right section, it first appears from unlabeled pathological images. Perturbation image Probability map of cell nucleus location and the probability map of cell nucleus locations generated from the perturbation image Extract pathological local images for adversarial training.
[0102] Probability map of cell nucleus location Perform nonmaximum suppression to obtain the m-th unlabeled pathological image sample. and its perturbation image Pseudo-position coordinates of the cell nucleus in, This indicates that the j-th cell nucleus is in the image. The x-coordinate in the middle, This indicates that the j-th cell nucleus is in the image. The x-coordinate in the diagram, K m Representing an image The total number of cell nuclei; in specific implementation, the radius and threshold in the non-maximum inhibition operation are set to 6 and 0.2, respectively;
[0103] According to the local region coordinates of the j-th cell nucleus From respectively as well as Extract the corresponding local pathological image blocks and Local location probability map and
[0104] Define tags when When, it indicates that the probability map is derived from the local location. and local pathological image patches of perturbed images The stitching together yields a local pathological image; when When, it indicates that the probability map is derived from the local location. and local pathological image blocks The images were stitched together to obtain local pathological images;
[0105] Step 4.3: Extract local pathological image blocks and The input is processed in the discriminator network, resulting in the output of the Fth fully connected layer: the local prediction probability of the cell nucleus in the original image.
[0106] Local location probability map and The input discriminator network performs corresponding processing, resulting in the output of the Fth fully connected layer showing the local prediction probability of the augmented image cell nucleus.
[0107] Step 4.4: Use equation (6) to establish the identification loss L for unlabeled pathological images. ul_D This is used for adversarial learning of the discriminator network;
[0108]
[0109] Step 4.5: Use equation (7) to establish the labelless pathological image generation loss L. ul_G , used for adversarial learning of cell nucleus detection networks;
[0110]
[0111] Step 4.6: Use equation (8) to establish the local adversarial loss L of the final unlabeled pathological image. ul_adv :
[0112] L ul_adv =L ul_D +L ul_G (8)
[0113] Step 5: Construct the overall loss function and use an alternating iterative optimization method to semi-supervised train the detection model to obtain the trained cell nucleus detection model;
[0114] Step 5.1: Construct the overall loss function L of the detection model using equation (9). total :
[0115] L total =L det +αL G_cons +βL L_adv (9)
[0116] In equation (9), L L_adv Indicates localized combat losses, and L L_adv =L sl_adv +L ul_adv α and β represent the global regularization loss L, respectively. G_cons and local combat losses L L_adv The corresponding weighting coefficients T, in specific implementations, α and β are taken as 10.0 and 1.0 respectively;
[0117] Step 5.2: Set the total number of training steps to T, which is set to 100 in practice; set the initial learning rate to l1, which is set to 10 in practice. -2 The decay steps of the learning rate at the three internal locations are set as t1, t2, and t3, respectively. In the specific implementation, t1, t2, and t3 are 30, 50, and 70, respectively, and the corresponding learning rates after decay are l2, l3, and l4, each with a value of 10. -3 10 -4 and 10 -5 The number of labeled and unlabeled pathological images sampled in each training session is set to B1 and B2, respectively. In practice, both B1 and B2 are set to 4.
[0118] Step 5.3: Following the initial learning rate l1, perform t4 supervised learning iterations on the cell nucleus detection network using labeled pathological image samples, and calculate the detection loss function L. detTo update the network parameters, unlabeled pathological image samples were added, and the detection model was trained alternately with supervised and unsupervised learning according to learning rates l2, l3, and l4, and decay steps t1, t2, and t3. The overall loss function L was calculated. total The model parameters are updated until the total number of training steps T is reached, thus obtaining a trained detection model.
[0119] Step 5.4: Use the trained detection model to perform cell nucleus detection on the pathological image data to obtain the corresponding detection results.
[0120] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described pathological image cell nucleus detection method, and the processor is configured to execute the program stored in the memory.
[0121] In this embodiment, a computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the above-described pathological image cell nucleus detection method.
[0122] The detection results of this invention are as follows Figure 5 As shown in the figure, circles represent the actual labeled areas, and dots represent the predicted cell nucleus locations. Furthermore, to quantitatively evaluate the performance of the proposed method, this embodiment presents a performance comparison of the proposed method with supervised cell nucleus detection methods such as U-Net, SFCN-OPI, and POI, as well as semi-supervised cell nucleus detection methods such as Self-training, Mean Teacher, and HydraMix on the Semi-BCN dataset, as shown in Table 1.
[0123] Table 1 Comparison with other methods
[0124]
[0125] In this invention, when performing cell nucleus detection, a global consistency regularization technique is first used to train the detection model on unlabeled data. This successfully alleviates the problem of increased computational cost caused by extracting image patches for detection in existing consistency methods. Furthermore, it fully utilizes the contextual information at the pathological image level, improving the inference efficiency of the detection network and its ability to represent cell nucleus features. Building upon this, to overcome the impact of significant background noise in pathological images on performance, a local consistency adversarial learning technique is used to constrain the detection results in local regions of the cell nucleus. This enhances the local spatial continuity and higher-order consistency of the cell nucleus location probability map, thereby improving the network's detection performance.
Claims
1. A method for detecting cell nuclei in pathological images that combines global regularization and local adversarial learning, characterized in that, Includes the following steps: Step 1: Obtain cell nucleus detection data and perform preprocessing; Several pathological images were acquired and normalized and augmented to obtain a sample set; the pathological image samples in the sample set were then randomly divided into labeled sample sets. and unlabeled sample set in, G represents the nth labeled pathological image sample. n Representing an image The corresponding actual distribution map of cell nuclei. Let m represent the m-th unlabeled pathological image sample, where n = 1, 2, ..., N, and m = 1, 2, ..., M; N represents the number of labeled samples, and M represents the number of unlabeled samples. The pathological image sample The coordinates of the location of any cell nucleus in the array are denoted as in, Indicates that the i-th cell nucleus is in The x-coordinate in the middle, Indicates that the i-th cell nucleus is in The x-coordinate in the diagram, K n Representing pathological image samples The total number of cell nuclei; Step 2: Construct a cell nucleus detection model consisting of a cell nucleus detection network and a discriminator network for local pathological images; Step 2.1: Construct a cell nucleus detection network consisting of an encoder and a decoder; The encoder contains C1 convolutional layers and Q residual coding modules, and the decoder contains C2 deconvolutional layers; each residual coding module consists of W concatenated residual blocks, and each residual block consists of C3 convolutional layers and their skip connections. Set the stride of all convolutional layers to c, and the stride of the deconvolutional layers to 2c; set the activation function of the first C1-1 convolutional layers and C2 deconvolutional layers to ReLU; set the activation function of the C1th convolutional layer to Softmax. Step 2.2: Construct a discriminator network for local pathological images; The discriminator network comprises C4 convolutional layers, R max-pooling layers, and F fully connected layers. The stride of all convolutional layers is set to c. The activation functions of all convolutional layers and the first F-1 fully connected layers are ReLU functions. The activation function of the Fth fully connected layer is the Softmax function. Step 3: Supervised learning of labeled pathological image samples is performed by combining supervised cell nucleus detection and local adversarial learning; Step 3.1: Take the nth labeled pathological image sample The input to the cell nucleus detection network first passes through The processing of each convolutional layer is then sequentially processed by Q residual coding modules before being input into the remaining... The process is carried out in several convolutional layers. The feature map output from the second residual coding module is concatenated with the feature map output from the C1th convolutional layer after a convolution, according to their dimensions, to output the final encoded feature map. The encoded feature map After being processed by C2 deconvolution layers in the decoder, the output I is... n l Corresponding cell nucleus location probability map Step 3.2: Establish the detection loss function L using equation (1). det , used for supervised learning of cell nuclear detection networks; In equation (1), l bce (·) denotes the binary cross-entropy loss function; Step 3.3: From the labeled pathological images Probability map of cell nucleus location and the actual distribution map of cell nuclei G n Extract pathological local images for adversarial training. According to the local region coordinates of the i-th cell nucleus From respectively G n Extracting local pathological image patches from the corresponding source Local location probability map Local distribution map of cell nuclei Among them, h and w are the height and width of the local region; Define tags when When, it indicates that the probability map is derived from the local location. and local pathological image blocks The stitching together yields a local pathological image; when At that time, it indicates the distribution of local cell nuclei. and local pathological image blocks The images were stitched together to obtain local pathological images; Step 3.5: Transfer the local pathological image block and local location probability map In the discriminator network, the data first passes through the first three convolutional layers, then through R max-pooling layers, and finally through the remaining C4-3 convolutional layers and F fully connected layers. The Fth fully connected layer outputs the local prediction probability of the cell nucleus. The local pathological image patch and local distribution diagram of cell nuclei The input is processed in the discriminator network, resulting in the output of the Fth fully connected layer showing the local prediction probability of the true cell nucleus. Step 3.6: Use equation (2) to establish the identification loss L of labeled pathological images. sl_D , used for adversarial learning of the discriminator network; Step 3.7: Use equation (3) to establish the labeled pathological image generation loss L sl_G , used for adversarial learning of cell nucleus detection networks; Step 3.8: Use equation (4) to establish the local adversarial loss L of the labeled pathological image. sl_adv : L sl_adv =L sl_D +L sl_G (4) Step 4: Perform unsupervised learning on unlabeled pathological images by combining global consistency regularization and local adversarial learning; Step 4.1: Take the m-th unlabeled pathological image sample Perform a flipping operation to generate the corresponding perturbation image. Will and After being input into the cell nucleus detection network for appropriate processing, the results are obtained. Corresponding cell nucleus location probability map and Corresponding cell nucleus location probability map Step 4.2: Use equation (5) to establish the global regularization loss L G_cons This is used for unsupervised learning of the cell nucleus detection network; In equation (5), l mse (·) represents the mean squared error loss function; Represents the probability diagram of cell nucleus location Perform a flip operation; Step 4.2, from the unlabeled pathological images Perturbation image Probability map of cell nucleus location and the probability map of cell nucleus locations generated from the perturbation image Extract pathological local images for adversarial training. The probability map of the cell nucleus location Perform nonmaximum suppression to obtain the m-th unlabeled pathological image sample. and its perturbation image Pseudo-position coordinates of the cell nucleus in, This indicates that the j-th cell nucleus is in the image. The x-coordinate in the middle, This indicates that the j-th cell nucleus is in the image. The x-coordinate in the diagram, K m Representing an image The total number of cell nuclei; According to the local region coordinates of the j-th cell nucleus From respectively as well as Extract the corresponding local pathological image blocks and Local location probability map and Define tags when When, it indicates that the probability map is derived from the local location. and local pathological image patches of perturbed images The stitching together yields a local pathological image; when When, it indicates that the probability map is derived from the local location. and local pathological image blocks The images were stitched together to obtain local pathological images; Step 4.3: Transfer the local pathological image block and The input is processed in the discriminator network, resulting in the output of the Fth fully connected layer: the local prediction probability of the cell nucleus in the original image. Local location probability map and The input discriminator network performs corresponding processing, resulting in the output of the Fth fully connected layer showing the local prediction probability of the augmented image cell nucleus. Step 4.4: Use equation (6) to establish the identification loss L for unlabeled pathological images. ul_D , used for adversarial learning of the discriminator network; Step 4.5: Use equation (7) to establish the labelless pathological image generation loss L. ul_G , used for adversarial learning of cell nucleus detection networks; Step 4.6: Use equation (8) to establish the local adversarial loss L of the final unlabeled pathological image. ul_adv : L ul_adv =L ul_D +L ul_G (8) Step 5: Construct the overall loss function and use an alternating iterative optimization method to semi-supervised train the cell nucleus detection model to obtain the trained cell nucleus detection model, which is used to detect cell nuclei in pathological image data to obtain the corresponding detection results.
2. The method for detecting cell nuclei in pathological images by combining global regularization and local adversarial learning according to claim 1, characterized in that, Step 5 includes the following steps: Step 5.1: Construct the overall loss function L of the cell nucleus detection model using equation (9). total : L total =L det +αL G_cons +βL L_adv (9) In equation (9), L L_adv Indicates localized combat losses, and L L_adv =L sl_adv +L ul_adv α and β represent the global regularization loss L, respectively. G_cons and localized combat losses L L_adv The corresponding weighting coefficients; Step 5.2: Set the total number of training steps to T; set the initial learning rate to l1; set the decay steps of the learning rate at the three internal locations to t1, t2 and t3 respectively, and the corresponding learning rates after decay to l2, l3 and l4 respectively; set the number of labeled pathological images and unlabeled pathological images sampled in each training session to B1 and B2 respectively. Step 5.3: Following the initial learning rate l1, perform t4 supervised learning iterations on the cell nucleus detection network using labeled pathological image samples, and calculate the detection loss function L. det To update the network parameters, unlabeled pathological image samples were added, and the cell nucleus detection model was trained alternately with supervised and unsupervised learning according to learning rates l2, l3, and l4, and decay steps t1, t2, and t3, and the overall loss function L was calculated. total The model parameters are updated until the total number of training steps T is reached, thereby obtaining a trained cell nucleus detection model. Step 5.4: Use the trained cell nucleus detection model to perform cell nucleus detection on the pathological image data to obtain the corresponding detection results.
3. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing the pathological image cell nucleus detection method of claim 1 or 2, and the processor is configured to execute the program stored in the memory.
4. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when run by a processor, performs the steps of the pathological image cell nucleus detection method according to claim 1 or 2.