Weakly supervised cell nuclei segmentation method based on wavelet difference convolution and region expansion

Abstract

The application discloses a weakly supervised cell nucleus segmentation method based on wavelet difference convolution and region expansion, and relates to the technical field of image processing, and comprises the following steps: obtaining a cell nucleus image sample, and marking a cell nucleus position by using a center point annotation; designing a wavelet difference convolution module, extracting multi-scale features through discrete wavelet transform, and combining the difference convolution module to enhance cell nucleus boundaries and detail information; constructing a region expansion module, generating a pseudo label based on point annotation iteration, gradually expanding a complete cell nucleus region, and reducing noise and nucleus missing detection problems; building a segmentation network, adding the wavelet difference convolution module to extract detail features, and using the pseudo label as a weakly supervised signal to optimize network performance; performing segmentation prediction, and outputting accurate positions and shapes of cell nuclei; thereby realizing high-precision segmentation under a small amount of annotation information, reducing annotation dependence, and achieving remarkable effects in reducing adhesion of adjacent cell nucleus boundaries and missing detection of small cell nuclei.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion. Background Art

[0002] The size, shape, and distribution of cell nuclei are crucial features in cancer diagnosis, often used to determine the cancer's malignancy. In traditional pathological diagnosis, pathologists observe tissue sections under a microscope and perform qualitative analysis based on characteristics such as the morphology and arrangement of the nuclei to reach a diagnosis. However, due to the dense density and adhesion of cells, the boundaries of the nuclei are often blurred, making manual segmentation a significant challenge. This not only increases the risk of misdiagnosis and missed diagnoses, but also significantly increases the pathologist's workload and compromises diagnostic efficiency.

[0003] With the continuous development of computer vision technology, digital pathology has become an important research direction in the fields of computer-aided diagnosis (CAD) and prognosis analysis. Analysis of digital pathology images not only improves diagnostic accuracy but also significantly enhances efficiency when processing large numbers of cases. Deep learning, particularly convolutional neural networks (CNNs), has made significant progress in the application of pathology images. CNNs can automatically learn useful features from images, eliminating the need for manual feature extraction, resulting in excellent performance in tasks such as cell nucleus segmentation. Through in-depth analysis of pathology images, these methods are expected to reduce the workload of pathologists, reduce misdiagnosis and missed diagnoses, and improve diagnostic accuracy and efficiency.

[0004] However, fully supervised learning methods rely on extensive pixel-level annotations. Medical image data acquisition and annotation are expensive, and concerns about patient privacy further increase costs. To address this challenge, weakly supervised learning has gained increasing attention. As a transition between unsupervised and fully supervised learning, weakly supervised learning uses limited or incompletely annotated data to train models and achieve better generalization. Current research focuses on achieving efficient segmentation with fewer nucleus annotations, thereby reducing the annotation burden.

[0005] Currently, the existing technology uses point annotations to train weakly supervised segmentation networks, uses point annotations to generate Voronoi and cluster labels to guide model training, and adopts dense conditional random field (CRF) loss to improve the model; the existing technology also uses a multi-task learning method that combines three different losses and learns from one of them in each iteration, which also shows performance improvement; the existing technology also designs instance loss to help the neural network learn continuous peak maps from point annotations; uses pseudo edges generated by Sobel filters to help the network perform segmentation; proposes a coarse-to-fine learning strategy to gradually improve segmentation performance through self-supervised learning; and proposes a three-stage network to learn boundary information using a multi-instance learning framework.

[0006] Although the above weakly supervised segmentation methods achieve relatively strong segmentation performance, they lack sufficient information to delineate the boundaries between adjacent nuclei, and the iteratively trained network increases the complexity of the method. The high noise level in the pseudo-labels generated in this process still has a negative impact on model training, resulting in problems such as missed and false detection of cell nuclei. Summary of the Invention

[0007] In order to solve the above technical problems, the present invention provides a weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion, comprising the following steps:

[0008] S1. Get an image sample X∈R containing the cell nucleus 3×H×W , and use the center point annotation P={p1,p2,...,p n}calibrate the position of the cell nucleus, where H represents the height, W represents the width, n represents the number of cell nuclei in an image, and p i Represents the point annotation coordinates of the i-th cell nucleus in the sample; the point annotations are used to generate the Voronoi label set V and the cluster label set C;

[0009] S2. Construct a wavelet differential convolution module to extract multi-scale features of different sub-bands by performing discrete wavelet transform on the input feature map X; extract features of the boundaries and details of the cell nucleus through the differential convolution module. The differential convolution module includes central differential convolution, horizontal differential convolution, vertical differential convolution, and diagonal differential convolution;

[0010] S3, build the region extension module, based on the initial point annotation P={p1,p2,...,p n}, by iteratively generating pseudo labels R, the complete cell nucleus region is gradually expanded to cover more high-confidence regions;

[0011] S4. Build a cell nucleus segmentation network, add wavelet differential convolution to extract detail features, and use pseudo labels as weak supervision signals;

[0012] S5. Segment and predict the cell nucleus in the image and output the position and shape prediction map of the cell nucleus Y∈R 1×H×W .

[0013] The technical solution further defined in the present invention is:

[0014] Furthermore, step S2 specifically includes the following sub-steps:

[0015] S2.1. Perform a two-dimensional discrete wavelet transform on the input feature map X to obtain a low-frequency subband cA and three high-frequency subbands. The three high-frequency subbands are a horizontal high-frequency subband cH, a vertical high-frequency subband cV, and a diagonal high-frequency subband cD.

[0016] S2.2, perform center differential convolution operation on low-frequency sub-band cA, and perform differential convolution operation on high-frequency sub-bands cH, cV, and cD in the same direction to extract multi-directional detail features, which are F cdc 、F hdc 、F vdc and F adc ;

[0017] S2.3. Perform inverse wavelet transform on the output features after differential convolution to reconstruct the enhanced feature map for subsequent region expansion and segmentation network. See the following formula for details:

[0018] X′=IDWT{F cdc ,(F hdc ,F vdc ,F adc )}

[0019] Among them, X′ represents the enhanced feature map.

[0020] As mentioned above, in the weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion, the discrete wavelet basis used in step S2.1 is the Harr wavelet, as shown in the following formula:

[0021] {cA,(cH,cV,cD)}=DWT(X).

[0022] As described above, in the weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion, in step S2.2, a central differential convolution operation is performed on the low-frequency subband cA, a horizontal differential convolution operation is performed on the high-frequency subband cH in the horizontal direction, a vertical differential convolution operation is performed on the high-frequency subband cV in the vertical direction, and a diagonal differential convolution operation is performed on the high-frequency subband cD in the diagonal direction, as shown in the following formula:

[0023] F cdc =CDC(cA),F hdc =HDC(cH),F vdc =VDC(cV),F adc =ADC(cD)

[0024] Among them, F cdc represents the detail features obtained after performing the center difference convolution operation on the low-frequency sub-band cA, F hdc Represents the detail features obtained by performing horizontal differential convolution on the high-frequency sub-band cH in the horizontal direction, F vdcRepresents the detail features obtained by performing vertical differential convolution on the high-frequency sub-band cV in the vertical direction, F adc It represents the detail features obtained after performing diagonal differential convolution operation on the high-frequency sub-band cD in the diagonal direction.

[0025] As described above, the weakly supervised cell nucleus segmentation method based on wavelet difference convolution and region expansion, step S3 specifically includes the following sub-steps:

[0026] S3.1. Generate a Gaussian mask M centered on the point annotation to ensure that the pixels around each annotation point are considered part of the cell nucleus region during initialization. The Gaussian mask is defined as follows:

[0027]

[0028] Among them, M i represents the i-th Gaussian heatmap, σ represents the standard deviation of the Gaussian kernel, r1 and r2 represent the hyperparameters defining the foreground and background region radii, respectively; D i represents the distance between the i-th pixel and its nearest point annotation, as shown in the following formula:

[0029]

[0030] Among them, N represents the set of all point annotations, x i represents the spatial position of the i-th pixel, p j Represents the spatial coordinates of the j-th point;

[0031] Then generate an initial region label R, and convert the pixel value M in the Gaussian heat map into ij The area greater than zero is considered to be the initial cell nucleus, as shown in the following formula:

[0032]

[0033] S3.2. During the region expansion process, we first identify boundary pixels by performing a neighborhood operation using a 3×3 convolution kernel. Boundary pixels are defined as kernel pixels that are in contact with the background or other categories. The convolution kernel K is defined as follows:

[0034]

[0035] Apply convolution operation to obtain the number N of kernel pixels near each pixel, N = R × K;

[0036] Boundary pixels are identified by the following conditions:

[0037] E(x,y)=(R(x,y)=0)∧(N(x,y)>0)

[0038] wherein R represents the current expanded mask, E represents the identified boundary pixels, i.e. the core pixels in contact with the background or other classes;

[0039] S3.3, generating an expanded mask based on the boundary pixels E(x, y) and the model predicted probability map P(x, y); for each boundary pixel, if its corresponding predicted probability P exceeds a threshold τ and the pixel is not located on the Voronoi boundary, it is marked as part of the core region, see the following formula:

[0040] expand(x, y) = (E(x, y) = 1) Λ (P(x, y) > τ) Λ (V(x, y) ≠ 0)

[0041] R[expand] = 1

[0042] S3.4, repeating steps S3.2 to S3.3, gradually adding high-confidence pixels to the core region, and the expansion process is iterated until no new pixels are expanded or the maximum number of iterations T is reached max .

[0043] The weakly supervised cell nucleus segmentation method based on wavelet difference convolution and region expansion as described above, step S4 specifically comprises the following steps:

[0044] S4.1, constructing a cell nucleus segmentation network based on a ResUNet34 backbone network, the cell nucleus segmentation network comprising an encoder, a decoder, a wavelet difference convolution module and a skip connection;

[0045] S4.2, in the encoding stage, using the encoder of ResUNet34 to perform multi-scale feature extraction on the input image, the encoder of ResUNet34 comprising a first encoding layer, a second encoding layer, a third encoding layer and a fourth encoding layer, each encoding layer generating a feature map of a different scale, denoted as F 1 ,F 2 ,F 3 ,F 4 ; a wavelet difference convolution module is inserted after the operation of the second encoding layer, the third encoding layer and the fourth encoding layer; a bridge for feature integration and transition is connected between the encoder and the decoder to compress and enhance the final output high-dimensional abstract features, and the result is taken as the initial input B of the decoder;

[0046] S4.3, in the decoding stage, using layer-by-layer upsampling and skip connection to gradually restore the features extracted in the encoding stage to the original image resolution; the decoder comprises a first decoding layer, a second decoding layer, a third decoding layer and a fourth decoding layer corresponding to the first encoding layer, the second encoding layer, the third encoding layer and the fourth encoding layer respectively, and the feature maps of each decoding layer are denoted as G 1 ,G2 ,G 3 ,G 4 The decoder reconstructs the segmentation map of the cell nucleus by gradually upsampling the feature map and combining it with the features of the corresponding encoder. It also uses a 1×1 convolutional layer to adjust the number of channels of the encoder's output feature map to obtain the final predicted segmentation result.

[0047] As described above, in the weakly supervised cell nucleus segmentation method based on wavelet difference convolution and region expansion, in step S4.3, in the first decoding layer, B is upsampled and then compared with F 4 Perform skip connection and generate G through convolution 1 ; In the second decoding layer, G 1 After upsampling, with F 3 Perform skip connection and generate G through convolution 2 ; In the third decoding layer, G 2 After upsampling, with F 2 Perform skip connection and generate G through convolution 3 ; In the fourth decoding layer, G 3 After upsampling, with F 1 Perform skip connection and generate G through convolution 4 .

[0048] As described above, the weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion, step S5 specifically includes the following sub-steps:

[0049] S5.1. The total loss L total Defined as the area expansion loss L exp , clustering loss L clu and the Voronoi loss L vor A linear combination of , which measures the difference between the predicted results and the true annotations;

[0050] S5.2. Use the Adam optimizer to optimize parameters, adjust the learning rate, and accelerate model convergence;

[0051] S5.3. Save the optimal model parameters based on the AJI index obtained on the validation set;

[0052] S5.4. Evaluate the model on the test set, output the precise location and shape of the cell nucleus, and verify the segmentation effect of the model.

[0053] As described above, in the weakly supervised nucleus segmentation method based on wavelet differential convolution and region expansion, in step S5.1, the total loss L total As shown in the following formula:

[0054]

[0055] Among them, α, β, and γ represent the balance weights of the control area expansion loss, clustering loss, and Voronoi loss ratio.

[0056] As described above, in the weakly supervised nucleus segmentation method based on wavelet differential convolution and region expansion, in step S5.1, once the region expansion is completed, the region expansion loss L is calculated using the binary cross entropy loss function expressed as follows: exp :

[0057]

[0058] Among them, Ω represents the set of all pixels, P i is the prediction of the i-th pixel, R i The label of the expanded area;

[0059] At the same time, clustering loss and Voronoi loss are introduced. Clustering loss L clu and Voronoi loss L vor The expression is as follows:

[0060]

[0061] Among them, Ω c and Ω v represents the set of pixels that are not ignored in the cluster labels and Voronoi labels; C i represents the cluster label, which guides the model to learn the cluster structure of the kernel; V i Represents the Voronoi marker, which is used to distinguish the nucleus from the background.

[0062] The beneficial effects of the present invention are:

[0063] In this invention, by adopting weakly supervised learning, the dependence on a large number of pixel-level annotations is reduced, the annotation cost is reduced, and the application flexibility of the model is improved; when processing complex morphological cell nuclei, the feature extraction and boundary recognition capabilities of the model are optimized to improve the recognition accuracy of morphologically complex and low-contrast cell nuclei, ensuring that the segmentation results are more accurate and stable, thereby making the model have good robustness; through innovations in model structure and training strategies, the segmentation accuracy is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0065] Figure 2 Schematic diagram of a specific process of the weakly supervised cell nucleus segmentation method in an embodiment of the present invention;

[0066] Figure 3 Schematic diagram of the structure of the wavelet difference convolution module in an embodiment of the present invention;

[0067] Figure 4 Schematic diagram of the structure of the expansion module in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] This embodiment provides a weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion, such as Figures 1 to 2 As shown, the following steps are included:

[0069] S1. Get an image sample X∈R containing the cell nucleus 3×H×W , and use the center point annotation P={p1,p2,...,p n}calibrate the position of the cell nucleus, where H represents the height, W represents the width, n represents the number of cell nuclei in an image, and p i Represents the point annotation coordinates of the i-th cell nucleus in the sample; the point annotations are used to generate the Voronoi label set V and the cluster label set C.

[0070] S2, build Figure 3 The Wavelet-based Differential Convolution Module block (WDCM block) shown in the figure extracts multi-scale features of different subbands by performing discrete wavelet transform (DWT) on the input feature map X; the boundary and details of the cell nucleus are extracted through the differential convolution module. The differential convolution module includes central difference convolution (CDC), horizontal differential convolution (HDC), vertical differential convolution (VDC) and diagonal differential convolution (ADC) to enhance the feature extraction of the boundary and details of the cell nucleus.

[0071] Step S2 specifically includes the following sub-steps:

[0072] S2.1. Perform a two-dimensional discrete wavelet transform (DWT) on the input feature map X to obtain a low-frequency subband cA and three high-frequency subbands. The three high-frequency subbands are the horizontal high-frequency subband cH, the vertical high-frequency subband cV, and the diagonal high-frequency subband cD. The discrete wavelet basis used is the Harr wavelet, as shown in the following formula:

[0073] {cA,(cH,cV,cD)}=DWT(X).

[0074] S2.2. Perform a central differential convolution operation on the low-frequency sub-band cA to enhance the global information, and perform differential convolution operations consistent with their directions on the high-frequency sub-bands cH, cV, and cD to enhance the detail information in a specific direction. Specifically, a horizontal differential convolution (HDC) operation is performed on the high-frequency sub-band cH in the horizontal direction to enhance the horizontal details, a vertical differential convolution (VDC) operation is performed on the high-frequency sub-band cV in the vertical direction to enhance the vertical details, and a diagonal differential convolution (ADC) operation is performed on the high-frequency sub-band cD in the diagonal direction to capture the details in the diagonal direction.

[0075] Extract multi-directional detail features, namely F cdc 、F hdc 、F vdc and F adc , these features will be used to enhance the expression of the cell nuclear boundary, as shown in the following formula:

[0076] F cdc =CDC(cA),F hdc =HDC(cH),F vdc =VDC(cV),F adc =ADC(cD)

[0077] Among them, F cdc represents the detail features obtained after performing the center difference convolution operation on the low-frequency sub-band cA, F hdc Represents the detail features obtained by performing horizontal differential convolution on the high-frequency sub-band cH in the horizontal direction, F vdc Represents the detail features obtained by performing vertical differential convolution on the high-frequency sub-band cV in the vertical direction, F adc It represents the detail features obtained after performing diagonal differential convolution operation on the high-frequency sub-band cD in the diagonal direction.

[0078] S2.3. Perform inverse wavelet transform on the output features after differential convolution to reconstruct an enhanced version of the feature map for subsequent region expansion and segmentation network. See the following formula for details:

[0079] X′=IDWT{F cdc ,(F hdc ,F vdc ,F adc )}

[0080] Among them, X′ represents the enhanced feature map.

[0081] S3, build the region extension module, based on the initial point annotation P={p1,p2,...,p n}, by iteratively generating pseudo labels R, the complete cell nucleus region is gradually expanded to cover more high-confidence regions and reduce noise and nuclear missed detection problems, such as Figure 4As shown, it specifically includes the following steps:

[0082] S3.1. Generate a Gaussian mask M centered on the point annotation to ensure that the pixels around each annotation point are considered part of the cell nucleus region during initialization. The Gaussian mask is defined as follows:

[0083]

[0084] Among them, M i represents the i-th Gaussian heatmap, σ represents the standard deviation of the Gaussian kernel, and r1 and r2 represent the hyperparameters defining the radii of the foreground and background regions, respectively.

[0085] In the above formula, D i represents the distance between the i-th pixel and its nearest point annotation, as shown in the following formula:

[0086]

[0087] Among them, N represents the set of all point annotations, x i represents the spatial position of the i-th pixel, p j Represents the spatial coordinates of the j-th point annotation.

[0088] Then generate an initial region label R, and convert the pixel value M in the Gaussian heat map into ij The area greater than zero is considered to be the initial cell nucleus, as shown in the following formula:

[0089] .

[0090] S3.2. During the region expansion process, we first identify boundary pixels by performing a neighborhood operation using a 3×3 convolution kernel. Boundary pixels are defined as kernel pixels that are in contact with the background or other categories. The convolution kernel K is defined as follows:

[0091]

[0092] Apply convolution operation to obtain the number N of kernel pixels near each pixel, N = R × K;

[0093] Boundary pixels are identified by the following conditions:

[0094] E(x,y)=(R(x,y)=0)∧(N(x,y)>0)

[0095] Among them, R represents the current expanded mask and E represents the identified boundary pixels, that is, the core pixels that are in contact with the background or other categories.

[0096] S3.3. Generate an extended mask based on the boundary pixels E(x, y) and the probability map P(x, y) predicted by the model. For each boundary pixel, if its corresponding predicted probability P exceeds the threshold τ and the pixel is not located on the Voronoi boundary (the boundary pixel value is 0), then mark it as part of the kernel region, as shown in the following formula:

[0097] expand(x,y)=(E(x,y)=1)∧(P(x,y)>τ)∧(V(x,y)≠0)

[0098] R[expand]=1.

[0099] S3.4. Repeat steps S3.2 to S3.3. This process can gradually add high-confidence pixels to the cell nucleus area to avoid the noise effect caused by directly using low-confidence pixel expansion. The expansion process is iterated continuously until no new pixels are expanded or the maximum number of iterations T is reached. max .

[0100] S4. Build a cell nucleus segmentation network, add wavelet differential convolution to extract detail features, use pseudo labels as weak supervision signals, and further optimize network performance. The specific steps include the following:

[0101] S4.1. Construct a cell nucleus segmentation network based on the ResUNet34 backbone network. The cell nucleus segmentation network includes an encoder, a decoder, a wavelet difference convolution module, and a skip connection.

[0102] S4.2. In the encoding stage, the ResUNet34 encoder is used to extract multi-scale features from the input image. The ResUNet34 encoder includes the first encoding layer, the second encoding layer, the third encoding layer, and the fourth encoding layer. Each encoding layer generates feature maps of different scales, which are respectively denoted as F 1 ,F 2 ,F 3 ,F 4 ; The spatial resolution of each level decreases layer by layer, and the feature semantic information is abstracted layer by layer; the feature semantic level output by the first-layer encoder is relatively shallow. As the network deepens, the semantics are enhanced, and multi-directional convolution of WDCM is required to capture the high-level boundaries between objects. Therefore, WDCM blocks are inserted after the second, third, and fourth coding layers to further capture the boundary information and detailed features of the image in multiple directions; a bridge for feature integration and transition is connected between the encoder and decoder, which compresses and enhances the high-dimensional abstract features of the final output, and the result is used as the initial input B of the decoder.

[0103] S4.3. In the decoding stage, the features extracted in the encoding stage are gradually restored to the original image resolution by using layer-by-layer upsampling and skip connections. The decoder includes the first decoding layer, the second decoding layer, the third decoding layer, and the fourth decoding layer, which correspond to the first encoding layer, the second encoding layer, the third encoding layer, and the fourth encoding layer, respectively. The feature map of each decoding layer is denoted as G. 1 ,G 2 ,G 3 ,G 4 .

[0104] The specific process is as follows:

[0105] In the first decoding layer, B is upsampled and then compared with F 4 Perform skip connection and generate G through convolution 1 ;

[0106] In the second decoding layer, G 1 After upsampling, with F 3 Perform skip connection and generate G through convolution 2 ;

[0107] In the third decoding layer, G 2 After upsampling, with F 2 Perform skip connection and generate G through convolution 3 ;

[0108] In the fourth decoding layer, G 3 After upsampling, with F 1 Perform skip connection and generate G through convolution 4 .

[0109] The decoder reconstructs the accurate segmentation map of the cell nucleus by gradually upsampling the feature map and combining it with the features of the corresponding encoder, and uses a 1×1 convolutional layer to adjust the number of channels of the encoder's output feature map to obtain the final predicted segmentation result.

[0110] S5. Segment and predict the cell nucleus in the image and output the precise position and shape prediction map Y∈R of the cell nucleus 1 ×H×W , specifically including the following steps:

[0111] S5.1. The total loss L total Defined as the area expansion loss L exp , clustering loss L clu and the Voronoi loss L vor The linear combination of , which measures the difference between the predicted results and the true annotations, is as follows:

[0112]

[0113] Among them, α, β, and γ represent the balance weights of the control area expansion loss, clustering loss, and Voronoi loss ratio.

[0114] Once the region expansion is completed, the region expansion loss L is calculated using the binary cross entropy loss function BCELoss expressed as follows: exp :

[0115]

[0116] Among them, Ω represents the set of all pixels, P i is the prediction of the i-th pixel, R i The label of the expanded area.

[0117] In addition to the region expansion loss, clustering loss and Voronoi loss are also introduced to use additional label information to enhance the overall morphological consistency of the kernel. The clustering loss L clu and Voronoi loss L vor The expression is as follows:

[0118]

[0119] Among them, Ω c and Ω v represents the set of pixels that are not ignored in the cluster labels and Voronoi labels; C i represents the cluster label, which guides the model to learn the cluster structure of the kernel; V i Represents the Voronoi marker, which is used to distinguish the nucleus from the background.

[0120] S5.2. Use the Adam optimizer to optimize parameters, adjust the learning rate, and accelerate model convergence.

[0121] S5.3. The Aggregated Jaccard Index (AJI) is a region-based evaluation metric specifically used to measure the accuracy of multi-target instance segmentation. The optimal model parameters are saved based on the AJI obtained on the validation set.

[0122] S5.4. Evaluate the model on the test set, output the precise location and shape of the cell nucleus, and verify the segmentation effect of the model.

[0123] For the task of cell nucleus segmentation, a single model often has difficulty handling complex image features and variable nuclear morphology, especially under weak supervision conditions; wavelet differential convolution can significantly promote the separation of adjacent nuclei and allow multi-scale and multi-directional feature extraction; wavelet transform captures fine details and structural information, while differential convolution further refines it by focusing on local changes, especially those around edges; they jointly enhance boundary delineation; by iteratively generating pseudo-labels to expand the cell nucleus area, the segmentation accuracy is gradually improved; this method can effectively deal with problems such as cell nucleus boundary adhesion and missed detection, reduce dependence on large amounts of labeled data, and achieve efficient and accurate segmentation results.

[0124] Based on this idea, this embodiment proposes a weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion. Multi-scale features are extracted through wavelet transform, boundary information is enhanced by combining differential convolution, and pseudo labels are iteratively generated using the region expansion module for training, giving full play to the advantages of wavelet transform and differential convolution in feature extraction. The cell nucleus segmentation model has strong robustness and can achieve high-precision cell nucleus segmentation under weak supervision conditions, reducing dependence on accurately labeled data. At the same time, it can effectively cope with challenges such as dense cell nuclei, boundary adhesion, and missed detection of small cell nuclei, so as to improve the efficiency and accuracy of image analysis and achieve high-precision cell nucleus segmentation.

[0125] In addition to the above embodiments, the present invention may also have other implementations. Any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection required by the present invention.

Claims

1. A weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion, characterized by: The following steps are involved: S1. Get an image sample X∈R containing the cell nucleus 3×H×W , and use the center point annotation P={p1,p2,...,p n } calibrate the position of the cell nucleus, where H represents the height, W represents the width, n represents the number of cell nuclei in an image, and p i Represents the point annotation coordinates of the i-th cell nucleus in the sample; Use point annotations to generate Voronoi label set V and cluster label set C; S2. Construct a wavelet differential convolution module to extract multi-scale features of different sub-bands by performing discrete wavelet transform on the input feature map X; extract features of the boundaries and details of the cell nucleus through the differential convolution module. The differential convolution module includes central differential convolution, horizontal differential convolution, vertical differential convolution, and diagonal differential convolution; S3, build the region extension module, based on the initial point annotation P={p1,p2,...,p n }, by iteratively generating pseudo labels R, the complete cell nucleus region is gradually expanded to cover more high-confidence regions; S4. Build a cell nucleus segmentation network, add wavelet differential convolution to extract detail features, and use pseudo labels as weak supervision signals; S5. Segment and predict the cell nucleus in the image and output the position and shape prediction map of the cell nucleus Y∈R 1×H×W ; Step S4 specifically includes the following sub-steps: S4.

1. Construct a cell nucleus segmentation network based on the ResUNet34 backbone network. The cell nucleus segmentation network includes an encoder, a decoder, a wavelet difference convolution module, and skip connections. S4.

2. In the encoding stage, the ResUNet34 encoder is used to extract multi-scale features from the input image. The ResUNet34 encoder includes the first encoding layer, the second encoding layer, the third encoding layer, and the fourth encoding layer. Each encoding layer generates feature maps of different scales, which are respectively denoted as F 1 ,F 2 ,F 3 ,F 4 ; Insert the wavelet difference convolution module after the second, third and fourth coding layers; Connect the bridge for feature integration and transition between the encoder and decoder to compress and enhance the high-dimensional abstract features of the final output, and the result is used as the initial input B of the decoder; S4.

3. In the decoding stage, the features extracted in the encoding stage are gradually restored to the original image resolution by using layer-by-layer upsampling and skip connections. The decoder includes the first decoding layer, the second decoding layer, the third decoding layer, and the fourth decoding layer, which correspond to the first encoding layer, the second encoding layer, the third encoding layer, and the fourth encoding layer, respectively. The feature map of each decoding layer is denoted as G. 1 ,G 2 ,G 3 ,G 4 The decoder reconstructs the segmentation map of the cell nucleus by gradually upsampling the feature map and combining it with the features of the corresponding encoder. It also uses a 1×1 convolutional layer to adjust the number of channels of the encoder's output feature map to obtain the final predicted segmentation result.

2. The weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion according to claim 1, characterized in that: The step S2 specifically includes the following sub-steps: S2.

1. Perform a two-dimensional discrete wavelet transform on the input feature map X to obtain a low-frequency subband cA and three high-frequency subbands. The three high-frequency subbands are a horizontal high-frequency subband cH, a vertical high-frequency subband cV, and a diagonal high-frequency subband cD. S2.2, perform center differential convolution operation on low-frequency sub-band cA, and perform differential convolution operation on high-frequency sub-bands cH, cV, and cD in the same direction to extract multi-directional detail features, which are F cdc 、F hdc 、F vdc and F adc ; S2.

3. Perform inverse wavelet transform on the output features after differential convolution to reconstruct the enhanced feature map for subsequent region expansion and segmentation network. See the following formula for details: X′=IDWT{F cdc ,(F hdc ,F vdc ,F adc )} Among them, X′ represents the enhanced feature map.

3. The weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion according to claim 2, characterized in that: In step S2.1, the discrete wavelet basis used is the Harr wavelet, as shown in the following formula: {cA,(cH,cV,cD)}=DWT(X).

4. The weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion according to claim 2, characterized in that: In step S2.2, a central differential convolution operation is performed on the low-frequency sub-band cA, a horizontal differential convolution operation is performed on the high-frequency sub-band cH in the horizontal direction, a vertical differential convolution operation is performed on the high-frequency sub-band cV in the vertical direction, and a diagonal differential convolution operation is performed on the high-frequency sub-band cD in the diagonal direction, as shown in the following formula: F cdc =CDC(cA),F hdc =HDC(cH),F vdc =VDC(cV),F adc =ADC(cD) Among them, F cdc represents the detail features obtained after performing the center difference convolution operation on the low-frequency sub-band cA, F hdc Represents the detail features obtained by performing horizontal differential convolution on the high-frequency sub-band cH in the horizontal direction, F vdc Represents the detail features obtained by performing vertical differential convolution on the high-frequency sub-band cV in the vertical direction, F adc It represents the detail features obtained after performing diagonal differential convolution operation on the high-frequency sub-band cD in the diagonal direction.

5. The weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion according to claim 1, characterized in that: The step S3 specifically includes the following sub-steps: S3.

1. Generate a Gaussian mask M centered on the point annotation to ensure that the pixels around each annotation point are considered part of the cell nucleus region during initialization. The Gaussian mask is defined as follows: Among them, M i represents the i-th Gaussian heatmap, σ represents the standard deviation of the Gaussian kernel, r1 and r2 represent the hyperparameters defining the foreground and background region radii, respectively; D i represents the distance between the i-th pixel and its nearest point annotation, as shown in the following formula: Among them, N represents the set of all point annotations, x i represents the spatial position of the i-th pixel, p j Represents the spatial coordinates of the j-th point; Then generate an initial region label R, and convert the pixel value M in the Gaussian heat map into ij The area greater than zero is considered to be the initial cell nucleus, as shown in the following formula: S3.

2. During the region expansion process, we first identify boundary pixels by performing a neighborhood operation using a 3×3 convolution kernel. Boundary pixels are defined as kernel pixels that are in contact with the background or other categories. The convolution kernel K is defined as follows: Apply convolution operation to obtain the number N of kernel pixels near each pixel, N = R × K; Boundary pixels are identified by the following conditions: E(x,y)=(R(x,y)=0)∧(N(x,y)>0) Where R represents the currently expanded mask, E represents the identified boundary pixels, that is, the core pixels that are in contact with the background or other categories; N(x,y) represents the number of adjacent core pixels at each position; S3.

3. Generate an extended mask based on the boundary pixels E(x, y) and the probability map P(x, y) predicted by the model; for each boundary pixel, if its corresponding predicted probability P exceeds the threshold τ and the pixel is not located on the Voronoi boundary, it is marked as part of the kernel region, as shown in the following formula: expand(x,y)=(E(x,y)=1)∧(P(x,y)>τ)∧(V(x,y)≠0) R[expand]=1 Where expand(x, y) represents a logical conditional expression used to determine whether the pixel at position (x, y) satisfies the expansion condition; V(x, y) represents the Voronoi boundary map, and pixels equal to 0 represent the boundary; expand in R[expand]=1 represents the set of all pixel positions that satisfy expand(x, y)=1; S3.

4. Repeat steps S3.2 to S3.3, gradually adding high-confidence pixels to the cell nucleus area. The expansion process continues to iterate until no new pixels are expanded or the maximum number of iterations T is reached. max .

6. The weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion according to claim 1, characterized in that: In step S4.3, in the first decoding layer, after upsampling B, the 4 Perform skip connection and generate G through convolution 1 ; In the second decoding layer, G 1 After upsampling, with F 3 Perform skip connection and generate G through convolution 2 ; In the third decoding layer, G 2 After upsampling, with F 2 Perform skip connection and generate G through convolution 3 ; In the fourth decoding layer, G 3 After upsampling, with F 1 Perform skip connection and generate G through convolution 4 .

7. The weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion according to claim 1, characterized in that: The step S5 specifically includes the following sub-steps: S5.

1. The total loss L total Defined as the area expansion loss L exp , clustering loss L clu and the Voronoi loss L vor A linear combination of , which measures the difference between the predicted results and the true annotations; S5.

2. Use the Adam optimizer to optimize parameters, adjust the learning rate, and accelerate model convergence; S5.

3. Save the optimal model parameters based on the AJI index obtained on the validation set; S5.

4. Evaluate the model on the test set, output the precise location and shape of the cell nucleus, and verify the segmentation effect of the model.

8. The weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion according to claim 7, characterized in that: In step S5.1, the total loss L total As shown in the following formula: Among them, α, β, and γ represent the balance weights of the control area expansion loss, clustering loss, and Voronoi loss ratio.

9. The weakly supervised cell nucleus segmentation method based on wavelet differential convolution and region expansion according to claim 8, characterized in that: In step S5.1, once the region expansion is completed, the region expansion loss L is calculated using the binary cross entropy loss function represented by the following formula: exp : Among them, Ω represents the set of all pixels, P i is the prediction of the i-th pixel, R i The label of the expanded area; At the same time, clustering loss and Voronoi loss are introduced. Clustering loss L clu and Voronoi loss L vor The expression is as follows: Among them, Ω c and Ω v represents the set of pixels that are not ignored in the cluster labels and Voronoi labels; C i represents the cluster label, which guides the model to learn the cluster structure of the kernel; V i Represents the Voronoi marker, which is used to distinguish the nucleus from the background.

Images