A method for constructing a cell nucleus segmentation model, a cell nucleus segmentation method, and a construction device

Abstract

The present application relates to the technical field of data processing, and discloses a cell nucleus segmentation model construction method, a cell nucleus segmentation method and a construction device, the construction method comprising: obtaining a cell image dataset; constructing a cell nucleus segmentation network structure, the cell nucleus segmentation network structure comprising a patch embedding layer, an encoder module, a tokenized KAN network module, a decoder module and a projection layer connected in sequence, and the encoder module and the decoder module are further connected through a frequency domain learnable module; wherein the frequency domain learnable module is used for transmitting low-level features in the encoder module to the decoder module; the tokenized KAN network module is used for deep feature extraction; and the cell nucleus segmentation network is trained based on the cell image dataset to obtain a cell nucleus segmentation model. The cell nucleus segmentation model provided by the present application pays more attention to the cell nucleus boundary effect, greatly improving the cell nucleus segmentation precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a nucleus segmentation model construction method, a nucleus segmentation method and a construction device. BACKGROUND

[0002] Nucleus segmentation is an important biomedical technology, which mainly carries out microscopic image processing and analysis on cells in collected tissue or body fluid samples, so as to obtain single nuclei after separation. Nucleus segmentation technology plays an important role in determining the position, shape and number of nuclei, and further researches on gene expression and protein expression, and has wide application in the fields of tumor liquid biopsy, spatial multi-omics analysis and pathological diagnosis.

[0003] However, the obtained microscopic images of tissue cells have problems such as strong cell heterogeneity, cell stacking and uneven staining, which cause poor accuracy and low precision of nucleus recognition and segmentation. At present, traditional algorithms such as watershed algorithm or interactive software are used for cell or nucleus segmentation. Although the watershed algorithm does not need to rely on a large amount of training data to learn features when segmenting nuclei, it has high dependence on pre-defined parameters, poor applicability and low accuracy in processing complex tissue images. When interactive software such as Ilastik (Ilastik is an open source software for biomedical image analysis) is used for nucleus segmentation, multiple labeling interactions and multiple iterations are required to obtain good segmentation results, and the degree of automation is low.

[0004] At present, nucleus segmentation methods based on deep learning have also been developed, such as Cellpose based on Unet, which can realize universal cell segmentation in different application scenarios by constructing diversified and multi-scene training data sets. However, when the shape of the nucleus is complex, the edge is blurred or there is overlap, the segmentation boundary of the nucleus will not be accurate enough. In recent years, improved UNet++, Transformer, SwinUNet and other deep learning methods have been developed. Due to the problems of overfitting, high computational cost and strong dependence on data labeling, it is difficult to meet the requirements of segmentation accuracy for analysis. SUMMARY

[0005] Therefore, the present application provides a nucleus segmentation model construction method, a nucleus segmentation method and a construction device to solve the problem of low nucleus segmentation precision in the prior art.

[0006] In a first aspect, the present application provides a nucleus segmentation model construction method, which comprises:

[0007] obtaining a cell image data set;

[0008] The cell nucleus segmentation network structure comprises a patch embedding layer, an encoder module, a tokenized KAN network module, a decoder module and a projection layer connected in sequence, and the encoder module and the decoder module are further connected through a frequency domain learnable module; wherein the frequency domain learnable module is used for transmitting low-level features in the encoder module to the decoder module; the tokenized KAN network module is used for deep feature extraction;

[0009] The cell nucleus segmentation network is trained based on the cell image data set to obtain a cell nucleus segmentation model.

[0010] In this embodiment, a deep learning network model FfvmUkan for cell nucleus recognition and segmentation is proposed. On the basis of the VM-UNet framework, a frequency domain learnable module sensitive to the image frequency domain is introduced for the boundary segmentation task to improve the cell nucleus boundary segmentation level by learning the image frequency domain features. In this embodiment, a tokenized KAN network module is also introduced to meet the deeper feature extraction requirement and increase the ability of the model to capture multi-scale features. Compared with the traditional method, the cell nucleus segmentation model provided by the present application pays more attention to the cell nucleus boundary effect, greatly improving the cell nucleus segmentation precision.

[0011] In an optional implementation, the encoder module comprises a plurality of layers of encoders connected in sequence, each layer of encoder comprises a visual state space module; the input end of the first layer of encoder is connected with the patch embedding layer, and the output end of the last layer of encoder is connected with the tokenized KAN network module; each layer of encoder further comprises a patch merging layer except the last layer of encoder, and the output end of the visual state space module is connected with the patch merging layer;

[0012] The decoder module comprises a plurality of layers of decoders connected in sequence, each layer of decoder comprises a visual state space module; the input end of the first layer of decoder is connected with the tokenized KAN network module, and the output end of the last layer of decoder is connected with the projection layer; each layer of decoder further comprises a patch expansion layer except the first layer of decoder, and the input end of the visual state space module is connected with the patch expansion layer.

[0013] In this embodiment, the visual state space module is used in the shallow layer, which can dynamically adjust the sampling strategy according to the different features of the input data, adapt to different resolutions and scales, and thus improve the adaptability of the model to diversified data. On the basis of feature extraction, the tokenized KAN network module is added in the deep layer to refine the features, which not only increases the network depth but also increases the ability of the model to capture multi-scale features, and further improves the cell nucleus segmentation precision in combination with the nonlinear modeling of the tokenized KAN network module.

[0014] In an optional embodiment, the input end of each encoder is connected to the output end of each decoder through a frequency domain learnable module.

[0015] In the embodiment, a frequency domain learnable module is further connected to the input end of each visual state space module, which helps to transfer low-level features such as edges and textures in the encoder to the decoder, and combine them with high-level features such as semantic information, so as to generate more accurate segmentation maps and further improve the accuracy of cell nucleus segmentation.

[0016] In an optional embodiment, the frequency domain learnable module comprises a feature input module, a fast Fourier transform module, a frequency domain feature map module, an inverse fast Fourier transform module and a feature output module connected in sequence.

[0017] The frequency domain learnable module further comprises a self-learning attention module, which comprises a parameter-learnable attention map for performing element-wise multiplication with the output result of the frequency domain feature map module and outputting the multiplication result to the inverse fast Fourier transform module.

[0018] The feature input module is further connected to the feature output module for adding the output result of the inverse fast Fourier transform module to the original features of the feature input module.

[0019] In the embodiment, a residual connection mechanism is introduced to superimpose the original features and the features processed by the first branch, so as to retain the ability to finely capture the details of the cell nucleus edges and optimize the overall segmentation.

[0020] In an optional embodiment, the tokenized KAN network module comprises a tokenized KAN network encoding block and a tokenized KAN network decoding block.

[0021] The output end of the last layer of encoder is connected to the tokenized KAN network encoding block, the output end of the tokenized KAN network decoding block is connected to the input end of the first layer of decoder, and the tokenized KAN network encoding block is connected to the tokenized KAN network decoding block of the same layer.

[0022] The tokenized KAN network encoding block comprises a tokenization layer, a first normalization layer, a KAN network layer, a second normalization layer and a reshaped output layer connected in sequence, wherein the tokenization layer is used to tokenize the feature map into a plurality of two-dimensional blocks, and the KAN network layer is used to perform nonlinear transformation.

[0023] In an alternative embodiment, the encoder module comprises four layers of encoders connected in sequence, each layer of encoders comprising two visual state space modules; the decoder module comprises four layers of decoders connected in sequence, each layer of decoders comprising two visual state space modules except for the last layer of decoders, and the last layer of decoders comprises one visual state space module.

[0024] The constructed cell nucleus segmentation model, i.e., the FfvmUkan model for cell or cell nucleus segmentation, has higher segmentation accuracy, and avoids complex parameter setting and interactive annotation workload. The model can better distinguish cell nucleus positions and background positions through frequency domain learning, and enhance features through residual connection. The model not only considers features after a series of transformations, but also retains original input features, helps the network learn image features at different levels, captures more fine and robust feature representations, significantly improves the network's ability to process cell nucleus edge details, and effectively improves the segmentation accuracy, especially in the boundary fuzzy or overlapping area.

[0025] In a second aspect, the present application provides a cell nucleus segmentation method, comprising:

[0026] obtaining a to-be-segmented image;

[0027] inputting the to-be-segmented image into the cell nucleus segmentation model constructed according to the construction method of any one of the above embodiments to obtain a cell nucleus segmentation result corresponding to the to-be-segmented image.

[0028] In a third aspect, the present application provides a cell nucleus segmentation model construction device, comprising:

[0029] an acquisition module configured to acquire a cell image dataset;

[0030] a construction module configured to construct a cell nucleus segmentation network structure, the cell nucleus segmentation network structure comprising a patch embedding layer, an encoder module, a tokenization KAN network module, a decoder module and a projection layer connected in sequence, and the encoder module and the decoder module are further connected through a frequency domain learnable module; wherein the frequency domain learnable module is configured to transfer low-level features in the encoder module to the decoder module; and the tokenization KAN network module is configured to extract deep-level features;

[0031] a training module configured to train the cell nucleus segmentation network based on the cell image dataset to obtain a cell nucleus segmentation model.

[0032] In a fourth aspect, the present application provides a computer device, comprising: a memory and a processor, which are connected in communication with each other, and the memory stores computer instructions, and the processor executes the cell nucleus segmentation model construction method of the first aspect or any of the corresponding embodiments thereof by executing the computer instructions.

[0033] In a fifth aspect, the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the cell nucleus segmentation model construction method of the first aspect or any of the corresponding embodiments thereof.

[0034] It should be noted that the cell nucleus segmentation model construction device, the computer device and the computer readable storage medium provided by the present application correspond to the cell nucleus segmentation model construction method described above. Therefore, the beneficial effects of the cell nucleus segmentation model construction device, the computer device and the computer readable storage medium will be described in the corresponding beneficial effects of the cell nucleus segmentation model construction method described above, and will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0036] Figure 1 is a flowchart of the cell nucleus segmentation model construction method according to an embodiment of the present application;

[0037] Figure 2 is a structural diagram of the cell nucleus segmentation model according to an embodiment of the present application;

[0038] Figure 3 is a structural diagram of the frequency domain learnable module according to an embodiment of the present application;

[0039] Figure 4 is a structural diagram of the tokenization KAN network module according to an embodiment of the present application;

[0040] Figure 5 is a structural diagram of the KAN network layer according to an embodiment of the present application;

[0041] Figure 6 is a comparison diagram of the cell nucleus segmentation performance under the private data set according to an embodiment of the present application;

[0042] Figure 7This is a schematic diagram comparing the cell nucleus segmentation performance on the MoNuSeg dataset according to an embodiment of the present invention;

[0043] Figure 8 This is a structural block diagram of a cell nucleus segmentation model construction device according to an embodiment of the present invention;

[0044] Figure 9 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] According to an embodiment of the present invention, a method for constructing a cell nucleus segmentation model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0047] This embodiment provides a method for constructing a cell nucleus segmentation model, which can be executed by devices such as servers, terminals, and mobile terminals. Figure 1 This is a flowchart of a method for constructing a cell nucleus segmentation model according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0048] Step S101: Obtain the cell image dataset.

[0049] In this embodiment, the cell image dataset can be based on the public dataset MoNuSeg (MoNuSeg is an open dataset for cell image segmentation) and a self-built private dataset, and preprocessing is performed. The public dataset in this embodiment consists of a training set (30 images) and a test set (14 images). The image size is 1000×1000 pixels. Four 512×512 pixel image patches are extracted from the four corners of each image. Then, each image patch is evenly divided into four parts with a 1:1 aspect ratio, resulting in a training set of 480 256×256 pixel images and a test set of 224 256×256 pixel images.

[0050] Taking mouse kidney tissue section H&E staining images as an example, the mouse kidney tissue section H&E staining images are cropped to 256*256 pixels, and the cell nucleus positions are manually labeled and corrected one by one. In order to accurately distinguish adjacent cells, a one-pixel-wide marking line is introduced at the edge position as a separator to ensure the clarity of the cell nucleus boundary. The final private dataset consists of a training set (276 images), a validation set (59 images) and a test set (32 images).

[0051] Further, data augmentation is performed on the public dataset and the private dataset. The image can be flipped with a probability of 50%, randomly flipped vertically with a probability of 50%, and randomly rotated by an angle of 0°-360° with a probability of 50%.

[0052] In step S102, a cell nucleus segmentation network structure is constructed, which includes a patch embedding layer, an encoder module, a tokenized KAN network module (Tok-KAN), a decoder module and a projection layer connected in sequence. The encoder module and the decoder module are further connected through a frequency domain learning module (PFDLM). The frequency domain learning module is used to transfer low-level features in the encoder module to the decoder module. The tokenized KAN network module is used for deep feature extraction.

[0053] Referring to Figure 2 As shown in the figure, the cell nucleus segmentation network structure in the embodiment is improved on the basis of the VM-UNet architecture. The VSS (Visual State Space) block is used in the encoder module to capture context information, and an asymmetric encoder-decoder structure is constructed. The features are embedded by the patch embedding layer before entering the encoder, and after decoding the features, the size of the features is restored by a final projection layer to match the segmentation target.

[0054] In the embodiment, an asymmetric structure is used. The patch embedding layer maps the input image x∈R H×W×3 into non-overlapping patches (patches can be understood as smaller regions) with a size of 4*4, and then maps the dimensions of the image to C, and C is 96 by default. This process obtains the embedded image Then, the embedded image x' is normalized by layer, and then input into the encoder model for feature extraction. In this embodiment, a frequency domain learnable module jump connection is arranged in the U-shaped structure, and the frequency domain learnable module helps to pass the low-level features such as edges and textures in the encoder to the decoder, and combine with the high-level features such as semantic information, which helps to generate more accurate segmentation map. The tokenization KAN network module is arranged at the bottleneck position of the network structure, which can refine the features, not only increase the network depth, but also increase the ability of the model to capture multi-scale features, and further improve the accuracy of the cell nucleus segmentation.

[0055] In step S103, the cell nucleus segmentation network is trained based on the cell image dataset to obtain a cell nucleus segmentation model.

[0056] In this embodiment, a deep learning network model FfvmUkan for cell nucleus recognition and segmentation is proposed. In the VM-UNet framework, an image frequency domain sensitive frequency domain learnable module is introduced for boundary segmentation task to improve the cell nucleus boundary segmentation level by learning image frequency domain features. In this embodiment, a tokenization KAN network module is also introduced to meet the deeper feature extraction requirement and increase the ability of the model to capture multi-scale features. Compared with the traditional method, the cell nucleus segmentation model provided by the present application pays more attention to the role of cell nucleus boundary, which greatly improves the accuracy of cell nucleus segmentation.

[0057] In some optional embodiments, the encoder module includes a plurality of layers of encoders connected in sequence, each layer of encoder includes a visual state space module; the input end of the first layer of encoder is connected with the patch embedding layer, and the output end of the last layer of encoder is connected with the tokenization KAN network module; in addition to the last layer of encoder, each layer of encoder further includes a patch merging layer, and the output end of the visual state space module is connected with the patch merging layer.

[0058] The decoder module includes a plurality of layers of decoders connected in sequence, each layer of decoder includes a visual state space module; the input end of the first layer of decoder is connected with the tokenization KAN network module, and the output end of the last layer of decoder is connected with the projection layer; in addition to the first layer of decoder, each layer of decoder further includes a patch expansion layer, and the input end of the visual state space module is connected with the patch expansion layer.

[0059] In some optional embodiments, the encoder module includes four layers of encoders connected in sequence, each layer of encoder includes two visual state space modules; the decoder module includes four layers of decoders connected in sequence, in addition to the last layer of decoder, each layer of decoder includes two visual state space modules; the last layer of decoder includes one visual state space module.

[0060] In this embodiment, the encoder module can be composed of four stages, each stage using 2 VSS blocks, and the number of channels is [C, 2C, 4C, 8C] respectively. The first three stages apply a patch merging operation after the VSS block. Through the patch merging layer, the information between different layers can be effectively integrated, the representation ability of the features can be improved, the height and width of the input features can be reduced, and the number of channels can be increased. The decoder module is also divided into four stages, and four stages use [2, 2, 2, 1] VSS blocks respectively, and the number of channels of each stage is [8C, 4C, 2C, C]. The last three stages use a patch expanding operation before the VSS block. Through the patch expanding layer, the decoder can generate high-dimensional features from low-dimensional features and recover more rich information, and also reduce the number of feature channels while increasing the height and width of the feature channels. The final projection layer after the decoder module can be used to restore the size of the features to match the segmentation target. Specifically, the height and width of the features are restored through 4 times of upsampling by patch expanding, and then the number of channels is restored through the projection layer.

[0061] In this embodiment, the visual state space module is used in the shallow layer, which can dynamically adjust the sampling strategy according to different features of the input data, adapt to different resolutions and scales, and thus improve the adaptability of the model to diversified data. On the basis of feature extraction, the deep layer increases the tokenization KAN network module to refine the features, which not only increases the network depth, but also increases the ability of the model to capture multi-scale features, and further improves the accuracy of the cell nucleus segmentation in combination with the nonlinear modeling of the tokenization KAN network module.

[0062] In some optional embodiments, each input end of each encoder is connected with each output end of each decoder through a frequency domain learnable module.

[0063] In this embodiment, a frequency domain learnable module is further arranged in a skip connection manner at the input end of each visual state space module. The frequency domain learnable module helps to transfer low-level features such as edges and textures in the encoder to the decoder, and combine with high-level features such as semantic information of the decoder, which helps to generate more accurate segmentation maps, and thus further improves the accuracy of the cell nucleus segmentation.

[0064] In some optional embodiments, the frequency domain learnable module comprises: a feature input module, a Fast Fourier Transform (FFT) module, a frequency domain feature map module, an Inverse Fast Fourier Transform (IFFT) module and a feature output module connected in sequence.

[0065] The frequency domain learnable module further comprises a self-learning attention module, which comprises a parameter-learnable attention map, used for element-wise multiplication with the output result of the frequency domain feature map module, and outputs the multiplication result to the inverse fast Fourier transform module.

[0066] The feature input module is further connected with the feature output module, used for adding the output result of the inverse fast Fourier transform module and the original feature of the feature input module.

[0067] The skip connection of the frequency domain learnable module in the network structure helps to pass the low-level features such as edges and textures in the encoder to the decoder, and combine with the high-level features such as semantic information in the encoder, which helps to generate more accurate segmentation maps. The improvement at the skip connection part adds a frequency domain learnable module, which can adaptively learn the features important for the segmentation task in the frequency domain, highlight the edges of the cell nuclei, and help make correct segmentation judgments at positions where the cell nucleus boundaries are not clear or overlap each other. The specific implementation of the frequency domain learnable module is described in detail in the following Figure 3 The implementation of the frequency domain learnable module is as follows:

[0068] First, 2D fast Fourier transform is performed along the spatial dimension:

[0069]

[0070] where F[·] represents two-dimensional fast Fourier transform (FFT), and then the parameter-learnable attention map A ∈ C H×W×C is multiplied by M to modulate its spectrum:

[0071] where □ represents element-wise multiplication, and then the inverse process (IFFT) of fast Fourier transform is used to restore M' to the spatial domain:

[0072] Finally, in the second branch, the residual connection is used to realize the fusion and enhancement of the features by adding the original feature m and the feature map m' processed by the first branch: m out = m + m'.

[0073] The frequency domain learnable module is a learnable version of the frequency domain filter, which globally adjusts the components of specific frequencies, and the learnability is conducive to the edge and texture features of the cell nucleus segmentation.

[0074] The first single branch significantly improves the ability to distinguish different cell nucleus instances by focusing on adaptive feature learning in the frequency domain, which is crucial for accurate segmentation of cell nucleus boundaries. However, using only this branch to input the encoder into the decoder at the same level can cause a slight decrease in accuracy. To solve this problem, a residual connection mechanism is introduced in this embodiment, which superimposes the original features with the features processed by the first branch, preserving the ability to finely capture the details of the cell nucleus edge and optimizing the overall segmentation.

[0075] In some optional embodiments, the tokenized KAN network module includes a tokenized KAN network encoding block and a tokenized KAN network decoding block.

[0076] The output end of the last layer encoder is connected to the tokenized KAN network encoding block; the output end of the tokenized KAN network decoding block is connected to the input end of the first layer decoder; and the tokenized KAN network encoding block is connected to the tokenized KAN network decoding block of the same layer.

[0077] The tokenized KAN network encoding block includes a tokenized layer, a first normalization layer, a KAN network layer, a second normalization layer, and a reshaping output layer connected in sequence; wherein the tokenized layer is used to tokenize the feature map into a plurality of two-dimensional blocks, and the KAN network layer is used for nonlinear transformation.

[0078] Still taking the above example, after four-stage VSS block encoding, the KAN network encoding structure in the tokenized KAN network module is entered, two layers of tokenized KAN network encoding blocks are additionally added in the feature extraction stage, and then transmitted into two layers of tokenized KAN network decoding blocks, and then four layers of VSS blocks are entered for decoding.

[0079] Referring to Figure 4 In the tokenized KAN network module, the feature map from the fourth stage of the encoder is tokenized into a series of two-dimensional blocks, wherein the size of each two-dimensional block is P x P, and the number of feature blocks is N = H L x W L / stride 2 Here P = 3, stride = 2, and the feature map is mapped to a latent embedding space Z0 through a trainable linear projection The embedded tensor is sent to the KAN network block (KAN Block), and the stacked KAN network layer (Stacked KAN Layer) in the KAN network block is three layers, which plays a role of nonlinear transformation in the network. The KAN network layer refers to Figure 5 ​As shown, the features sequentially pass through a KAN nonlinear layer (KANLinear), a deep convolutional layer (DwConv), and a batch normalization layer (BatchNorm), and then are subjected to ReLU activation. This process is repeated three times to constitute a stacked KAN network layer structure (Stacked KAN), where one layer is denoted as Stacked_KAN_Layer_1 / 3. After passing through the KAN network layer, the original token is added back as a residual using a residual connection. Finally, layer normalization, i.e., the second normalization layer in this embodiment, is applied to the output features, which are passed to the next tokenization KAN network encoding block or tokenization KAN network decoding block or the VSS block thereafter.

[0080] In the structure of the tokenization KAN network decoding block (Tok-KAN decoding block), the features first pass through a decoding structure composed of two convolutional layers, two batch normalization layers, and two ReLU activation functions in sequence. Then, bilinear upsampling is used to add the output result to the same layer tokenization KAN network encoding block (Tok-KAN encoding block) in the tokenization KAN network module.

[0081] In this embodiment, the tokenization KAN network module includes two tokenization KAN network decoding blocks, where k refers to the first or the second, and k in the decoder is the same.

[0082] If the output features of the same layer kth tokenization KAN network encoding block are denoted as Z k , the features of the tokenization KAN network decoding block are denoted as Z′ k , and the features of the previous layer tokenization KAN network decoding block are denoted as Z′ k+1 , then the features Z′ k of the kth tokenization KAN network decoding block are:

[0083] Z′ k = Cat(Z′ k+1 , Z k ),

[0084] where Cat(·) represents a feature concatenation operation.

[0085] In the KAN network block (KAN Block), the KANBlock includes a normalization layer (Layer Norm), a stacked KAN network layer (Stacked KAN Layer), and a residual connection. The features output by the stacked KAN network block are:

[0086] KANBlock(X) = Z K-1 + Stacked_KAN_Layer(LN(X))

[0087] LN is the abbreviation of Layer Norm.

[0088] The Kth Tok-KAN block is expressed as follows:

[0089] Z k = LN(tokenized(Z K-1 )+Stacked_KAN_Layer(LN(tokenized(Z K-1 ))))

[0090] wherein:

[0091] Stacked_KAN_Layer_1 / 3(X) = (Relu(BN(DwConv(KANLinear(X))))) wherein BN is the abbreviation of BatchNorm.

[0092] The cell nucleus segmentation model constructed by the application, that is, the FfvmUkan model for cell or cell nucleus segmentation has higher segmentation accuracy, and avoids complex parameter setting and interactive labeling workload. The model can better distinguish the cell nucleus position and the background position through frequency domain learning, and enhances the features through residual connection. Not only the features after a series of transformations are considered, but also the original input features are reserved, helping the network to learn image features at different levels, capturing more fine and robust feature representations, significantly improving the network's ability to handle cell nucleus edge details, especially in the boundary fuzzy or overlapping area, effectively improving the segmentation accuracy.

[0093] In the present embodiment, a cell nucleus segmentation method is provided, which can be used in servers, terminals, mobile terminals such as mobile phones, tablet computers and the like. The cell nucleus segmentation method comprises the following steps:

[0094] In step S201, a to-be-segmented image is obtained.

[0095] In step S201, the to-be-segmented image is input into the cell nucleus segmentation model constructed according to the construction method of any one of the above embodiments to obtain a cell nucleus segmentation result corresponding to the to-be-segmented image.

[0096] Compared with the VM-UNet, the nuclear segmentation model FfvmUkan constructed by the application can improve the nuclear segmentation accuracy on a private data set from 0.8454 to 0.8595 in terms of mIOU (Mean Intersection over Union, a commonly used index for evaluating image segmentation tasks); and on a public data set MoNuSeg, the mIOU can be improved from 0.6654 to 0.6866. As shown in Table 1, compared with the performance of deep learning models such as UNet, UNet++, Swinunet and the like, the FfvmUkan model is superior to other models in terms of mean intersection over union, Dice similarity coefficient, accuracy and specificity. Compared with UNet, UNet++, Swinunet and VM-UNet, the mean intersection over union is increased by 5.0%, 4.4%, 5.9% and 1.7% on the private data set, and by 13.6%, 14.3%, 9.5% and 3.2% on the public data set. UNet and UNet++ perform poorly at the boundary position, it is difficult for them to distinguish overlapping or close nuclear instances, and there is a problem of identifying background regions as nuclear regions. Although the specificity of Swinunet is the highest, it can be seen from the mIOU and DSC scores and the performance of the segmentation map that it identifies many nuclear regions as background regions.

[0097] Referring to Figure 6 Fig. 1 is a comparison chart of nuclear segmentation performance on a private data set, Figure 7 Fig. 2 is a comparison chart of nuclear segmentation performance on a MoNuSeg data set. Figure 6 The selected region in the first row shows that a large number of nuclei are classified as background regions, so although good results can be achieved at the boundary position of the nucleus, the overall segmentation is not ideal. Figure 6 The selected region in the upper right corner of each figure in the second row shows that compared with Swin-Unet, FfvmUkan achieves more accurate segmentation results. The selected region in the middle of each figure shows the obvious advantage of FfvmUkan over UNet and UNet++ at the boundary position. It shows that the FfvmUkan algorithm can provide more fine and accurate segmentation results from multiple angles when dealing with complex cell structures.

[0098] Table 1

[0099]

[0100] Further description of the nuclear segmentation model is the same as the above corresponding embodiments, which will not be repeated here.

[0101] In the embodiment, a cell nucleus segmentation model construction device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described herein again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0102] The embodiment provides a cell nucleus segmentation model construction device, as shown in Figure 8 , comprising:

[0103] The acquisition module 301 is configured to acquire a cell image dataset.

[0104] The construction module 302 is configured to construct a cell nucleus segmentation network structure, the cell nucleus segmentation network structure comprising a patch embedding layer, an encoder module, a tokenization KAN network module, a decoder module and a projection layer connected in sequence, and the encoder module and the decoder module are further connected through a frequency domain learnable module; wherein the frequency domain learnable module is configured to pass low-level features in the encoder module to the decoder module; and the tokenization KAN network module is configured to extract deep-level features.

[0105] The training module 303 is configured to train the cell nucleus segmentation network based on the cell image dataset to obtain a cell nucleus segmentation model.

[0106] The cell nucleus segmentation model construction device in the embodiment is presented in the form of functional units, and the units herein refer to ASIC circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0107] Further function descriptions of the above-mentioned modules and units are the same as those of the corresponding embodiments described above, and will not be described herein again.

[0108] The embodiment of the present application also provides a computer device with the cell nucleus segmentation model construction device shown in Figure 8 .

[0109] Please refer to Figure 9 , Figure 9 is a structural schematic diagram of a computer device provided by an optional embodiment of the present application, as shown in Figure 9As shown, the computer device includes one or more processors 10, memory 20, and interfaces 30 for the various components to communicate with one another. The various components communicate through one or more buses, and can be mounted on a common motherboard or in other manners as appropriate. The processor 10 can execute instructions, for example, stored in the memory 20 to display graphical information for a GUI on an external input / output device, such as a display device coupled to the interface. In some optional implementations, multiple processors and / or multiple buses can be employed as appropriate, such as about the memory 20. Also, various components can be distributed, such as over a network to provide for parallel processing. Figure 9 The processor 10 is taken as an example.

[0110] The processor 10 can be a central processing unit, a network processor, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic device, a general array logic, or any combination thereof.

[0111] The memory 20 stores instructions that are executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods illustrated in the above embodiments.

[0112] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required by at least one function, and the like. The data storage area can store data created by the computer device, and the like. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid state memory device. In some optional implementations, the memory 20 can optionally include a memory that is remotely located with respect to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0113] The memory 20 can include a volatile memory, such as a random access memory, and can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid state disk. The memory 20 can further include a combination of the above-mentioned types of memories.

[0114] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or communication networks.

[0115] The embodiments of the present application further provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or be implemented as computer codes stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded through a network and stored in a local storage medium, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer codes, when the software or computer codes are accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0116] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method for constructing a cell nucleus segmentation model, characterized in that, The construction method comprises: acquiring a cell image data set; constructing a cell nucleus segmentation network structure, the cell nucleus segmentation network structure comprising a patch embedding layer, an encoder module, a tokenized KAN network module, a decoder module and a projection layer connected in sequence, and the encoder module and the decoder module being further connected through a frequency domain learnable module; wherein the frequency domain learnable module is used for transmitting low-level features in the encoder module to the decoder module; the tokenized KAN network module is used for deep feature extraction; wherein the encoder module comprises multiple layers of encoders connected in sequence, each layer of encoder comprising a visual state space module; the decoder module comprises multiple layers of decoders connected in sequence, each layer of decoder comprising a visual state space module; the frequency domain learnable module comprises a self-learning attention module, and further comprises a feature input module, a fast Fourier transform module, a frequency domain feature map module, an inverse fast Fourier transform module and a feature output module connected in sequence; the self-learning attention module comprises a parameter-learnable attention map, used for element-wise multiplication with an output result of the frequency domain feature map module, and outputs a multiplication result to the inverse fast Fourier transform module; the feature input module is further connected with the feature output module, and used for adding an output result of the inverse fast Fourier transform module with original features of the feature input module; the tokenized KAN network module comprises a tokenized KAN network encoding block and a tokenized KAN network decoding block; an output end of the last layer of encoder is connected with the tokenized KAN network encoding block; an output end of the tokenized KAN network decoding block is connected with an input end of the first layer of decoder; and the tokenized KAN network encoding block is connected with the tokenized KAN network decoding block of the same layer; the tokenized KAN network encoding block comprises a tokenization layer, a first normalization layer, a KAN network layer, a second normalization layer and a reshaped output layer connected in sequence; wherein the tokenization layer is used for tokenizing a feature map into multiple two-dimensional blocks, and the KAN network layer is used for nonlinear transformation; wherein the encoder module comprises four layers of encoders connected in sequence, each layer of encoder comprising two blocks of the visual state space module; the decoder module comprises four layers of decoders connected in sequence, each layer of decoder comprising two blocks of the visual state space module except for the last layer of decoder; the last layer of decoder comprises one block of the visual state space module; an input end of the first layer of encoder is connected with the patch embedding layer, and an output end of the last layer of encoder is connected with the tokenized KAN network module; each layer of encoder further comprises a patch merging layer except for the last layer of encoder, and an output end of the visual state space module is connected with the patch merging layer; an input end of the first layer of decoder is connected with the tokenized KAN network module, and an output end of the last layer of decoder is connected with the projection layer; each layer of decoder further comprises a patch expansion layer except for the first layer of decoder, and an input end of the visual state space module is connected with the patch expansion layer; An input end of each encoder is connected with an output end of each decoder through the frequency domain learnable module; The nucleus segmentation network is trained based on the cell image dataset, and a nucleus segmentation model is obtained.

2. A method of cell nucleus segmentation, the method comprising: The nucleus segmentation method comprises: An image to be segmented is acquired; The nucleus segmentation model constructed according to the construction method of claim 1 is input with the image to be segmented, and a nucleus segmentation result corresponding to the image to be segmented is obtained.

3. A nucleus segmentation model construction apparatus, characterized by comprising: The device comprises: An acquisition module is configured to acquire a cell image dataset; The construction module is used for constructing a cell nucleus segmentation network structure, and the cell nucleus segmentation network structure comprises a patch embedding layer, an encoder module, a tokenized KAN network module, a decoder module and a projection layer which are connected in sequence, and the encoder module and the decoder module are further connected through a frequency domain learnable module; wherein the frequency domain learnable module is used for transmitting low-level features in the encoder module to the decoder module; the tokenized KAN network module is used for deep feature extraction; wherein the encoder module comprises multiple layers of encoders which are connected in sequence, and each layer of encoder comprises a visual state space module; the decoder module comprises multiple layers of decoders which are connected in sequence, and each layer of decoder comprises a visual state space module; the frequency domain learnable module comprises a self-learning attention module, and further comprises a feature input module, a fast Fourier transform module, a frequency domain feature map module, an inverse fast Fourier transform module and a feature output module which are connected in sequence; the self-learning attention module comprises a parameter-learnable attention map, is used for performing element-by-element multiplication with an output result of the frequency domain feature map module, and outputs a multiplication result to the inverse fast Fourier transform module; the feature input module is further connected with the feature output module, and is used for adding an output result of the inverse fast Fourier transform module with original features of the feature input module; the tokenized KAN network module comprises a tokenized KAN network encoding block and a tokenized KAN network decoding block; an output end of the last layer of encoder is connected with the tokenized KAN network encoding block; an output end of the tokenized KAN network decoding block is connected with an input end of the first layer of decoder; and the tokenized KAN network encoding block is connected with the tokenized KAN network decoding block of the same layer; the tokenized KAN network encoding block comprises a tokenization layer, a first normalization layer, a KAN network layer, a second normalization layer and a reshaping output layer which are connected in sequence; wherein the tokenization layer is used for tokenizing a feature map into multiple two-dimensional blocks, and the KAN network layer is used for nonlinear transformation; wherein the encoder module comprises four layers of encoders which are connected in sequence, and each layer of encoder comprises two blocks of the visual state space module; the decoder module comprises four layers of decoders which are connected in sequence, and each layer of decoder comprises two blocks of the visual state space module except the last layer of decoder; the last layer of decoder comprises one block of the visual state space module; an input end of the first layer of encoder is connected with the patch embedding layer, and an output end of the last layer of encoder is connected with the tokenized KAN network module; each layer of encoder further comprises a patch merging layer except the last layer of encoder, and an output end of the visual state space module is connected with the patch merging layer; an input end of the first layer of decoder is connected with the tokenized KAN network module, and an output end of the last layer of decoder is connected with the projection layer; each layer of decoder further comprises a patch expansion layer except the first layer of decoder, and an input end of the visual state space module is connected with the patch expansion layer; An input end of each encoder is connected with an output end of each decoder through the frequency domain learnable module; A training module is configured to train the nucleus segmentation network based on the cell image dataset to obtain a nucleus segmentation model.

4. A computer device, comprising: Comprise: A memory and a processor in communication connection with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the cell nucleus segmentation model construction method in claim 1.

5. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the computer to perform the cell nucleus segmentation model construction method in claim 1. The computer readable storage medium stores computer instructions for causing the computer to perform the cell nucleus segmentation model construction method in claim 1.

Images