Cell nucleus segmentation model construction method, cell nucleus segmentation method and construction device

Abstract

The invention relates to the technical field of data processing, and discloses a cell nucleus segmentation model construction method, a cell nucleus segmentation method and a construction device.The construction method comprises the steps that a cell image data set is acquired; a cell nucleus segmentation network structure is constructed, the cell nucleus segmentation network structure comprises a patch embedding layer, an encoder module, a tokenized KAN network module, a decoder module and a projection layer which are connected in sequence, and the encoder module and the decoder module are further connected through a frequency domain learnable module; wherein the frequency domain learnable module is used for transmitting low-level features in the encoder module to the decoder module; the tokenized KAN network module is used for deep feature extraction; and training the cell nucleus segmentation network based on the cell image data set to obtain a cell nucleus segmentation model. The cell nucleus segmentation model provided by the invention pays more attention to the cell nucleus boundary effect, and the cell nucleus segmentation precision is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a cell nucleus segmentation model construction method, a cell nucleus segmentation method and a construction device. Background Art

[0002] Nuclear segmentation is an important biomedical technology, which mainly involves microscopic image processing and analysis of cells in collected tissue or body fluid samples to obtain separated single cell nuclei. Nuclear segmentation technology plays an important role in determining the location, morphology, and number of cell nuclei, and then conducting research on gene expression and protein expression. It has wide applications in tumor liquid biopsy, spatial multi-omics analysis, pathological diagnosis and other fields.

[0003] However, the acquired tissue cell microscopic images have strong cell heterogeneity, cell stacking, uneven staining and other problems, resulting in poor accuracy of cell nucleus recognition and segmentation and low segmentation precision. At present, cell or cell nucleus segmentation mostly uses traditional algorithms, such as the watershed algorithm or relies on interactive software to perform segmentation. When the watershed algorithm is used to segment the cell nucleus, although it does not need to rely on a large amount of training data to learn features, it has a high dependence on predefined parameters, poor applicability, and low accuracy in processing complex tissue images. When using interactive software such as Ilastik (Ilastik is an open source software for biomedical image analysis) to perform cell nucleus segmentation, multiple annotation interactions and multiple rounds of iterations are required to obtain a good segmentation effect, and its automation level is low.

[0004] At present, cell nucleus segmentation methods based on deep learning have also been developed, such as Cellpose based on Unet (Cellpose is a deep learning model based on U-Net structure), etc., which achieve universal cell segmentation capabilities in different application scenarios by constructing diverse and multi-scenario training data sets. However, when the cell nucleus has a complex shape, blurred edges or overlaps, the cell nucleus segmentation boundary will be inaccurate. In recent years, improved UNet++, Transformer, SwinUNet and other deep learning methods have been developed one after another. Due to problems such as overfitting, high computational cost, and strong dependence on data annotation, it is difficult to meet the segmentation accuracy requirements required for analysis. Summary of the invention

[0005] In view of this, the present invention provides a cell nucleus segmentation model construction method, a cell nucleus segmentation method and a construction device to solve the problem of low cell nucleus segmentation accuracy in the cell nucleus segmentation method in the prior art.

[0006] In a first aspect, the present invention provides a method for constructing a cell nucleus segmentation model, the method comprising:

[0007] Obtain cell image dataset;

[0008] Constructing a cell nucleus segmentation network structure, which includes a patch embedding layer, an encoder module, a tokenized KAN network module, a decoder module, and a projection layer connected in sequence. The encoder module and the decoder module are also connected through a frequency domain learnable module; the frequency domain learnable module is used to transfer low-level features in the encoder module to the decoder module; the tokenized KAN network module is used for deep-level feature extraction;

[0009] The cell nucleus segmentation network is trained based on the cell image dataset to obtain the cell nucleus segmentation model.

[0010] In this embodiment, a deep learning network model FfvmUkan for cell nucleus recognition and segmentation is proposed. Based on the VM-UNet framework, a frequency domain learnable module that is sensitive to the image frequency domain is introduced for the boundary segmentation task, and the level of cell nucleus boundary segmentation is improved by learning the image frequency domain features. In this embodiment, a tokenized KAN network module is also introduced to meet the needs of deeper feature extraction and increase the model's ability to capture multi-scale features. Compared with traditional methods, the cell nucleus segmentation model provided by the present invention pays more attention to the role of the cell nucleus boundary and greatly improves the accuracy of cell nucleus segmentation.

[0011] In an optional embodiment, the encoder module includes multiple layers of encoders connected in sequence, each layer of encoders includes a visual state space module; the input end of the first layer of encoders is connected to the patch embedding layer, and the output end of the last layer of encoders is connected to the tokenized KAN network module; except for the last layer of encoders, each layer of encoders also includes a patch merging layer, and the output end of the visual state space module is connected to the patch merging layer;

[0012] The decoder module includes multiple layers of decoders connected in sequence, and each layer of decoders includes a visual state space module; the input end of the first layer of decoders is connected to the tokenized KAN network module, and the output end of the last layer of decoders is connected to the projection layer; except for the first layer of decoders, each layer of decoders also includes a patch expansion layer, and the input end of the visual state space module is connected to the patch expansion layer.

[0013] In this embodiment, the visual state space module is used in the shallow layer, and the sampling strategy can be dynamically adjusted according to the different features of the input data to adapt to different resolutions and scales, thereby improving the model's adaptability to diverse data. On the basis of feature extraction, the tokenized KAN network module is added in the deep layer to refine the features, which not only increases the network depth, but also increases the model's ability to capture multi-scale features. The nonlinear modeling combined with the tokenized KAN network module further improves the accuracy of cell nucleus segmentation.

[0014] In an optional implementation, the input end of each encoder and the output end of each decoder are connected via a frequency domain learnable module.

[0015] In this embodiment, a jump connection with a frequency domain learnable module is also set at the input end of each visual state space module. The frequency domain learnable module helps to transfer low-level features such as edges and textures in the encoder to the decoder, and combines with high-level features of the decoder such as semantic information, which helps to generate a more accurate segmentation map, thereby further improving the accuracy of cell nucleus segmentation.

[0016] In an optional implementation, the frequency domain learnable module includes: a feature input module, a fast Fourier transform module, a frequency domain feature map module, an inverse fast Fourier transform module, and a feature output module connected in sequence;

[0017] The frequency domain learnable module also includes a self-learning attention module, which includes an attention map with learnable parameters, and is used to perform element-by-element multiplication with the output result of the frequency domain feature map module, and output the product result to the inverse fast Fourier transform module;

[0018] The feature input module is also connected to the feature output module for adding the output result of the inverse fast Fourier transform module to the original feature of the feature input module.

[0019] In this embodiment, a residual connection mechanism is introduced to superimpose the original features with the features processed by the first branch, thereby retaining the ability to finely capture the details of the edge of the cell nucleus and achieving optimization in the overall segmentation.

[0020] In an optional embodiment, the tokenized KAN network module includes: a tokenized KAN network encoding block and a tokenized KAN network decoding block;

[0021] The output of the last layer encoder is connected to the tokenized KAN network encoding block; the output of the tokenized KAN network decoding block is connected to the input of the first layer decoder; and the tokenized KAN network encoding block is connected to the tokenized KAN network decoding block of the same layer;

[0022] The tokenized KAN network coding block includes: a tokenization layer, a first normalization layer, a KAN network layer, a second normalization layer and a reshaped output layer connected in sequence; wherein the tokenization layer is used to tokenize the feature map into multiple two-dimensional blocks, and the KAN network layer is used to perform nonlinear transformation.

[0023] In an optional embodiment, the encoder module includes four layers of encoders connected in sequence, each layer of encoders includes two visual state space modules; the decoder module includes four layers of decoders connected in sequence, except for the last layer of decoders, each layer of decoders includes two visual state space modules; the last layer of decoders includes a visual state space module.

[0024] The cell nucleus segmentation model constructed by the present invention, that is, the FfvmUkan model for cell or cell nucleus segmentation, has higher segmentation accuracy, while avoiding complex parameter settings and interactive annotation workload. The model can achieve better distinction between the cell nucleus position and the background position through frequency domain learning, and enhance features through residual connection. It not only considers the features after a series of transformations, but also retains the original input features, helping the network to learn image features at different levels, capturing more refined and robust feature representations, and significantly improving the network's ability to process cell nucleus edge details, especially in areas with blurred or overlapping boundaries, effectively improving segmentation accuracy.

[0025] In a second aspect, the present invention provides a method for cell nucleus segmentation, the method for cell nucleus segmentation comprising:

[0026] Obtain the image to be segmented;

[0027] The image to be segmented is input into the cell nucleus segmentation model constructed according to the construction method described in any one of the above embodiments to obtain a cell nucleus segmentation result corresponding to the image to be segmented.

[0028] In a third aspect, the present invention provides a device for constructing a cell nucleus segmentation model, the device comprising:

[0029] An acquisition module, used to acquire a cell image dataset;

[0030] A construction module is used to construct a cell nucleus segmentation network structure, which includes a patch embedding layer, an encoder module, a tokenized KAN network module, a decoder module, and a projection layer connected in sequence. The encoder module and the decoder module are also connected through a frequency domain learnable module; wherein the frequency domain learnable module is used to transfer low-level features in the encoder module to the decoder module; and the tokenized KAN network module is used for deep-level feature extraction;

[0031] The training module is used to train the cell nucleus segmentation network based on the cell image dataset to obtain the cell nucleus segmentation model.

[0032] In a fourth aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the cell nucleus segmentation model construction method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0033] In a fifth aspect, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the cell nucleus segmentation model construction method of the first aspect or any corresponding embodiment thereof.

[0034] It should be noted that, since the cell nucleus segmentation model construction device, computer device and computer readable storage medium provided by the present invention correspond to the above-mentioned cell nucleus segmentation model construction method. Therefore, for the beneficial effects of the cell nucleus segmentation model construction device, computer device and computer readable storage medium, please refer to the description of the corresponding beneficial effects of the cell nucleus segmentation model construction method above, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 is a schematic flow chart of a method for constructing a cell nucleus segmentation model according to an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of the structure of a cell nucleus segmentation model according to an embodiment of the present invention;

[0038] Figure 3 is a schematic diagram of the structure of a frequency domain learnable module according to an embodiment of the present invention;

[0039] Figure 4 is a schematic diagram of the structure of a tokenized KAN network module according to an embodiment of the present invention;

[0040] Figure 5 is a schematic diagram of the structure of a KAN network layer according to an embodiment of the present invention;

[0041] Figure 6 is a schematic diagram of comparing cell nucleus segmentation performance under a private dataset according to an embodiment of the present invention;

[0042] Figure 7is a schematic diagram of comparing the cell nucleus segmentation performance under the MoNuSeg dataset according to an embodiment of the present invention;

[0043] Figure 8 is a structural block diagram of a cell nucleus segmentation model construction device according to an embodiment of the present invention;

[0044] Fig. 9 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] According to an embodiment of the present invention, an embodiment of a method for constructing a cell nucleus segmentation model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0047] In this embodiment, a method for constructing a cell nucleus segmentation model is provided, which can be executed by a server, a terminal, a mobile terminal, etc. Figure 1 is a flow chart of a method for constructing a cell nucleus segmentation model according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0048] Step S101, obtaining a cell image dataset.

[0049] In this embodiment, the public data set MoNuSeg (MoNuSeg is an open data set for cell image segmentation) and the self-built private data set can be used as the cell image data set, and preprocessing work can be carried out. In this embodiment, the public data set consists of a training set (30 images) and a test set (14 images), the image size is 1000×1000 pixels, and four 512×512 pixel image blocks are extracted from the four corners of each image, and then the image block is evenly divided into four parts according to the aspect ratio of 1:1, and finally the training set is 480 256×256 pixel images, and the test set is 224 256×256 pixel images.

[0050] The self-built private dataset takes the H&E staining images of mouse kidney tissue sections as an example. The images are cropped to 256×256 pixels, and the positions of the cell nuclei are manually annotated and corrected one by one. In order to accurately distinguish adjacent cells, a one-pixel wide annotation line is introduced at the edge position as a separator to ensure the clarity of the cell nucleus boundary. The private dataset finally obtained consists of a training set (276 images), a validation set (59 images), and a test set (32 images).

[0051] Furthermore, data augmentation is performed on the public datasets and the self-owned datasets, and the images can be flipped horizontally with a probability of 50%, randomly flipped vertically with a probability of 50%, and randomly rotated by 0°-360° with a probability of 50%.

[0052] Step S102, constructing a cell nucleus segmentation network structure, the cell nucleus segmentation network structure includes a patch embedding layer, an encoder module, a tokenized Kolmogorov-Arnold Network (Tok-KAN), a decoder module and a projection layer connected in sequence, the encoder module and the decoder module are also connected through a parallel frequency domain learning module (PFDLM); wherein the frequency domain learnable module is used to transfer low-level features in the encoder module to the decoder module; the tokenized KAN network module is used for deep feature extraction.

[0053] Reference Figure 2 As shown, the cell nucleus segmentation network structure in this embodiment is improved on the basis of the VM-UNet architecture. The VSS (Visual State Space) block is used in the encoder module to capture context information and an asymmetric encoder-decoder structure is constructed. The feature is embedded through the patch embedding layer before entering the encoder. After the feature is decoded, the size of the feature is restored through a final projection layer to match the segmentation target.

[0054] In this embodiment, an asymmetric structure is adopted, where the patch embedding layer converts the input image x∈R H×W×3 It is divided into non-overlapping patches of size 4×4 (patches can be understood as smaller areas), and then the dimension of the image is mapped to C, which defaults to 96. This process obtains the embedded image Next, the embedded image x′ is layer normalized and then input into the encoder model for feature extraction. In this embodiment, a frequency domain learnable module jump connection is set in the U-shaped structure. The frequency domain learnable module helps to transfer low-level features such as edges and textures in the encoder to the decoder, and combines with high-level features such as semantic information to help generate a more accurate segmentation map. A tokenized KAN network module is set at the bottleneck position of the network structure to refine the features, which can not only increase the network depth, but also increase the model's ability to capture multi-scale features, further improving the accuracy of cell nucleus segmentation.

[0055] Step S103, training the cell nucleus segmentation network based on the cell image data set to obtain a cell nucleus segmentation model.

[0056] In this embodiment, a deep learning network model FfvmUkan for cell nucleus recognition and segmentation is proposed. Based on the VM-UNet framework, a frequency domain learnable module that is sensitive to the image frequency domain is introduced for the boundary segmentation task, and the level of cell nucleus boundary segmentation is improved by learning the image frequency domain features. In this embodiment, a tokenized KAN network module is also introduced to meet the needs of deeper feature extraction and increase the model's ability to capture multi-scale features. Compared with traditional methods, the cell nucleus segmentation model provided by the present invention pays more attention to the role of the cell nucleus boundary and greatly improves the accuracy of cell nucleus segmentation.

[0057] In some optional embodiments, the encoder module includes multiple layers of encoders connected in sequence, each layer of encoder includes a visual state space module; the input end of the first layer of encoder is connected to the patch embedding layer, and the output end of the last layer of encoder is connected to the tokenized KAN network module; except for the last layer of encoder, each layer of encoder also includes a patch merging layer, and the output end of the visual state space module is connected to the patch merging layer.

[0058] The decoder module includes multiple layers of decoders connected in sequence, and each layer of decoders includes a visual state space module; the input end of the first layer of decoders is connected to the tokenized KAN network module, and the output end of the last layer of decoders is connected to the projection layer; except for the first layer of decoders, each layer of decoders also includes a patch expansion layer, and the input end of the visual state space module is connected to the patch expansion layer.

[0059] In some optional embodiments, the encoder module includes four layers of encoders connected in sequence, each layer of encoders includes two visual state space modules; the decoder module includes four layers of decoders connected in sequence, except for the last layer of decoders, each layer of decoders includes two visual state space modules; the last layer of decoders includes a visual state space module.

[0060] In this embodiment, the encoder module can be composed of four stages, each of which uses 2 VSS blocks, and the number of channels is [C, 2C, 4C, 8C]. The first three stages apply a patch merging operation after the VSS block. Through the patch merging layer, information can be effectively integrated between different layers, the representation ability of features can be improved, the height and width of input features can be reduced, and the number of channels can be increased. The decoder module is also divided into four stages, and the four stages use [2, 2, 2, 1] VSS blocks respectively, and the number of channels in each stage is [8C, 4C, 2C, C]. The last three stages use a patch expansion operation before the VSS block. Through the patch expansion layer, the decoder can generate high-dimensional features from low-dimensional features and recover richer information. It also reduces the number of feature channels and increases the height and width of feature channels. The final projection layer after the decoder module can be used to restore the size of the feature to match the segmentation target. Specifically, the height and width of the feature are restored by upsampling 4 times through patch expansion, and then the number of channels is restored through the projection layer.

[0061] In this embodiment, the visual state space module is used in the shallow layer, and the sampling strategy can be dynamically adjusted according to the different features of the input data to adapt to different resolutions and scales, thereby improving the model's adaptability to diverse data. On the basis of feature extraction, the tokenized KAN network module is added in the deep layer to refine the features, which not only increases the network depth, but also increases the model's ability to capture multi-scale features. The nonlinear modeling combined with the tokenized KAN network module further improves the accuracy of cell nucleus segmentation.

[0062] In some optional implementations, the input end of each encoder and the output end of each decoder are connected via a frequency domain learnable module.

[0063] In this embodiment, a jump connection with a frequency domain learnable module is also set at the input end of each visual state space module. The frequency domain learnable module helps to transfer low-level features such as edges and textures in the encoder to the decoder, and combines with high-level features of the decoder such as semantic information, which helps to generate a more accurate segmentation map, thereby further improving the accuracy of cell nucleus segmentation.

[0064] In some optional embodiments, the frequency domain learnable module includes: a feature input module, a fast Fourier transform module (Fast Fourier Transform, FFT for short), a frequency domain feature map module, an inverse fast Fourier transform module (Inverse Fast Fourier Transform, IFFT for short) and a feature output module connected in sequence.

[0065] The frequency domain learnable module also includes a self-learning attention module, which includes an attention map with learnable parameters, which is used to perform element-by-element multiplication with the output result of the frequency domain feature map module and output the product result to the inverse fast Fourier transform module.

[0066] The feature input module is also connected to the feature output module for adding the output result of the inverse fast Fourier transform module to the original feature of the feature input module.

[0067] The jump connection of the frequency domain learnable module in the network structure helps to transfer low-level features such as edges and textures in the encoder to the decoder, and combined with high-level features in the encoder, such as semantic information, helps to generate a more accurate segmentation map. By improving the jump connection part and adding the frequency domain learnable module, the features that are crucial to the segmentation task can be adaptively learned in the frequency domain, highlighting the edges of the cell nucleus, and helping to make correct segmentation judgments in locations where the cell nucleus boundaries are unclear or overlap. For the specific implementation of the frequency domain learnable module, refer to Figure 3 As shown, the implementation of the frequency domain learnable module is as follows:

[0068] First, perform a 2D fast Fourier transform along the spatial dimension:

[0069]

[0070] where F[·] represents a two-dimensional fast Fourier transform (FFT), and then the attention map A∈C is learned by parameterizing H×W×C Multiply by M to modulate its spectrum:

[0071] Among them, □ represents the element-by-element product, and then the inverse fast Fourier transform (IFFT) is used to restore M' to the spatial domain:

[0072] Finally, in the second branch, residual connection is used to add the original feature m to the feature map m' processed by the first branch to achieve feature fusion and enhancement: out =m+m′.

[0073] The frequency domain learnable module is a learnable version of the frequency domain filter. It globally adjusts the components of specific frequencies and can learn features such as edges and textures that are beneficial to cell nucleus segmentation.

[0074] The first single branch significantly improves the ability to distinguish different cell nucleus instances by focusing on adaptive feature learning in the frequency domain, which is crucial for accurate segmentation of cell nucleus boundaries. However, using only this branch to input the encoder into the decoder at the same level will result in a slight decrease in accuracy. To solve this problem, a residual connection mechanism is introduced in this embodiment to superimpose the original features with the features processed by the first branch, retaining the ability to capture the fine details of the cell nucleus edge and achieving optimization in overall segmentation.

[0075] In some optional implementations, the tokenized KAN network module includes: a tokenized KAN network encoding block and a tokenized KAN network decoding block.

[0076] The output of the last layer encoder is connected to the tokenized KAN network encoding block; the output of the tokenized KAN network decoding block is connected to the input of the first layer decoder; and the tokenized KAN network encoding block is connected to the tokenized KAN network decoding block of the same layer.

[0077] The tokenized KAN network coding block includes: a tokenized layer (Tokenized), a first normalization layer (Layer Norm), a KAN network layer (KAN Layer), a second normalization layer (Layer Norm) and a reshaped output layer (reshape) connected in sequence; wherein the tokenized layer is used to tokenize the feature map into multiple two-dimensional blocks, and the KAN network layer is used to perform nonlinear transformation.

[0078] Taking the above example as an example, after four stages of VSS block encoding, it enters the KAN network coding structure in the tokenized KAN network module. Two additional layers of tokenized KAN network coding blocks are added in the feature extraction stage, and then passed to the two-layer tokenized KAN network decoding block, and then enters the four-layer VSS block for decoding.

[0079] Reference Figure 4 As shown, in the tokenized KAN network module, the feature map from the fourth stage of the encoder is tokenized into a series of two-dimensional blocks, where the size of each two-dimensional block is P×P and the number of feature blocks is N=H L ×W L / stride 2 , where P = 3, stride = 2, through a trainable linear projection Mapped into a latent embedding space Z0, The embedded tensor is sent to the KAN network block (KAN Block). The stacked KAN layer in the KAN network block is three layers, which plays a nonlinear transformation role in the network. Figure 5As shown, the features pass through the KAN nonlinear layer (KANLinear), the deep convolution layer (DwConv) and the batch normalization layer (BatchNorm) in sequence, and then ReLU activation is performed. This process is repeated three times to form a stacked KAN network layer structure (Stacked KAN), where one layer is represented as Stacked_KAN_Layer_1 / 3. After passing through the KAN network layer, the original token is added back as the residual using a residual connection. Finally, layer normalization is applied, that is, the second normalization layer in this embodiment processes the output features and passes them to the next tokenized KAN network encoding block or the tokenized KAN network decoding block or the subsequent VSS block.

[0080] In the Tok-KAN decoding block structure, the features first pass through a decoding structure, which is a composite convolutional block consisting of two convolutional layers, two batch normalization layers, and two ReLU activation functions in sequence. Then, bilinear upsampling is used, and the output result is added to the Tok-KAN encoding block of the same layer in the Tokenized KAN Network module by skip connection.

[0081] In this embodiment, the tokenized KAN network module includes two tokenized KAN network decoding blocks, wherein the kth refers to the first or the second, and the same applies to the k in the decoder.

[0082] If the output feature of the kth tokenized KAN network coding block in the same layer is Z k , the tokenized KAN network decoding block is characterized by Z′ k , the feature of the decoding block of the tokenized KAN network in the previous layer is Z′ k+1 , then the feature Z′ of the kth tokenized KAN network decoding block k for:

[0083] Z′ k = Cat(Z′ k+1 ,Z k ),

[0084] Among them, Cat(·) represents the feature concatenation operation;

[0085] In the KAN network block (KAN Block), KANBlock contains a normalization layer (Layer Norm), a stacked KAN network layer (Stacked KAN Layer) and a residual connection, and the characteristics of the stacked KAN network block output are:

[0086] KANBlock(X)=Z K-1 +Stacked_KAN_Layer(LN(X))

[0087] Among them, LN is the abbreviation of Layer Norm.

[0088] The Kth Tok-KAN block is represented as follows:

[0089] Z k =LN(tokenized(Z K-1 )+Stacked_KAN_Layer(LN(tokenized(Z K-1 ))))

[0090] in:

[0091] Stacked_KAN_Layer_1 / 3(X)=(Relu(BN(DwConv(KANLinear(X))))), where BN is the abbreviation of BatchNorm.

[0092] The cell nucleus segmentation model constructed by the present invention, that is, the FfvmUkan model for cell or cell nucleus segmentation, has higher segmentation accuracy, while avoiding complex parameter settings and interactive annotation workload. The model can achieve better distinction between the cell nucleus position and the background position through frequency domain learning, and enhance features through residual connection. It not only considers the features after a series of transformations, but also retains the original input features, helping the network to learn image features at different levels, capturing more refined and robust feature representations, and significantly improving the network's ability to process cell nucleus edge details, especially in areas with blurred or overlapping boundaries, effectively improving segmentation accuracy.

[0093] In this embodiment, a cell nucleus segmentation method is provided, which can be used in a server, a terminal, a mobile terminal, such as a mobile phone, a tablet computer and other devices. The cell nucleus segmentation method includes the following steps:

[0094] Step S201, obtaining an image to be segmented.

[0095] Step S201: input the image to be segmented into the cell nucleus segmentation model constructed according to the construction method described in any one of the above embodiments to obtain the cell nucleus segmentation result corresponding to the image to be segmented.

[0096] Compared with VM-UNet, the cell nucleus segmentation model constructed by the present invention, namely the FfvmUkan model, has a cell nucleus segmentation accuracy of mIOU (Mean Intersection over Union, a common indicator for evaluating image segmentation tasks) on private datasets, which is improved from 0.8454 to 0.8595; on the public dataset MoNuSeg, mIOU is improved from 0.6654 to 0.6866. The performance comparison with deep learning models such as UNet, UNet++, and Swinunet is shown in Table 1. The FfvmUkan model outperforms other models in terms of average intersection over union, Dice similarity coefficient, accuracy, and specificity. Compared with Unet, Unet++, Swinunet, and VM-UNet, the average intersection over union is 5.0%, 4.4%, 5.9%, and 1.7% higher on private datasets. On public datasets, it is 13.6%, 14.3%, 9.5%, and 3.2% higher. Unet and Unet++ perform poorly at the boundary, it is difficult to distinguish overlapping or close nucleus instances, and there is a problem of identifying background areas as nucleus areas. Although Swinunet has the highest specificity, it can be seen from the performance of mIOU and DSC scores and segmentation maps that it identifies many nucleus areas as background areas.

[0097] Reference Figure 6 As shown, it is a comparison chart of cell nucleus segmentation performance under private datasets. Figure 7 This is a comparison chart of cell nucleus segmentation performance under the MoNuSeg dataset. Figure 6 In the boxed area in the first row, the cell nuclei are largely classified as background areas. Therefore, although a good effect can be achieved at the boundary of the cell nucleus, the overall segmentation is not ideal. Figure 6 In the upper right corner of each image in the second row, FfvmUkan achieves a more accurate segmentation effect compared with Swin-Unet. The middle boxed area of ​​each image shows the obvious advantages of FfvmUkan over Unet and Unet++ at the boundary position. This shows that the FfvmUkan algorithm can provide more detailed and accurate segmentation results from multiple angles when processing complex cell structures.

[0098] Table 1

[0099]

[0100] The further description of the cell nucleus segmentation model is the same as that of the above corresponding embodiment and will not be repeated here.

[0101] In this embodiment, a cell nucleus segmentation model construction device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0102] This embodiment provides a device for constructing a cell nucleus segmentation model. Figure 8 As shown, including:

[0103] An acquisition module 301 is used to acquire a cell image dataset;

[0104] A construction module 302 is used to construct a cell nucleus segmentation network structure, which includes a patch embedding layer, an encoder module, a tokenized KAN network module, a decoder module, and a projection layer connected in sequence, and the encoder module and the decoder module are also connected through a frequency domain learnable module; wherein the frequency domain learnable module is used to transfer low-level features in the encoder module to the decoder module; and the tokenized KAN network module is used for deep-level feature extraction;

[0105] The training module 303 is used to train the cell nucleus segmentation network based on the cell image data set to obtain a cell nucleus segmentation model.

[0106] The cell nucleus segmentation model construction device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0107] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0108] The embodiment of the present invention also provides a computer device having the above Figure 8 The cell nucleus segmentation model construction device shown.

[0109] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig. 9As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0110] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0111] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0112] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0113] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0114] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0115] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0116] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for constructing a cell nucleus segmentation model, characterized in that: The construction method comprises: Obtain cell image dataset; Constructing a cell nucleus segmentation network structure, the cell nucleus segmentation network structure includes a patch embedding layer, an encoder module, a tokenized KAN network module, a decoder module and a projection layer connected in sequence, the encoder module and the decoder module are also connected through a frequency domain learnable module; wherein the frequency domain learnable module is used to transfer low-level features in the encoder module to the decoder module; the tokenized KAN network module is used for deep-level feature extraction; The cell nucleus segmentation network is trained based on the cell image data set to obtain a cell nucleus segmentation model.

2. The method according to claim 1, characterized in that The encoder module includes multiple layers of encoders connected in sequence, each layer of encoders includes a visual state space module; the input end of the first layer of encoders is connected to the patch embedding layer, and the output end of the last layer of encoders is connected to the tokenized KAN network module; except for the last layer of encoders, each layer of encoders also includes a patch merging layer, and the output end of the visual state space module is connected to the patch merging layer; The decoder module includes multiple layers of decoders connected in sequence, each layer of decoders includes a visual state space module; the input end of the first layer of decoders is connected to the tokenized KAN network module, and the output end of the last layer of decoders is connected to the projection layer; In addition to the first layer decoder, each layer decoder further comprises a patch expansion layer, and an input end of the visual state space module is connected to the patch expansion layer.

3. The method according to claim 2, characterized in that The input end of each encoder and the output end of each decoder are connected through the frequency domain learnable module.

4. The method according to claim 2, characterized in that: The frequency domain learnable module includes: a feature input module, a fast Fourier transform module, a frequency domain feature map module, an inverse fast Fourier transform module and a feature output module connected in sequence; The frequency domain learnable module also includes a self-learning attention module, which includes an attention map with learnable parameters, and is used to perform element-by-element multiplication with the output result of the frequency domain feature map module, and output the product result to the inverse fast Fourier transform module; The feature input module is also connected to the feature output module, and is used to add the output result of the inverse fast Fourier transform module to the original feature of the feature input module.

5. The method according to claim 2, characterized in that: The tokenized KAN network module includes: a tokenized KAN network encoding block and a tokenized KAN network decoding block; The output end of the last layer encoder is connected to the tokenized KAN network encoding block; the output end of the tokenized KAN network decoding block is connected to the input end of the first layer decoder; and the tokenized KAN network encoding block is connected to the tokenized KAN network decoding block of the same layer; The tokenized KAN network coding block includes: a tokenization layer, a first normalization layer, a KAN network layer, a second normalization layer and a reshaping output layer connected in sequence; wherein the tokenization layer is used to tokenize the feature map into multiple two-dimensional blocks, and the KAN network layer is used to perform nonlinear transformation.

6. The method according to claim 2, characterized in that The encoder module includes four layers of encoders connected in sequence, each layer of encoders includes two visual state space modules; the decoder module includes four layers of decoders connected in sequence, except for the last layer of decoders, each layer of decoders includes two visual state space modules; the last layer of decoders includes one visual state space module.

7. A cell nucleus segmentation method, characterized in that: The cell nucleus segmentation method comprises: Obtain the image to be segmented; The image to be segmented is input into the cell nucleus segmentation model constructed according to the construction method according to any one of claims 1 to 6 to obtain a cell nucleus segmentation result corresponding to the image to be segmented.

8. A cell nucleus segmentation model construction device, characterized in that: The device comprises: An acquisition module, used to acquire a cell image dataset; A construction module is used to construct a cell nucleus segmentation network structure, wherein the cell nucleus segmentation network structure includes a patch embedding layer, an encoder module, a tokenized KAN network module, a decoder module and a projection layer connected in sequence, wherein the encoder module and the decoder module are also connected via a frequency domain learnable module; wherein the frequency domain learnable module is used to transfer low-level features in the encoder module to the decoder module; and the tokenized KAN network module is used for deep-level feature extraction; A training module is used to train the cell nucleus segmentation network based on the cell image data set to obtain a cell nucleus segmentation model.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the cell nucleus segmentation model construction method according to any one of claims 1 to 6 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the cell nucleus segmentation model construction method according to any one of claims 1 to 6.

Images