Cell segmentation method and system based on multi-scale feature map
By using multi-scale feature maps and generative adversarial networks in cell segmentation methods, the shortcomings of traditional methods in feature fusion and non-adjacent hierarchy feature utilization are solved, and a higher accuracy and application range of cell segmentation is achieved.
Patent Information
- Application Number
- CN202510155760.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-12
- Publication Date
- 2025-05-13
Smart Images
Figure CN119992098A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of cell segmentation, and in particular relates to a cell segmentation method and system based on a multi-scale feature map. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] Cell segmentation is one of the key tasks in biomedical image processing. It has important application value in the fields of cell counting, pathological analysis, and drug research. Traditional cell segmentation methods usually rely on manually annotated training data or complex feature engineering, which limits their application scope and effectiveness. In recent years, deep learning-based cell segmentation methods have made significant progress, but the traditional network directly connects the connection layer and the convolution layer, lacking sufficient connection, resulting in poor fusion of features from different levels at different scales, and cannot fully utilize the features of feature layers at non-adjacent levels. Summary of the invention
[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a cell segmentation method and system based on a multi-scale feature map, which fully utilizes features of different levels at different scales to improve the accuracy of subsequent cell segmentation.
[0005] To achieve the above object, the first aspect of the present invention provides: a cell segmentation method based on a multi-scale feature map, comprising:
[0006] For the acquired cell images to be segmented, clustering algorithms with different fuzzy factors are used to obtain cell images of different scales;
[0007] For cell images of different scales, feature reconstruction is performed based on the pixel position according to odd and even numbers, and the reconstructed features are stacked to obtain vertex features.
[0008] The vertex features corresponding to the cell images of different scales are reconstructed again based on the pixel positions according to the odd and even numbers, and the reconstructed features are convolved to obtain the image features;
[0009] According to the image features, a generative adversarial network is used to obtain a segmentation result of the cell image to be segmented.
[0010] A second aspect of the present invention provides a cell segmentation system based on a multi-scale feature map, comprising:
[0011] An acquisition module is used to obtain cell images of different scales by using clustering algorithms with different fuzzy factors on the acquired cell images to be segmented;
[0012] The first reconstruction module is used to reconstruct features of cell images of different scales based on the odd and even pixel positions, and stack the reconstructed features to obtain vertex features;
[0013] The second reconstruction module is used to reconstruct the vertex features corresponding to the cell images of different scales according to the odd and even pixel positions, and perform convolution operation on the reconstructed features to obtain image features;
[0014] The segmentation module is used to obtain the segmentation result of the cell image to be segmented by using a generative adversarial network according to the image features.
[0015] The third aspect of the present invention provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, a cell segmentation method based on a multi-scale feature map is executed.
[0016] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, executes a cell segmentation method based on a multi-scale feature map.
[0017] A fifth aspect of the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements a cell segmentation method based on a multi-scale feature map.
[0018] One or more of the above technical solutions have the following beneficial effects:
[0019] In the present invention, cell images of different scales are obtained by using clustering algorithms with different fuzzy factors for the cell images to be segmented; for the cell images of different scales, feature reconstruction is performed based on the pixel positions according to odd and even numbers, and the reconstructed features are stacked to obtain vertex features; the vertex features corresponding to the cell images of different scales are reconstructed based on the pixel positions according to odd and even numbers, and the reconstructed features are convolved to obtain image features, thereby realizing the segmentation of the cell images. The method of the present invention makes full use of features of different levels at different scales to improve the accuracy of subsequent cell segmentation.
[0020] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0022] Figure 1 This is a flow chart of a cell segmentation method based on a multi-scale feature map in Example 1 of the present invention;
[0023] Figure 2 This is a schematic diagram of the feature reconstruction generator fusion process in the first embodiment of the present invention;
[0024] Figure 3 This is an overall block diagram of cell segmentation in Example 1 of the present invention. DETAILED DESCRIPTION
[0025] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0026] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.
[0027] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.
[0028] Embodiment 1
[0029] This embodiment discloses a cell segmentation method based on a multi-scale feature map, comprising:
[0030] For the acquired cell images to be segmented, clustering algorithms with different fuzzy factors are used to obtain cell images of different scales;
[0031] For cell images of different scales, feature reconstruction is performed based on the pixel position according to odd and even numbers, and the reconstructed features are stacked to obtain vertex features.
[0032] The vertex features corresponding to the cell images of different scales are reconstructed again based on the pixel positions according to the odd and even numbers, and the reconstructed features are convolved to obtain the image features;
[0033] According to the image features, a generative adversarial network is used to obtain a segmentation result of the cell image to be segmented.
[0034] This embodiment uses image distance measurement and multi-scale analysis to achieve accurate cell segmentation and reduce dependence on labeled data.
[0035] Combine the following Figure 1A cell segmentation method based on a multi-scale feature map proposed in this embodiment is described in detail, specifically including:
[0036] Step 1: Dataset selection:
[0037] We selected image datasets containing a variety of cell types and different scales from publicly available cell image databases, such as biomedical image datasets or cell microscopy image datasets, to ensure that the datasets are diverse and challenging enough to fully evaluate the performance of our methods.
[0038] Step 2: Data Preprocessing:
[0039] The selected images are preprocessed, including denoising and normalization. First, a median filter is used to remove salt and pepper noise and Gaussian noise from the image. Then, the image is normalized to map the pixel values into a suitable range to ensure consistent input conditions.
[0040] Step 3: Scale distance map generation:
[0041] Based on the preprocessed image, a multiscale filter is used to extract image responses of different scales. Due to the complex cell environment in the cell image, the large number of cells, and the presence of many edge areas, the pixel values of the pixels on both sides of the edge will be very different. Bilateral filtering is used for image filtering, and C-means clustering fuzzy is used for multi-scale generation. The multiscale filter mainly consists of a bilateral filter and a combination of multiple C-means clustering fuzzy filters that can be processed in parallel. Among them, the parallel combination of C-means clustering fuzzy is mainly used to generate feature images of different scales for the same cell image to capture the feature information at different scales in the original cell image.
[0042] The following is the process of extracting feature information at a scale from a cell image using C-means clustering fuzzy. First, the original cell image is fuzzified by C-means clustering to obtain cell image A to reduce image noise and reduce the level of detail.
[0043] Different cells may exist simultaneously in the cell image A, such as red blood cells, white blood cells, muscle cells, etc. Different cells are labeled with different classification categories.
[0044] The fuzzy principle of C-means clustering is as follows:
[0045] For a given pixel set H = {H1, H2, H3 ... H k}, C is the classification category, a m (m=1,2,…C) is the center of each cluster, μ m (X n) is the membership function of the mth class corresponding to the nth sample, then the clustering loss function based on the membership function can be written as:
[0046]
[0047] Among them, q is the smoothing factor, which takes a value of 2.
[0048] To find the center of the cluster, we need to find M f The minimum value of is as follows:
[0049]
[0050] Iterate successively, let a in the above formula m and μ m (X n ) is 0, and M is obtained by iteration. f The minimum value of is the cluster center.
[0051] By changing the size of the smoothing factor q, the corresponding cell image A after blurring under different smoothing factors is obtained. i (i=1, 2, 3, ..., k; k is the number of smoothing factors q).
[0052] Since the connection layer and the convolution layer are directly connected in the traditional network, there is a lack of sufficient connection, which makes the fusion effect of features from different levels at different scales poor, and the features of feature layers at non-adjacent levels cannot be fully utilized. Therefore, in order to solve the above problem, this embodiment proposes a multi-peak feature fusion module for fusing features of different cell scale images, which is used to fuse cell images A from different smoothing factors. i image features are fused.
[0053] The structure of the model proposed in this embodiment includes an image parity feature extractor and a feature reconstruction generator model architecture as shown in Figure 3 As shown. The image parity feature extractor is mainly used to extract cell image A i The odd-numbered feature information and even-numbered feature information in the image pave the way for subsequent feature fusion. The feature reconstruction generator is mainly used to reconstruct the cell image A i The odd-numbered feature information and the even-numbered feature information are reconstructed so that the feature information at different scales can be fused together to form a feature map with different scales.
[0054] The principle of the feature reconstruction generator is as follows Figure 2 As shown, Figure 2 The numbers in the figure represent cell images A. iBy performing odd-even fusion on feature information of multiple images at different scales, a feature map with information of different scales is generated.
[0055] Among them, the cell image A under different smoothing factors is subjected to feature fusion through the position converter, and finally a multi-scale distance map is obtained.
[0056] Among them, the principle of image parity feature extractor is as follows:
[0057] (I) Image feature A i Convert to vertex features, the steps are as follows:
[0058] 1. Image column features The step size is 2;
[0059] 2. In image feature A i Select even-numbered columns of features Y e ;
[0060] 3. In image feature A i Select the odd-numbered feature Y o , where Y e and Y o is a pair of adjacent odd-even column features;
[0061] 4. Y e , Y o Refactored to and The reconstruction method is to transform the even-numbered column feature Y e The two adjacent feature information in the averaging process are processed to shorten the feature information length for subsequent stacking. The odd-numbered feature Y o The reconstruction method and even-numbered column feature Y e same;
[0062] 5. Reconstruct the even-numbered column feature Y e and even-numbered columns feature Y o Stacking is done to get vertex features: Among them, the stacking method is to splice the features front and back.
[0063] (II) The principle of the feature reconstruction generator is as follows:
[0064] Different image features A i The corresponding vertex features are converted into image features. The steps are as follows:
[0065] 1. Vertex Features The step size is 2;
[0066] 2. Vertex Features Select even-numbered columns of features
[0067] 3. Vertex Features Select the odd number of features in, and is a pair of adjacent odd-even column features;
[0068] 4. Even-numbered column features Odd column features Refactored to and The reconstruction method is to transform the even-numbered column features The two adjacent feature information in the averaging process are processed to shorten the feature information length for subsequent stacking. Reconstruction method and even-numbered column features same;
[0069] 5. Reconstructed even-numbered column features and odd-numbered column features Perform convolution operation to obtain The convolution kernel size is 3 and the value is 1, thus obtaining the image features. Among them, R represents all the feature information of the cell image, H1 and H2 represent the width of the image features, and W represents the length of the image features.
[0070] Among them, (i) mainly processes the image features obtained after a smoothing factor processing, converting the original image features into vertex features in the form of "columns" for subsequent image reconstruction; then the image features obtained after different smoothing factors are processed by step (i), so as to obtain vertex features after different smoothing factors, and the different vertex features corresponding to the scale are arranged in order from small to large, and then reconstructed and convolved through step (ii), so as to finally obtain the cell image B at multi-scale distances.
[0071] Step 4: Adaptive threshold selection and unsupervised segmentation:
[0072] In this section, we introduce a generative adversarial network (GAN) based on image content adaptation. Through interactive learning between the generator and the discriminator, the optimal threshold is adaptively generated, which further improves the processing ability of complex image content. This method has strong generalization ability and flexibility. This solves the problem that traditional threshold determination methods rely on image histogram analysis or are based on some empirical rules. In order to reduce the problem of difficult labeling in cell images, the generator in the generative adversarial network selects a flow generator, and the discriminator selects a discriminator based on an adversarial autoencoder to perform unsupervised learning on the samples.
[0073] First, the cell image B obtained above is annotated. These data include not only the image itself, but also the corresponding artificial optimal segmentation threshold. Then, the cell feature information is extracted through the flow generator, and combined with the optimal segmentation threshold manually annotated, so that the flow generator can generate a threshold according to the input image features, and then the threshold generated by the generator is input into the discriminator based on the adversarial autoencoder. The role of the discriminator is to evaluate the effectiveness of the threshold generated by the generator, that is, to determine whether this threshold can effectively segment the image. Finally, the threshold is generated according to the above process for image segmentation to obtain the final image segmentation result.
[0074] The principle of its stream generator is to use a series of reversible transformations to gradually change the distribution of input data to generate the required data samples.
[0075]
[0076] Where x is a sample in the tower data structure, z is a transformed sample, and the relationship between x and z is defined by a reversible function f, z = f(x). is the Jacobian matrix The absolute value of the determinant of .
[0077] Its discriminator based on adversarial autoencoder is mainly composed of encoder, decoder and discriminator. The encoder is mainly used to map the input data, the decoder is used to reconstruct the mapped data, and the discriminator is mainly used to judge whether the mapped data meets expectations.
[0078] Embodiment 2
[0079] The purpose of this embodiment is to provide a cell segmentation system based on a multi-scale feature map, including:
[0080] An acquisition module is used to obtain cell images of different scales by using clustering algorithms with different fuzzy factors on the acquired cell images to be segmented;
[0081] The first reconstruction module is used to reconstruct features of cell images of different scales based on the odd and even pixel positions, and stack the reconstructed features to obtain vertex features;
[0082] The second reconstruction module is used to reconstruct the vertex features corresponding to cell images of different scales according to the odd and even pixel positions, and perform convolution operations on the reconstructed features to obtain image features;
[0083] The segmentation module is used to obtain the segmentation result of the cell image to be segmented by using a generative adversarial network according to the image features.
[0084] In further embodiments, there is also provided:
[0085] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in Embodiment 1 is performed. For the sake of brevity, no further description is given here.
[0086] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0087] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.
[0088] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method described in embodiment 1 is completed.
[0089] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.
[0090] A computer program product includes a computer program, and when the computer program is executed by a processor, the method described in the first embodiment is implemented.
[0091] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer executable instructions, such as instructions included in a program module, which are executed in a device on a real or virtual processor of the target to perform the process / method as described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of program modules can be combined or divided between program modules as needed. Machine executable instructions for program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.
[0092] The computer program code for implementing the method of the present invention can be written in one or more programming languages. These computer program codes can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the computer or other programmable data processing device, causes the function / operation specified in the flow chart and / or block diagram to be implemented. The program code can be executed completely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer or completely on a remote computer or server.
[0093] In the context of the present invention, computer program codes or related data may be carried by any appropriate carrier to enable a device, apparatus or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, etc. Examples of signals may include electrical, optical, radio, acoustic or other forms of propagation signals, such as carrier waves, infrared signals, etc.
[0094] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0095] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.
Claims
1. A cell segmentation method based on a multi-scale feature map, characterized in that: The invention is characterized by comprising: For the acquired cell images to be segmented, clustering algorithms with different fuzzy factors are used to obtain cell images of different scales; For cell images of different scales, feature reconstruction is performed based on the pixel position according to odd and even numbers, and the reconstructed features are stacked to obtain vertex features. The vertex features corresponding to the cell images of different scales are reconstructed again based on the pixel positions according to the odd and even numbers, and the reconstructed features are convolved to obtain the image features; According to the image features, a generative adversarial network is used to obtain a segmentation result of the cell image to be segmented.
2. A cell segmentation method based on a multi-scale feature map as claimed in claim 1, characterized in that: The cell images to be segmented were filtered using a bilateral filter, and cell images of different scales were extracted from the filtered cell images based on the C-means clustering fuzzy algorithm with different fuzzy factors.
3. A cell segmentation method based on a multi-scale feature map as claimed in claim 1, characterized in that: For cell images of different scales, feature reconstruction is performed based on the pixel position according to odd and even numbers, and the reconstructed features are stacked to obtain vertex features, specifically: Determine image even column features and image odd column features based on pixel positions; Based on the mean processing method of adjacent features, the image even-numbered column features and the image odd-numbered column features are reconstructed respectively, and the reconstructed features are spliced to obtain the vertex features.
4. The cell segmentation method based on a multi-scale feature map according to claim 1, characterized in that: The vertex features corresponding to cell images of different scales are reconstructed again based on the pixel positions according to the odd and even numbers, and the reconstructed features are convolved to obtain the image features, which are as follows: Determine the image even column features and the image odd column features of all vertex features based on the pixel positions; Based on the mean processing method of adjacent features, the image even-numbered column features and the image odd-numbered column features of all vertex features are reconstructed respectively, and the reconstructed features are spliced to obtain the image features.
5. A cell segmentation method based on a multi-scale feature map as claimed in claim 1 or 2, characterized in that: The C-means clustering fuzzy algorithm based on different fuzzy factors extracts cell images of different scales from the filtered cell images to be segmented, specifically: Based on a given set of pixels and cell classification categories; Calculate the minimum value of the clustering loss function based on the membership function and obtain the cluster center; Change the size of the smoothing factor and repeat the above operation to obtain cell images of different scales corresponding to the blurring under different smoothing factors.
6. A cell segmentation system based on multi-scale feature maps, characterized in that: include: An acquisition module is used to obtain cell images of different scales by using clustering algorithms with different fuzzy factors on the acquired cell images to be segmented; The first reconstruction module is used to reconstruct features of cell images of different scales based on the odd and even pixel positions, and stack the reconstructed features to obtain vertex features; The second reconstruction module is used to reconstruct the vertex features corresponding to cell images of different scales according to the odd and even pixel positions, and perform convolution operations on the reconstructed features to obtain image features; The segmentation module is used to obtain the segmentation result of the cell image to be segmented by using a generative adversarial network according to the image features.
7. A cell segmentation system based on a multi-scale feature map as claimed in claim 6, characterized in that: In the first reconstruction module, for cell images of different scales, feature reconstruction is performed based on the pixel positions according to odd and even numbers, and the reconstructed features are stacked to obtain vertex features, specifically: Determine image even column features and image odd column features based on pixel positions; Based on the mean processing method of adjacent features, the image even-numbered column features and the image odd-numbered column features are reconstructed respectively, and the reconstructed features are spliced to obtain the vertex features.
8. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 5 is completed.
9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method described in any one of claims 1 to 5.
10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.