Tumor cell image extraction method and system based on deep learning

Abstract

The invention discloses a tumor cell image extraction method and system based on deep learning, and the method comprises the steps: obtaining tumor cell image data in a microscope, and carrying out the data preprocessing of the tumor cell image data; eliminating non-uniform illumination of slices in the initial tumor cell image data by using a Retinex theory in combination with guided filtering, and enhancing a cell nucleus contrast ratio; capturing the morphological characteristics of the multi-scale cells based on a DCPM cavity convolution pyramid module, and strengthening the topological relation between the cells by using a lightweight module of an SE compression-excitation mechanism; and through gating attention mechanism dynamic weight fusion double branch output, calculating the cell mass center distance constraint of the characteristic tumor cell image based on an adhesion cell separation algorithm of connected domain analysis, and iteratively cutting an over-segmentation region, so that tumor adhesion cells can be accurately separated. The problems of over-segmentation and under-segmentation are avoided in combination with a minimum cut algorithm and area and shape constraint conditions.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method and system for extracting tumor cell images based on deep learning. Background Art

[0002] Currently, tumor cell image extraction faces numerous technical challenges. First, during microscopy imaging, due to differences in sample preparation and uneven illumination, the resulting cell images often suffer from uneven illumination across the slices and low contrast between the cell nucleus and the background. This leads to blurred cell edges and loss of detail, significantly impacting the accuracy of subsequent analysis. Second, traditional image extraction methods, based on algorithms such as threshold segmentation and region growing, often struggle to accurately distinguish tumor cells from normal cells, or cells from the background, when dealing with complex backgrounds and adherent cells. This results in poor segmentation and is prone to over- or under-segmentation. Summary of the Invention

[0003] The purpose of the present invention is to solve the above problems and design a tumor cell image extraction method based on deep learning.

[0004] To achieve the above-mentioned purpose, the technical solution of the present invention is as follows: further, in the above-mentioned tumor cell image extraction method based on deep learning, the tumor cell image extraction method includes the following steps: Acquiring tumor cell image data from a microscope, and performing data preprocessing on the tumor cell image data to obtain initial tumor cell image data; Using Retinex theory combined with guided filtering to eliminate uneven illumination of slices in the initial tumor cell image data, enhance cell nucleus contrast, and obtain enhanced tumor cell image data; Based on the DCPM dilated convolutional pyramid module to capture multi-scale cell morphological features and the SE compression-excitation mechanism to strengthen the topological relationship between cells, a DCPM-SE dual-branch feature fusion network model was established; Inputting the enhanced tumor cell image data into the DCPM-SE dual-branch feature fusion network model for recognition, dynamically weighting and fusing the dual-branch outputs through a gated attention mechanism to suppress background noise and obtain a characteristic tumor cell image; The adhesion cell separation algorithm based on connected domain analysis calculates the cell centroid distance constraint of the characteristic tumor cell image, and iteratively cuts the over-segmented area to obtain the target tumor cell image.

[0005] Furthermore, in the above-mentioned tumor cell image extraction method based on deep learning, the step of acquiring tumor cell image data from a microscope and performing data preprocessing on the tumor cell image data to obtain initial tumor cell image data includes: acquire tumor cell image data by using an image sensor to take three-dimensional stereoscopic multi-angle photos of the tumor cell sample, including 360° rotation photos in the horizontal direction and layer cutting photos every 1 μm in the vertical direction; convert the tumor cell image data into a unified standard image format, crop out irrelevant blank areas in the image, retain the effective area containing cells, and obtain a cropped tumor cell image; scale the cropped tumor cell image to a fixed size by using a bilinear interpolation algorithm, and obtain a scaled tumor cell image; label the scaled tumor cell image, including at least tumor cell boundaries, cell types, and cell states, divide the labeled image data, and obtain initial tumor cell image data.

[0006] Further, in the above-mentioned tumor cell image extraction method based on deep learning, the step of eliminating uneven illumination in the initial tumor cell image data by using the Retinex theory combined with guided filtering, enhancing the contrast of cell nuclei, and obtaining enhanced tumor cell image data includes: eliminate the initial tumor cell image data by using the Retinex theory combined with guided filtering, construct Gaussian filter kernels of different scales, and set the scale parameters to 15, 80, and 250 respectively, which are used to simulate illumination changes in different ranges; perform convolution operations on the input image and the three different scale Gaussian filter kernels respectively to obtain illumination estimation images at three scales; normalize the illumination estimation images at each scale so that the gray scale range is between [0, 1]; calculate the reflection image, perform dynamic range compression on the reflection image, map the gray scale value to the [0, 255] interval, and obtain a preliminary processed image; use guided filtering to smooth the preliminary processed image, and obtain enhanced tumor cell image data.

[0007] Further, in the above-mentioned tumor cell image extraction method based on deep learning, the DCPM hollow convolution pyramid module captures multi-scale cell morphological features, and the lightweight Transformer module using the SE compression-excitation mechanism strengthens the topological relationship between cells, and a DCPM-SE double-branch feature fusion network model is established, which includes: construct the DCPM hollow convolution pyramid module, design four hollow convolution layers with different hollow rates, and the structure of each hollow convolution layer is that the input channel number is dynamically adjusted according to the output of the previous layer, the output channel number is set to 64, and the convolution kernel size is 3×3; The extracted different scale feature maps are fused, and four feature maps are connected in the channel dimension by using channel splicing to obtain a feature map of multi-scale information.

[0008] Further, in the above-mentioned deep learning-based tumor cell image extraction method, the DCPM hollow convolution pyramid module captures multi-scale cell morphological features, and the lightweight Transformer module using the SE compression-excitation mechanism strengthens the topological relationship between cells, and a DCPM-SE double-branch feature fusion network model is established, further comprising: 8 attention heads are set, and the dimension of each head is 64. The input feature map is mapped to the input dimension of the multi-head attention mechanism through linear transformation, and the topological relationship between cells is learned through multi-head attention calculation to obtain a feature map; Global average pooling is performed on the feature map to compress the features of each channel into a scalar value. A channel-level weight vector is generated by learning the dependency between channels through two fully connected layers.

[0009] Further, in the above-mentioned deep learning-based tumor cell image extraction method, the enhanced tumor cell image data is input into the DCPM-SE double-branch feature fusion network model for recognition, and the double-branch output is dynamically weighted and fused through the gated attention mechanism to suppress background noise and obtain a feature tumor cell image, further comprising: The enhanced tumor cell image data is input into the trained DCPM-SE double-branch feature fusion network model to extract multi-scale morphological features of tumor cells, and a lightweight Transformer module is combined with an SE compression-excitation mechanism to extract intercellular topological relationship features; The feature outputs of the two branches are dynamically weighted and fused through the gated attention mechanism to obtain a fused feature representation; The gated attention mechanism calculates the weights of the two branch features according to the content of the input image, maps the fused features to the corresponding class labels, and obtains a feature tumor cell image.

[0010] Further, in the above-mentioned deep learning-based tumor cell image extraction method, the connected component analysis-based adherent cell separation algorithm calculates the cytocenter distance constraint of the feature tumor cell image and iteratively cuts the over-segmented region to obtain a target tumor cell image, comprising: The feature tumor cell image is binarized, and the Otsu threshold segmentation algorithm is used to automatically calculate the optimal threshold to convert the image into a black and white binary image; Connected component analysis is performed using 8 connectivity definition, and a connected component labeling algorithm based on scan line is used to traverse each pixel point in the black and white binary image. The centroid coordinates of each connected component are calculated to obtain a target tumor cell image.

[0011] Further, in a tumor cell image extraction system based on deep learning, the tumor cell image extraction system comprises the following modules: a tumor image acquisition module, configured to acquire tumor cell image data in a microscope, and perform data preprocessing on the tumor cell image data to obtain initial tumor cell image data; an image data enhancement module, configured to eliminate uneven illumination of a slice in the initial tumor cell image data by using Retinex theory combined with guided filtering, and enhance the contrast of a cell nucleus to obtain enhanced tumor cell image data; a fusion model establishment module, configured to capture multi-scale cell morphological features based on a DCPM hollow convolution pyramid module, and strengthen the topological relationship between cells by using a lightweight Transformer module with an SE compression-excitation mechanism, to establish a DCPM-SE double-branch feature fusion network model; a cell feature extraction module, configured to input the enhanced tumor cell image data into the DCPM-SE double-branch feature fusion network model for identification, dynamically weight and fuse the double-branch outputs by a gate attention mechanism, and suppress background noise to obtain a feature tumor cell image; a cell image segmentation module, configured to calculate the cytocenter distance constraint of the feature tumor cell image based on a connected component analysis-based adherent cell separation algorithm, and iteratively cut the over-segmented region to obtain a target tumor cell image.

[0012] Further, in the tumor cell image extraction system based on deep learning, the fusion model establishment module comprises the following sub-modules: a construction sub-module, configured to set 8 attention heads, each head having a dimension of 64, and input feature maps being mapped to the input dimension of the multi-head attention mechanism through linear transformation, and the topological relationship between cells being learned through multi-head attention calculation to obtain a feature map; a generation sub-module, configured to perform global average pooling on the feature map, compress the features of each channel into a scalar value, and generate a channel-level weight vector by learning the dependency relationship between channels through two fully connected layers.

[0013] Further, in the tumor cell image extraction system based on deep learning, the cell image segmentation module comprises the following sub-modules: a processing sub-module, configured to perform binaryzation processing on the feature tumor cell image, and automatically calculate an optimal threshold value by using an Otsu threshold segmentation algorithm to convert the image into a black-and-white binary image; a calculation sub-module, configured to perform connected component analysis by using 8-connectivity definition, traverse each pixel point in the black-and-white binary image based on a scan line-based connected component labeling algorithm, and calculate the centroid coordinates of each connected component to obtain a target tumor cell image.

[0014] The beneficial effects are that 1. the cell nucleus edge details are maximally reserved, the contrast of the cell nucleus and the background and other cell structures is significantly enhanced, the intercellular topological relationship is strengthened, the response to important features is enhanced through channel level weight adjustment, irrelevant information is inhibited, and the model is more accurate and comprehensive in extracting tumor cell features. 2. In a complex background, the cell morphological features are highlighted; in a cell dense area, the intercellular topological relationship is strengthened, the background noise is effectively inhibited, and the adaptability and recognition accuracy of the model to tumor cell images in different scenes are improved. 3. It can accurately separate adherent cells. Combined with the minimum cut algorithm and area and shape constraint conditions, the over-segmentation and under-segmentation problems are avoided, and it is ensured that each tumor cell can be accurately segmented, providing reliable image data for subsequent medical diagnosis and research. BRIEF DESCRIPTION OF DRAWINGS

[0015] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included only to illustrate preferred embodiments and are not intended to limit the present application.

[0016] Figure 1 The first embodiment schematic diagram of a tumor cell image extraction method based on deep learning in an embodiment of the present application; Figure 2 The second embodiment schematic diagram of a tumor cell image extraction method based on deep learning in an embodiment of the present application; Figure 3 The first embodiment schematic diagram of a tumor cell image extraction system based on deep learning in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical scheme and advantages of the present application clearer, the following further describes the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0018] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an" and "the" used herein also include the plural forms. It should be further understood that the use of the word "comprise" in the specification of the present application means that the features, integers, steps, operations, elements and / or components exist, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0019] The present application will be described in detail below in combination with the drawings, such as Figure 1As shown, a deep learning-based tumor cell image extraction method, the tumor cell image extraction method comprises the following steps: Step 101, obtaining tumor cell image data in a microscope, pre-processing the tumor cell image data to obtain initial tumor cell image data; Specifically, in this embodiment, the image sensor is used to take three-dimensional stereoscopic multi-angle pictures of the tumor cell sample, including 360° rotation shooting in the horizontal direction and layer cutting shooting every 1 μm in the vertical direction, to obtain tumor cell image data; The tumor cell image data is converted into a unified standard image format, the irrelevant blank area in the image is cropped, and the effective area containing the cells is retained to obtain a cropped tumor cell image; The cropped tumor cell image is scaled to a fixed size using a bilinear interpolation algorithm to obtain a scaled tumor cell image; The scaled tumor cell image is labeled, at least including tumor cell boundary, cell type and cell state, and the labeled image data is divided to obtain initial tumor cell image data.

[0020] Specifically; 1.1 Cell image data acquisition: A professional biological microscope (Nikon Eclipse Ni-E, Olympus BX53) equipped with a high-resolution CCD or CMOS image sensor is used, and unified and appropriate shooting parameters are set. The specific parameters are: the optical magnification is fixed at 40 times or 100 times oil lens, the exposure time is adjusted according to the sample fluorescence intensity and sensor sensitivity, and the range is 50-500 ms; the focal length adopts the mode of automatic focusing combined with manual fine adjustment to ensure clear cell image. The tumor cell sample is taken three-dimensional stereoscopic multi-angle pictures, including 360° rotation shooting in the horizontal direction and layer cutting shooting every 1 μm in the vertical direction, to obtain multi-view, multi-layer cell image data.

[0021] During the shooting process, the sample is placed on the object table in a constant temperature (37℃) and constant humidity (5% CO2) culture environment, and a special culture solution for live cell imaging is used to maintain the physiological activity of the sample to avoid distortion of the cell morphology in the image due to changes in the sample state. At the same time, the image acquisition software matched with the microscope is used to record the metadata of each image in real time, including sample number, shooting time, shooting position coordinates, fluorescence channel information, etc., and store it as a structured data file for subsequent data management and retrieval.

[0022] 1.2 Data pre-processing: 1. Image format conversion: Use Python's OpenCV library or MATLAB image processing toolbox to convert the original image data (microscope-specific.czi,.nd2 format) into standard JPEG or PNG format. During the conversion process, set the image quality parameters, and set the JPEG format quality factor to 90 to ensure minimal loss of image information while controlling file size for subsequent processing.

[0023] 2. Image cropping and scaling: First, automatically identify the effective area containing cells in the image through edge detection algorithm (Canny edge detection) and crop out the blank area around the image. Then, scale the cropped image to a fixed size, such as 224x224 pixels. When scaling, choose the interpolation algorithm according to the image resolution and content complexity. For low-resolution images, use bicubic interpolation to improve image clarity; for high-resolution images, use bilinear interpolation to reduce the sawtooth effect while ensuring processing efficiency.

[0024] 3. Data annotation: Assemble an annotation team composed of medical image physicians and professional annotators, and use professional image annotation tools such as LabellImg and CVAT to manually annotate tumor cells in cell images. The annotation content includes the accurate boundary of tumor cells (using polygon annotation method), cell type (lung cancer cells, colon cancer cells, etc.), and cell state (proliferation period, apoptosis period, etc.). Establish a strict annotation quality control process, first have two annotators independently annotate the same batch of images, calculate the overlap rate (IoU) of the annotation results, and when IoU≥0.85, consider the annotation as qualified. For images with differences, refer to medical experts for arbitration and review to ensure annotation accuracy.

[0025] 4. Data division: Use stratified sampling method to divide the annotated image data into 70% training set, 15% validation set, and 15% test set. To ensure the balanced proportion of different types of tumor cells in each set, first stratify the image data by cell type, then randomly select images in each layer according to the proportion. At the same time, to enhance the generalization ability of the model, randomly shuffle the training set data and save the divided data set index file for subsequent training and evaluation.

[0026] Step 102, use Retinex theory combined with guided filtering to eliminate uneven illumination in the initial tumor cell image data, enhance the contrast of cell nuclei, and obtain enhanced tumor cell image data; Specifically, in this embodiment, the initial tumor cell image data is processed using Retinex theory combined with guided filtering, and Gaussian filter kernels of different scales are constructed, with scale parameters set to 15, 80, and 250, respectively, to simulate different ranges of illumination changes. Convolution operation is performed on the input image with three different scale Gaussian filter kernels to obtain the illumination estimation images under three scales; The illumination estimation images under each scale are normalized to have a gray scale range between [0, 1]; The reflection image is calculated, and dynamic range compression is performed on the reflection image to map the gray value to the interval [0, 255] to obtain the preliminary processed image; The preliminary processed image is used as the input image, and the initial tumor cell image data is used as the guide image. Guided filtering is used for smoothing to obtain the enhanced tumor cell image data.

[0027] Specifically; 2.1 Application of Retinex theory: The core of the Retinex theory is to decompose the image I(x, y) into a reflection image R(x, y) and an illumination image L(x, y). In practical applications, the multi-scale Retinex algorithm (MSR) is used to process the initial tumor cell image. The specific steps are as follows: Gaussian filter kernels of different scales are constructed, with scale parameters set to 15, 80, and 250, respectively, to simulate different ranges of illumination changes.

[0028] Convolution operation is performed on the input image with three different scale Gaussian filter kernels to obtain the illumination estimation images L_1(x, y), L_2(x, y), and L_3(x, y) under three scales.

[0029] The illumination estimation images under each scale are normalized to have a gray scale range between [0, 1];

[0030] The reflection image is calculated, and dynamic range compression is performed on the reflection image to map the gray value to the interval [0, 255] to obtain the preliminary processed image;

[0031] 2.2 Guided filtering combination: The reflection image R(x, y) processed by Retinex is used as the input image, and the original initial tumor cell image I(x, y) is used as the guide image. Guided filtering is used for smoothing.

[0032] In practical applications, the filtering effect is controlled by adjusting the radius parameter r and the regularization parameter epsilon of the guided filtering. According to the characteristics of the tumor cell image, the radius parameter r is set to 5-15 pixels to control the size of the filtering window, and the regularization parameter epsilon is set to 0.01^2-0.1^2 to balance the smoothing effect and edge preservation ability. After multiple experiments and comparisons, when r = 10 and epsilon = 0.05^2, the uneven illumination of the slice can be effectively eliminated while the cell nucleus edge details are preserved to the greatest extent, the contrast between the cell nucleus and the background and other cell structures is significantly enhanced, and thus the enhanced tumor cell image data is obtained.

[0033] Step 103, based on the DCPM hollow convolution pyramid module, capture multi-scale cell morphology features and use the SE compression-excitation mechanism to strengthen the topological relationship between cells, and establish a DCPM-SE double-branch feature fusion network model; Specifically, in the present embodiment, a DCPM hollow convolution pyramid module is constructed, four hollow convolution layers with different hollow rates are designed, the structure of each hollow convolution layer is that the input channel number is dynamically adjusted according to the output of the previous layer, the output channel number is set to 64, and the convolution kernel size is 3×3. The extracted feature maps of different scales are fused, and the four feature maps are connected in the channel dimension by using channel splicing to obtain a feature map with multi-scale information.

[0034] Eight attention heads are set, each head has a dimension of 64, the input feature map is mapped to the input dimension of the multi-head attention mechanism through linear transformation, the topological relationship between cells is learned through multi-head attention calculation to obtain a feature map; The feature map is globally averaged and pooled to compress the features of each channel into a scalar value; the dependency between channels is learned through two fully connected layers to generate a channel-level weight vector.

[0035] Specifically; 3.1 DCPM hollow convolution pyramid module: In constructing the DCPM hollow convolution pyramid module, four hollow convolution layers with different hollow rates are designed, and the hollow rates are 1, 2, 4 and 8. The structure of each hollow convolution layer is that the input channel number is dynamically adjusted according to the output of the previous layer, the output channel number is uniformly set to 64, the convolution kernel size is 3×3, and the padding mode is "same" to keep the feature map size unchanged.

[0036] For each dilated convolution layer, Batch Normalization and ReLU activation function are adopted to accelerate network convergence and alleviate the problem of gradient vanishing. The different scale feature maps extracted by the four dilated convolution layers are fused. The channel concatenation method is used to connect the four feature maps in the channel dimension, obtaining a feature map containing rich multi-scale information with 64 x 4 = 256 channels.

[0037] 3.2 Lightweight Transformer module combined with SE compression-activation mechanism: The lightweight Transformer module is constructed based on the multi-head attention mechanism, with 8 attention heads and a dimension of 64 for each head. The input feature map is first mapped to the input dimension of the multi-head attention mechanism through linear transformation, and then the topological relationship between cells is learned through multi-head attention calculation. To reduce the computational load, a 1 x 1 convolution layer is added after the multi-head attention mechanism to compress the feature map channel number to 128.

[0038] The SE compression-activation mechanism is introduced to adjust the channel-level weight of the feature map output by the lightweight Transformer module. The specific process is as follows: first, the feature map is globally averaged to compress the features of each channel into a scalar value; then, through two fully connected layers, the first layer has 128 / 16 = 8 neurons, and the second layer has 128 neurons, learning the dependency between channels to generate a channel-level weight vector; finally, the weight vector is multiplied with the original feature map to enhance the response to important channel features, suppress irrelevant channel information, and strengthen the topological relationship between cells.

[0039] 3.3 DCPM-SE dual-branch feature fusion network model construction: The DCPM dilated convolution pyramid module and the lightweight Transformer module are used as the two branches of the network, and the enhanced tumor cell image data is input into them. Before the model output layer, a gated attention mechanism is used to dynamically weight and fuse the outputs of the two branches. The structure of the gated attention mechanism includes two fully connected layers and a Sigmoid activation function, with the input being the concatenation of the output feature maps of the two branches and the output being a fusion weight vector.

[0040] Through training, the gated attention mechanism learns to adaptively adjust the weights of the outputs of the two branches according to the input features, allowing the network to flexibly select and fuse feature information according to the image content. Finally, a global average pooling layer and a fully connected layer are added, with the number of neurons in the fully connected layer being set according to the number of classes in the tumor cell classification task, for example, for a 10-class tumor cell classification task, the number of neurons is set to 10, and the output feature is the classification result or feature description of the tumor cell image.

[0041] Step 104, input the enhanced tumor cell image data into the DCPM-SE double-branch feature fusion network model for recognition, dynamically weight the fusion of double-branch outputs through the gating attention mechanism, suppress background noise, and obtain a feature tumor cell image; Specifically, in this embodiment, the enhanced tumor cell image data is input into the trained DCPM-SE double-branch feature fusion network model, the multi-scale morphological features of tumor cells are extracted, and the lightweight Transformer module is combined with the SE compression-excitation mechanism to extract the topological relationship features between cells. The feature outputs of the two branches are dynamically weighted and fused through the gating attention mechanism to obtain the fused feature representation. The gating attention mechanism calculates the weights of the two branch features according to the content of the input image, maps the fused features to the corresponding class labels, and obtains a feature tumor cell image.

[0042] Specifically; In the model training phase, the cross-entropy loss function is used as the objective function, the Adam optimizer is selected as the optimizer, the initial learning rate is set to 0.001, and the cosine annealing learning rate decay strategy is used to dynamically adjust the learning rate during training. The training batch size is set to 32, and the training number of rounds is determined according to the performance of the validation set. When the validation set loss does not decrease for 5 consecutive rounds, the training is stopped.

[0043] In the inference phase, the enhanced tumor cell image data is input into the trained DCPM-SE double-branch feature fusion network model. The DCPM hollow convolution pyramid module first extracts the multi-scale morphological features of tumor cells, and the lightweight Transformer module combined with the SE compression-excitation mechanism extracts the topological relationship features between cells. The feature outputs of the two branches are dynamically weighted and fused through the gating attention mechanism to obtain the fused feature representation.

[0044] The gating attention mechanism automatically calculates the weights of the two branch features according to the content of the input image. For images containing complex background, the weight of the DCPM module feature is increased to highlight the cell morphological features. For images with dense cell regions, the weight of the lightweight Transformer module feature is increased to strengthen the topological relationship between cells. Finally, through the output layer of the network, the fused features are mapped to the corresponding class labels or feature descriptions to obtain a feature tumor cell image.

[0045] Step 105, based on the connected domain analysis-based adhesion cell separation algorithm, calculate the cell center distance constraint of the feature tumor cell image, and iteratively cut the over-segmented region to obtain a target tumor cell image.

[0046] Specifically, in this embodiment, the feature tumor cell image is binarized, the Otsu threshold segmentation algorithm is used to automatically calculate the optimal threshold, and the image is converted into a black and white binary image. The 8-connectivity definition is used for connected component analysis, and the connected component labeling algorithm based on scan line is used to traverse each pixel point in the black and white binary image. The centroid coordinates of each connected component are calculated to obtain the target tumor cell image.

[0047] Specifically; 5.1 Connected component analysis: The feature tumor cell image is binarized, the Otsu threshold segmentation algorithm is used to automatically calculate the optimal threshold, and the image is converted into a black and white binary image, highlighting the tumor cell region. Then, the 8-connectivity definition is used for connected component analysis, and the connected component labeling algorithm based on scan line (two-pass scan algorithm) is used to traverse each pixel point in the image.

[0048] In the first pass, each pixel point is assigned a temporary label, and equivalent label pairs are recorded. In the second pass, the connected components are merged according to the equivalent label pairs, and each connected component is assigned a unique final label, and the tumor cells are segmented into different connected regions. At the same time, the area, perimeter and other geometric features of each connected component are calculated, which are used for subsequent adhesion cell judgment.

[0049] 5.2 Cell centroid distance constraint calculation: For each connected component, the centroid coordinates (x_c, y_c) are calculated.

[0050] The beneficial effects are: 1. Maximum preservation of cell nucleus edge details, significant enhancement of cell nucleus and background and other cell structure contrast, strengthening of intercellular topological relationship, enhancement of response to important features through channel level weight adjustment, suppression of irrelevant information, making the model more accurate and comprehensive in extracting tumor cell features. 2. In complex background, highlight the morphological features of cells; in cell dense area, strengthen the intercellular topological relationship, effectively suppress the background noise, improve the adaptability and recognition accuracy of the model to tumor cell images in different scenes. 3. Can accurately separate tumor adhesion cells. Combined with the minimum cut algorithm and area and shape constraint conditions, the over-segmentation and under-segmentation problems are avoided, ensuring that each tumor cell can be accurately segmented, providing reliable tumor cell image data for subsequent medical diagnosis and research.

[0051] Please refer to Figure 2 In a tumor cell image extraction method based on deep learning, tumor cell image data in a microscope is obtained, and the tumor cell image data is preprocessed to obtain initial tumor cell image data, including the following steps: Step 201, using an image sensor to take three-dimensional stereoscopic multi-angle photos of the tumor cell sample, including 360° rotation photos in the horizontal direction and layer cutting photos every 1 μm in the vertical direction, to obtain tumor cell image data; Step 202, converting the tumor cell image data into a unified standard image format, cropping out irrelevant blank areas in the image, retaining the effective area containing cells, and obtaining a cropped tumor cell image; Step 203, using a bilinear interpolation algorithm to scale the cropped tumor cell image to a fixed size, and obtaining a scaled tumor cell image; Step 204, labeling the scaled tumor cell image, including at least tumor cell boundaries, cell types, and cell states, and dividing the labeled image data into data, to obtain initial tumor cell image data.

[0052] Please refer to Figure 3 In a tumor cell image extraction system based on deep learning, the tumor cell image extraction system includes the following modules: A tumor image acquisition module is configured to acquire tumor cell image data in a microscope, and to obtain initial tumor cell image data by preprocessing the tumor cell image data. An image data enhancement module is configured to eliminate uneven illumination in the initial tumor cell image data using Retinex theory combined with guided filtering, and to enhance the contrast of cell nuclei, to obtain enhanced tumor cell image data. A fusion model establishment module is configured to capture multi-scale cell morphological features based on a DCPM hollow convolution pyramid module, and to strengthen the topological relationship between cells using a lightweight Transformer module with SE compression-excitation mechanism, to establish a DCPM-SE double-branch feature fusion network model. A cell feature extraction module is configured to input the enhanced tumor cell image data into the DCPM-SE double-branch feature fusion network model for recognition, to dynamically weight and fuse the double-branch outputs through a gated attention mechanism, and to suppress background noise, to obtain a feature tumor cell image. A cell image segmentation module is configured to calculate the cytoplasmic center distance constraint of the feature tumor cell image based on a connected domain analysis-based adhesion cell separation algorithm, and to iteratively cut the over-segmented regions, to obtain a target tumor cell image.

[0053] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Various changes and improvements can be made to the present application without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A tumor cell image extraction method based on deep learning, characterized in that: The tumor cell image extraction method comprises the following steps: Acquiring tumor cell image data from a microscope, and performing data preprocessing on the tumor cell image data to obtain initial tumor cell image data; Using Retinex theory combined with guided filtering to eliminate uneven illumination of slices in the initial tumor cell image data, enhance the contrast of cell nuclei, and obtain enhanced tumor cell image data; Based on the DCPM dilated convolutional pyramid module to capture multi-scale cell morphological features and the SE compression-excitation mechanism to strengthen the topological relationship between cells, a DCPM-SE dual-branch feature fusion network model was established; Inputting the enhanced tumor cell image data into the DCPM-SE dual-branch feature fusion network model for recognition, dynamically weighting and fusing the dual-branch outputs through a gated attention mechanism to suppress background noise and obtain a characteristic tumor cell image; The adhesion cell separation algorithm based on connected domain analysis calculates the cell centroid distance constraint of the characteristic tumor cell image, and iteratively cuts the over-segmented area to obtain the target tumor cell image.

2. The method for extracting tumor cell images based on deep learning according to claim 1, wherein: The step of acquiring tumor cell image data from a microscope and performing data preprocessing on the tumor cell image data to obtain initial tumor cell image data includes: The image sensor is used to perform three-dimensional, multi-angle photography of tumor cell samples, including 360-degree rotational photography in the horizontal direction and vertical slice photography every 1 μm, to obtain tumor cell image data; converting the tumor cell image data into a unified standard image format, cropping irrelevant blank areas in the image, and retaining valid areas containing cells to obtain a cropped tumor cell image; scaling the cropped tumor cell image to a fixed size using a bilinear interpolation algorithm to obtain a scaled tumor cell image; The zoomed tumor cell image is annotated, including at least tumor cell boundaries, cell types, and cell states, and the annotated image data is divided to obtain initial tumor cell image data.

3. The method for extracting tumor cell images based on deep learning according to claim 1, wherein: The method of using Retinex theory combined with guided filtering to eliminate uneven illumination of slices in the initial tumor cell image data and enhance the contrast of cell nuclei to obtain enhanced tumor cell image data includes: The initial tumor cell image data was processed by using Retinex theory combined with guided filtering to construct Gaussian filter kernels of different scales, with scale parameters set to 15, 80, and 250, respectively, to simulate illumination changes in different ranges; Perform convolution operations on the input image with Gaussian filter kernels of three different scales to obtain illumination estimation images at three scales; Normalize the illumination estimation image at each scale so that the grayscale range is between [0, 1]; Calculate the reflection image, perform dynamic range compression on the reflection image, map the grayscale value to the [0,255] interval, and obtain a preliminary processed image; The preliminary processed image is used as the input image, the initial tumor cell image data is used as the guide image, and the guided filter is used for smoothing to obtain the enhanced tumor cell image data.

4. The method for extracting tumor cell images based on deep learning according to claim 1, wherein: The DCPM-SE dual-branch feature fusion network model is established by capturing multi-scale cell morphological features based on the DCPM dilated convolutional pyramid module and strengthening the topological relationship between cells using the SE compression-excitation mechanism. Construct a DCPM dilated convolutional pyramid module and design four dilated convolutional layers with different dilation rates. The structure of each dilated convolutional layer is that the number of input channels is dynamically adjusted according to the output of the previous layer, the number of output channels is set to 64, and the convolution kernel size is 3×3. The extracted feature maps of different scales are fused, and the four feature maps are connected in the channel dimension using channel splicing to obtain a feature map of multi-scale information.

5. The method for extracting tumor cell images based on deep learning according to claim 1, wherein: The DCPM-SE dual-branch feature fusion network model is established by capturing multi-scale cell morphological features based on the DCPM dilated convolutional pyramid module and strengthening the topological relationship between cells using the SE compression-excitation mechanism. Set 8 attention heads, each with a dimension of 64. The input feature map is mapped to the input dimension of the multi-head attention mechanism through linear transformation. Through multi-head attention calculation, the topological relationship between cells is learned to obtain the feature map; Global average pooling is performed on the feature map to compress the features of each channel into a scalar value; the dependency between channels is learned through two fully connected layers to generate a channel-level weight vector.

6. The method for extracting tumor cell images based on deep learning according to claim 1, wherein: The enhanced tumor cell image data is input into the DCPM-SE dual-branch feature fusion network model for identification, and the dual-branch outputs are dynamically weighted and fused through a gated attention mechanism to suppress background noise to obtain a characteristic tumor cell image, further comprising: The enhanced tumor cell image data is input into the trained DCPM-SE dual-branch feature fusion network model to extract the multi-scale morphological features of tumor cells. The lightweight Transformer module is combined with the SE compression-excitation mechanism to extract the topological relationship features between cells. The feature outputs of the two branches are dynamically weighted fused through the gated attention mechanism to obtain the fused feature representation; The gated attention mechanism calculates the weights of the two branch features according to the content of the input image, maps the fused features to the corresponding category labels, and obtains the characteristic tumor cell image.

7. The method for extracting tumor cell images based on deep learning according to claim 1, wherein: The adhesion cell separation algorithm based on connected domain analysis calculates the cell centroid distance constraint of the characteristic tumor cell image and iteratively cuts the over-segmented region to obtain the target tumor cell image, including: The characteristic tumor cell image is binarized, and the optimal threshold is automatically calculated using the Otsu threshold segmentation algorithm to convert the image into a black and white binary image; Connected domain analysis was performed using the 8-connectivity definition, and a scan-line-based connected domain labeling algorithm traversed each pixel in the black-and-white binary image. The centroid coordinates of each connected domain were calculated to obtain the target tumor cell image.

8. A tumor cell image extraction system based on deep learning, characterized in that: The tumor cell image extraction system includes the following modules: a tumor image acquisition module, configured to acquire tumor cell image data from a microscope, and perform data preprocessing on the tumor cell image data to obtain initial tumor cell image data; An image data enhancement module is used to eliminate uneven illumination of slices in the initial tumor cell image data by using Retinex theory combined with guided filtering, enhance the contrast of cell nuclei, and obtain enhanced tumor cell image data; A fusion model building module is used to capture multi-scale cell morphological features based on the DCPM dilated convolutional pyramid module and strengthen the topological relationship between cells using the lightweight Transformer module with SE compression-excitation mechanism, thereby establishing a DCPM-SE dual-branch feature fusion network model; A cell feature extraction module is used to input the enhanced tumor cell image data into the DCPM-SE dual-branch feature fusion network model for identification, dynamically weighted fusion of the dual-branch outputs through a gated attention mechanism, suppress background noise, and obtain a characteristic tumor cell image; The cell image segmentation module is used to calculate the cell centroid distance constraint of the characteristic tumor cell image based on the adhesion cell separation algorithm of the connected domain analysis, and iteratively cut the segmented area to obtain the target tumor cell image.

9. The tumor cell image extraction system based on deep learning according to claim 8, characterized in that: The fusion model building module includes the following submodules: Construct a submodule to set up 8 attention heads, each with a dimension of 64. The input feature map is mapped to the input dimension of the multi-head attention mechanism through linear transformation. Through multi-head attention calculation, the topological relationship between cells is learned to obtain the feature map; The generation submodule is used to perform global average pooling on the feature map to compress the features of each channel into a scalar value; The dependencies between channels are learned through two fully connected layers to generate channel-level weight vectors.

10. The tumor cell image extraction system based on deep learning according to claim 8, characterized in that: The cell image segmentation module includes the following submodules: The processing submodule is used to perform binary processing on the characteristic tumor cell image, automatically calculate the optimal threshold using the Otsu threshold segmentation algorithm, and convert the image into a black and white binary image; The calculation submodule is used to perform connected domain analysis using the 8-connectivity definition, and traverse each pixel point in the black and white binary image based on the scan line connected domain labeling algorithm; the centroid coordinates of each connected domain are calculated to obtain the target tumor cell image.

Images