A deep learning-based tumor cell image extraction method and system

Abstract

The application discloses a kind of based on deep learning's tumor cell image extraction method and system, by obtaining tumor cell image data in microscope, data preprocessing is carried out to the tumor cell image data;Using Retinex theory is combined with guided filter to eliminate the uneven illumination in the initial tumor cell image data slice, enhance the contrast of cell nucleus;Based on DCPM hollow convolution pyramid module captures multiscale cell morphological characteristics and utilizes SE compression-excitation mechanism light weight module to strengthen intercellular topological relationship;Through the dynamic weighting fusion of double-branch output of gate attention mechanism, the cytoplasmic center distance constraint of the feature tumor cell image is calculated based on the connected component analysis algorithm of adherent cell separation algorithm, can accurately separate tumor adherent cell.Combined with minimum cut algorithm and area, shape constraint condition, avoid over-segmentation and under-segmentation problem.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and system for extracting tumor cell images based on deep learning. Background Technology

[0002] Currently, tumor cell image extraction faces numerous technical challenges. On the one hand, during microscopic imaging, due to differences in sample preparation and uneven illumination, the acquired cell images generally suffer from uneven slice illumination and low contrast between cell nuclei and background, resulting in blurred cell edges and loss of details, which greatly affects the accuracy of subsequent analysis. On the other hand, traditional image extraction methods, based on threshold segmentation and region growing algorithms, often struggle to accurately distinguish between tumor cells and normal cells, or between cells and background, when dealing with complex backgrounds and adherent cells. This leads to poor segmentation results and a tendency for oversegmentation or undersegmentation. Summary of the Invention

[0003] The purpose of this invention is to solve the above-mentioned problems by designing a tumor cell image extraction method based on deep learning.

[0004] To achieve the above objectives, the technical solution of the present invention further includes the following steps in the aforementioned deep learning-based tumor cell image extraction method:

[0005] Acquire tumor cell image data from a microscope, and perform data preprocessing on the tumor cell image data to obtain initial tumor cell image data;

[0006] By using Retinex theory combined with guided filtering to eliminate uneven illumination of the slices in the initial tumor cell image data and enhance the contrast of cell nuclei, enhanced tumor cell image data is obtained.

[0007] The DCPM dilated convolutional pyramid module captures multi-scale cell morphological features, and the lightweight Transformer module with SE compression-excitation mechanism enhances the topological relationship between cells. The lightweight Transformer module is set with 8 attention heads, each with a dimension of 64. The DCPM dilated convolutional pyramid module and the lightweight Transformer module are used as two branches of the network to establish a DCPM-SE dual-branch feature fusion network model.

[0008] The enhanced tumor cell image data is input into the DCPM-SE dual-branch feature fusion network model for recognition. Dynamic weighted fusion is performed through a gated attention mechanism, and the fused features are mapped to the corresponding category labels to obtain the feature tumor cell image.

[0009] The adhesion cell separation algorithm based on connected domain analysis calculates the cytocenter distance constraint of the feature tumor cell image to obtain a target tumor cell image.

[0010] Further, in the above-mentioned tumor cell image extraction method based on deep learning, the tumor cell image data in the microscope is obtained, and the tumor cell image data is preprocessed to obtain initial tumor cell image data, including:

[0011] The tumor cell sample is photographed in three dimensions and multiple angles by using an image sensor, including 360° rotation photographing in the horizontal direction and layer cutting photographing every 1 μm in the vertical direction, to obtain tumor cell image data;

[0012] The tumor cell image data is converted into a unified standard image format, and the irrelevant blank area in the image is cropped to retain the effective area containing cells, to obtain a cropped tumor cell image;

[0013] The cropped tumor cell image is scaled to a fixed size by using a bilinear interpolation algorithm to obtain a scaled tumor cell image;

[0014] The scaled tumor cell image is labeled, including at least tumor cell boundaries, cell types and cell states, and the labeled image data is divided to obtain initial tumor cell image data.

[0015] Further, in the above-mentioned tumor cell image extraction method based on deep learning, the Retinex theory is used in combination with guided filtering to eliminate uneven illumination in the initial tumor cell image data, and the contrast of the cell nucleus is enhanced to obtain enhanced tumor cell image data, including:

[0016] The Retinex theory is used in combination with guided filtering to filter the initial tumor cell image data, and Gaussian filter kernels of different scales are constructed, with scale parameters being 15, 80 and 250 respectively, to simulate illumination changes in different ranges;

[0017] The initial tumor cell image is convolved with the Gaussian filter kernels of three different scales to obtain illumination estimation images under three scales;

[0018] The illumination estimation images under each scale are normalized to have a gray scale range of [0, 1] to obtain reflection images;

[0019] The reflection images are calculated, and the dynamic range compression is performed on the reflection images to map the gray scale values to the interval [0, 255] to obtain preliminary processing images;

[0020] The preliminary processing image is taken as an input image, and the initial tumor cell image data is taken as a guide image, and smoothing processing is performed using a guide filter to obtain enhanced tumor cell image data.

[0021] Further, in the above-mentioned deep learning-based tumor cell image extraction method, the DCPM hollow convolution pyramid module captures multi-scale cell morphological features, and the lightweight Transformer module with SE compression-excitation mechanism strengthens the topological relationship between cells, the lightweight Transformer module is provided with 8 attention heads, each head has a dimension of 64, the DCPM hollow convolution pyramid module and the lightweight Transformer module are taken as two branches of the network, and a DCPM-SE double-branch feature fusion network model is established, which comprises:

[0022] The DCPM hollow convolution pyramid module is constructed, four hollow convolution layers with different hollow rates are designed, the structure of each hollow convolution layer is that the input channel number is dynamically adjusted according to the output of the previous layer, the output channel number is set to 64, and the convolution kernel size is 3*3;

[0023] The extracted feature maps of different scales are fused, four feature maps are connected in the channel dimension by using the channel splicing method, and a feature map with multi-scale information is obtained.

[0024] Further, in the above-mentioned deep learning-based tumor cell image extraction method, the DCPM hollow convolution pyramid module captures multi-scale cell morphological features, and the lightweight Transformer module with SE compression-excitation mechanism strengthens the topological relationship between cells, the lightweight Transformer module is provided with 8 attention heads, each head has a dimension of 64, the DCPM hollow convolution pyramid module and the lightweight Transformer module are taken as two branches of the network, and a DCPM-SE double-branch feature fusion network model is established, which further comprises:

[0025] The lightweight Transformer module is provided with 8 attention heads, each head has a dimension of 64, the input feature map is mapped to the input dimension of the multi-head attention mechanism through linear transformation, the topological relationship between cells is learned through multi-head attention calculation to obtain a feature map;

[0026] The feature map is subjected to global average pooling to compress the features of each channel into a scalar value; the dependence between channels is learned through two fully connected layers to generate a channel-level weight vector.

[0027] Further, in the above-mentioned tumor cell image extraction method based on deep learning, the enhanced tumor cell image data is input into the DCPM-SE double-branch feature fusion network model for recognition, dynamic weighted fusion is performed through the gated attention mechanism, the fused features are mapped to the corresponding class labels, and a feature tumor cell image is obtained.

[0028] The enhanced tumor cell image data is input into the trained DCPM-SE double-branch feature fusion network model, multi-scale morphological features of tumor cells are extracted, and a lightweight Transformer module is combined with an SE compression-excitation mechanism to extract intercellular topological relationship features.

[0029] The feature outputs of the two branches are dynamically weighted and fused through the gated attention mechanism to obtain fused feature representations.

[0030] The gated attention mechanism calculates the weights of the two branch features according to the content of the input image, maps the fused features to the corresponding class labels, and obtains a feature tumor cell image.

[0031] Further, in the above-mentioned tumor cell image extraction method based on deep learning, the connected component analysis-based adherent cell separation algorithm calculates the cytocenter distance constraint of the feature tumor cell image to obtain a target tumor cell image, including:

[0032] The feature tumor cell image is binarized, and an Otsu threshold segmentation algorithm is used to automatically calculate the optimal threshold to convert the image into a black-and-white binary image.

[0033] Connected component analysis is performed using 8-connectivity definition, and a connected component labeling algorithm based on scan lines is used to traverse each pixel point in the black-and-white binary image; the centroid coordinates of each connected component are calculated to obtain a target tumor cell image.

[0034] Further, in a tumor cell image extraction system based on deep learning, the tumor cell image extraction system comprises the following modules:

[0035] A tumor image acquisition module is configured to acquire tumor cell image data in a microscope, perform data preprocessing on the tumor cell image data, and obtain initial tumor cell image data.

[0036] An image data enhancement module is configured to use Retinex theory combined with guided filtering to eliminate uneven illumination in the initial tumor cell image data, enhance the contrast of cell nuclei, and obtain enhanced tumor cell image data.

[0037] The fusion model establishing module is configured to capture multi-scale cell morphological features based on the DCPM hollow convolution pyramid module, and to strengthen the topological relationship between cells by using a lightweight Transformer module with an SE compression-excitation mechanism, wherein the lightweight Transformer module is configured with 8 attention heads, each head having a dimension of 64, and the DCPM hollow convolution pyramid module and the lightweight Transformer module are taken as two branches of the network to establish a DCPM-SE double-branch feature fusion network model.

[0038] The cell feature extraction module is configured to input the enhanced tumor cell image data into the DCPM-SE double-branch feature fusion network model for identification, dynamically weight fuse through a gated attention mechanism, map the fused features to corresponding class labels, and obtain a feature tumor cell image.

[0039] The cell image segmentation module is configured to calculate the cytocenter distance constraint of the feature tumor cell image based on a connected component analysis-based adherent cell separation algorithm, and obtain a target tumor cell image.

[0040] Further, in the above-mentioned deep learning-based tumor cell image extraction system, the fusion model establishing module comprises the following sub-modules:

[0041] The construction sub-module is configured to set the lightweight Transformer module with 8 attention heads, each head having a dimension of 64, and map the input feature map to the input dimension of the multi-head attention mechanism through linear transformation, learn the topological relationship between cells through multi-head attention calculation, and obtain a feature map.

[0042] The generation sub-module is configured to perform global average pooling on the feature map, compress the features of each channel into a scalar value, learn the dependency relationship between channels through two fully connected layers, and generate a channel-level weight vector.

[0043] Further, in the above-mentioned deep learning-based tumor cell image extraction system, the cell image segmentation module comprises the following sub-modules:

[0044] The processing sub-module is configured to perform binarization processing on the feature tumor cell image, automatically calculate the optimal threshold value by using an Otsu threshold segmentation algorithm, and convert the image into a black-and-white binary image.

[0045] The calculation sub-module is configured to perform connected component analysis by using 8-connectivity definition, traverse each pixel point in the black-and-white binary image based on a scan line-based connected component labeling algorithm, calculate the centroid coordinates of each connected component, and obtain a target tumor cell image.

[0046] The beneficial effects are that 1. the cell nucleus edge details are maximally reserved, the contrast of the cell nucleus and the background and other cell structures is significantly enhanced, the intercellular topological relationship is strengthened, the response to important features is enhanced through channel level weight adjustment, irrelevant information is inhibited, and the model is more accurate and comprehensive in extracting tumor cell features. 2. In a complex background, the cell morphological features are highlighted; in a cell dense area, the intercellular topological relationship is strengthened, the background noise is effectively inhibited, and the adaptability and recognition accuracy of the model to tumor cell images in different scenes are improved. 3. It can accurately separate adherent cells. Combined with the minimum cut algorithm and area and shape constraint conditions, the over-segmentation and under-segmentation problems are avoided, and it is ensured that each tumor cell can be accurately segmented, providing reliable image data for subsequent medical diagnosis and research. BRIEF DESCRIPTION OF DRAWINGS

[0047] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included only to illustrate preferred embodiments and are not intended to limit the present application.

[0048] Figure 1 The first embodiment schematic diagram of a tumor cell image extraction method based on deep learning in an embodiment of the present application is shown in the figure.

[0049] Figure 2 The second embodiment schematic diagram of a tumor cell image extraction method based on deep learning in an embodiment of the present application is shown in the figure.

[0050] Figure 3 The first embodiment schematic diagram of a tumor cell image extraction system based on deep learning in an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the present application clearer, the following further describes the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0052] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an" and "the" used herein also include the plural forms. It should be further understood that the use of the word "comprise" in the specification of the present application means that the features, integers, steps, operations, elements and / or groups exist, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, groups and / or their combinations.

[0053] The present application will be described in detail below in combination with the drawings, such as Figure 1As shown, a deep learning-based tumor cell image extraction method includes the following steps:

[0054] Step 101, obtaining tumor cell image data in a microscope, pre-processing the tumor cell image data to obtain initial tumor cell image data;

[0055] Specifically, in this embodiment, the image sensor is used to take three-dimensional stereoscopic multi-angle pictures of the tumor cell sample, including 360° rotation shooting in the horizontal direction and layer cutting shooting every 1 μm in the vertical direction, to obtain tumor cell image data;

[0056] The tumor cell image data is converted into a unified standard image format, the irrelevant blank area in the image is cropped, the effective area containing cells is retained, and a cropped tumor cell image is obtained;

[0057] The cropped tumor cell image is scaled to a fixed size using a bilinear interpolation algorithm to obtain a scaled tumor cell image;

[0058] The scaled tumor cell image is labeled, including at least tumor cell boundary, cell type and cell state, and the labeled image data is divided into data to obtain initial tumor cell image data.

[0059] Specifically;

[0060] 1.1 Cell image data acquisition;

[0061] A professional biological microscope (Nikon Eclipse Ni-E, Olympus BX53) equipped with a high-resolution CCD or CMOS image sensor is used, and unified and appropriate shooting parameters are set. The specific parameters are: the optical magnification is fixed at 40x or 100x oil lens, the exposure time is adjusted according to the sample fluorescence intensity and sensor sensitivity, ranging from 50-500ms; the focal length uses an automatic focusing combined with manual fine tuning method to ensure clear cell images. Three-dimensional stereoscopic multi-angle shooting is performed on the tumor cell sample, including 360° rotation shooting in the horizontal direction and layer cutting shooting every 1 μm in the vertical direction, to obtain multi-view, multi-layer cell image data.

[0062] During shooting, the sample is placed on the object stage in a constant temperature (37℃) and humidity (5% CO2) culture environment, and a special culture solution for live cell imaging is used to maintain the physiological activity of the sample to avoid distortion of the cell morphology in the image due to changes in the sample state. At the same time, the image acquisition software matched with the microscope is used to record the metadata of each image in real time, including sample number, shooting time, shooting position coordinates, fluorescence channel information, etc., and store it as a structured data file for subsequent data management and retrieval.

[0063] 1.2 Data preprocessing;

[0064] 1. Image format conversion: Convert the original image data (microscope-specific.czi,.nd2 format) to standard JPEG or PNG format using Python's OpenCV library or MATLAB image processing toolbox. During the conversion process, set the image quality parameters, and set the JPEG format quality factor to 90 to ensure minimal loss of image information while controlling file size for subsequent processing.

[0065] 2. Image cropping and scaling: First, automatically identify the effective area containing cells in the image through edge detection algorithm (Canny edge detection), and crop out the blank area around the image. Then, uniformly scale the cropped image to a fixed size, such as 224x224 pixels. When scaling, choose the interpolation algorithm according to the image resolution and content complexity. For low-resolution images, use bicubic interpolation to improve image clarity; for high-resolution images, use bilinear interpolation to reduce jagged effects while ensuring processing efficiency.

[0066] 3. Data annotation: Assemble a labeling team composed of medical image physicians and professional annotators, and use professional image annotation tools such as LabellImg and CVAT to manually annotate tumor cells in cell images. The annotation content includes the accurate boundary of tumor cells (using polygon annotation method), cell type (lung cancer cells, colon cancer cells, etc.), and cell state (proliferation period, apoptosis period, etc.). Establish a strict annotation quality control process, first have two annotators independently annotate the same batch of images, calculate the overlap rate (IoU) of the annotation results, and when IoU≥0.85, consider the annotation as qualified; for images with differences, submit them to medical experts for arbitration and review to ensure annotation accuracy.

[0067] 4. Data division: Use stratified sampling method to divide the annotated image data into 70% training set, 15% validation set, and 15% test set. To ensure the balanced proportion of different types of tumor cells in each set, first stratify the image data by cell type, then randomly sample images in each layer according to the proportion. At the same time, to enhance the generalization ability of the model, randomly shuffle the training set data and save the divided data set index file for subsequent training and evaluation.

[0068] Step 102, use Retinex theory combined with guided filtering to eliminate uneven illumination in the initial tumor cell image data, enhance the contrast of cell nuclei, and obtain enhanced tumor cell image data;

[0069] Specifically, in the embodiment, the Retinex theory is combined with the guided filter to eliminate the initial tumor cell image data for processing, a Gaussian filter core of different scales is constructed, and scale parameters are respectively set as 15, 80 and 250, which are used to simulate illumination changes in different ranges.

[0070] The initial tumor cell image is respectively convolved with the Gaussian filter cores of three different scales to obtain illumination estimation images under three scales.

[0071] The illumination estimation images under each scale are normalized to have a gray scale range of [0, 1] to obtain reflection images.

[0072] The reflection images are calculated, and the dynamic range compression is performed on the reflection images to map the gray scale values to the interval [0, 255] to obtain preliminary processed images.

[0073] The preliminary processed images are used as input images, and the initial tumor cell image data is used as a guide image, and the guided filter is used for smoothing to obtain enhanced tumor cell image data.

[0074] Specifically;

[0075] 2.1 Application of Retinex theory;

[0076] The core of the Retinex theory is to decompose the image I(x, y) into a reflection image R(x, y) and an illumination image L(x, y). In actual application, the multi-scale Retinex algorithm (MSR) is used to process the initial tumor cell image. The specific steps are as follows:

[0077] A Gaussian filter core of different scales is constructed, and scale parameters are respectively set as 15, 80 and 250, which are used to simulate illumination changes in different ranges.

[0078] The initial tumor cell image is respectively convolved with the Gaussian filter cores of three different scales to obtain illumination estimation images L_1(x, y), L_2(x, y) and L_3(x, y) under three scales.

[0079] The illumination estimation images under each scale are normalized to have a gray scale range of [0, 1].

[0080] The reflection images are calculated, and the dynamic range compression is performed on the reflection images to map the gray scale values to the interval [0, 255] to obtain preliminary processed images.

[0081] 2.2 Combination of guided filter;

[0082] The reflection image R(x, y) processed by Retinex is used as an input image, and the original initial tumor cell image I(x, y) is used as a guide image, and smoothing processing is performed using a guide filter.

[0083] In practical applications, the filtering effect is controlled by adjusting the radius parameter r and the regularization parameter epsilon of the guide filter. According to the characteristics of the tumor cell image, the radius parameter r is set to 5-15 pixels to control the size of the filtering window, and the regularization parameter epsilon is set to 0.01^2-0.1^2 to balance the smoothing effect and edge preservation ability. After multiple experiments and comparisons, when r=10 and epsilon=0.05^2, the uneven illumination of the slice can be effectively eliminated while the cell nucleus edge details are maximally preserved, the contrast between the cell nucleus and the background and other cell structures is significantly enhanced, and thus the enhanced tumor cell image data is obtained.

[0084] Step 103, based on the DCPM hollow convolution pyramid module, capture multi-scale cell morphology features, use the lightweight Transformer module with SE compression-excitation mechanism to strengthen the topological relationship between cells, set 8 attention heads for the lightweight Transformer module, and the dimension of each head is 64; the DCPM hollow convolution pyramid module and the lightweight Transformer module are used as two branches of the network;

[0085] Specifically, in the present embodiment, the DCPM hollow convolution pyramid module is constructed, four hollow convolution layers with different hollow rates are designed, the structure of each hollow convolution layer is that the input channel number is dynamically adjusted according to the output of the previous layer, the output channel number is set to 64, and the convolution kernel size is 3x3;

[0086] The extracted feature maps of different scales are fused, and the four feature maps are connected in the channel dimension by using the channel splicing method to obtain a feature map with multi-scale information.

[0087] 8 attention heads are set, and the dimension of each head is 64. The input feature map is mapped to the input dimension of the multi-head attention mechanism through linear transformation, and the topological relationship between cells is learned through multi-head attention calculation to obtain a feature map;

[0088] The feature map is globally averaged and pooled to compress the features of each channel into a scalar value; the dependence between channels is learned through two fully connected layers to generate a channel-level weight vector.

[0089] Specifically;

[0090] 3.1 DCPM hollow convolution pyramid module;

[0091] In the construction of the DCPM dilated convolution pyramid module, four layers of dilated convolution layers with different dilation rates are designed, and the dilation rates are 1, 2, 4, and 8 respectively. The structure of each dilated convolution layer is as follows: the number of input channels is dynamically adjusted according to the output of the previous layer, the number of output channels is uniformly set to 64, the convolution kernel size is 3x3, and the padding method is "same" to keep the feature map size unchanged.

[0092] For each dilated convolution layer, Batch Normalization (Batch Normalization) and ReLU activation function are adopted to accelerate network convergence and alleviate the problem of gradient vanishing. The different scale feature maps extracted by the four dilated convolution layers are fused, and the channel splicing method is adopted to connect the four feature maps in the channel dimension to obtain a feature map containing rich multi-scale information, and the number of channels is 64x4=256.

[0093] 3.2 Lightweight Transformer module combined with SE compression-excitation mechanism;

[0094] The lightweight Transformer module is based on the multi-head attention mechanism, with 8 attention heads and a dimension of 64 for each head. The input feature map is first mapped to the input dimension of the multi-head attention mechanism through linear transformation, and then the topological relationship between cells is learned through multi-head attention calculation. To reduce the amount of calculation, a 1x1 convolution layer is added after the multi-head attention mechanism to compress the feature map channel number to 128.

[0095] The SE compression-excitation mechanism is introduced to adjust the channel-level weight of the feature map output by the lightweight Transformer module. The specific process is as follows: first, the feature map is globally averaged to compress the features of each channel into a scalar value; then through two fully connected layers, the first layer has 128 / 16=8 neurons, and the second layer has 128 neurons, to learn the dependence between channels and generate a channel-level weight vector; finally, the weight vector is multiplied with the original feature map to enhance the response to important channel features, suppress irrelevant channel information, and strengthen the topological relationship between cells.

[0096] 3.3 DCPM-SE dual-branch feature fusion network model construction;

[0097] The DCPM dilated convolution pyramid module and the lightweight Transformer module are used as the two branches of the network, and the enhanced tumor cell image data is input into them. Before the output layer of the model, a gated attention mechanism is used to dynamically weight and fuse the outputs of the two branches. The structure of the gated attention mechanism includes two fully connected layers and a Sigmoid activation function, the input is the splicing of the output feature maps of the two branches, and the output is the fusion weight vector.

[0098] Through training, the gating attention mechanism learns to adaptively adjust the weight of the output of the two branches according to the input features, so that the network can flexibly select and fuse feature information according to the image content. Finally, a global average pooling layer and a fully connected layer are added, and the number of neurons in the fully connected layer is set according to the number of classes of the tumor cell classification task, for example, for a 10-class tumor cell classification task, the number of neurons is set to 10, and the output feature is the classification result or feature description of the tumor cell image.

[0099] Step 104, input the enhanced tumor cell image data into the DCPM-SE double-branch feature fusion network model for recognition, dynamically weighted fusion through the gating attention mechanism, and map the fused features to the corresponding class label to obtain a feature tumor cell image.

[0100] Specifically, in this embodiment, the enhanced tumor cell image data is input into the trained DCPM-SE double-branch feature fusion network model, multi-scale morphological features of tumor cells are extracted, and a lightweight Transformer module is combined with an SE compression-excitation mechanism to extract intercellular topological relationship features.

[0101] The feature outputs of the two branches are dynamically weighted and fused through the gating attention mechanism to obtain the fused feature representation.

[0102] The gating attention mechanism calculates the weights of the two branch features according to the content of the input image, maps the fused features to the corresponding class label, and obtains a feature tumor cell image.

[0103] Specifically;

[0104] In the model training phase, the cross-entropy loss function is used as the objective function, the Adam optimizer is selected as the optimizer, the initial learning rate is set to 0.001, and the cosine annealing learning rate decay strategy is used to dynamically adjust the learning rate during training. The training batch size is set to 32, and the number of training rounds is determined according to the performance of the validation set. When the validation set loss does not decrease for 5 consecutive rounds, the training is stopped.

[0105] In the inference phase, the enhanced tumor cell image data is input into the trained DCPM-SE double-branch feature fusion network model. The DCPM hollow convolution pyramid module first extracts multi-scale morphological features of tumor cells, and the lightweight Transformer module is combined with the SE compression-excitation mechanism to extract intercellular topological relationship features. The feature outputs of the two branches are dynamically weighted and fused through the gating attention mechanism to obtain the fused feature representation.

[0106] The gating attention mechanism automatically calculates the weight of the features of the two branches according to the content of the input image, increases the weight of the DCPM module feature for an image containing a complex background, highlights the cell morphological features, and increases the weight of the lightweight Transformer module feature for an image of a cell dense region, to strengthen the topological relationship between cells. Finally, the fused features are mapped to the corresponding class label or feature description through the output layer of the network, to obtain the feature tumor cell image.

[0107] In step 105, a connected component analysis-based adherent cell separation algorithm is used to calculate the cytocenter distance constraint of the feature tumor cell image, to obtain the target tumor cell image.

[0108] Specifically, in the present embodiment, the feature tumor cell image is binarized, and the Otsu threshold segmentation algorithm is used to automatically calculate the optimal threshold, to convert the image into a black and white binary image.

[0109] Connected component analysis is performed using the 8-connectivity definition, and a connected component labeling algorithm based on scanning lines is used to traverse each pixel point in the black and white binary image. The centroid coordinates of each connected component are calculated to obtain the target tumor cell image.

[0110] Specifically,

[0111] 5.1 Connected component analysis

[0112] The feature tumor cell image is binarized, and the Otsu threshold segmentation algorithm is used to automatically calculate the optimal threshold, to convert the image into a black and white binary image, highlighting the tumor cell region. Then, connected component analysis is performed using the 8-connectivity definition, and a connected component labeling algorithm based on scanning lines (two-pass scanning algorithm) is used to traverse each pixel point in the image.

[0113] In the first pass of scanning, each pixel point is assigned a temporary label, and equivalent label pairs are recorded. In the second pass of scanning, the connected components are merged according to the equivalent label pairs, and each connected component is assigned a unique final label, to segment the tumor cells into different connected regions. At the same time, the area, perimeter and other geometric features of each connected component are calculated, for subsequent adherent cell judgment.

[0114] 5.2 Cytocenter distance constraint calculation

[0115] For each connected component, the centroid coordinates (x_c, y_c) are calculated.

[0116] Its beneficial effects are that 1. The cell nucleus edge details are maximally preserved, the contrast between the cell nucleus and the background and other cell structures is significantly enhanced, the intercellular topological relationship is strengthened, the response to important features is enhanced through channel-level weight adjustment, irrelevant information is suppressed, and the model is more accurate and comprehensive in extracting tumor cell features. 2. In a complex background, the cell morphological features are highlighted; in a cell-dense area, the intercellular topological relationship is strengthened, the background noise is effectively suppressed, and the adaptability and recognition accuracy of the model to tumor cell images in different scenes are improved. 3. It can accurately separate tumor adherent cells. Combined with the minimum cut algorithm and area and shape constraints, the over-segmentation and under-segmentation problems are avoided, and each tumor cell can be accurately segmented, providing reliable tumor cell image data for subsequent medical diagnosis and research.

[0117] Please refer to Figure 2 In a deep learning-based tumor cell image extraction method, tumor cell image data in a microscope is obtained, and the tumor cell image data is preprocessed to obtain initial tumor cell image data, including the following steps:

[0118] Step 201, using an image sensor to take three-dimensional stereo multi-angle photos of the tumor cell sample, including 360° rotation photos in the horizontal direction and layer cutting photos every 1 μm in the vertical direction, to obtain tumor cell image data;

[0119] Step 202, converting the tumor cell image data into a unified standard image format, cropping out irrelevant blank areas in the image, and retaining the effective area containing cells to obtain a cropped tumor cell image;

[0120] Step 203, using a bilinear interpolation algorithm to scale the cropped tumor cell image to a fixed size to obtain a scaled tumor cell image;

[0121] Step 204, labeling the scaled tumor cell image, including at least tumor cell boundaries, cell types and cell states, and dividing the labeled image data into initial tumor cell image data.

[0122] Please refer to Figure 3 In a deep learning-based tumor cell image extraction system, the tumor cell image extraction system includes the following modules:

[0123] A tumor image acquisition module is configured to obtain tumor cell image data in a microscope, pre-process the tumor cell image data, and obtain initial tumor cell image data;

[0124] An image data enhancement module is configured to eliminate uneven illumination in the initial tumor cell image data by using Retinex theory combined with guided filtering, enhance the contrast of cell nuclei, and obtain enhanced tumor cell image data;

[0125] A fusion model establishment module is configured to capture multi-scale cell morphological features based on a DCPM hollow convolution pyramid module, strengthen the topological relationship between cells by using a lightweight Transformer module with SE compression-excitation mechanism, set 8 attention heads for the lightweight Transformer module, with the dimension of each head being 64, take the DCPM hollow convolution pyramid module and the lightweight Transformer module as two branches of the network, and establish a DCPM-SE double-branch feature fusion network model.

[0126] A cell feature extraction module is configured to input the enhanced tumor cell image data into the DCPM-SE double-branch feature fusion network model for identification, dynamically weight and fuse the features by using a gated attention mechanism, map the fused features to corresponding class labels, and obtain a feature tumor cell image.

[0127] A cell image segmentation module is configured to calculate the cytoplasmic center distance constraint of the feature tumor cell image based on a connected component analysis-based adherent cell separation algorithm, and obtain a target tumor cell image.

[0128] The basic principles, main features and advantages of the present application are shown and described above. It should be understood by those skilled in the art that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for extracting tumor cell images based on deep learning, characterized in that, The tumor cell image extraction method includes the following steps: Acquire tumor cell image data from a microscope, and perform data preprocessing on the tumor cell image data to obtain initial tumor cell image data; By using Retinex theory combined with guided filtering to eliminate uneven illumination of the slices in the initial tumor cell image data and enhance the contrast of cell nuclei, enhanced tumor cell image data is obtained. The DCPM dilated convolutional pyramid module captures multi-scale cell morphological features, and the lightweight Transformer module with SE compression-excitation mechanism enhances the topological relationship between cells. The lightweight Transformer module is set with 8 attention heads, each with a dimension of 64. The DCPM dilated convolutional pyramid module and the lightweight Transformer module are used as two branches of the network to establish a DCPM-SE dual-branch feature fusion network model. The enhanced tumor cell image data is input into the DCPM-SE dual-branch feature fusion network model for recognition. Dynamic weighted fusion is performed through a gated attention mechanism, and the fused features are mapped to the corresponding category labels to obtain the feature tumor cell image. An algorithm for separating adherent cells based on connected component analysis calculates the centroid distance constraint of the characteristic tumor cell image to obtain the target tumor cell image.

2. The tumor cell image extraction method based on deep learning as described in claim 1, characterized in that, The process of acquiring tumor cell image data from a microscope and preprocessing the tumor cell image data to obtain initial tumor cell image data includes: The tumor cell sample was captured in three dimensions from multiple angles using an image sensor, including 360° rotation in the horizontal direction and slice-by-slice imaging at 1μm intervals in the vertical direction, to obtain tumor cell image data. The tumor cell image data is converted into a unified standard image format, and irrelevant blank areas in the image are cropped out, while retaining the effective areas containing cells, to obtain a cropped tumor cell image. The cropped tumor cell image is scaled to a fixed size using a bilinear interpolation algorithm to obtain a scaled tumor cell image. The scaled tumor cell image is labeled, including at least the tumor cell boundary, cell type and cell state. The labeled image data is then divided to obtain initial tumor cell image data.

3. The tumor cell image extraction method based on deep learning as described in claim 1, characterized in that, The DCPM-based dilated convolutional pyramid module captures multi-scale cell morphological features, and the lightweight Transformer module using the SE compression-excitation mechanism enhances the topological relationships between cells. The lightweight Transformer module has eight attention heads, each with a dimension of 64. The DCPM-SE dual-branch feature fusion network model is established by using the DCPM-dilated convolutional pyramid module and the lightweight Transformer module as two branches of the network, including: Construct a DCPM dilated convolution pyramid module and design four dilated convolution layers with different dilation rates. The structure of each dilated convolution layer is such that the number of input channels is dynamically adjusted according to the output of the previous layer, the number of output channels is set to 64, and the convolution kernel size is 3×3. The extracted feature maps at different scales are fused together, and the four feature maps are connected in the channel dimension by channel splicing to obtain a feature map with multi-scale information.

4. The tumor cell image extraction method based on deep learning as described in claim 1, characterized in that, The DCPM-based dilated convolutional pyramid module captures multi-scale cell morphological features, and the lightweight Transformer module using the SE compression-excitation mechanism enhances the topological relationships between cells. The lightweight Transformer module has eight attention heads, each with a dimension of 64. The DCPM-SE dual-branch feature fusion network model is established by using the DCPM-dilated convolutional pyramid module and the lightweight Transformer module as two branches of the network. The model also includes: The lightweight Transformer module sets up 8 attention heads, each with a dimension of 64. The input feature map is linearly transformed and mapped to the input dimension of the multi-head attention mechanism. Through multi-head attention calculation, the topological relationship between cells is learned to obtain the feature map. Global average pooling is performed on the feature maps to compress the features of each channel into a scalar value; the dependencies between channels are learned through two fully connected layers to generate channel-level weight vectors.

5. The tumor cell image extraction method based on deep learning as described in claim 1, characterized in that, The step of inputting the enhanced tumor cell image data into the DCPM-SE dual-branch feature fusion network model for recognition, performing dynamic weighted fusion through a gated attention mechanism, and mapping the fused features to corresponding category labels to obtain a featured tumor cell image, further includes: Enhanced tumor cell image data is input into a trained DCPM-SE dual-branch feature fusion network model to extract multi-scale morphological features of tumor cells. The lightweight Transformer module is combined with the SE compression-excitation mechanism to extract inter-cell topological relationship features. The feature outputs of the two branches are dynamically weighted and fused through a gated attention mechanism to obtain the fused feature representation; The gated attention mechanism calculates the weights of the two branch features based on the content of the input image, maps the fused features to the corresponding category labels, and obtains the feature tumor cell image.

6. The tumor cell image extraction method based on deep learning as described in claim 1, characterized in that, The adhering cell separation algorithm based on connected component analysis calculates the cell centroid distance constraint of the feature tumor cell image to obtain the target tumor cell image, including: The characteristic tumor cell image is binarized, and the optimal threshold is automatically calculated using the Otsu threshold segmentation algorithm to convert the image into a black and white binary image. Connectivity analysis is performed using the 8-connectivity definition. A connected component labeling algorithm based on scan lines is used to traverse each pixel in the black-and-white binary image. The centroid coordinates of each connected component are calculated to obtain the target tumor cell image.

7. A tumor cell image extraction system based on deep learning, characterized in that, The tumor cell image extraction system includes the following modules: The tumor image acquisition module is used to acquire tumor cell image data in a microscope, and to perform data preprocessing on the tumor cell image data to obtain initial tumor cell image data. The image data enhancement module is used to eliminate uneven illumination of the slices in the initial tumor cell image data by combining Retinex theory with guided filtering, thereby enhancing the contrast of cell nuclei and obtaining enhanced tumor cell image data. The fusion model building module is used to capture multi-scale cell morphological features based on the DCPM dilated convolutional pyramid module and enhance the topological relationship between cells using the SE compression-excitation mechanism of the lightweight Transformer module. The lightweight Transformer module is set with 8 attention heads, each with a dimension of 64. The DCPM dilated convolutional pyramid module and the lightweight Transformer module are used as two branches of the network to build a DCPM-SE dual-branch feature fusion network model. The cell feature extraction module is used to input the enhanced tumor cell image data into the DCPM-SE dual-branch feature fusion network model for recognition, and to perform dynamic weighted fusion through a gating attention mechanism, mapping the fused features to the corresponding category labels to obtain the feature tumor cell image; The cell image segmentation module is used to calculate the centroid distance constraint of the feature tumor cell image based on the connected component analysis-based adhering cell separation algorithm, so as to obtain the target tumor cell image.

8. The tumor cell image extraction system based on deep learning as described in claim 7, characterized in that, The fusion model establishment module includes the following sub-modules: A submodule is constructed to set up 8 attention heads for the lightweight Transformer module, each head having a dimension of 64. The input feature map is mapped to the input dimension of the multi-head attention mechanism through linear transformation. The topological relationship between cells is learned through multi-head attention calculation to obtain the feature map. The generation submodule is used to perform global average pooling on the feature map, compressing the feature of each channel into a scalar value; Channel-level weight vectors are generated by learning the dependencies between channels through two fully connected layers.

9. The tumor cell image extraction system based on deep learning as described in claim 7, characterized in that, The cell image segmentation module includes the following sub-modules: The processing submodule is used to binarize the feature tumor cell image, and automatically calculates the optimal threshold using the Otsu threshold segmentation algorithm to convert the image into a black and white binary image. The computational submodule is used to perform connected component analysis using the 8-connectivity definition. It traverses each pixel in the black-and-white binary image based on the connected component labeling algorithm of the scan line; and calculates the centroid coordinates of each connected component to obtain the target tumor cell image.

Images