A pan-cancer cell detection method, system and storage medium

Abstract

A pan-cancer cell detection method, system and storage medium, comprising: obtaining training data, using a multi-scale neighbor perception function to obtain a true cell neighbor perception map of the training data, using a cell neighbor perception map prediction network to obtain a predicted cell neighbor perception map of the training data; calculating a first loss between the true cell neighbor perception map and the predicted cell neighbor perception map, and optimizing the cell neighbor perception map prediction network through the first loss; obtaining detection data, using the optimized cell neighbor perception map prediction network to obtain neighbor perception features of the detection data; using a cell detection classification network to obtain cell features, using a feature fusion module to fuse the cell features with the neighbor perception features to obtain fused features; obtaining a cell detection map and a cell classification map according to the fused features, and obtaining a cell position and a cell category according to the cell detection map and the cell classification map. The accuracy and robustness of the cell detection task can be significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical imaging, more particularly to a pan-cancer cell detection method and system and a storage medium. BACKGROUND

[0002] As the "gold standard" for cancer diagnosis, one of the key steps of pathological examination is to observe and analyze the cells in pathological sections. Pathologists can accurately diagnose the type of cancer and develop appropriate diagnosis and treatment plans by carefully studying the number, morphology and distribution of different types of cells in pathological sections. However, manually labeling and counting cells in pathological sections is not only time-consuming and labor-intensive, but also susceptible to subjective factors, leading to misdiagnosis and missed diagnosis. With the rise of deep learning technology, it has made remarkable achievements in the field of computer vision, and also brought new opportunities for cell detection and classification technology. Most existing deep learning methods focus on the morphological features of individual cells, ignoring the spatial context information between cells, which is particularly important in cell classification tasks. For example, cancerous cells often cluster into cancer nests, and cells within cancer nests are more likely to be cancerous, while single cancerous cells in the stromal region outside the cancer nest are less common. Therefore, relying solely on morphological features for classification is difficult to achieve satisfactory results. SUMMARY

[0003] The present application is proposed to solve the above problems. According to one aspect of the present application, a pan-cancer cell detection method is provided, which comprises:

[0004] obtaining training data, obtaining a real cell proximity perception map of the training data using a multi-scale proximity perception function, and obtaining a predicted cell proximity perception map of the training data using a cell proximity perception map prediction network;

[0005] calculating a first loss between the real cell proximity perception map and the predicted cell proximity perception map, and optimizing the cell proximity perception map prediction network through the first loss;

[0006] obtaining detection data, and obtaining proximity perception features of the detection data using the optimized cell proximity perception map prediction network;

[0007] obtaining cell features using a cell detection and classification network, fusing the cell features with the proximity perception features using a feature fusion module, and obtaining fused features;

[0008] obtaining a cell detection map and a cell classification map according to the fused features, and obtaining cell positions and cell categories according to the cell detection map and the cell classification map.

[0009] In one embodiment of the present application, the method for obtaining a real cell proximity perception map of the training data using a multi-scale proximity perception function comprises:

[0010] calculating, by the multi-scale neighborhood perception function, a neighborhood perception value of each pixel position of the training data in each cell category, each perception scale, and each perception direction;

[0011] arranging the neighborhood perception values according to the pixel positions to obtain a real cell neighborhood perception map of the training data.

[0012] In an embodiment of the present application, the calculating, by the multi-scale neighborhood perception function, a neighborhood perception value of each pixel position of the training data in each cell category, each perception scale, and each perception direction, comprises:

[0013] for a specified pixel, traversing a specified category cell position around the specified pixel position;

[0014] respectively calculating a projection distance of the specified category cell position and the specified pixel position in a specified perception direction;

[0015] processing the projection distance using a perception function to obtain a function value of a specified perception scale;

[0016] multiplying the function value of the specified perception scale by a direction consistency factor to obtain a perception value of a cell;

[0017] adding the perception values of all cells in the specified category to obtain a neighborhood perception value of the specified pixel.

[0018] In an embodiment of the present application, after obtaining the training data, further comprising:

[0019] obtaining a real cell detection map and a real cell classification map of the training data;

[0020] obtaining a predicted cell detection map and a predicted cell classification map of the training data using a cell detection and classification network;

[0021] calculating a second loss between the real cell detection map and the predicted cell detection map, and calculating a third loss between the real cell classification map and the predicted cell classification map, and optimizing the cell detection and classification network through the second loss and the third loss.

[0022] In an embodiment of the present application, the cell neighborhood perception map prediction network comprises a neighborhood perception feature extractor, a spatial distribution modeling module, and a cell neighborhood perception map predictor, and the obtaining a predicted cell neighborhood perception map of the training data using a cell neighborhood perception map prediction network comprises:

[0023] obtaining a first neighborhood perception feature of the training data using the neighborhood perception feature extractor;

[0024] enhancing the first neighboring-aware feature using the spatial distribution modeling module to obtain a second neighboring-aware feature;

[0025] inputting the second neighboring-aware feature into the cell neighboring-aware map predictor to obtain a predicted cell neighboring-aware map of the training data.

[0026] In an embodiment of the present application, the feature fusion module includes a channel attention module and a spatial cross-attention module, and the feature fusion module is used to fuse the cell feature and the neighboring-aware feature to obtain a fused feature, including:

[0027] The channel attention module is used to extract key semantic channels of the cell feature and the neighboring-aware feature respectively to obtain a channel-enhanced cell feature and a channel-enhanced neighboring-aware feature;

[0028] The spatial cross-attention module is used to align and fuse the channel-enhanced cell feature and the channel-enhanced neighboring-aware feature in the spatial dimension to obtain a spatial-enhanced feature;

[0029] The cell feature and the neighboring-aware feature are spliced in the channel dimension to obtain a spliced feature, and the spliced feature and the spatial-enhanced feature are added to obtain the fused feature.

[0030] In an embodiment of the present application, the cell detection and classification network further includes a cell detector and a cell classifier, and the cell detection map and the cell classification map are obtained according to the fused feature, including:

[0031] The fused feature is input into the cell detector to obtain the cell detection map;

[0032] The fused feature is input into the cell classifier to obtain the cell classification map.

[0033] In an embodiment of the present application, the cell position and the cell category are obtained according to the cell detection map and the cell classification map, including:

[0034] The cell detection map is thresholded to calculate the centroid positions of each connected region greater than a specified threshold on the cell detection map, and the centroid positions are taken as the cell positions;

[0035] A channel of a specified value at the cell position is found on the cell classification map, and the cell category is obtained according to the channel.

[0036] According to another aspect of the present application, a pan-cancer cell detection system is provided, including an optimization module, a feature acquisition module and a detection module:

[0037] The optimization module is configured to obtain training data, obtain a real cell proximity perception map of the training data using a multi-scale proximity perception function, obtain a predicted cell proximity perception map of the training data using a cell proximity perception map prediction network, calculate a first loss between the real cell proximity perception map and the predicted cell proximity perception map, and optimize the cell proximity perception map prediction network based on the first loss.

[0038] The feature acquisition module is configured to obtain detection data, obtain proximity perception features of the detection data using the optimized cell proximity perception map prediction network, obtain cell features using a cell detection and classification network, and fuse the cell features and the proximity perception features using a feature fusion module to obtain fused features.

[0039] The detection module is configured to obtain a cell detection map and a cell classification map based on the fused features, and obtain cell positions and cell categories based on the cell detection map and the cell classification map.

[0040] According to another aspect of the present application, a storage medium is provided, and the storage medium stores a computer program. When the computer program is executed, the above-mentioned pan-cancer cell detection method is performed.

[0041] The present application calculates a real cell proximity perception map using a multi-scale proximity perception function, optimizes a cell proximity perception map prediction network, comprehensively describes the spatial distribution relationship of different types of cells in a pathological image, extracts proximity perception features using the optimized cell proximity perception map prediction network, and fuses proximity perception features and cell features using a cell detection and classification network, thereby significantly improving the accuracy and robustness of the cell detection and classification task. BRIEF DESCRIPTION OF DRAWINGS

[0042] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application taken in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of embodiments of the present application and constitute a part of the specification, which together with the description, serve to explain the present application. The drawings are not intended to limit the present application, and the same reference numerals are generally used to represent the same elements throughout the drawings.

[0043] Figure 1 A schematic flowchart of a pan-cancer cell detection method according to an embodiment of the present application is shown.

[0044] Figure 2 A training data annotation map according to an embodiment of the present application is shown.

[0045] Figure 3 A real cell proximity perception map of epithelial cells according to an embodiment of the present application is shown.

[0046] Figure 4A real cell proximity perception map of inflammatory cells is shown according to an embodiment of the present application.

[0047] Figure 5 A real cell proximity perception map of fibroblasts is shown according to an embodiment of the present application.

[0048] Figure 6 A system block diagram of a pan-cancer cell detection system is shown according to an embodiment of the present application.

[0049] Figure 7 A basic flowchart of a pan-cancer cell detection method is shown according to an embodiment of the present application.

[0050] Figure 8 A cell annotation processing schematic diagram is shown according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present application more obvious, the following will describe the example embodiments according to the present application in detail with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application described in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the protection scope of the present application.

[0052] First, referring to Figure 1 and Figure 7 to describe a pan-cancer cell detection method for implementing an embodiment of the present application. Figure 1 A schematic flowchart of a pan-cancer cell detection method according to an embodiment of the present application is shown. Figure 7 A basic flowchart of a pan-cancer cell detection method according to an embodiment of the present application is shown.

[0053] In step S100, training data is acquired, a real cell proximity perception map of the training data is acquired using a multi-scale proximity perception function, and a predicted cell proximity perception map of the training data is acquired using a cell proximity perception map prediction network.

[0054] In step S200, a first loss between the real cell proximity perception map and the predicted cell proximity perception map is calculated, and the cell proximity perception map prediction network is optimized through the first loss.

[0055] In step S300, detection data is acquired, and proximity perception features of the detection data are acquired using the optimized cell proximity perception map prediction network.

[0056] At step S400, a cell feature is obtained using the cell detection classification network, and the cell feature is fused with the proximity perception feature using a feature fusion module to obtain a fused feature.

[0057] At step S500, a cell detection map and a cell classification map are obtained according to the fused feature, and a cell position and a cell category are obtained according to the cell detection map and the cell classification map.

[0058] The present application calculates a real cell proximity perception map by a multi-scale proximity perception function to optimize a cell proximity perception map prediction network, comprehensively depicts the spatial distribution relationship of different types of cells in a pathological image, extracts proximity perception features by the optimized cell proximity perception map prediction network, and fuses the proximity perception features with cell features by a cell detection classification network, which can significantly improve the accuracy and robustness of the cell detection and classification task.

[0059] In one embodiment, in step S100 of obtaining training data, a real cell proximity perception map of the training data is obtained using a multi-scale proximity perception function, and a predicted cell proximity perception map of the training data is obtained using a cell proximity perception map prediction network.

[0060] The training data is a <cell annotation, data> pair composed of each cell position and each cell category as cell annotation information and a corresponding pan-cancer digital pathology image. By scanning and annotating the pathological section images of patients, the <cell annotation, data> is obtained, so as to obtain the generated training data and detection data.

[0061] Wherein the data is the pan-cancer digital pathology image obtained after scanning each pan-cancer pathological section, and the cell annotation is each cell position and each cell category corresponding to the digital pathology image. As shown in Figure 2 The blue dots, red dots and green dots represent epithelial cells, inflammatory cells and fibroblasts, respectively. In one embodiment, taking a pan-cancer digital pathology image as an example, the digital pathology image has a size of 500x500 pixels, and is a hematoxylin-eosin (H&E) staining colorectal cancer pathological image scanned under a 20x lens. The corresponding cell annotation file contains the image coordinate positions of the center points of each cell, such as (132, 24), (256, 246), (48, 258), etc., and the category labels thereof, such as "lymphocyte", "epithelial cell", "stromal cell", etc. The above digital pathology image and each cell position and each cell category corresponding to the digital pathology image are all data provided by a hospital. The training data in the training process is the <cell annotation, data> pair obtained by the above processing, which aims to train the neural network by the training data.

[0062] In one embodiment, the real cell neighborhood perception map of the training data is obtained using the multi-scale neighborhood perception function, including: calculating, by the multi-scale neighborhood perception function, a neighborhood perception value of each pixel position of the training data in each cell category, each perception scale, and each perception direction; and arranging the neighborhood perception values according to the pixel positions to obtain the real cell neighborhood perception map of the training data.

[0063] As shown in FIG. 1, Figure 8 the cell annotation is processed to obtain the real cell neighborhood perception map. For a digital pathology image, a real cell neighborhood perception map corresponding to the digital pathology image has the same size and a channel number of The real cell neighborhood perception map is obtained by the following method: for each pixel position of the digital pathology image, a neighborhood perception value of the pixel position in each cell category, each perception scale, and each perception direction is calculated using the multi-scale neighborhood perception function. The real cell neighborhood perception map is a structured representation map for modeling the spatial organization structure of cells in a pathology image. The map takes pixels as the basic unit and can accurately depict the neighborhood relationship and spatial distribution of different categories of cells in multiple scales and multiple directions through the calculation of the multi-scale neighborhood perception function. As shown in FIGS. 1, Figure 3 , 4 , 5, Figure 3 is a real cell neighborhood perception map with the cell category being epithelial cells, the perception scale being 15, and the perception direction being 0°. Figure 4 is a real cell neighborhood perception map with the cell category being inflammatory cells, the perception scale being 15, and the perception direction being 0°. Figure 5 is a real cell neighborhood perception map with the cell category being fibroblasts, the perception scale being 15, and the perception direction being 0°. The number of perception scales and perception directions can be set according to requirements.

[0064] In one embodiment, the neighborhood perception value of each pixel position of the training data in each cell category, each perception scale, and each perception direction is calculated by the multi-scale neighborhood perception function, including: for a specified pixel, traversing the specified category cell positions around the specified pixel position; respectively calculating the projection distances of the specified category cell positions and the specified pixel position in the specified perception direction; using a perception function to process the projection distances to obtain a function value of the specified perception scale; multiplying the function value of the specified perception scale by a direction consistency factor to obtain a perception value of one cell; and adding the perception values of all cells in the specified category to obtain the neighborhood perception value of the specified pixel. The perception function can be a Gaussian kernel function.

[0065] The formula of the multi-scale neighborhood perception function is:

[0066] ;

[0067] wherein, is a neighboring perception value of the pixel position, is a cell category, is a perception scale, is a preset perception scale set, is a perception direction, is a preset perception direction set, is a pixel position in a real cell neighboring perception map, is a pixel position corresponding to a surrounding cell, is a set of pixel positions corresponding to the category cell position in the image, is a set of pixel positions corresponding to the category cell position in the image, is a pixel position and a projection distance in the perception direction , is a cosine function. is a perception scale related Gaussian kernel function, , is a simple representation of the projection distance, is a direction consistency factor, used to measure the alignment degree of the pixel position and relative to the perception direction . , represents the direction angle from to .

[0068] The function can adaptively give different weights to neighboring cells with different distances and directions, taking into account the spatial distance and orientation information of the cells, thereby comprehensively describing the spatial distribution relationship of different types of cells in the pathological image. Compared with the single-scale and directionless modeling method of the prior art, it has higher expression ability and spatial recognition.

[0069] As shown in Figure 7 , in one embodiment, the cell neighboring perception map prediction network includes a neighboring perception feature extractor, a spatial distribution modeling module, and a cell neighboring perception map predictor. The predicted cell neighboring perception map of the training data is obtained using the cell neighboring perception map prediction network, including: obtaining the first neighboring perception feature of the training data using the neighboring perception feature extractor; enhancing the first neighboring perception feature to obtain the second neighboring perception feature using the spatial distribution modeling module; inputting the second neighboring perception feature into the cell neighboring perception map predictor to obtain the predicted cell neighboring perception map of the training data.

[0070] The digital pathology image is input into a neighborhood-aware feature extractor to obtain first neighborhood-aware features. The neighborhood-aware feature extractor is composed of a neighborhood-aware feature encoder and a neighborhood-aware feature decoder, and the features extracted by the neighborhood-aware feature encoder and the neighborhood-aware feature decoder are connected by a skip connection. The neighborhood-aware feature encoder is composed of a plurality of convolutional layers and a plurality of pooling layers in alternation, and the neighborhood-aware feature decoder is composed of a plurality of convolutional layers and a plurality of pooling layers in alternation.

[0071] Since the receptive field size of the single-size convolutional layer in the neighborhood-aware feature encoder is limited, it cannot effectively model long-distance spatial dependencies. To further improve the spatial modeling capability of the neighborhood-aware features, the first neighborhood-aware features are input into a spatial distribution modeling module for enhancement. The operation process of the spatial distribution modeling module is as follows:

[0072] The first neighborhood-aware features are subjected to multi-scale convolutional embedding processing, and local feature information with different receptive fields is extracted through a plurality of convolutional kernels of different sizes, and is mapped to multi-scale embedding feature representations with consistent dimensions through linear projection.

[0073] The multi-scale embedding feature maps are divided into a plurality of non-overlapping local region sets according to a fixed window size division strategy, and a window-based self-attention mechanism is applied in each local region to model the spatial dependencies within the local cell population. A cross-window cross-attention mechanism is introduced between each local region to realize the global spatial dependencies between cell populations.

[0074] The features of all local windows are globally pooled and aggregated to generate a cell spatial distribution vector representing the entire pathology image; and through a cross-attention guiding mechanism, the cell spatial distribution vector is fused with the first neighborhood-aware features to guide the local region features to focus on the key structural patterns and their spatial semantic relationships in the image, to obtain fused features.

[0075] The fused features are connected in residual connection with the first neighborhood-aware features, and then the fused features are weighted and adjusted by a gated fusion module to dynamically adjust the importance and fusion ratio of local and global features, and finally output enhanced neighborhood-aware features, i.e. second neighborhood-aware features.

[0076] The second neighborhood-aware features enhanced by the spatial distribution modeling module are input into a cell neighborhood-aware map predictor to output a predicted cell neighborhood-aware map. By introducing a spatial distribution modeling module, the network can effectively model the dependencies of local and global spaces, thereby extracting high-quality neighborhood-aware features to assist the cell detection and classification task. And by calculating the neighborhood-aware map prediction loss between the predicted cell neighborhood-aware map and the real cell neighborhood-aware map, the optimization of the cell neighborhood-aware map prediction network is realized.

[0077] In one embodiment, the first loss between the calculated real cell proximity map and the predicted cell proximity map in step S200 is used to optimize the cell proximity map prediction network.

[0078] The first loss The cell proximity map similarity loss And the L1 loss The calculation method is as follows:

[0079] ;

[0080] Wherein, is the weight coefficient of adjusting and .

[0081] In one embodiment, the calculation method of the cell proximity map similarity loss is as follows:

[0082] ;

[0083] ;

[0084] ;

[0085] ;

[0086] Wherein, and respectively represent the predicted cell proximity map and the real cell proximity map; , , The brightness difference, contrast difference, and structure difference between the predicted cell proximity map and the real cell proximity map; and respectively represent the average value of and ; represents the covariance of and ; and respectively represent the mean square error of and ; , and respectively represent the constant coefficient for maintaining the stability of the calculation result. is the first digital pathology image, is each pixel point of the digital pathology image.

[0087] In one embodiment, the L1 loss is calculated as:

[0088] ;

[0089] in, is the number of pixels.

[0090] In one embodiment, after obtaining the training data, the method further includes: obtaining a real cell detection map and a real cell classification map of the training data; using a cell detection and classification network to obtain a predicted cell detection map and a predicted cell classification map of the training data; calculating a second loss between the real cell detection map and the predicted cell detection map, calculating a third loss between the real cell classification map and the predicted cell classification map, and optimizing the cell detection and classification network through the second loss and the third loss.

[0091] like Figure 8 As shown, the cell annotation is processed to obtain the real cell detection map and the real cell classification map. For digital pathology images, the corresponding real cell detection map is obtained as follows: In the pathology image, for each pixel point , with its coordinates is the center and the radius is Within the region, if there are cells, the value of the corresponding pixel in the real cell detection image is set to 1, otherwise it is set to 0. For digital pathology images, the corresponding real cell classification map is obtained as follows: In the pathology image, for each pixel , with its coordinates is the center and the radius is Within the area of ​​, if there is a category The cells in the real cell classification map are Set the value of the corresponding pixel in each channel to 1, otherwise it is set to 0.

[0092] The cell classification network is optimized by calculating the second loss between the predicted cell detection map and the true cell detection map, and the third loss between the predicted cell classification map and the true cell classification map.

[0093] In one embodiment, the second loss The calculation method is:

[0094] ;

[0095] in: and Respectively represent The predicted cell detection graph pixel values ​​and The real cell detection diagram a pixel value, is the number of pixels.

[0096] In one embodiment, the third loss is calculated as:

[0097]

[0098] wherein: and respectively represent the value of the first pixel on the channel of the first predicted cell proximity map and the value of the first pixel on the channel of the first real cell proximity map.

[0099] The cell proximity map prediction network and the cell detection classification network are trained respectively using the Adam optimizer to minimize the first loss, the second loss and the third loss, and the cell proximity map prediction network and the cell detection classification network are iterated respectively, and after several rounds of training, the optimal cell proximity map prediction network and cell detection classification network are selected for detecting and classifying cells in the pathological image.

[0100] The difference between the network running process and the training process is that the detection data in the running process is a pan-cancer digital pathology image whose cell positions and cell categories are unknown, and the output of the network is the predicted cell positions and categories of the digital pathology image.

[0101] Compared with the training process, the running process no longer needs to calculate the real cell proximity map, the real cell detection map and the real cell classification map. The reason is that the above calculation results are used to improve the performance of the network in the training process, and the running process is for the trained network, so the calculation process is no longer needed. The process of the network predicting the cell detection map and the cell classification map from the pathological image is consistent with the steps in the training process.

[0102] In one embodiment, in the step S300 of obtaining the detection data, the proximity perception feature of the detection data is obtained using the optimized cell proximity map prediction network. By training the neural network to predict the cell proximity map, the neural network can learn the cell proximity perception feature that can reflect the spatial distribution of cells. The proximity perception feature extractor is used to input the pathological image to obtain the proximity perception feature. The cell proximity map prediction network is independent of the detection classification backbone network, avoids interference between tasks, can focus on modeling spatial distribution information, and has higher accuracy and training stability compared with the multi-task learning method.

[0103] As Figure 7 ​​​​​​​As shown, in one embodiment, in step S400, a cell detection and classification network is used to obtain cell features, and a feature fusion module is used to fuse the cell features with neighboring perception features to obtain fused features.

[0104] The cell detection and classification network is used to extract, detect and classify cells in pathological images. The cell detection and classification network consists of a cell feature extractor, a feature fusion module, a cell detector and a cell classifier. The cell feature extractor is used to input pathological images to obtain cell features. The digital pathological image is input into the cell feature extractor to obtain discriminative cell features. The cell feature extractor consists of a cell feature encoder and a cell feature decoder, and the features extracted by the cell feature encoder and the cell feature decoder are connected using jump connections. In one embodiment, the cell feature encoder is composed of multiple convolutional layers and multiple pooling layers alternating. In one embodiment, the cell feature decoder is composed of multiple convolutional layers and multiple pooling layers alternating. The feature fusion module is used to fuse the cell features extracted by the cell feature extractor and the neighboring perception features extracted by the neighboring perception feature extractor to obtain fused features.

[0105] In one embodiment, the feature fusion module includes a channel attention module and a spatial cross attention module, and the feature fusion module is used to fuse the cell features and the neighboring perception features to obtain a fused feature, including: using the channel attention module to extract the key semantic channels of the cell features and the neighboring perception features respectively to obtain channel-enhanced cell features and channel-enhanced neighboring perception features; using the spatial cross attention module to align and fuse the channel-enhanced cell features and the channel-enhanced neighboring perception features in the spatial dimension to obtain a spatially enhanced feature; splicing the cell features and the neighboring perception features in the channel dimension to obtain a spliced ​​feature, and adding the spliced ​​feature and the spatially enhanced feature to obtain a fused feature.

[0106] The cell features and neighboring perception features are input into the feature fusion module for feature fusion. The feature fusion module includes the channel attention module and , respectively used for channel enhancement of neighboring perception features and cell features; spatial cross attention module , which is used to model the spatial semantic interaction between two types of features. The operation process of the feature fusion module is as follows:

[0107] The channel attention module extracts the key semantic channels of neighboring perception features and cell features respectively, and obtains channel enhanced features:

[0108] ;

[0109] ;

[0110] in, and Represent proximity perception features and cell features, respectively. and They represent channel-enhanced proximity perception features and channel-enhanced cell features, respectively.

[0111] Subsequently, the spatial cross-attention module dynamically aligns and fuses the channel-enhanced neighboring perception features and channel-enhanced cell features in the spatial dimension to obtain the spatially enhanced features:

[0112] ;

[0113] Finally, the original proximity-aware features Cell characteristics Concatenate in the channel dimension and combine with spatial enhancement features Add to get fusion features , the operation is as follows:

[0114] ;

[0115] in, Represents a concatenation operation on the channel dimension.

[0116] The feature fusion module uses a dual-channel attention mechanism and a spatial cross-attention mechanism to dynamically model the semantic complementary relationship between the input cell features and the neighboring perception features. It can significantly improve the sensitivity of cell features to spatial structure and enhance classification accuracy. It is particularly suitable for pan-cancer images with strong heterogeneity.

[0117] like Figure 7 As shown, in one embodiment, in step S500, a cell detection map and a cell classification map are obtained based on the fused features, and cell positions and cell categories are obtained based on the cell detection map and the cell classification map. The feature fusion mechanism enhances the detection and classification network's ability to utilize cell proximity perception features.

[0118] In one embodiment, the cell detection and classification network further includes a cell detector and a cell classifier. The network generates a cell detection map and a cell classification map based on the fused features, including: inputting the fused features into the cell detector to generate a cell detection map; and inputting the fused features into the cell classifier to generate a cell classification map. The cell detector is used to input the fused features to detect cells in the pathology image; and the cell classifier is used to input the fused features to classify cells in the pathology image.

[0119] In one embodiment, the cell position and cell category are obtained based on the cell detection map and the cell classification map, including: thresholding the cell detection map, calculating the centroid position of each connected area on the cell detection map that is greater than a specified threshold, and using the centroid position as the cell position; finding a channel with a specified value on the cell position on the cell classification map, and obtaining the cell category based on the channel.

[0120] The pan-cancer cell detection method of the present invention calculates the true cell proximity perception map through a multi-scale proximity perception function to optimize the cell proximity perception map prediction network, comprehensively characterizes the spatial distribution relationship of different types of cells in pathological images, extracts proximity perception features through the optimized cell proximity perception map prediction network, and fuses proximity perception features with cell features through a cell detection and classification network, which can significantly improve the accuracy and robustness of cell detection and classification tasks.

[0121] The present application also provides a pan-cancer cell detection system 600, such as Figure 6 As shown, it includes an optimization module 610, a feature acquisition module 620 and a detection module 630: the optimization module 610 is used to obtain training data, use a multi-scale proximity perception function to obtain a real cell proximity perception map of the training data, and use a cell proximity perception map prediction network to obtain a predicted cell proximity perception map of the training data; calculate the first loss between the real cell proximity perception map and the predicted cell proximity perception map, and optimize the cell proximity perception map prediction network through the first loss; the feature acquisition module 620 is used to obtain detection data, use the optimized cell proximity perception map prediction network to obtain the proximity perception features of the detection data; use the cell detection and classification network to obtain cell features, and use the feature fusion module to fuse the cell features with the proximity perception features to obtain fused features; the detection module 630 is used to obtain a cell detection map and a cell classification map based on the fused features, and obtain cell positions and cell categories based on the cell detection map and the cell classification map.

[0122] In addition, the present application also provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes the pan-cancer cell detection method according to the embodiment of the present application as described above. The storage medium may include, for example, a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0123] Various component embodiments of the present application can be implemented in hardware, or as software modules running in one or more processors, or in combinations thereof. As will be appreciated by one skilled in the art, a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functionality of some of the modules according to embodiments of the present application. The present application can also be implemented as a program (for example, a computer program and computer program product) for executing any one of the methods described herein on a computer. Such a program can be stored on a computer readable medium, or can be in the form of one or more signals. Such a signal can be downloaded from an Internet website, or can be available for pur- chase on a carrier medium. The program can also be provided on a carrier medium which is not downloaded but which is provided for reading directly from the carrier medium.

[0124] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, several devices can be listed with a conjunction like 'or', but it is to be understood that a device can also be implemented using only some of the devices recited. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures can not be used to advantage. The use of relative terms like 'about', 'approximately','substantially' and the like is intended to broadly describe and account for small variations in the indicated values. The scope of the application is not intended to be limited to the embodiments described herein but is intended to include any alterations without departing from the scope of the present application. The scope of the application is defined by the claims.

[0125] The above description is only specific embodiments of the present application or specific explanations of the specific embodiments, and the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, and all of them should be covered within the protection scope of the present application. The protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A pan-cancer cell detection method, characterized in that: The method comprises: Acquiring training data, acquiring a real cell proximity perception map of the training data using a multi-scale proximity perception function, and acquiring a predicted cell proximity perception map of the training data using a cell proximity perception map prediction network; Calculating a first loss between the true cell neighborhood perception map and the predicted cell neighborhood perception map, and optimizing a cell neighborhood perception map prediction network using the first loss; Acquiring detection data, and using the optimized cell proximity perception map prediction network to acquire proximity perception features of the detection data; Using a cell detection and classification network to obtain cell features, and using a feature fusion module to fuse the cell features with neighboring perception features to obtain fused features; A cell detection map and a cell classification map are obtained based on the fusion features, and cell positions and cell categories are obtained based on the cell detection map and the cell classification map.

2. The pan-cancer cell detection method according to claim 1, wherein: The method of using a multi-scale proximity perception function to obtain a true cell proximity perception map of the training data includes: Calculating the proximity perception value of each pixel position of the training data in each cell category, each perception scale, and each perception direction by the multi-scale proximity perception function; The proximity perception values ​​are arranged according to the pixel positions to obtain a real cell proximity perception map of the training data.

3. The pan-cancer cell detection method according to claim 2, wherein: The calculating the proximity perception value of each pixel position of the training data in each cell category, each perception scale, and each perception direction by the multi-scale proximity perception function includes: For a specified pixel point, traverse the specified category of cell positions around the specified pixel point position; Calculating the projection distance between the position of the specified category cell and the position of the specified pixel point in the specified perception direction respectively; Processing the projection distance using a perception function to obtain a function value of a specified perception scale; The perception value of a cell is obtained by multiplying the function value of the specified perception scale by the direction consistency factor; The perception values ​​of all cells in the specified category are added together to obtain the neighboring perception value of the specified pixel point.

4. The pan-cancer cell detection method according to claim 1, wherein: After obtaining the training data, it also includes: Obtaining a real cell detection image and a real cell classification image of the training data; Using a cell detection and classification network to obtain a predicted cell detection map and a predicted cell classification map of the training data; Calculate the second loss between the true cell detection map and the predicted cell detection map, calculate the third loss between the true cell classification map and the predicted cell classification map, and optimize the cell detection and classification network through the second loss and the third loss.

5. The pan-cancer cell detection method according to claim 1, wherein: The cell proximity perception map prediction network includes a proximity perception feature extractor, a spatial distribution modeling module, and a cell proximity perception map predictor. The method of using the cell proximity perception map prediction network to obtain a predicted cell proximity perception map of the training data includes: obtaining a first proximity-aware feature of the training data using the proximity-aware feature extractor; Using the spatial distribution modeling module to enhance the first proximity perception feature to obtain a second proximity perception feature; The second proximity perception feature is input into the cell proximity perception map predictor to obtain a predicted cell proximity perception map of the training data.

6. The pan-cancer cell detection method according to claim 1, wherein: The feature fusion module includes a channel attention module and a spatial cross attention module. The feature fusion module is used to fuse the cell feature with the neighboring perception feature to obtain a fusion feature, including: Using the channel attention module to extract the key semantic channels of the cell features and the neighbor perception features respectively, to obtain channel-enhanced cell features and channel-enhanced neighbor perception features; Using the spatial cross attention module to align and fuse the channel-enhanced cell features and the channel-enhanced neighboring perception features in the spatial dimension to obtain spatially enhanced features; The cell feature and the neighboring perception feature are spliced ​​in the channel dimension to obtain a spliced ​​feature, and the spliced ​​feature is added to the spatial enhancement feature to obtain a fusion feature.

7. The pan-cancer cell detection method according to claim 1, wherein: The cell detection and classification network further includes a cell detector and a cell classifier, and obtaining a cell detection map and a cell classification map based on the fusion features includes: Inputting the fusion features into a cell detector to obtain a cell detection map; The fusion features are input into a cell classifier to obtain a cell classification map.

8. The pan-cancer cell detection method according to claim 1, wherein: The obtaining of cell positions and cell categories according to the cell detection map and the cell classification map includes: performing thresholding processing on the cell detection map, calculating the centroid position of each connected area on the cell detection map that is greater than a specified threshold, and using the centroid position as the cell position; A channel of a specified value at the position of the cell is found on the cell classification map, and the cell category is obtained according to the channel.

9. A pan-cancer cell detection system, characterized in that: Including optimization module, feature acquisition module and detection module: an optimization module, configured to obtain training data, obtain a true cell proximity perception map of the training data using a multi-scale proximity perception function, and obtain a predicted cell proximity perception map of the training data using a cell proximity perception map prediction network; calculate a first loss between the true cell proximity perception map and the predicted cell proximity perception map, and optimize the cell proximity perception map prediction network using the first loss; A feature acquisition module is used to acquire detection data, obtain proximity perception features of the detection data using the optimized cell proximity perception map prediction network; obtain cell features using the cell detection and classification network, and fuse the cell features with the proximity perception features using the feature fusion module to obtain fused features; The detection module is used to obtain a cell detection map and a cell classification map according to the fusion feature, and obtain cell positions and cell categories according to the cell detection map and the cell classification map.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when running, executes the pan-cancer cell detection method according to any one of claims 1 to 8.

Images