Pathological cell image classification method and device, electronic equipment and storage medium

Abstract

The invention relates to a pathological cell image classification method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the image collection of a target pathological section, obtaining a stained cell image comprising a plurality of cell instances, and inputting the stained cell image into an image classification model; a position prediction result, a distance prediction result and a first class prediction result are obtained through prediction of a prediction module. And constructing a cell graph structure by extracting morphological characteristics of each cell instance in the stained cell image, and determining graph node attributes corresponding to the cell graph structure according to the position prediction result, the distance prediction result and the first category prediction result, so as to determine a second category prediction result through a graph convolution module. And according to the first category prediction result and the second category prediction result, judging a cell category corresponding to each pixel position in the stained cell image. According to the invention, classification result prediction is carried out on the stained cell image input into the model through two prediction modes, and the classification result is obtained by integrating the two prediction results, so that the accuracy of the classification result is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a pathological cell image classification method, device, electronic device and storage medium. Background Art

[0002] The distribution and proportion of diverse cells such as tumor cells, B cells and T cells are of extremely important scientific value for a deep understanding of the composition of the tumor microenvironment, its evolution mechanism in neoadjuvant therapy, and treatment response. At present, accurate classification of cells on HE (hematoxylin-eosin) images can not only reveal the complexity of the microenvironment of the tertiary lymphoid structure, but also provide a key reference for optimizing neoadjuvant therapy strategies. However, in the case of image-based cell classification through artificial intelligence models, due to technical limitations or sample characteristics, the automatically generated HE image cell labels often carry noise, that is, the classification labels of some cells may be wrong or inaccurate. This has restricted the in-depth research and the accuracy of model training to a certain extent. Summary of the invention

[0003] In view of this, the present disclosure proposes a pathological cell image classification method, device, electronic device and storage medium, aiming to improve the accuracy of the classification results of stained cell images.

[0004] According to a first aspect of the present disclosure, a pathological cell imaging method is provided, the method comprising:

[0005] Performing image acquisition on the target pathological section to obtain a stained cell image including a plurality of cell instances;

[0006] Input the stained cell image into an image classification model, and obtain a position prediction result, a distance prediction result, and a first category prediction result through a prediction module therein, wherein the position prediction result is used to characterize whether each pixel position is the position of a cell instance, and the distance prediction result is used to characterize the distance between each pixel position and the cell centroid in the cell instance;

[0007] Constructing a cell graph structure by extracting the morphological features of each cell instance in the stained cell image, wherein each node in the cell graph structure is used to represent a cell instance in the stained cell image, and the edge is used to represent the neighbor relationship between different cell instances;

[0008] Obtaining graph node attributes corresponding to the cell graph structure by performing feature fusion on the position prediction result, the distance prediction result and the first category prediction result;

[0009] Inputting the cell graph structure and the graph node attributes into a graph convolution module in the image classification model to obtain a second category prediction result;

[0010] The cell category corresponding to each pixel position in the stained cell image is determined according to the first category prediction result and the second category prediction result.

[0011] In a possible implementation, the prediction module includes an encoder, a first decoder, a second decoder, and a third decoder, and the stained cell image is input into an image classification model, and the prediction module is used to predict the position prediction result, the distance prediction result, and the first category prediction result, including:

[0012] Inputting the stained cell image into an image classification model, and encoding the stained cell image through an encoder to obtain an image encoding result;

[0013] The first decoder, the second decoder and the third decoder respectively perform decoding based on the image encoding result to obtain corresponding position prediction results, distance prediction results and first category prediction results.

[0014] In a possible implementation, constructing a cell map structure by extracting morphological features of each cell instance in the stained cell image includes:

[0015] Extracting the morphological features of each cell instance in the stained cell image to obtain a corresponding morphological feature vector;

[0016] Calculate the degree of discreteness between every two cell instances according to the morphological feature vector of each of the cell instances;

[0017] Determine that there is a neighbor relationship between each of the cell instances and a preset number of cell instances with the largest corresponding discreteness;

[0018] The cell graph structure is determined by taking each cell instance as a node and the neighbor relationship as an edge.

[0019] In one possible implementation,

[0020] The degree of dispersion is expressed by the formula Calculated, among which, For characterizing all cell instances included in the stained cell image, L 2 (c i , c j ) are any two cell instances c i and c j The degree of dispersion between il and c jl Characterize cell instances c i and c j The eigenvalue of the lth dimension in the corresponding morphological eigenvector.

[0021] In a possible implementation manner, the morphological features include morphological features and internal texture features.

[0022] In a possible implementation, the obtaining of graph node attributes corresponding to the cell graph structure by performing feature fusion on the position prediction result, the distance prediction result and the first category prediction result includes:

[0023] Performing feature concatenation on the position prediction result, the distance prediction result, and the first category prediction result to obtain a first feature attribute;

[0024] Determine a second feature attribute according to the morphological feature vector corresponding to each of the cell instances;

[0025] The first characteristic attribute and the second characteristic attribute are fused to obtain a graph node attribute.

[0026] In a possible implementation manner, the first category prediction result includes a first category and a corresponding confidence level for each pixel position, and the second category prediction result includes a second category and a corresponding confidence level for each pixel position;

[0027] The step of determining the cell category corresponding to each pixel position in the stained cell image according to the first category prediction result and the second category prediction result includes:

[0028] In response to the first category and the second category of the pixel position being the same, determining that the cell category is the same as the first category and the second category, and the corresponding confidence is an average of the confidence of the first category and the confidence of the second category;

[0029] In response to the first category and the second category of the pixel position being different, it is determined that the cell category is the same as the category with a greater confidence between the first category and the second category, and the confidence is the confidence of the same category.

[0030] In a possible implementation, the method further includes:

[0031] The first category prediction result is corrected based on the position prediction result and the distance prediction result through a watershed algorithm.

[0032] In a possible implementation, the training process of the image classification model includes:

[0033] Determine a training set and a test set including a plurality of sample cell images and corresponding annotated category information;

[0034] Determine a first test set and a second test set according to the test set, wherein the sample cell images in the first test set and the second test set are mutually exclusive;

[0035] Preliminarily training the image classification model based on the training set and the first test set, wherein the sample cell images in the first test set are input into the image classification model, and the corresponding predicted classification results are output;

[0036] In response to the pixel position corresponding to the same target category in the prediction classification results for a consecutive preset number of times and the confidence level being greater than a preset value, and the target category being different from the labeled category of the pixel position in the labeled category information, modifying the labeled category to the target category;

[0037] The image classification model after preliminary training is fine-tuned based on the training set and the second test set.

[0038] In a possible implementation, the determining of a training set and a test set including a plurality of sample cell images and corresponding annotated category information includes:

[0039] Collect multiple pathological sections with corresponding sample cell images and immunofluorescence images;

[0040] Performing alignment processing on the sample cell image and the cell instance in the immunofluorescence image corresponding to each of the pathological sections, aligning the cell instance in the sample cell image with the cell instance in the corresponding immunofluorescence image;

[0041] Determine the corresponding category according to the color of the cell instance in the sample cell image in the immunofluorescence image, and obtain the labeled category information;

[0042] Randomly extract sample cell images and corresponding labeled category information to obtain training sets and test sets.

[0043] According to a second aspect of the present disclosure, a pathological cell imaging device is provided, the device comprising:

[0044] An image acquisition module is used to acquire images of target pathological sections to obtain stained cell images including multiple cell instances;

[0045] A first prediction module is used to input the stained cell image into an image classification model, and obtain a position prediction result, a distance prediction result and a first category prediction result through prediction by the prediction module, wherein the position prediction result is used to characterize whether each pixel position is the position of a cell instance, and the distance prediction result is used to characterize the distance between each pixel position and the cell centroid in the cell instance;

[0046] A graph structure determination module, used for constructing a cell graph structure by extracting morphological features of each cell instance in the stained cell image, wherein each node in the cell graph structure is used to represent a cell instance in the stained cell image, and an edge is used to represent a neighbor relationship between different cell instances;

[0047] An attribute determination module, used for obtaining a graph node attribute corresponding to the cell graph structure by performing feature fusion on the position prediction result, the distance prediction result and the first category prediction result;

[0048] A second prediction module, used for inputting the cell graph structure and the graph node attributes into the graph convolution module in the image classification model to obtain a second category prediction result;

[0049] A cell classification module is used to determine the cell category corresponding to each pixel position in the stained cell image according to the first category prediction result and the second category prediction result.

[0050] In a possible implementation, the prediction module includes an encoder, a first decoder, a second decoder, and a third decoder, and the first prediction module is further configured to:

[0051] Inputting the stained cell image into an image classification model, and encoding the stained cell image through an encoder to obtain an image encoding result;

[0052] The first decoder, the second decoder and the third decoder respectively perform decoding based on the image encoding result to obtain corresponding position prediction results, distance prediction results and first category prediction results.

[0053] In a possible implementation, the graph structure determination module is further used to:

[0054] Extracting the morphological features of each cell instance in the stained cell image to obtain a corresponding morphological feature vector;

[0055] Calculate the degree of discreteness between every two cell instances according to the morphological feature vector of each of the cell instances;

[0056] Determine that there is a neighbor relationship between each of the cell instances and a preset number of cell instances with the largest corresponding discreteness;

[0057] The cell graph structure is determined by taking each cell instance as a node and the neighbor relationship as an edge.

[0058] In one possible implementation,

[0059] The degree of dispersion is expressed by the formula Calculated, among which, For characterizing all cell instances included in the stained cell image, L 2 (c i , c j ) are any two cell instances c i and c j The degree of dispersion between il and c jl Characterize cell instances c i and c j The eigenvalue of the lth dimension in the corresponding morphological eigenvector.

[0060] In a possible implementation manner, the morphological features include morphological features and internal texture features.

[0061] In a possible implementation, the attribute determination module is further configured to:

[0062] Performing feature concatenation on the position prediction result, the distance prediction result, and the first category prediction result to obtain a first feature attribute;

[0063] Determine a second feature attribute according to the morphological feature vector corresponding to each of the cell instances;

[0064] The first characteristic attribute and the second characteristic attribute are fused to obtain a graph node attribute.

[0065] In a possible implementation manner, the first category prediction result includes a first category and a corresponding confidence level for each pixel position, and the second category prediction result includes a second category and a corresponding confidence level for each pixel position;

[0066] The cell classification module is further used for:

[0067] In response to the first category and the second category of the pixel position being the same, determining that the cell category is the same as the first category and the second category, and the corresponding confidence is an average of the confidence of the first category and the confidence of the second category;

[0068] In response to the first category and the second category of the pixel position being different, it is determined that the cell category is the same as the category with a greater confidence between the first category and the second category, and the confidence is the confidence of the same category.

[0069] In a possible implementation manner, the device further includes:

[0070] A result correction module is used to correct the first category prediction result based on the position prediction result and the distance prediction result by using a watershed algorithm.

[0071] In a possible implementation, the training process of the image classification model includes:

[0072] Determine a training set and a test set including a plurality of sample cell images and corresponding annotated category information;

[0073] Determine a first test set and a second test set according to the test set, wherein the sample cell images in the first test set and the second test set are mutually exclusive;

[0074] Preliminary training of the image classification model is performed based on the training set and the first test set, wherein the sample cell images in the first test set are input into the image classification model, and the corresponding predicted classification results are output;

[0075] In response to the pixel position corresponding to the same target category in the prediction classification results for a consecutive preset number of times and the confidence level being greater than a preset value, and the target category being different from the labeled category of the pixel position in the labeled category information, modifying the labeled category to the target category;

[0076] The image classification model after preliminary training is fine-tuned based on the training set and the second test set.

[0077] In a possible implementation, the determining of a training set and a test set including a plurality of sample cell images and corresponding annotated category information includes:

[0078] Collect multiple pathological sections with corresponding sample cell images and immunofluorescence images;

[0079] Performing alignment processing on the sample cell image and the cell instance in the immunofluorescence image corresponding to each of the pathological sections, and aligning the cell instance in the sample cell image with the cell instance in the corresponding immunofluorescence image;

[0080] Determine the corresponding category according to the color of the cell instance in the sample cell image in the immunofluorescence image, and obtain the labeled category information;

[0081] Randomly extract sample cell images and corresponding labeled category information to obtain training sets and test sets.

[0082] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0083] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0084] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0085] In an embodiment of the present disclosure, an image of a stained cell image including multiple cell instances is acquired by image acquisition of a target pathological section and input into an image classification model, and a position prediction result, a distance prediction result, and a first category prediction result are predicted by the prediction module therein. A cell graph structure is constructed by extracting the morphological features of each cell instance in the stained cell image, and the graph node attributes corresponding to the cell graph structure are determined based on the position prediction result, the distance prediction result, and the first category prediction result, so as to determine the second category prediction result through a graph convolution module. Based on the first category prediction result and the second category prediction result, the cell category corresponding to each pixel position in the stained cell image is determined. The present disclosure predicts the classification results of the stained cell image of the input model through two prediction methods, and obtains the classification results by combining the two prediction results, thereby improving the accuracy of the classification results.

[0086] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0088] Figure 1 A flow chart showing a method for classifying pathological cell images according to an embodiment of the present disclosure is shown;

[0089] Figure 2 A schematic diagram showing a method of modifying a first category prediction result according to an embodiment of the present disclosure;

[0090] Figure 3 A schematic diagram showing a preliminary training process of an image classification model according to an embodiment of the present disclosure;

[0091] Figure 4 A schematic diagram showing a pathological cell image classification device according to an embodiment of the present disclosure;

[0092] Figure 5 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0093] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0094] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0095] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0096] The pathological cell image classification method of the embodiment of the present disclosure can be performed by an electronic device such as a terminal device or a server. Among them, the terminal device can be any fixed or mobile terminal such as a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be a single server or a server cluster composed of multiple servers. Any electronic device can implement the pathological cell image classification method of the embodiment of the present disclosure by a processor calling a computer-readable instruction stored in a memory.

[0097] Figure 1 FIG. 5 is a flow chart showing a method for classifying pathological cell images according to an embodiment of the present disclosure. Figure 1 As shown, the pathological cell image classification method of the embodiment of the present disclosure may include the following steps S10-S60.

[0098] Step S10: performing image acquisition on the target pathological section to obtain a stained cell image including a plurality of cell instances.

[0099] In a possible implementation, in a scenario where pathological cells need to be classified, the electronic device can acquire images of target pathological sections obtained after staining the pathological cells to obtain a stained cell image including multiple cell instances. The multiple cell instances included in the stained cell image are all cells stained with cell staining substances such as HE (hematoxylin-eosin).

[0100] Optionally, the electronic device can capture the target pathological slice through a built-in or connected image acquisition device to directly obtain the stained cell image. Alternatively, the target pathological slice can be imaged through an image acquisition device of other devices, and the acquired stained cell image is sent to the electronic device for pathological cell image classification. The format of the stained cell image can be any image format, which is not limited here.

[0101] Step S20: input the stained cell image into an image classification model, and obtain a position prediction result, a distance prediction result and a first category prediction result through the prediction module therein.

[0102] In a possible implementation, after determining the stained cell image, the electronic device inputs the stained cell image into an image classification model, so as to classify each cell instance in the stained cell image through the pre-trained image classification model, and obtain the cell category corresponding to each cell instance. The image classification model may include a prediction module and a graph convolution module, which are respectively used to predict the first category prediction result and the second category prediction result, so as to obtain the final cell category based on the two category prediction results.

[0103] Optionally, after the stained cell image is input into the image classification model, the stained cell image can be predicted by the prediction module therein to obtain the position prediction result, the distance prediction result and the first category prediction result. Among them, the position prediction result is used to characterize whether each pixel position in the stained cell image is the position of the cell instance, the distance prediction result is used to characterize the distance between the pixel position in each stained cell image and the cell centroid in the cell instance, and the first category prediction result is used to characterize the cell instance category of each pixel position in the stained cell image predicted by the prediction module.

[0104] Furthermore, the prediction module may include a deep learning network including an encoder, a first decoder, a second decoder, and a third decoder. In the process of processing the stained cell image, the stained cell image may be input into the image classification model, and the stained cell image may be encoded by the encoder to obtain an image encoding result. The first decoder, the second decoder, and the third decoder may be used to decode the image encoding result respectively to obtain the corresponding position prediction result, distance prediction result, and first category prediction result. Optionally, the encoder may be a RenseNet50 encoder, whose basic unit is constructed by short-circuiting, and the input of each layer depends on the output of the previous layer and the input of the previous layer, which can be obtained by formula X l =H l (X l-1 )+X l-1represents the data processing process of each layer in the encoder, X is the input of the corresponding layer, H represents the nonlinear change of the corresponding layer, and l represents the corresponding layer number. The decoder can be a Dense Unit decoder, whose basic unit adopts dense connection, and the input of each layer is obtained by combining the outputs of all layers before the current layer, that is, it can be obtained by X l =H l ([X 0 , X 1 , ..., X l-1 ]) represents the input of the lth layer, which is the same as the encoder, X is the input of the corresponding layer, H represents the nonlinear change of the corresponding layer, and l represents the corresponding layer number. [X 0 , X 1 , ..., X l-1 ] means combining the output feature maps from the 0th to the (l-1)th layers.

[0105] In a possible implementation, the distance prediction result includes a horizontal distance prediction result and a vertical distance prediction result, which are used to mark the distance between the pixel position in each stained cell image and the cell mass center in the cell instance in the horizontal direction and the vertical direction, respectively. The first category prediction result may also include the first category of each pixel position and the corresponding confidence level, that is, the cell category that each pixel position in the stained cell image may correspond to predicted by the prediction module, and the confidence level used to characterize the possibility of the cell category.

[0106] Furthermore, in the disclosed embodiment, after the electronic device obtains the position prediction result, the distance prediction result and the first category prediction result based on the prediction module, it can correct the first category prediction result to improve the accuracy of the prediction result. The first category prediction result can be corrected based on the position prediction result and the distance prediction result by a watershed algorithm.

[0107] Figure 2 FIG. 2 is a schematic diagram showing a method of modifying a first category prediction result according to an embodiment of the present disclosure. Figure 2As shown, the process of correcting the prediction results of the first category can be to combine the position prediction results and the distance prediction results, and use the marker-controlled watershed algorithm to refine the cell instance segmentation. Specifically, the position prediction results are used as the input of the watershed algorithm, and the watershed algorithm regards the position prediction results as a topographic map, and divides different areas by simulating the immersion process. At the same time, the distance prediction results are used as prior knowledge to control the flow rules of "water" in the watershed algorithm, determine which areas in the stained cell image should be merged or segmented, and obtain more accurate position prediction results to avoid over-segmentation or under-segmentation problems. Further, the prediction results of the first category are corrected according to the optimized position prediction results. The principle of this correction method can be to correct the first category corresponding to the area where the same cell instance is located.

[0108] Step S30, constructing a cell map structure by extracting the morphological features of each cell instance in the stained cell image.

[0109] In a possible implementation, the electronic device can also extract the morphological features of each cell instance from the dimension of the morphological features after determining the stained cell image to construct a cell graph structure for another cell category prediction. Wherein, each node in the cell graph structure is used to characterize a cell instance in the stained cell image, and each edge is used to characterize the neighbor relationship between different cell instances. The neighbor relationship between different cell instances can be determined by the morphological features of each cell instance, and the morphological features in the embodiment of the present disclosure may include morphological features and internal texture features. Morphological features may include area, perimeter, polygonal outline, etc., and internal texture features may include LBP features (Local Binary Pattern, local binary operator), GLCM features (grayscale co-occurrence matrix) and color features, etc.

[0110] Optionally, the electronic device first extracts the morphological features of each cell instance in the stained cell image, and performs feature concatenation on each type of extracted features to obtain a corresponding morphological feature vector. Then, each cell instance can be regarded as a node, and P( Characterize the node features, n is the number of cell instances in the stained cell image, and k is the cell feature vector extracted from each cell instance. The cell graph structure is further constructed based on the spatial relationship between the cell instances through the k-nearest neighbor algorithm. Specifically, the process of the graph structure can be to calculate the degree of discreteness between every two cell instances according to the morphological feature vector of each cell instance, determine that there is a neighbor relationship between each cell instance and the preset number of cell instances with the largest corresponding discreteness, and then use each cell instance as a node and the neighbor relationship as an edge to determine the cell graph structure.

[0111] For example, the electronic device may first Represent all cell instances in the stained cell image, and then use the Euclidean distance to measure the discrete degree L between cell instances 2 (c i , c j ), that is, we can use the formula The discrete degree between every two cell instances is calculated, where For characterizing all cell instances included in the stained cell image, L 2 (c i , c j ) are any two cell instances c i and c j The degree of dispersion between il and c jl Characterize cell instances c i and c i The eigenvalue of the lth dimension in the corresponding morphological feature vector. After calculating the discrete degree between every two cells, the discrete degrees of all cell instances are sorted, and the first k cell instances with the closest discrete degree of each cell instance are selected as the cell instances with a neighbor relationship {c 1 , c 2 , ..., c k}, k is the preset number. i The k nearest neighbors of cell instance c are i and c i If it is a k-nearest neighbor relationship, then add an edge (c i , c j ),

[0112] ε={(c i , c j )|c j ∈{c i1 , c i2 , ..., c ik}ot c i ∈{c j1 , c j2 , ..., c jk}}, finally determine the cell map structure Expressed as

[0113] Step S40: Obtain graph node attributes corresponding to the cell graph structure by performing feature fusion on the position prediction result, the distance prediction result and the first category prediction result.

[0114] In one possible implementation, after determining the cell graph structure, the electronic device can determine the graph node attribute corresponding to the cell graph structure through the position prediction result, distance prediction result and first category prediction result output by the prediction module. Optionally, the graph node attribute can be determined by splicing the above three prediction structures. For example, the position prediction result, the distance prediction result and the first category prediction result can be feature spliced ​​to obtain the first feature attribute, and then the second feature attribute can be determined based on the morphological feature vector corresponding to each cell instance, and finally the first feature attribute and the second feature attribute can be fused to obtain the graph node attribute.

[0115] Exemplarily, the position prediction result, the distance prediction result and the first category prediction result can all be obtained by using the feature map R Characterization, the above three feature maps are concatenated on the channel channel and the feature map R′ is obtained by global pooling and broadcasting As the first feature attribute, it is then combined with the second feature attribute composed of the morphological feature vector corresponding to each cell instance to obtain the graph node attribute P′ Furthermore, we can perform graph propagation learning on the representation of nodes based on the cell graph structure and the corresponding graph node attributes to obtain the corresponding classification results.

[0116] Step S50: input the cell graph structure and the graph node attributes into the graph convolution module in the image classification model to obtain a second category prediction result.

[0117] In a possible implementation, after obtaining the cell graph structure and the corresponding graph node attributes, the cell graph structure and the graph node attributes are input into the graph convolution module in the image classification model for graph propagation, and the representation of the nodes in the cell graph structure is learned to perform category prediction to obtain a second category prediction result. The second category prediction result may include the category predicted by the graph convolution module at each pixel position in the stained cell image as the second category, and the confidence level representing the possibility that each pixel position is the corresponding second category.

[0118] Step S60: Determine the cell category corresponding to each pixel position in the stained cell image according to the first category prediction result and the second category prediction result.

[0119] In one possible implementation, after the electronic device classifies by different dimensions through the prediction module and the graph convolution module, and obtains the first category prediction result and the second category prediction result, it can judge the cell category corresponding to each pixel position in the stained cell image according to the first category prediction result and the second category prediction result. Specifically, the disclosed embodiment can determine that the cell category is the same as the first category and the second category when the first category and the second category of the pixel position are the same, and the corresponding confidence is the average of the confidence of the first category and the confidence of the second category. When the first category and the second category of the pixel position are different, it is determined that the cell category is the same as the category with a greater confidence in the first category and the second category, and the confidence is the confidence of the same category.

[0120] Optionally, the training process of the image classification model for classifying cell instances in the stained cell image in the embodiment of the present disclosure can be a label update training process, that is, the label of the sample image used for training is updated in real time during the training process. Specifically, the electronic device can first determine a training set and a test set including multiple sample cell images and corresponding annotated category information. Then determine the first test set and the second test set according to the test set, and the sample cell images in the first test set and the second test set are mutually exclusive. The image classification model is preliminarily trained based on the training set and the first test set, wherein the sample cell image in the first test set is input into the image classification model, and the corresponding predicted classification result is output. When the target category corresponding to the pixel position in the predicted classification results for a preset number of consecutive times is the same and the confidence is greater than the preset value, and the target category is different from the annotated category of the pixel position in the annotated category information, the annotated category is modified to the target category. Finally, the image classification model after preliminary training is fine-tuned based on the training set and the second test set.

[0121] The process of determining the training set and the test set may include collecting a plurality of pathological sections with corresponding sample cell images and immunofluorescence images. Aligning the sample cell images and cell instances in the immunofluorescence images corresponding to each pathological section, aligning the cell instances in the sample cell images with the cell instances in the corresponding immunofluorescence images. Determining the corresponding category according to the color of the cell instance in the sample cell image in the immunofluorescence image, obtaining the labeled category information, and randomly extracting the sample cell images and the corresponding labeled category information to obtain the training set and the test set.

[0122] Exemplarily, the disclosed embodiment can collect a large number of staining images and fluorescent images corresponding to the pathological sections of patients for cutting and other image preprocessing processes, and obtain the HE staining images of each pathological section corresponding to the preset size as the sample cell image, and the fluorescent image as the immunofluorescence image. The cutting process can be preset, for example, the 20 magnification images of the two images are cut into images of 1000*1000 pixels. Due to the washing of stains and manual operations during the pathological section processing, the image will be offset and misaligned, so the HE staining image and its corresponding immunofluorescence image are not naturally paired. Further, the electronic device can align the sample cell image and the cell instance in the immunofluorescence image corresponding to the same area in the same pathological section by rotation, scaling, translation, etc., and can obtain each cell instance in the sample cell image by segmentation through the existing automatic segmentation system. Optionally, the corresponding classification label is determined according to the color of the corresponding cell instance in the immunofluorescence image corresponding to each cell instance, and the corresponding annotation category information is obtained. Specifically, the color threshold of each color channel corresponding to different categories can be set, and the corresponding classification label is assigned to the cell instance when the cell instance channel value is greater than the corresponding color threshold.

[0123] Furthermore, after obtaining the sample cell images and annotated category information corresponding to each pathological section, the existing sample cell images can be randomly divided into training sets and test sets according to a preset ratio. At the same time, data enhancement operations of slight rotation and translation can be performed on the data to be trained. Optionally, part of the data can be randomly selected from the test set to determine the first test set, and the other data can be used as the second test set. During the training process, the first test set is used as a test set including noise to update the label during the training process, and the second test set is used as a gold standard test set that does not contain any noise. In order to further improve the training effect, after obtaining the first test set, the classification labels of each cell instance in the sample cell image can be manually checked and corrected one by one to correct the incorrectly divided annotated category information due to different staining depths.

[0124] In the process of model training, the image classification model is first preliminarily trained based on the training set and the first test set, wherein the sample cell images in the first test set are input into the image classification model, and the corresponding predicted classification results are output. If the target category corresponding to the predicted classification results of the pixel position is the same for a consecutive preset number of times and the confidence is greater than the preset value, and the target category is different from the annotated category of the pixel position in the annotated category information, the annotated category is modified to the target category. Finally, the preliminarily trained image classification model is fine-tuned based on the training set and the second test set.

[0125] Figure 3FIG. 1 is a schematic diagram showing a preliminary training process of an image classification model according to an embodiment of the present disclosure. Figure 3 As shown, the electronic device can first perform preliminary training on the image classification model based on the training set and the first test set, and when the model loss of the image classification model tends to slowly decrease, and the confidence corresponding to the target category in each predicted classification result no longer increases and tends to be stable. If the target category and corresponding confidence of a cell instance both meet the label update rule, the corresponding target category is replaced with the labeled category information of the original test set.

[0126] Specifically, the updating rule of the labeled category information can be that the label update is started after the image classification model passes the training set and has been trained to a certain extent. That is, when the indicators of the training set and the first test set tend to be stable, the label update is started. In the process of label update, the target category in the predicted classification result of each cell instance in the first test set is recorded during three consecutive iterations. If a cell instance predicts a target category different from the labeled category information for three consecutive times, and the wrong category is the same category, it is further determined whether the corresponding confidence meets the threshold requirement. If the confidence threshold is set to 0.9, when the prediction probability of three consecutive iterations is greater than 0.9, it is considered that the corresponding label in the original labeled category information may be wrong. The corresponding label in the original labeled category information is replaced with the predicted target category. In the next training of the model, the latest updated labeled category information is loaded, and training continues for 3 rounds, and the label update algorithm continues to be used until the training set and the first test set become stable again, that is, the preliminary training is completed. After completing the preliminary training, the image classification model is fine-tuned using the training set and the second test set until the various indicators of the gold standard second test set no longer change and tend to be stable, completing the model training process and obtaining an image classification model that can perform accurate classification.

[0127] Based on the above technical features, the disclosed embodiment predicts the classification results of the stained cell images input to the model through two prediction methods: encoding and decoding, and graph convolution, and obtains the classification results by combining the two prediction results, thereby improving the accuracy of the classification results. At the same time, by correcting the training data by executing the label update algorithm during the model training process, the accuracy of the model training process is improved, and a more accurate image classification model can be obtained through training.

[0128] Figure 4 FIG. 2 is a schematic diagram showing a pathological cell image classification device according to an embodiment of the present disclosure. Figure 4 As shown, the pathological cell image classification device in the embodiment of the present disclosure may include:

[0129] An image acquisition module 40 is used to acquire images of target pathological sections to obtain a stained cell image including a plurality of cell instances;

[0130] A first prediction module 41 is used to input the stained cell image into an image classification model, and obtain a position prediction result, a distance prediction result and a first category prediction result through prediction by the prediction module therein, wherein the position prediction result is used to characterize whether each pixel position is the position of a cell instance, and the distance prediction result is used to characterize the distance between each pixel position and the cell mass center in the cell instance;

[0131] A graph structure determination module 42, configured to construct a cell graph structure by extracting morphological features of each cell instance in the stained cell image, wherein each node in the cell graph structure is used to represent a cell instance in the stained cell image, and an edge is used to represent a neighbor relationship between different cell instances;

[0132] An attribute determination module 43, configured to obtain a graph node attribute corresponding to the cell graph structure by performing feature fusion on the position prediction result, the distance prediction result and the first category prediction result;

[0133] A second prediction module 44, configured to input the cell graph structure and the graph node attributes into a graph convolution module in the image classification model to obtain a second category prediction result;

[0134] The cell classification module 45 is used to determine the cell category corresponding to each pixel position in the stained cell image according to the first category prediction result and the second category prediction result.

[0135] In a possible implementation, the prediction module includes an encoder, a first decoder, a second decoder and a third decoder, and the first prediction module 41 is further configured to:

[0136] Inputting the stained cell image into an image classification model, and encoding the stained cell image through an encoder to obtain an image encoding result;

[0137] The first decoder, the second decoder and the third decoder respectively perform decoding based on the image encoding result to obtain corresponding position prediction results, distance prediction results and first category prediction results.

[0138] In a possible implementation, the graph structure determination module 42 is further configured to:

[0139] Extracting the morphological features of each cell instance in the stained cell image to obtain a corresponding morphological feature vector;

[0140] Calculate the degree of discreteness between every two cell instances according to the morphological feature vector of each of the cell instances;

[0141] Determine that there is a neighbor relationship between each of the cell instances and a preset number of cell instances with the largest corresponding discreteness;

[0142] The cell graph structure is determined by taking each cell instance as a node and the neighbor relationship as an edge.

[0143] In one possible implementation,

[0144] The degree of dispersion is expressed by the formula Calculated, among which, For characterizing all cell instances included in the stained cell image, L 2 (c i , c j ) are any two cell instances c i and c j The degree of dispersion between il and c jl Characterize cell instances separately i and c j The eigenvalue of the lth dimension in the corresponding morphological eigenvector.

[0145] In a possible implementation manner, the morphological features include morphological features and internal texture features.

[0146] In a possible implementation, the attribute determination module 43 is further configured to:

[0147] Performing feature concatenation on the position prediction result, the distance prediction result, and the first category prediction result to obtain a first feature attribute;

[0148] Determine a second feature attribute according to the morphological feature vector corresponding to each of the cell instances;

[0149] The first characteristic attribute and the second characteristic attribute are fused to obtain a graph node attribute.

[0150] In a possible implementation manner, the first category prediction result includes a first category and a corresponding confidence level for each pixel position, and the second category prediction result includes a second category and a corresponding confidence level for each pixel position;

[0151] The cell classification module 45 is further used for:

[0152] In response to the first category and the second category of the pixel position being the same, determining that the cell category is the same as the first category and the second category, and the corresponding confidence is an average of the confidence of the first category and the confidence of the second category;

[0153] In response to the first category and the second category of the pixel position being different, it is determined that the cell category is the same as the category with a greater confidence between the first category and the second category, and the confidence is the confidence of the same category.

[0154] In a possible implementation manner, the device further includes:

[0155] A result correction module is used to correct the first category prediction result based on the position prediction result and the distance prediction result by using a watershed algorithm.

[0156] In a possible implementation, the training process of the image classification model includes:

[0157] Determine a training set and a test set including a plurality of sample cell images and corresponding annotated category information;

[0158] Determine a first test set and a second test set according to the test set, wherein the sample cell images in the first test set and the second test set are mutually exclusive;

[0159] Preliminary training of the image classification model is performed based on the training set and the first test set, wherein the sample cell images in the first test set are input into the image classification model, and the corresponding predicted classification results are output;

[0160] In response to the pixel position corresponding to the same target category in the prediction classification results for a consecutive preset number of times and the confidence level being greater than a preset value, and the target category being different from the labeled category of the pixel position in the labeled category information, modifying the labeled category to the target category;

[0161] The image classification model after preliminary training is fine-tuned based on the training set and the second test set.

[0162] In a possible implementation, the determining of a training set and a test set including a plurality of sample cell images and corresponding annotated category information includes:

[0163] Collect multiple pathological sections with corresponding sample cell images and immunofluorescence images;

[0164] Performing alignment processing on the sample cell image and the cell instance in the immunofluorescence image corresponding to each of the pathological sections, and aligning the cell instance in the sample cell image with the cell instance in the corresponding immunofluorescence image;

[0165] Determine the corresponding category according to the color of the cell instance in the sample cell image in the immunofluorescence image, and obtain the labeled category information;

[0166] Randomly extract sample cell images and corresponding labeled category information to obtain training sets and test sets.

[0167] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0168] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0169] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0170] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0171] Figure 5 1 is a schematic diagram showing an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Figure 5 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0172] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0173] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0174] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0175] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0176] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0177] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0178] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0179] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0180] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0181] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0182] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A pathological cell image classification method, characterized in that: The method comprises: Performing image acquisition on the target pathological section to obtain a stained cell image including a plurality of cell instances; Input the stained cell image into an image classification model, and obtain a position prediction result, a distance prediction result, and a first category prediction result through a prediction module therein, wherein the position prediction result is used to characterize whether each pixel position is the position of a cell instance, and the distance prediction result is used to characterize the distance between each pixel position and the cell centroid in the cell instance; Constructing a cell graph structure by extracting the morphological features of each cell instance in the stained cell image, wherein each node in the cell graph structure is used to represent a cell instance in the stained cell image, and the edge is used to represent the neighbor relationship between different cell instances; Obtaining graph node attributes corresponding to the cell graph structure by performing feature fusion on the position prediction result, the distance prediction result and the first category prediction result; Inputting the cell graph structure and the graph node attributes into a graph convolution module in the image classification model to obtain a second category prediction result; The cell category corresponding to each pixel position in the stained cell image is determined according to the first category prediction result and the second category prediction result.

2. The method according to claim 1, characterized in that The prediction module includes an encoder, a first decoder, a second decoder and a third decoder. The stained cell image is input into the image classification model, and the prediction module predicts the position prediction result, the distance prediction result and the first category prediction result, including: Inputting the stained cell image into an image classification model, and encoding the stained cell image through an encoder to obtain an image encoding result; The first decoder, the second decoder and the third decoder respectively perform decoding based on the image encoding result to obtain corresponding position prediction results, distance prediction results and first category prediction results.

3. The method according to claim 1 or 2, characterized in that: The step of constructing a cell map structure by extracting the morphological features of each cell instance in the stained cell image comprises: Extracting the morphological features of each cell instance in the stained cell image to obtain a corresponding morphological feature vector; Calculate the degree of discreteness between every two cell instances according to the morphological feature vector of each of the cell instances; Determine that there is a neighbor relationship between each of the cell instances and a preset number of cell instances with the largest corresponding discreteness; The cell graph structure is determined by taking each cell instance as a node and the neighbor relationship as an edge.

4. The method according to claim 3, characterized in that The degree of dispersion is expressed by the formula Calculated, where c = {c1, c2, ..., c n } is used to characterize all cell instances included in the stained cell image, L2(c i , c j ) are any two cell instances c i and c j The degree of dispersion between il and c jl Characterize Cell Example C i and C j The eigenvalue of the lth dimension in the corresponding morphological eigenvector.

5. The method according to claim 3 or 4, characterized in that: The morphological features include morphological features and internal texture features.

6. The method according to any one of claims 3 to 5, characterized in that: The method of obtaining the graph node attributes corresponding to the cell graph structure by performing feature fusion on the position prediction result, the distance prediction result and the first category prediction result includes: Performing feature concatenation on the position prediction result, the distance prediction result, and the first category prediction result to obtain a first feature attribute; Determine a second feature attribute according to the morphological feature vector corresponding to each of the cell instances; The first characteristic attribute and the second characteristic attribute are fused to obtain a graph node attribute.

7. The method according to any one of claims 1 to 6, characterized in that: The first category prediction result includes a first category and a corresponding confidence level for each pixel position, and the second category prediction result includes a second category and a corresponding confidence level for each pixel position; The step of determining the cell category corresponding to each pixel position in the stained cell image according to the first category prediction result and the second category prediction result includes: In response to the first category and the second category of the pixel position being the same, determining that the cell category is the same as the first category and the second category, and the corresponding confidence is an average of the confidence of the first category and the confidence of the second category; In response to the first category and the second category of the pixel position being different, it is determined that the cell category is the same as the category with a greater confidence between the first category and the second category, and the confidence is the confidence of the same category.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: The first category prediction result is corrected based on the position prediction result and the distance prediction result through a watershed algorithm.

9. The method according to any one of claims 1 to 8, characterized in that: The training process of the image classification model includes: Determine a training set and a test set including a plurality of sample cell images and corresponding annotated category information; Determine a first test set and a second test set according to the test set, wherein the sample cell images in the first test set and the second test set are mutually exclusive; Preliminarily training the image classification model based on the training set and the first test set, wherein the sample cell images in the first test set are input into the image classification model, and the corresponding predicted classification results are output; In response to the pixel position corresponding to the same target category in the prediction classification results for a consecutive preset number of times and the confidence level being greater than a preset value, and the target category being different from the labeled category of the pixel position in the labeled category information, modifying the labeled category to the target category; The image classification model after preliminary training is fine-tuned based on the training set and the second test set.

10. The method according to claim 9, characterized in that The determination includes a training set and a test set including a plurality of sample cell images and corresponding annotated category information, including: Collect multiple pathological sections with corresponding sample cell images and immunofluorescence images; Performing alignment processing on the sample cell image and the cell instance in the immunofluorescence image corresponding to each of the pathological sections, aligning the cell instance in the sample cell image with the cell instance in the corresponding immunofluorescence image; Determine the corresponding category according to the color of the cell instance in the sample cell image in the immunofluorescence image, and obtain the labeled category information; Randomly extract sample cell images and corresponding labeled category information to obtain training sets and test sets.

11. A pathological cell image classification device, characterized in that: The device comprises: An image acquisition module is used to acquire images of target pathological sections to obtain stained cell images including multiple cell instances; A first prediction module is used to input the stained cell image into an image classification model, and obtain a position prediction result, a distance prediction result and a first category prediction result through prediction by the prediction module, wherein the position prediction result is used to characterize whether each pixel position is the position of a cell instance, and the distance prediction result is used to characterize the distance between each pixel position and the cell centroid in the cell instance; A graph structure determination module, used for constructing a cell graph structure by extracting morphological features of each cell instance in the stained cell image, wherein each node in the cell graph structure is used to represent a cell instance in the stained cell image, and an edge is used to represent a neighbor relationship between different cell instances; An attribute determination module, used for obtaining a graph node attribute corresponding to the cell graph structure by performing feature fusion on the position prediction result, the distance prediction result and the first category prediction result; A second prediction module, used for inputting the cell graph structure and the graph node attributes into the graph convolution module in the image classification model to obtain a second category prediction result; A cell classification module is used to determine the cell category corresponding to each pixel position in the stained cell image according to the first category prediction result and the second category prediction result.

12. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 1 to 10 when executing the instructions stored in the memory.

13. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images