Method for predicting maturity of tertiary lymphoid structures in pathological images based on cell segmentation

Abstract

The embodiment of the application discloses a pathological image three-level lymph structure maturity prediction method based on cell segmentation, and comprises the following steps: acquiring a three-level lymph structure pathological image to be processed; performing cell segmentation on the pathological image by using a cell instance segmentation model based on deep learning to obtain a cell distance map of at least one cell type, wherein the at least one cell type comprises KI67, PANCK, CD3, CD20, CD21, CD23 and DAPI; and processing each cell distance map by using a lymph maturity prediction model based on deep learning to obtain the maturity of the three-level lymph structure. The embodiment improves the accuracy of lymph maturity prediction.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of intelligent medical treatment, and in particular to a pathological image tertiary lymph structure maturity prediction method based on cell segmentation. BACKGROUND

[0002] Tertiary lymph structure is an organized immune cell aggregate formed in non-lymph tissue, which appears in many cancers. Tertiary lymph structure is an important prognostic factor in the immunotherapy of cancer, and among many characteristics of tertiary lymph structure, maturity is considered to affect the prognostic effect of tertiary lymph structure.

[0003] At present, the gold standard for judging the maturity of tertiary lymph structure is an immunofluorescence staining image, and the maturity can be judged according to the cell components of CD21 and CD23 in the tertiary lymph structure. However, the cost of immunofluorescence staining is very high, and the scope of use is limited. In contrast, the most commonly used and lowest cost in clinical practice is H&E (Hematoxylin and Eosin, hematoxylin-eosin) section. Pathologists can also identify the maturity of tertiary lymph structure on H&E, but there is a certain difference between the identification results and immunofluorescence, and it is affected by subjective factors, and there is also a large difference between the identification results of different doctors. Therefore, an objective method is needed to identify the maturity of tertiary lymph structure on H&E section. SUMMARY

[0004] Embodiments of the present application provide a pathological image tertiary lymph structure maturity prediction method based on cell segmentation to solve the above problems.

[0005] In a first aspect, embodiments of the present application provide a pathological image tertiary lymph structure maturity prediction method based on cell segmentation, comprising:

[0006] Obtaining a tertiary lymph structure pathological image to be processed;

[0007] Segmenting cells in the pathological image using a deep learning-based cell instance segmentation model to obtain a cell distance map of at least one cell type, the at least one cell type including KI67, PANCK, CD3, CD20, CD21, CD23, and DAPI;

[0008] Processing each cell distance map using a deep learning-based lymph maturity prediction model to obtain the maturity of the tertiary lymph structure.

[0009] In a second aspect, embodiments of the present application provide an electronic device, comprising:

[0010] One or more processors;

[0011] a memory storing one or more programs,

[0012] When the one or more programs are executed by the one or more processors, the one or more processors implement the cell segmentation-based pathological image tertiary lymphatic structure maturity prediction method described in any embodiment.

[0013] In a third aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the cell segmentation-based pathological image tertiary lymphatic structure maturity prediction method described in any embodiment.

[0014] The embodiments of the present application provide a cell segmentation-based pathological image tertiary lymphatic structure maturity prediction method. First, cell instance segmentation is performed, and then maturity prediction of the tertiary lymphatic structure is performed based on the cell recognition result, so as to improve the accuracy of the maturity prediction of the tertiary lymphatic structure. The embodiments can predict the maturity of the tertiary lymphatic structure with high precision on the H&E stained pathological section, greatly reduce the cost of the tertiary lymphatic maturity determination, and the method based on cell recognition conforms to the biological principle of the maturity definition and has good interpretability. In addition, the embodiments can also provide the corresponding cell segmentation result, which can assist in improving the reliability of the prediction. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0016] Figure 1 is a flowchart of a cell segmentation-based pathological image tertiary lymphatic structure maturity prediction method provided by the embodiments of the present application;

[0017] Figure 2 is a flowchart of another cell segmentation-based pathological image tertiary lymphatic structure maturity prediction method provided by the embodiments of the present application;

[0018] Figure 3 is a schematic diagram of cell instance segmentation provided by the embodiments of the present application;

[0019] Figure 4 is a schematic diagram of a cell instance segmentation result obtained by a watershed algorithm provided by the embodiments of the present application;

[0020] Figure 5is a flowchart of another cell segmentation-based pathological image tertiary lymphoid structure maturity prediction method provided by an embodiment of the present application;

[0021] Figure 6 is a schematic diagram of cell labeling provided by an embodiment of the present application;

[0022] Figure 7 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0024] In the description of the present application, it should be noted that the orientations or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0025] In the description of the present application, it should also be noted that, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0026] Figure 1 is a flowchart of a cell segmentation-based pathological image tertiary lymphoid structure maturity prediction method provided by an embodiment of the present application. The method is suitable for predicting the maturity of the tertiary lymphoid structure through H&E sections (also known as H&E pathological images), and is executed by an electronic device.

[0027] As shown in Figure 1 , the method specifically includes:

[0028] S110, acquiring a tertiary lymphoid structure pathological image to be processed.

[0029] The pathological image here refers to an H&E pathological image, as shown in FIG. 1, which is the data source of the entire method. Figure 2

[0030] S120, performing cell segmentation on the pathological image by using a cell instance segmentation model based on deep learning to obtain a cell distance map of at least one cell type.

[0031] The cell type here refers to a cell type related to a tertiary lymphoid structure, which can optionally include KI67 (Kiel67), PANCK (pan-Cytokeratin), CD3 (Cluster of differentiation 3), CD20 (Cluster of differentiation 20), CD21 (Cluster of differentiation 21), CD23 (Cluster of differentiation 23), and DAPI (4', 6-diamidino-2-phenylindole) and other immune cell subtypes. In the cell distance map, the pixel value in each cell represents the Euclidean distance from the point to the nearest point outside the cell, so the value of the cell center in the cell distance map will be larger, and the value of the cell edge will be smaller. Optionally, the pixel value in the cell distance map can be normalized to 0-1. This step uses the trained cell instance segmentation model to process the pathological image to obtain the cell distance map of each subtype cell as the basis for predicting the maturity of the tertiary lymphoid structure.

[0032] Optionally, the backbone of the cell instance segmentation model adopts a U-Net architecture, including an encoder and a decoder, as shown in FIG. 2. In a specific embodiment, the cell instance segmentation model uses ResNet-50 to replace the original encoder of CiscNet, and removes the first convolutional layer of ResNet-50, because ResNet will immediately down-sample the image by 4 times when input, which is not suitable for fine segmentation tasks such as cell segmentation that require high resolution. At the same time, using ResNet-50 is deeper than the original encoder network of CiscNet, and is more suitable for batch processing of pathological images in practical applications. The decoder can use deconvolutional layers for up-sampling. Figure 3

[0033] Optionally, for a pathological image of a tertiary lymphoid structure, it can be first cut into image blocks of the same size as the cell instance segmentation model, for example, Figure 3 ​​The encoder input size shown is 256*256, and the pathological image is cut into 256*256 image blocks. Then, the cell distance map prediction is performed on each image block using the cell instance segmentation model described above, and finally the cell distance maps of 7 cell subtypes and the cell distance map of the cell nucleus are obtained, which are collectively used as the basis for lymph structure maturity prediction. In particular, the distance map of the cell nucleus contains all cells, including cells other than the above-mentioned 7 subtypes. When using the model, all cells can be found according to the identification result of the cell nucleus, and the specific type of the cell can be determined according to the subtype identification result, so the cell nucleus distance map is also very important. At the same time, after obtaining the cell distance map, the watershed algorithm can be used to segment each type of cell to obtain the final cell instance segmentation result, such as Figure 4 shown.

[0034] S130, processing each cell distance map using a deep learning-based lymph maturity prediction model to obtain the maturity of the three-level lymph structure.

[0035] As described above, if the pathological image is cut in S120, the cell distance maps of the same type are first spliced in this step to obtain the cell distance map of the entire H&E image. Then, each type of cell distance map is scaled to a size (for example, 512*512) matching the lymph maturity prediction model, and is input into the lymph maturity prediction model for maturity prediction. Optionally, the task performed by the maturity prediction model is a classification task, and the output includes three maturity levels, namely early, primary, and secondary. The following provides two optional implementation modes for the structure and data processing process of the lymph maturity prediction model:

[0036] The first optional implementation mode is that the backbone model of the lymph maturity prediction model can use a convolutional neural network (CNN), such as ResNet-18, and the input channels of the first layer of the ResNet-18 network are modified to be consistent with the number of cell distance maps, and a fully connected network is connected behind as a classifier. For example, in the case of 8 cell distance maps obtained in S120, the input channels of the first layer of the ResNet-18 network can be modified to 8, and the input size is 512*512*8.

[0037] Correspondingly, the prediction process of the maturity includes: inputting each cell distance map into each channel of the improved ResNet-18 to extract deep cell features; inputting the deep cell features into the fully connected network for dimension reduction, and classifying the maturity of the three-level lymph structure.

[0038] In a second alternative implementation, the lymphatic maturity prediction model comprises a cell branch and a tissue branch. The cell branch is configured to predict the lymphatic maturity based on cell features. Optionally, the structure of the cell branch can be the same as the cell instance segmentation model in the first alternative implementation. The tissue branch is configured to predict the lymphatic maturity based on tissue features. The backbone structure of the tissue branch can employ a graph-based neural network (GNN), such as a graph attention network, followed by a fully connected network as a classifier. It should be noted that the fully connected networks in the two branches are independent of each other and run separately.

[0039] Based on the above model structure, in combination with Figure 5 the prediction process of the maturity can include the following steps:

[0040] For the cell branch, the cell distance map is input into the branch to extract deep cell features, and the maturity of the tertiary lymphatic structure is classified based on the deep cell features. The specific data processing procedure is the same as that in the first alternative implementation, which will not be described here.

[0041] For the tissue branch, first, the tissue structure graph of the pathological image is constructed by the following steps:

[0042] Step 1: Use a feature extraction network to extract the tissue structure graph of the pathological image. Optionally, the encoder in the above cell instance segmentation model is reused as the feature extraction network here, and the last layer features of the encoder are extracted as the cell feature map of the pathological image.

[0043] Step 2: Perform superpixel clustering on the pathological image to obtain superpixel blocks representing each tissue region. Optionally, unsupervised tissue segmentation is performed, and simple linear iterative clustering is used to convert the cell feature map into uniform superpixels.

[0044] Step 3: Take each superpixel block as a node, and generate edges between the nodes of adjacent superpixel blocks to obtain a tissue structure graph representing the relationship between tissue regions. Optionally, adjacent superpixels are merged and the same color is used to represent the same tissue region. Each tissue region can be understood as a type of tissue. The region adjacency graph method is applied to generate edges of the image, i.e., if two regions are adjacent in space (share a common edge), an edge is connected between the nodes of the two regions to obtain a tissue structure graph with nodes and edges. The node features of the graph are the average values of the corresponding regions of the cell feature map in Step 1, representing the cell composition of the tissue region. The graph can represent the position and connection relationship between different tissues, providing a global perspective for predicting the maturity of the tertiary lymphatic structure.

[0045] Then, the tissue structure graph is input into a graph attention network to extract deep tissue features. The graph attention network is composed of three graph attention layers and adopts LeakyReLU (leaky rectified linear unit) as an activation function. The graph attention network can aggregate regional features in the tissue structure graph, capture spatial relationships and global topological information between tissue regions, and realize the conversion from cell features to tissue features. Finally, the deep tissue features are input into a fully connected layer for dimension reduction, and the deep cell features after dimension reduction are input into a multilayer perceptron to classify the maturity of the three-level lymphatic structure.

[0046] Meanwhile, the deep cell features and the deep tissue features are fused, and the fused features are used to classify the maturity of the three-level lymphatic structure. According to the results of the three classifications, the final maturity of the three-level lymphatic structure is determined. Optionally, a voting method can be used for fusion, and the majority result is the final result. Of course, the results of the three classifications can also be constrained during the network training stage, and only the fused features are used for classification during the prediction stage, and the present embodiment does not make specific limitations.

[0047] In summary, the present embodiment provides a three-level lymphatic structure maturity prediction method based on cell segmentation. First, cell instance segmentation is performed, and then the maturity of the three-level lymphatic structure is predicted based on the cell recognition results to improve the accuracy of three-level lymphatic structure maturity prediction. The present embodiment can predict the maturity of the three-level lymphatic structure with high precision on H&E stained pathological sections, greatly reducing the cost of three-level lymphatic maturity determination. The method based on cell recognition conforms to the biological principles of maturity definition and has good interpretability. In addition, the present embodiment can also provide corresponding cell segmentation results to assist in improving the reliability of prediction.

[0048] On the basis of the above-mentioned embodiments, the training process of each model is refined in the present embodiment. In a specific implementation, the training process can include the following steps:

[0049] Step one, obtain a plurality of three-level lymphatic structure pathological images and corresponding immunofluorescence images. Optionally, first, multiple immunofluorescence staining is performed on the tissue sections of lung cancer, using KI67, PANCK, CD3, CD20, CD21, CD23 and DAPI markers, and scanning under 10X magnification. Then, for the same slice, H&E staining is performed, and scanning is performed under 40X magnification. Here, no decolorization is required because the interaction between H&E reagent and fluorescent group is small. At this point, the H&E staining image and the corresponding immunofluorescence image of the same tissue section can be obtained.

[0050] Step 2: Based on each immunofluorescence staining image, label at least one cell type and the maturity of tertiary lymphoid structures in each pathological image. Optionally, for each H&E image, select the location of its tertiary lymphoid structures and the corresponding immunofluorescence section. Based on the CD21 and CD23 staining results of the immunofluorescence sections, label the maturity of the tertiary lymphoid structures, i.e., three maturity levels: early, primary, and secondary. In addition, cell type labeling can also be performed. Referring to the immunofluorescence, label the tertiary lymphoid structures and surrounding cells as 7 cell types and delineate the cell boundaries. A multi-label setting can be used, i.e., the same cell can belong to multiple categories. After labeling, the image is cut into 256*256 image blocks (matching the input size of the cell instance segmentation model). This results in two datasets: a cell instance segmentation dataset and a tertiary lymphoid structure maturity classification dataset.

[0051] Step 3: Train the deep learning-based cell instance segmentation model using the labeled pathological images. The trained model takes the pathological images as input and outputs cell distance maps of at least one cell type within the pathological images. Optionally, firstly, as... Figure 6 As shown, cell segmentation labels are converted into cell distance maps, and the pixels in the map are normalized to between 0 and 1 to generate labels for the cell segmentation task. Optionally, the cell distance map includes 8 channels (7 cell types plus 1 nucleus channel). During training, Ranger is used as the optimizer with an initial learning rate of 6e-3. During training, if the loss on the validation set does not decrease for 10 consecutive epochs, the learning rate is reduced to one-third of its original value. The training is performed for a total of 300 epochs with a batch size of 32. Data augmentation can also be performed during training using random flipping, random rotation, random brightness changes, and random contrast changes.

[0052] Step four, using the trained cell instance segmentation model to segment cells in each labeled pathological image; using the segmentation result to train a lymphoid maturation prediction model based on deep learning, and the trained lymphoid maturation prediction model takes at least one cell distance map of a cell type of a three-level lymphoid structure pathological image as input and takes the maturity of the three-level lymphoid structure as output. Optionally, for the H&E image of the whole three-level lymphoid structure, it is cut into non-overlapping image blocks of 256*256, and the trained cell instance segmentation model is used to predict the cell distance map for each image block. The prediction result is spliced to obtain the cell distance map prediction result of the whole H&E image, and is scaled to 512*512*8 input lymphoid maturation prediction model. In the training, the cross-entropy function can be used, the ResNet-18 network can be initialized using the pre-trained weight of ImageNet, and various data enhancement methods can be used, including random flipping, random rotation, affine transformation, etc. The optimizer used in the training is Adam, the learning rate is fixed at 0.0003, and the iteration number is 50 rounds. In Figure 4 the model structure, the following loss function can be used:

[0053] L maturation =a×L cell +b×L tissue +c×L fusion

[0054] Wherein, L maturation represents the total loss, L cell , L tissue and L fusion represent the loss predicted by the cell branch, the tissue branch and the fusion feature respectively, and a, b and c represent the corresponding weights.

[0055] In summary, the embodiment overcomes two main difficulties of three-level lymphoid structure maturity prediction: first, the morphological difference between three-level lymphoid structures of different maturity types is small, even a pathologist is difficult to accurately judge the maturity on the H&E slice; second, due to the high price of immunofluorescence staining, it is difficult to obtain enough labeled samples, so the task has a small sample problem. The present application converts the problem into a multi-sample problem by identifying cells, and automatically extracts features using deep learning methods, thereby improving the accuracy of three-level lymphoid structure identification. In more detail, because the number of three-level lymphoid structures is small, but the number of cells is large enough for the deep learning model to train, the model is first trained as a proxy task to identify cells, and then migrated to the maturity prediction problem.

[0056] Figure 7 A structural schematic diagram of an electronic device provided by the embodiment of the present application is shown in Figure 7As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more, Figure 7 The processor 60 in the device is taken as an example in the embodiment of the present application; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected through a bus or other means, Figure 7 The connection through the bus is taken as an example in the embodiment of the present application.

[0057] The memory 61 is a kind of computer readable storage medium, which can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the pathological image three-lymph structure maturity prediction method based on cell segmentation in the embodiment of the present application. The processor 60 executes the software programs, instructions and modules stored in the memory 61, thereby performing various functional applications and data processing of the device, i.e. implementing the pathological image three-lymph structure maturity prediction method based on cell segmentation.

[0058] The memory 61 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 61 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the memory 61 can further include a memory remotely arranged with respect to the processor 60, which can be connected to the device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0059] The input device 62 can be used to receive input digital or character information, and generate key signal input related to user settings and function control of the device. The output device 63 can include a display device such as a display screen.

[0060] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the pathological image three-lymph structure maturity prediction method based on cell segmentation of any embodiment.

[0061] The computer storage medium of the embodiments of the present application can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the computer-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0062] The computer-readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the computer-readable program code is contained. Such propagated data signal can take a variety of forms, including but not limited to electro-magnetic, optical or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can be used to carry or propagate program code for use by or in connection with an instruction execution system, apparatus or device.

[0063] The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0064] The computer program code for carrying out operations of the present application can be written in one or more programming languages or combinations of languages including an object oriented programming language such as Python, Java, C++ or conventional procedural programming languages such as C or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0065] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application.

Claims

1. A method for predicting the maturity of three-level lymphoid structures in pathological images based on cell segmentation, characterized in that, include: Obtain pathological images of the tertiary lymphoid structures to be processed; The pathological image is segmented using a deep learning-based cell instance segmentation model to obtain a cell distance map of at least one cell type, wherein the at least one cell type includes KI67, PANCK, CD3, CD20, CD21, CD23 and DAPI; The maturity of the three-level lymphatic structure is obtained by processing the cell distance map using a deep learning-based lymphatic maturity prediction model. Specifically, the cell distance map is input into the cell branch of the lymphatic maturity prediction model to extract deep cell features. The maturity of the tertiary lymphoid structures is classified using the deep cell features; a tissue structure map of the pathological image is constructed, and the tissue structure map is input into the tissue branch of the lymphoid maturity prediction model to extract deep tissue features; The maturity of the tertiary lymphoid structures is classified using the aforementioned deep tissue features. The deep cellular features and the deep tissue features are fused together, and the maturity of the tertiary lymphoid structures is classified using the fused features. Based on the results of the three classifications, the final maturity of the tertiary lymphoid structures is determined.

2. The method according to claim 1, characterized in that, The process of segmenting the pathological image using a deep learning-based cell instance segmentation model to obtain a cell distance map for at least one cell type includes: The pathological image is segmented using a deep learning-based cell instance segmentation model to obtain cell distance maps of at least one cell type and cell distance maps of cell nuclei.

3. The method according to claim 1, characterized in that, Before performing cell segmentation on the pathological image using a deep learning-based cell instance segmentation model, the method further includes: Multiple pathological images of tertiary lymphoid structures and corresponding immunofluorescence staining images were obtained; Based on each immunofluorescence staining image, at least one cell type is labeled for each pathological image; A deep learning-based cell instance segmentation model is trained using the labeled pathological images. The trained model takes the pathological images as input and outputs cell distance maps of at least one cell type in the pathological images.

4. The method according to claim 1, characterized in that, The lymph node maturity prediction model includes an improved ResNet-18 network and a fully connected network, wherein the improved ResNet-18 modifies the input channels of the first layer of the ResNet-18 network to be consistent with the number of cell distance maps; The process of using a deep learning-based lymph node maturity prediction model to process the cell distance map yields the maturity of the tertiary lymph node structure, including: Each cell distance map is input into each channel of the improved ResNet-18 to extract deep cell features; The deep cell features are input into the fully connected network for dimensionality reduction, and the maturity of the tertiary lymphoid structures is classified.

5. The method according to claim 1, characterized in that, The tissue structure diagram for constructing the pathological image includes: Cell feature maps of the pathological images are extracted using a feature extraction network; Superpixel clustering is performed on the pathological images to obtain superpixel blocks that characterize each tissue region; Using each superpixel block as a node, edges are generated between nodes of adjacent superpixel blocks to obtain a tissue structure diagram that characterizes the relationship between tissue regions. The node features of the diagram are the average value of the corresponding region of each superpixel in the cell feature diagram, representing the cellular components of the region.

6. The method according to claim 5, characterized in that, The cell instance segmentation model includes an encoder and a decoder; The step of extracting cell feature maps from the pathological images using a feature extraction network includes: The last layer of features from the encoder is extracted as the cell feature map of the pathological image.

7. The method according to claim 1, characterized in that, Before processing the cell distance map using a deep learning-based lymph node maturity prediction model to obtain the maturity of the tertiary lymph node structure, the following steps are also included: Multiple pathological images of tertiary lymphoid structures and corresponding immunofluorescence staining images were obtained; Based on each immunofluorescence staining image, at least one cell type and the maturity of tertiary lymphoid structures in each pathological image were labeled; The deep learning-based cell instance segmentation model is trained using the labeled pathological images. The trained cell instance segmentation model takes the pathological images as input and outputs a cell distance map of at least one cell type in the pathological images. The trained cell instance segmentation model is used to segment cells in each labeled pathological image. The segmentation results are used to train a deep learning-based lymph node maturity prediction model. The trained lymph node maturity prediction model takes the cell distance map of at least one cell type in the pathological image of the tertiary lymph node structure as input and the maturity of the tertiary lymph node structure as output.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method for predicting the maturity of three-level lymphoid structures in pathological images based on cell segmentation as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method for predicting the maturity of three-level lymphoid structures in pathological images based on cell segmentation as described in any one of claims 1-7.

Images