Pathological prognosis modeling method based on intra-tumor nerve neighborhood cell level graph convolution

Abstract

The invention provides a pathological prognosis modeling method based on intra-tumor nerve neighborhood cell level graph convolution, which comprises the following steps of: extracting nerve areas of a nerve structure, and marking coordinates of neuropathology slices in a pathological image one by one; performing neighborhood expansion operation on the nerve region, and reserving an expansion result of the intersection with the tumor tissue to obtain an intra-tumor nerve neighborhood; and outputting the cell set, constructing a pathological cell image set, and inputting the pathological cell image set into the prediction model to obtain prognosis prediction. By adopting the method, the limitation of subjective dependence and binary judgment of traditional manual film reading is avoided, and high accuracy and objectivity of a detection result are ensured. The accuracy of risk prediction is improved, and the speed of detection and prognosis evaluation is also greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to pathological prognosis modeling, in particular to a pathological prognosis modeling method based on intratumoral nerve neighborhood cell level graph convolution. BACKGROUND

[0002] Head and neck squamous cell carcinoma (HNSCC) is the sixth most common cancer worldwide. The high density of head and neck peripheral nerve network makes head and neck squamous cell carcinoma have rich and complex nerve features. In recent years, studies have shown that the tumor peripheral nerve microenvironment plays a key role in the occurrence of peripheral nerve invasion (PNI). Intratumoral nerves are not only passive channels for tumor cell invasion, but also further promote tumor growth, invasion and malignant evolution through various regulatory mechanisms such as secretion of neurotransmitters, neurotrophic factors and nerve electrical signals. In addition, Schwann cells (SCs) are one of the most important types of glial cells in the tumor peripheral nerve microenvironment. When tumor cells invade and cause nerve damage, Schwann cells can be activated and chemotactically migrate to the periphery of tumor cells, thereby constructing a potential pathway for early metastasis. In head and neck squamous cell carcinoma, peripheral nerve invasion has been established as a classic pathological indicator of its invasiveness.

[0003] Although peripheral nerve invasion is considered a classic pathological prognostic indicator of head and neck squamous cell carcinoma, it has obvious limitations in practice, especially in the following aspects:

[0004] 1. Subjectivity of traditional pathological indicators: Current classic PNI indicators rely heavily on the subjective judgment of pathologists, and only consider the status of PNI (i.e., observing random multiple fields to detect whether there are intratumoral nerve structures, and determining the wrapping of tumor around nerve structures) in clinical practice. However, such a rough binary determination method not only makes it difficult to accurately quantify the strength of tumor and nerve interaction, but also lacks sufficient repeatability and objectivity.

[0005] 2. Lack of efficient and fine prognosis detection tools: Current PNI evaluation relies on traditional staining and manual reading, which is a cumbersome process with low throughput. A single indicator is difficult to reflect the complex interaction of tumor nerve microenvironment, resulting in insufficient accuracy of risk stratification prediction. Existing models also ignore spatial heterogeneity and invasion depth, and lack quantitative capture of PNI micro-level.

[0006] Therefore, there is an urgent need for a pathological prognosis modeling method based on intratumoral nerve neighborhood cell level graph convolution. SUMMARY

[0007] To overcome the existing technical problems, the present application provides a pathological prognosis modeling method based on intratumoral nerve neighborhood cell level graph convolution.

[0008] The present application adopts the following technical solutions.

[0009] A pathological prognostic modeling method based on intratumoral neural neighborhood cell-level graph convolution includes the following steps:

[0010] The pathological images are preprocessed to obtain multiple standardized pathological slides. Tumor tissue is identified and the presence of neural structures in each standardized pathological slide is determined by a pathological slide classification model. If so, the neural regions of the neural structures are extracted, and the standardized pathological slide is regarded as a neuropathological slide. All neuropathological slides are merged, and the coordinates of each neuropathological slide in the pathological image are marked.

[0011] Neighborhood expansion is performed on the neural region, preserving the expansion results that intersect with the tumor tissue, to obtain the intratumoral neural neighborhood M. ROI ;

[0012] Each cell in the neuropathological slide was identified and segmented using a detection segmentation model, and the category of each cell was identified using a category recognition model. Cells located in the intratumoral neural neighborhood M were then screened. ROI The cells inside, the output cell set C ROI ;

[0013] According to cell set C ROI Constructing a pathological cell atlas G (i) , pathological cell atlas G (i) Input the GCN prediction model to obtain the prognostic prediction.

[0014] As a further improvement of the present invention, the specific steps of preprocessing include: performing HE staining and manual annotation on the pathological image, delineating candidate regions of nerves within the tumor, cropping the pathological image into multiple pathological slides, and performing staining standardization using the Macenko method.

[0015] As a further improvement of the present invention, the pathological slide classification model is a binary classification model built based on ResNet-101;

[0016] The pathological slide classification model is based on a label setting method, which assigns binary labels to all standardized pathological slides within the candidate region delineating intratumoral nerves as input for model training data.

[0017] The specific steps of the label setting method include: manually annotating the neural structure mask, calculating the pixel ratio of neural structures in each standardized pathological slide, and labeling the standardized pathological slide as positive if the pixel ratio is greater than or equal to 50% and as negative if the pixel ratio is less than 50%.

[0018] As a further improvement of the present application, the specific step of judging whether each standardized pathological section has nerve structure is: predicting each standardized pathological section, outputting the probability value of each standardized pathological section belonging to nerve structure, if the probability value is greater than the dynamic nerve judgment threshold δ i , then judging that the standardized pathological section has nerve structure;

[0019]

[0020] Wherein, M i is the tumor tissue shielding impact score of the i-th standardized pathological section, D tumo r is the pixel number of tumor tissue in the i-th standardized pathological section, D 总 is the total pixel number of the i-th standardized pathological section, β1 is the shielding conversion factor, M max is the maximum impact threshold of tumor tissue shielding, Z i is the spot inspection model confidence score, n k is the number of the same spot inspection prediction results in the last k times, N k is the total number of the last k times of spot inspection prediction results, η min is the minimum prediction threshold of nerve structure, α1 and α2 are nerve judgment weight factors.

[0021] As a further improvement of the present application, the specific steps of performing neighborhood dilation operation on the nerve region include:

[0022] The coordinate set of the neuropathological section in the pathological image is: P nerve ={p1, p2, ···, p n}, the binary mask of each neuropathological section p i is M(p i ), a plurality of nerve structures are divided by edge continuity algorithm, and the total nerve mask M nerve ,

[0023]

[0024] Wherein, M(p i )=1 indicates that the pixel is a nerve region, and M(p i )=0 indicates that the pixel is background.

[0025] Respectively, the two-dimensional morphological dilation M dilated is performed to obtain a plurality of dilation masks,

[0026]

[0027] Wherein, represents the dilation operation, S rIt is a two-dimensional spherical structure with a radius r.

[0028] As a further improvement of the present invention, the expansion result at the intersection with the tumor tissue is retained to obtain the intratumoral neural neighborhood M. ROI The specific steps include: performing aggregation processing on multiple expansion masks and tumor tissue masks to obtain the intratumoral neural neighborhood M. ROI ,

[0029]

[0030] in, Let M represent the j-th neural structure. tumor It is a mask for tumor tissue. Clip() means to preserve the part where the expansion mask and the tumor tissue mask intersect.

[0031] As a further improvement of the present invention, while identifying and segmenting each cell in the neuropathological slide using a detection segmentation model, a binary mask for the corresponding cell is generated. and centroid coordinates (x) k ,y k );

[0032] The specific steps for generating centroid coordinates include:

[0033]

[0034] in, This indicates that the numerical value in the binary mask represents the total number of pixels in that cell;

[0035] Screening for M located in the intratumoral neural neighborhood ROI The cells inside, the output cell set C ROI The specific steps include:

[0036] If the centroid coordinates of the kth cell are (x) k ,y k ) Falling into the nerve neighborhood of the tumor M ROI If the cell falls within the neural neighborhood of the tumor, then the output cell set C is considered to be located within the tumor. ROI ,

[0037]

[0038] in, It is the cell category of the k-th cell. It is the binary mask information of the k-th cell.

[0039] As a further improvement to the present invention, cell collection C ROI This includes the centroid coordinates of each cell, cell type, and binary mask information;

[0040] According to cell set CROI Constructing a pathological cell atlas G (i) The specific steps include: treating cells of the same cell type as a node, and calculating the Euclidean distance W between any two nodes. ij If the Euclidean distance W ij Less than the preset dynamic construction threshold W t If these two nodes are connected by an edge, then a pathological cell graph G corresponding to multiple cell types can be constructed based on the nodes and edges. (i) ;

[0041] Dynamically constructing the threshold W t The expression is:

[0042]

[0043] Among them, R k N is the average size of this cell type. i N represents the total number of pixels counted using the binary mask information for this cell category. k R represents the total number of cells of this type. 基 It is the size value of the baseline cell, L 基 This is the base distance value.

[0044] As a further improvement of the present invention, the specific steps for inputting the GCN prediction model to obtain prognostic predictions include: performing cell density ratio analysis on cells of the same cell type. calculate,

[0045]

[0046] in, A is the number of cells of the i-th cell type. ROI It is the area calculation of the neural neighborhood within the tumor, ρ i It is the density weighting correction factor for the i-th cell type;

[0047] Tumor cells located at the edge of the tumor tissue are extracted to obtain marginal tumor cells. The straight-line distance between the marginal tumor cell and the nearest nerve cell is calculated. If the straight-line distance If the value is less than the preset diffusion threshold, then the pair of marginal tumor cells, nerve cells, the edges connected to the marginal tumor cells, and the edges connected to the nerve cells are extracted and integrated to form a diffusion propagation set.

[0048] Calculate the maximum directional similarity τ between each pair of marginal tumor cells and nerve cells in the diffusion propagation set. jk ,

[0049] u≠v;

[0050] in, is an edge direction vector of the jth edge tumor cell connected with the u th tumor cell in the diffusion propagation set, is an edge direction vector of the k th nerve cell connected with the v th tumor cell, the k th nerve cell and the jth edge tumor cell are a pair;

[0051] Calculate the risk diffusion index δ spread ,

[0052]

[0053] wherein ξ dir is the direction consistency index, N t is the total number of edge tumor cells or nerve cells in the diffusion propagation set, ξ dif is the risk diffusion distance index, is the straight line distance of the ith pair of edge tumor cells and nerve cells, η i is the diffusion distance dynamic weight coefficient, λ f is the distance adjustment factor, γ 1 and γ 2 are risk diffusion weight factors, ξ den is the density propagation index, is the cell density ratio of tumor cells, is the cell density ratio of nerve cells, and γ 3 is the scale conversion weight factor of the cell density ratio.

[0054] The risk diffusion index δ spread is compared with a plurality of risk value intervals, and the corresponding risk level and prognosis report are output.

[0055] The beneficial effects of the present application are: by performing a neighborhood inflation operation on the neural region and retaining the inflation result intersecting with the tumor tissue, the intratumoral neural neighborhood is obtained, which can capture the spatial relationship between the tumor and the nerve, avoid the subjective dependence and the limitations of binary determination of traditional manual film reading, and ensure the high accuracy and objectivity of the detection result. At the same time, combined with the pathological section classification model, the detection segmentation model, the category recognition model and the GCN prediction model, the cell set located in the intratumoral neural neighborhood can be screened out, and the pathological cell atlas can be constructed, the distance, position and type distribution between cells are quantified, the fine modeling of the complex interaction of the tumor nerve microenvironment is realized, not only the accuracy of risk prediction is improved, but also the speed of detection and prognosis evaluation is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below, and obviously, other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0057] Figure 1 is the main flow chart of the present application. DETAILED DESCRIPTION

[0058] The accompanying drawings are only used for illustrative purposes and cannot be understood as limiting the patent; in order to better illustrate the embodiments, some components of the drawings may be omitted, enlarged or reduced, and do not represent the actual size of the product.

[0059] It is understandable to some skilled persons in the art that some known structures and their descriptions in the drawings may be omitted. The technical solutions of the present application will be further described below in combination with the drawings and embodiments.

[0060] Referring to Figure 1 A tumor intraneural neighborhood cell level graph convolution-based pathological prognosis modeling method includes the following steps:

[0061] The pathological images are preprocessed to obtain a plurality of standardized pathological slides, tumor tissues are identified, and a pathological slide classification model is used to determine whether each standardized pathological slide contains a neural structure. If so, the neural region of the neural structure is extracted, and the standardized pathological slide is regarded as a neural pathological slide. All neural pathological slides are combined and labeled with the coordinates of the neural pathological slide in the pathological image.

[0062] In one embodiment of the present application, the method of identifying tumor tissues is similar to the method of extracting neural regions of neural structures by a pathological slide classification model, that is, the model is used to determine whether there is tumor tissue, and if so, the tumor tissue is extracted. However, the subsequent tumor tissue is determined and output by using a fixed threshold. Since the method of extracting specific regions by a model is very common, such as the patent with the authorization announcement number CN119784634B, the present application will not be described in more detail. In addition, the identification of tumor tissues obtains the mask information of tumor tissues.

[0063] As a further improvement of the present application, the specific steps of preprocessing include: manually annotating the pathological images by HE staining, delineating the candidate region of intratumoral nerves, cutting the pathological images into a plurality of pathological slides, and standardizing the staining by the Macenko method.

[0064] Delineating the candidate region of intratumoral nerves reduces the computational redundancy of invalid image regions and improves the efficiency of subsequent processing. The Macenko method is used for staining standardization to eliminate color deviations caused by different scanning equipment and staining techniques, ensuring the consistency of image color. Specifically, the pathological images have coordinate information. When cutting into a plurality of pathological slides, the coordinates of each corner of each pathological slide in the pathological image are recorded to determine the specific position of each neural pathological slide in the pathological image when the neural pathological slides are combined.

[0065] As a further improvement of the present application, the pathological section classification model is a binary classification model based on ResNet-101;

[0066] The pathological section classification model is based on a label setting method, which assigns a binary classification label to all standardized pathological sections in the candidate region of the intratumoral nerve demarcation, as input of the model training data;

[0067] The specific steps of the label setting method include: manually annotating the nerve structure mask, counting the pixel proportion of the nerve structure in each standardized pathological section, if the pixel proportion is greater than or equal to 50%, the standardized pathological section is marked as positive, if the pixel proportion is less than 50%, the standardized pathological section is marked as negative.

[0068] The label setting method ensures that the training label has clear morphological boundaries, which helps the model to more accurately distinguish between nerve and non-nerve regions, and further identify whether there is a nerve structure.

[0069] More specifically, ResNet-101 has a deep residual connection structure, which can effectively extract complex pathological morphological features and is suitable for identifying linear and slender nerve structures. The network parameters are initialized using the pre-trained ResNet-101 weights on ImageNet, which significantly improves the initial feature extraction capability of the model and speeds up the convergence speed of the model on the target task. During the model training process, the cross-entropy loss function is used to optimize the binary classification performance, and the Adam optimizer is used for gradient update, with an initial learning rate of 0.001, and the learning rate is dynamically adjusted combined with the cosine annealing scheduling strategy, thereby improving the training stability and generalization ability of the model.

[0070] As a further improvement of the present application, the specific steps of determining whether each standardized pathological section has a nerve structure are: predicting each standardized pathological section, outputting the probability value of each standardized pathological section belonging to a nerve structure, if the probability value is greater than a dynamic nerve determination threshold i , it is judged that the standardized pathological section has a nerve structure;

[0071]

[0072] Wherein, M i is the tumor tissue occlusion impact score of the i-th standardized pathological section, D tumo r is the pixel number of the tumor tissue in the i-th standardized pathological section, D 总 is the total pixel number of the i-th standardized pathological section, β1 is the occlusion conversion factor, M max is the maximum impact threshold of tumor tissue occlusion, Z i is the sampling model confidence score, n kis the number of the same prediction results in the last k times of sampling inspection, N k is the total number of prediction results in the last k times of sampling inspection, η min is the minimum prediction threshold of the neural structure, and α1 and α2 are neural determination weight factors.

[0073] By designing a dynamic neural determination threshold δ based on a two-dimensional parameter i , integrating tumor tissue shielding impact score M i and sampling model confidence score Z i , objective quantification of the neural structure is realized, and the subjective dependence of the traditional evaluation is overcome. The dynamic threshold mechanism is introduced to provide a repeatable quantification standard, and different dynamic neural determination thresholds δ i are generated for the specific conditions of each standardized pathological section, which significantly shortens the processing time. The reason for introducing the tumor tissue shielding impact score M i is that the tumor tissue will wrap around the neural structure, and if there is a large amount of tumor tissue in the standardized pathological section, part of the neural structure will be shielded, so that the neural structure cannot be well exposed, affecting the judgment of the model, therefore when there is a large amount of tumor tissue, the corresponding dynamic neural determination threshold δ i should be adjusted and lowered. The sampling model confidence score Z i is introduced to detect the accuracy of the model, when the accuracy is low, the dynamic neural determination threshold δ i should be lowered to avoid missing the standardized pathological section with the neural structure.

[0074] Specifically, the dynamic neural determination threshold δ i is set at about 80% under normal circumstances, and the following is a specific calculation example:

[0075] Suppose the tumor pixels occupy 10% of the standardized pathological section, and the tumor tissue shielding impact score M i is 1 when β1 is 10, the accuracy of the result is 40 times in the last 50 times of sampling inspection, the sampling model confidence score Z i is 0.8, M max is 1, α1 is 0.2, α2 is 0.7, η min is 0.5, in this case, the dynamic neural determination threshold δ i = max (0.2*1+0.7*0.8, 0.5) = 0.76.

[0076] The neighborhood inflation operation is performed on the neural region, and the inflation result intersected with the tumor tissue is retained to obtain the intratumoral neural neighborhood M ROI ;

[0077] As a further improvement of the present application, the specific steps of performing the neighborhood inflation operation on the neural region include:

[0078] The set of coordinates of the neuropathological slice in the pathological image is P nerve ={p1, p2, ···, p n}, the binary mask of each neuropathological slice p i is M(p i ), a plurality of nerve structures are divided by an edge continuity algorithm, and the total nerve mask M nerve of each nerve structure is calculated.

[0079]

[0080] M(p i ) = 1 indicates that the pixel is a nerve region, and M(p i ) = 0 indicates that the pixel is background.

[0081] The two-dimensional morphological dilation M dilated is respectively performed to obtain a plurality of dilated masks,

[0082]

[0083] wherein, represents a dilated operation, and S r is a two-dimensional spherical structure with a radius r.

[0084] Specifically, the value of the radius r is determined according to the histological scale, such as the number of pixels corresponding to 50 μm. By using this method, a plurality of nerve regions can be dilated. The logic algorithm of the edge continuity algorithm is to determine whether the edge of the nerve structure is connected to other nerve structures. If it is connected, it is the same nerve structure. If it is not connected, it is an independent nerve structure. Further, the edge recognition can use a canny edge detection algorithm.

[0085] As a further improvement of the present application, the intersection of the dilated result and the tumor tissue is retained to obtain the intratumoral nerve neighborhood M ROI The specific steps include: performing set processing on the plurality of dilated masks and the tumor tissue mask to obtain the intratumoral nerve neighborhood M ROI .

[0086]

[0087] wherein, represents the jth nerve structure, M tumor is the mask of the tumor tissue, and Clip() represents the intersection of the dilated mask and the tumor tissue mask.

[0088] According to the present application, the intratumoral nerve neighborhood M ROI can be divided for any pathological image.

[0089] Each cell in the neuropathological slice is identified and segmented by the detection segmentation model, and the class of each cell is identified by the class identification model, and cells located in the intratumoral neural neighborhood M ROI are screened out, and a cell set C ROI is output.

[0090] Specifically, the detection segmentation model is a target detection and segmentation model based on Mask R-CNN, ResNet-101 is selected as the feature extractor of Mask R-CNN to capture complex pathological features with its strong representation ability, and the backbone network of the model is initialized using the pre-trained weights of ImageNet, which can accelerate convergence and improve the initial performance of the model. Mask R-CNN has an RPN module for generating candidate regions. Here, the inventors adjust the anchor size and aspect ratio in the RPN to adapt to the specific cell morphology in the pathological image. Referring to the image in the middle part of the drawing. Figure 1

[0091] During the training process of Mask R-CNN, the following hyperparameters are adjusted according to experiments: the initial learning rate is set to 0.001, and a learning rate scheduling strategy with cosine annealing is used to balance training speed and convergence. The initial batch size is set to 2 and adjusted accordingly according to the GPU memory capacity. The Adam optimizer is adopted, which is suitable for processing sparse data and has strong adaptability. In the RPN stage, the ratio of positive and negative samples is set to 1:3 to ensure more accurate candidate region generation for cell regions. To accelerate model training, the inventors will use multi-GPU parallel computing. If the data volume is large, the inventors can consider using mixed precision training to reduce memory occupancy. The loss function is composed of classification loss, bounding box regression loss, and mask segmentation loss. Model training strategy: use 5-fold cross-validation method to ensure the robustness and generalization of the model. At the same time, early stopping mechanism is added during training to avoid overfitting.

[0092] For the class identification model, the Densenet121 model is adopted, and the pre-trained Densenet121 on ImageNet is used as the base model to inherit the feature extraction ability in the large-scale natural image dataset. This helps the model to converge and optimize quickly on pathological image data. In the initialization stage of the model weight, a transfer learning strategy matching the pathological data is adopted to smoothly transition to the specific task learning process, to ensure the stability and adaptability of the initial state of the model. The cross-entropy loss function is used for the multi-classification task of cell types to measure the difference between the predicted results and the actual class labels.

[0093] ​In order to dynamically adjust the learning rate, a cosine annealing learning rate strategy is used. The initial learning rate is set to 0.001, and the learning rate is gradually reduced as the training proceeds, so as to avoid the model falling into local optimum.

[0094] The formula of the learning rate changing with the training period is:

[0095]

[0096] Wherein, η tmin is set to 0, η tmax is set to 0.001, T cur is the current training period, and T is the total training period. The strategy can smoothly adjust the learning rate and improve the fine-tuning accuracy of the model at the end of the training.

[0097] Specifically, the present application can also extract the deep learning features of each cell through the second-to-last layer of the DenseNet121 model, which is denoted as here. And generate the cell set C ROI , record the deep learning features of the cell , and then facilitate the subsequent analysis and utilization of the cell.

[0098] As a further improvement of the present application, while detecting and segmenting each cell in the neuropathological slice by the segmentation model, the binary mask and the centroid coordinates (x k , y k ) of the corresponding cell are generated.

[0099] The specific steps of generating the centroid coordinates include:

[0100]

[0101] Wherein, represents the total number of pixels of the cell in the binary mask;

[0102] The specific steps of screening out the cells located in the intratumoral nerve neighborhood M ROI and outputting the cell set C ROI include:

[0103] If the centroid coordinates (x k , y k ) of the kth cell fall into the intratumoral nerve neighborhood M ROI , it is considered that the cell falls into the intratumoral nerve neighborhood, and the cell set C ROI is output.

[0104]

[0105] Wherein, is a cell class of the kth cell, is binary mask information of the kth cell.

[0106] By performing a neighborhood expansion operation on the neural region and retaining the expansion result intersecting with the tumor tissue, an intratumoral neural neighborhood can be obtained, which can capture the spatial relationship between the tumor and the nerve, avoid the subjective dependence and limitations of traditional manual film reading, and ensure the high accuracy and objectivity of the detection result.

[0107] According to the cell set C ROI constructing a pathological cell atlas G (i) , the pathological cell atlas G (i) is input into the GCN prediction model to obtain a prognosis prediction.

[0108] Combining the pathological film classification model, the detection segmentation model, the class recognition model and the GCN prediction model, a cell set located in the intratumoral neural neighborhood can be screened, and a pathological cell atlas can be constructed to quantify the distance, position and type distribution between cells, realize fine modeling of the complex interaction of the tumor neural microenvironment, not only improve the accuracy of risk prediction, but also greatly improve the speed of detection and prognosis evaluation.

[0109] As a further improvement of the present application, the cell set C ROI includes the centroid coordinates, cell class and binary mask information of each cell;

[0110] According to the cell set C ROI constructing a pathological cell atlas G (i) The specific steps include: regarding cells of the same cell class as a node, calculating the Euclidean distance W ij between any two nodes, if the Euclidean distance W ij is less than a preset dynamic construction threshold W t , the two nodes have a connected edge, and a pathological cell graph G (i) corresponding to multiple cell classes is constructed according to the nodes and edges;

[0111] The expression of the dynamic construction threshold W t is:

[0112]

[0113] Wherein, R k is the average size of the cell class, N i represents the total number of pixels counted through the binary mask information of the cell class, N k represents the total number of the cell class, R 基 is the size value of the reference cell, and L 基 is the basic distance value.

[0114] When the size of a certain type of cell is small, dynamically construct the threshold value W t is automatically reduced, which facilitates close connection between adjacent cells and accurately captures the cell aggregation phenomenon; otherwise, the threshold value W t is automatically increased, which prevents excessive connection from causing confusion in the graph structure and improves the generalization ability of the model.

[0115] As a further improvement of the present application, the specific steps of inputting the GCN prediction model to obtain the prognosis prediction include: calculating the cell density ratio of cells of the same cell type ,

[0116]

[0117] wherein, is the number of cells of the i-th cell type, A ROI is the area calculation of the intratumoral nerve neighborhood, ρ i is the density weight correction factor of the i-th cell type, ρ i is the density weight correction factor of the i-th cell type, which is calculated according to the average density of normal people of the cell type;

[0118] extracting tumor cells located at the edge of the tumor tissue to obtain edge tumor cells, and calculating the straight-line distance between the edge tumor cells and the nearest nerve cell If the straight-line distance is less than a preset diffusion threshold value, the pair of edge tumor cells, nerve cells, edges connected to the edge tumor cells, and edges connected to the nerve cells are integrated to form a diffusion propagation set;

[0119] calculate the maximum directional similarity value τ jk ,

[0120] u≠v;

[0121] wherein, is the edge direction vector of the j-th edge tumor cell and the u-th tumor cell connected in the diffusion propagation set, is the edge direction vector of the k-th nerve cell and the v-th tumor cell connected, and the k-th nerve cell and the j-th edge tumor cell are a pair;

[0122] calculate the risk diffusion index δ spread ,

[0123]

[0124] wherein, ξ dir is the directional consistency index, N tis the total number of edge tumor cells or nerve cells spread by diffusion propagation set, ξ dif is the risk diffusion distance index, is the straight line distance of the i-th pair of edge tumor cells and nerve cells, η i is the diffusion distance dynamic weight coefficient, λ f is the distance adjustment factor, γ1 and γ2 are risk diffusion weight factors, ξ den is the density propagation index, is the cell density ratio of tumor cells, is the cell density ratio of nerve cells, and γ3 is the scale conversion weight factor of the cell density ratio.

[0125] The risk diffusion index δ spread is compared with multiple risk numerical intervals, and the corresponding risk level and prognosis report are output.

[0126] The maximum direction similarity value τ jk The calculation principle is to quantify the biological invasion characteristics of tumor cells along the nerve, and reflect the malignant degree of tumor cells and nerve cell interaction. The higher the direction similarity of the edge connected by the nerve cell and the tumor cell, the higher the invasion risk. The diffusion distance dynamic weight coefficient η i rapidly decreases with the increase of distance, effectively reflecting the invasion difficulty of tumor cells and nerve cells. The setting of density can comprehensively reflect the diffusion speed of the tumor cells, and the risk diffusion index δ spread can be effectively calculated by comprehensive estimation from three dimensions.

[0127] Specifically, if the direction vector of a certain edge connected to a certain tumor cell is (3, 4), and the direction vector of a certain edge connected to a nerve cell paired with the tumor cell is (6, 8), the maximum direction similarity value τ jk is 1, and other tumor cells and nerve cells are calculated. Assuming that there are three pairs, the maximum direction similarity values τ jk of the remaining two are 0.5 and 0.1 respectively, the straight line distances of the three pairs of tumor cells and nerve cells are 5, 20 and 50, the tumor cell density ratios are assumed to be 0.8 and 1.1, and γ1 is 0.8, γ1 is 0.7, γ3 is 1. It can be concluded that the δ spread=(0.8*0.53+0.7*2.027)*0.95=1.75, the risk value interval can be divided into six intervals, no risk, low risk, low-medium risk, medium risk, high-medium risk, high risk, and the specific interval dividing value can be adjusted according to the actual situation, for example, the medium risk is 1 to 1.3, the high-medium risk is 1.3 to 1.5, the high risk is greater than 1.5, etc., and the prognosis report is a combination according to the existing treatment scheme and prognosis experience, and since different people have different treatment schemes and physical conditions, it is unnecessary to specifically discuss them, and therefore the present application will not be described in more details.

[0128] Obviously, the above embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the embodiments of the present application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, it is not necessary and impossible to exhaust all the embodiments. Any modification, equivalent replacement and improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. An intratumoral neural neighborhood cell-level graph convolution-based pathological prognosis modeling method, characterized in that, The method comprises the following steps: The pathological image is preprocessed to obtain a plurality of standardized pathological slides, tumor tissues are identified, and a pathological slide classification model is used to determine whether each standardized pathological slide contains a neural structure. If yes, the neural region of the neural structure is extracted, and the standardized pathological slide is regarded as a neuropathological slide. All neuropathological slides are combined and labeled with the coordinates of the neuropathological slide in the pathological image. performing a neighborhood dilation operation on the neural region and retaining the dilated result intersected with the tumor tissue to obtain an intratumoral neural neighborhood M ROI ; Each cell in the neuropathological slice is identified and segmented by detecting the segmentation model, and the class of each cell is identified by the class identification model, and the cells located in the intratumoral neural neighborhood M ROI are screened out, and the cell set C ROI is output. According to cell set C ROI Constructing a pathological cell atlas G (i) , pathological cell atlas G (i) Input the GCN prediction model to obtain the prognostic prediction.

2. The intratumoral neural neighborhood cell-level graph convolution-based pathology prognosis modeling method according to claim 1, characterized in that, The specific steps of preprocessing include: performing HE staining on the pathological image and manually annotating the tumor-intra neural candidate region, cutting the pathological image into a plurality of pathological slides, and performing staining standardization on the pathological slides by the Macenko method.

3. The intratumoral neural neighborhood cell-level graph convolution-based pathology prognosis modeling method according to claim 2, characterized in that, The pathological slide classification model is a binary classification model based on ResNet-101. The pathological slide classification model is based on a label setting method, which assigns a binary classification label to all standardized pathological slides in the tumor-intra neural candidate region as input of the model training data. The specific steps of the label setting method include: manually annotating the neural structure mask, counting the pixel proportion of the neural structure in each standardized pathological slide, and if the pixel proportion is greater than or equal to 50%, the standardized pathological slide is labeled as positive, and if the pixel proportion is less than 50%, the standardized pathological slide is labeled as negative.

4. The intratumoral neural neighborhood cell-level graph convolution-based pathology prognosis modeling method according to claim 1, characterized in that, The specific steps of judging whether each standardized pathological section has a nerve structure are: making a prediction for each standardized pathological section, outputting a probability value of each standardized pathological section belonging to a nerve structure, and if the probability value is greater than a dynamic nerve judgment threshold δ i , it is judged that the standardized pathological section has a nerve structure; wherein M i is the tumor tissue blocking impact score of the i-th standardized pathology slide, D tumor is the pixel number of the tumor tissue on the i-th standardized pathology slide, D 总 is the total pixel number of the i-th standardized pathology slide, β1is a blocking conversion factor, M max is the maximum impact threshold of the tumor tissue blocking, Z i is the spot-check model confidence score, n k is the number of the same prediction results of the last k times of spot-checking, N k is the total number of the last k times of spot-checking, η min is the minimum prediction threshold of the neural structure, and α1and α2are both neural determination weight factors.

5. The intratumoral neural neighborhood cell-level graph convolution-based pathology prognosis modeling method according to claim 1, characterized in that, The specific steps of the neighborhood expansion operation on the neural region include: The pathological image of the nerve slice is a set of coordinates: P nerve ={p1, p2, ···, p n} The binary mask of each nerve slice p i is M(p i ), and multiple nerve structures are divided by the edge continuity algorithm. The total nerve mask M nerve of each nerve structure is calculated, where M(p) = 1 indicates that the pixel is a region of interest, and M(p) = 0 indicates that the pixel is background. i i where M(p) = 1 indicates that the pixel is a region of interest, and M(p) = 0 indicates that the pixel is background.​ performing two-dimensional morphological dilation M dilated to obtain a plurality of dilated masks, wherein represents an expansion operation, S r is a two-dimensional spherical structure with radius r as a dimension.

6. The intratumoral neural neighborhood cell-level graph convolution-based pathology prognosis modeling method according to claim 5, characterized in that, Retain the dilation result intersected with the tumor tissue to obtain the intratumoral nerve neighborhood M ROI The specific steps include: performing set processing on a plurality of dilation masks and a tumor tissue mask to obtain the intratumoral nerve neighborhood M ROI , wherein, represents the jthneural structure, M tumor is a mask of tumor tissue, Clip() represents the part of the dilated mask that is retained in the intersection with the tumor tissue mask.

7. The intratumoral neural neighborhood cell-level graph convolution-based pathology prognosis modeling method according to claim 6, characterized in that, generate a binary mask of the corresponding cell while identifying and segmenting each cell in the neuropathology slide by detecting a segmentation model and the centroid coordinates (x k ,y k ) The specific steps of generating the centroid coordinates include: wherein, represents the total number of pixels of the cell in the binary mask; Screening cells located in the intratumoral neural neighborhood M ROI Outputting a collection of cells C ROI The specific steps include: If the centroid coordinates (x k ,y k ) of the kth cell fall into the intratumoral nerve neighborhood M ROI , it is considered that the cell falls in the intratumoral nerve neighborhood, and the cell set C ROI is output. wherein, is a cell class of the kth cell, is a binary mask information of the kth cell.

8. The intratumoral neural neighborhood cell-level graph convolution-based pathology prognosis modeling method according to claim 1, characterized in that, cell collection C ROI includes the centroid coordinates of each cell, the cell class, and the binary mask information; According to cell set C ROI Constructing a pathological cell atlas G (i) The specific steps include: treating cells of the same cell type as a node, and calculating the Euclidean distance W between any two nodes. ij If the Euclidean distance W ij Less than the preset dynamic construction threshold W t If these two nodes are connected by an edge, then a pathological cell graph G corresponding to multiple cell types can be constructed based on the nodes and edges. (i) ; The threshold value W is dynamically constructed t The expression is: where R k is the size average value of the cell class, N i represents the total number of pixels counted by the binary mask information of the cell class, N k represents the total number of the cell class, R 基 is the size value of the reference cell, L 基 is the base distance value.

9. The intratumoral neural neighborhood cell-level graph convolution-based pathology prognosis modeling method according to claim 8, characterized in that, The specific steps of inputting the GCN prediction model to obtain the prognosis prediction include: performing cell density ratio calculations, wherein, is the number of cells of the i-th cell type, A ROI is the area calculation of intratumoral neural neighborhood, p i is the density weight correction factor for the i-th cell type; extracting tumor cells located at the edge of the tumor tissue to obtain edge tumor cells, and calculating a straight-line distance between the edge tumor cells and the nearest nerve cell If the straight-line distance is less than a preset diffusion threshold, then a pair of the edge tumor cells, the nerve cell, an edge connected to the edge tumor cell, and an edge connected to the nerve cell are integrated to form a diffusion propagation set calculating the maximum directional similarity value τ for each pair of edge tumor cell and nerve cell under the diffusion propagation set jk , wherein, is the edge direction vector of the connection between the jth marginal tumor cell and the u th tumor cell in the diffusive propagation set, is the edge direction vector of the connection between the k th neuron cell and the v th tumor cell, the k th neuron cell and the jth marginal tumor cell being a pair; Computing a risk spread indicator δ spread , wherein, ξ dir is a direction consistency index, N t is the total number of edge tumor cells or nerve cells under the diffusion propagation set, ξ dif is a risk diffusion distance index, is the straight-line distance of the i-th pair of edge tumor cells and nerve cells, η i is a diffusion distance dynamic weight coefficient, λ f is a distance adjustment factor, γ1 and γ2 are risk diffusion weight factors, ξ den is a density propagation index, is the cell density ratio of tumor cells, is the cell density ratio of nerve cells, and γ3 is a scale conversion weight factor of the cell density ratio. The risk diffusion index δ spread In comparison with a plurality of risk numerical intervals, the corresponding risk level and prognosis report are output.

Images