Renal cell carcinoma sinus invasion prediction method based on prior information

Abstract

The invention belongs to the technical field of image classification, and particularly relates to a renal cell carcinoma sinus invasion prediction method based on prior information, and the method comprises the steps: obtaining a CT image, and constructing a renal cell carcinoma sinus invasion prediction data set according to the CT image; performing feature extraction on the data in the data set, and constructing a heterogeneous graph according to an extracted feature graph; inputting the heterogeneous graph into the trained priori-driven graph neural network model to obtain a renal cell carcinoma sinus invasion prediction result; evaluating the prediction result; according to the method, different potential appearances are generated in a data driving mode through a diffusion perturbation strategy, and disturbance invariant features are learned, so that reduction correction is carried out on network performance degradation caused by cross-dataset application.

Description

Technical Field

[0001] The invention belongs to the technical field of image classification, and in particular relates to a method for predicting sinus invasion of renal cell carcinoma based on prior information. Background Art

[0002] Renal cell carcinoma is the third most common malignant tumor of the urinary system, and its incidence has steadily increased in recent years, posing a serious health threat to patients. In the treatment of renal cell carcinoma, surgery remains one of the main treatment methods. In the surgical treatment model of renal cell carcinoma, the presence or absence of renal sinus invasion is a key factor in deciding whether to perform partial nephrectomy. By preserving the nephron, partial nephrectomy helps improve the patient's quality of life and renal function. However, in clinical practice, the diagnosis of renal sinus invasion usually relies on preoperative imaging examinations, although invasive pathological evaluation is considered the gold standard.

[0003] Computed tomography (CT) has become an important tool for the assessment of renal sinus invasion in renal cell carcinoma due to its widespread use in routine examinations and clinical diagnosis. However, a major limitation of CT imaging is its low soft tissue contrast, which makes it extremely difficult to accurately identify renal sinus invasion of renal cell carcinoma on CT images. Although modern imaging technology has made significant progress, the imaging manifestations of renal sinus invasion remain complex and varied, and there is significant heterogeneity between different patients.

[0004] In recent years, deep learning technology has shown good performance in cancer screening and detection in medical imaging, especially in the detection and identification of tumor regions. These methods can extract rich features when processing tumors, however, the diagnosis of renal sinus invasion still faces many challenges. This is because the invasive characteristics of renal cell carcinoma are very complex, and due to individual differences, the appearance, density, shape, and texture of the tumor may vary slightly in imaging. Around the tumor, there are differences in the boundaries between tumors with renal sinus invasion and tumors without renal sinus invasion, but these differences are often difficult to accurately distinguish through CT images. In addition, even within the tumor, tumors in the same position may present different imaging features due to the heterogeneity of the tumor. Tumors with renal sinus invasion usually show a tendency to endogenous growth in the microenvironment, which makes it impossible for models that simply rely on tumor regional features to effectively capture the relevant task features, limiting the accuracy of the model in practical applications. To complicate matters, a small number of tumors with renal sinus invasion may be very similar to tumors without renal sinus invasion in location, size, and texture. This similarity not only poses a great challenge to radiologists, but also easily leads to confusion in the diagnostic model of renal sinus invasion, thereby increasing the difficulty and error rate of the prediction task. Summary of the invention

[0005] In order to solve the problems existing in the above-mentioned prior arts, the present invention proposes a method for predicting sinus invasion of renal cell carcinoma based on prior information, the method comprising: acquiring CT images, constructing a renal cell carcinoma sinus invasion prediction data set according to the CT images; extracting features from the data in the data set, and constructing a heterogeneous graph according to the extracted feature graph; inputting the heterogeneous graph into a trained prior-driven graph neural network model to obtain a renal cell carcinoma sinus invasion prediction result; and evaluating the prediction result.

[0006] Beneficial effects of the present invention:

[0007] The present invention systematically integrates prior information (such as tissue differentiation, spatial proximity, and feature similarity) into a classification model based on a graph neural network, describing the imaging characteristics of the tumor from multiple levels such as intratumoral, peritumoral, and carrier environment, thereby capturing the discriminant features related to renal sinus invasion and achieving end-to-end high-precision patient-level classification prediction. At the same time, through a diffusion perturbation strategy, different potential appearances are generated in a data-driven manner, and perturbation-invariant features are learned, thereby correcting the network performance degradation caused by cross-dataset applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a flow chart of the method of the present invention;

[0009] Figure 2 It is the Patch level feature extraction graph of the present invention;

[0010] Figure 3 It is a structural diagram of the Patch level feature extractor of the present invention;

[0011] Figure 4 A diagram showing the construction process of a heterogeneous graph of the present invention;

[0012] Figure 5 This is a structural diagram of the prior-driven graph neural network of the present invention;

[0013] Figure 6 It is a structural diagram of the global perception graph aggregation layer of the present invention;

[0014] Figure 7 It is the evaluation index of the present invention in the task of predicting sinus invasion of renal cell carcinoma;

[0015] Figure 8 This is a comparison chart between the prediction results of the model of the present invention and the diagnosis results of human clinical experts. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] In medical image classification, the introduction of prior information is of great significance. Especially in the context of CT image analysis, data is often scarce and distributed in high dimensions. High dimensionality makes it difficult for sample distribution to approximate the true distribution characteristics, resulting in significant differences between the training set and the test set. Therefore, when transferring the model from the training environment to the actual test environment, the introduction of prior information can effectively reduce the model variance and alleviate the performance degradation. CT images contain rich prior knowledge, such as the characteristics of different tissue types. Integrating this knowledge into the deep learning framework can normalize the model during the training process, thereby improving its test performance in practical applications. However, there are also some challenges in the application of prior information. First, prior knowledge is usually based on historical data and may not reflect emerging pathological features, resulting in insufficient adaptability of the model to new cases. Second, the selection and construction process of prior information may be highly subjective, and different experts may come up with different prior assumptions, which affects the consistency of the model. Therefore, when embedding prior information in the model, it is necessary to comprehensively consider its advantages and limitations, and select appropriate processing methods and technical strategies based on actual conditions to improve the effect and application value of image classification.

[0018] In this embodiment, a method for predicting sinus invasion of renal cell carcinoma based on prior information is provided. Figure 1 As shown, the method includes: acquiring a CT image, constructing a renal cell carcinoma sinus invasion prediction dataset based on the CT image; extracting features from the data in the dataset, and constructing a heterogeneous graph based on the extracted feature graph; inputting the heterogeneous graph into a trained prior-driven graph neural network model to obtain a renal cell carcinoma sinus invasion prediction result; and evaluating the prediction result.

[0019] The process of making a renal cell carcinoma sinus invasion dataset includes:

[0020] Step 1: Use the Unet++ model to segment the CT image;

[0021] Step 2: Calculate the minimum bounding box of the kidney in the segmentation mask;

[0022] Step 3: Increase the minimum bounding box by 5px along the x and y directions to calculate the ROI;

[0023] Step 4: Starting from the upper left corner of the ROI, use the sliding window strategy to intercept a patch image with a fixed size of r×r (r is the side length of the patch);

[0024] Step 5: Repeat steps 1 to 4 until all case patch images are captured.

[0025] Preferably, the size of the patch image is 20×20.

[0026] Patch-level feature extraction Figure 2 As shown, including:

[0027] Step 1: Obtain a dataset of renal sinus invasion and the corresponding tumor annotations as Patch labels Y patch , the patch containing the tumor is marked as 1, and the patch containing only the kidney is marked as 0, and divided into training set, validation set and test set;

[0028] Step 2: Perform data enhancement on the patch images in the training set by using the diffusion perturbation strategy;

[0029] Step 3: Input the enhanced patch images in the training set into the patch-level feature extraction network to obtain the prediction results P patch ;

[0030] Step 3: According to Y patch and P patch Construct loss function;

[0031] Step 4: Update all parameters of the feature extraction network through the back-propagation mechanism according to the loss function;

[0032] Step 5: Use the validation set to evaluate the network and save the model parameter weights with the best performance.

[0033] Step 6: Repeat steps 2 to 5 until the training is completed;

[0034] Step 7: Remove the last fully connected layer of the feature extraction network as an encoder, load the optimal model parameter weights, and transform the Patch image x∈R r×r Input encoder to get Patch-level feature representation f∈R d (d is the dimension of the feature vector).

[0035] Preferably, the diffusion perturbation strategy marks the original patch image as x0, and according to the variance table β1,…,β T Gradually add noise to Patch, t-1 Add noise to get the Patch image x at time step t t , which is expressed as follows:

[0036]

[0037] Among them, β t is the variance table, ε~N(0,I) is the standard Gaussian noise.

[0038] Preferably, the feature extraction network at the patch level includes an encoder and a classifier, and the network structure is as follows: Figure 3 As shown, the input is a 2D Patchx∈R r×r The encoder consists of 6 2D convolutional blocks, a global maximum pooling layer and a multi-layer perceptron; the classifier consists of a fully connected layer. Each convolutional block consists of a convolutional layer, a normalization layer and an activation layer.

[0039] Preferably, the heterogeneous graph construction process is as follows Figure 4 As shown, including:

[0040] Step 1: Define all patch images of a case as a node set V in a heterogeneous graph;

[0041] Step 2: The type of patch image (tumor or kidney) is introduced into the node type as the prior information of tissue discrimination. The node with tumor is coded as 1, and the node containing only kidney is coded as 0, forming a node type set T.

[0042] Step 3: Input all nodes into the pre-trained encoder to obtain node features f∈R d (d is the dimension of the feature vector), and is combined with the corresponding node type to form a node feature matrix F∈R n×d (f is the dimension of the feature vector, n is the number of nodes);

[0043] Step 4: Define the edges and edge attributes according to the spatial proximity prior information of the patch in the original CT image, connect each node and its spatially adjacent nodes to form an edge set E; for each edge e∈E, calculate the Euclidean distance between the head node and the tail node, and then map these distances through the Gaussian distribution function, convert the distances into continuous probability distributions, and form a spatial proximity matrix A', and finally normalize the spatial proximity matrix to obtain the edge attribute matrix A; where the spatial distance d between the head node i and the tail node j is ij The expression is:

[0044]

[0045] Among them, (x i ,y i ,z i ) and (x j ,y j ,zj ) represent the upper left corner position coordinates of the i-th and j-th patches in the three-dimensional voxel space of the original CT image, respectively.

[0046] The expression of the edge attribute matrix A is:

[0047]

[0048] Where diag and n are the diagonal matrix and the number of nodes respectively.

[0049] Step 5: Define the heterogeneous graph G = (V, E, T, A) corresponding to the case, where V, E, T are the node set, edge set and node type set respectively, and A represents the edge attribute matrix;

[0050] Step 6: Repeat steps 1 to 5 until all cases have been processed.

[0051] In this embodiment, the prior-driven graph neural network module is as follows: Figure 5 As shown in Figure 1, it includes a global perception graph aggregation layer, a priori pooling, and a multi-layer perceptron classifier. After the heterogeneous graph is input into the two layers of global perception graph aggregation layer, the node features are fused through a priori pooling to generate a graph-level feature representation g, and the graph-level feature representation g is input into the multi-layer perceptron classifier to generate a prediction result.

[0052] Preferably, the structure of the global perception graph aggregation layer is as follows: Figure 6 As shown, for the heterogeneous graph G of the l-th layer input l-1 , put G l-1 The node feature matrix is ​​mapped to the query vector and key vector through two 1×1 convolution operations, and its expression is:

[0053]

[0054] Among them, F l-1 It's G l-1 The node feature matrix of and They represent the parameter matrices related to the two 1×1 convolution operations in the lth layer. The feature similarity prior information between nodes is calculated based on the dot product operation of the query vector and the key vector, and the similarity prior information is normalized by softmax to obtain the feature similarity matrix S of the nodes. l , which is expressed as follows:

[0055]

[0056] The node feature similarity matrix S lAdded to the edge attribute matrix A as the weight coefficient of each adjacent node in the aggregation process, the node feature is multiplied by the corresponding weight coefficient to update the feature of each node. l Updated node feature F l It can be expressed as:

[0057] F l =σ(( S l+A)F l-1 W l )

[0058] Among them, σ and W l is the activation function and weight parameter of the linear layer. The updated graph G l As the output of the lth global perception graph aggregation layer.

[0059] Prior pooling clusters node representations based on node type prior information. For each node type t, all node features belonging to this node type t are aggregated into a single vector h through maximum pooling. t The pooled features of each node type are then aggregated into the feature matrix P. Subsequently, the graph-level features g are determined by another pooling layer.

[0060] In this embodiment, the training process of the prior-driven graph neural network model includes:

[0061] Step 1: Obtain the renal cell carcinoma sinus invasion prediction dataset and the corresponding renal sinus invasion label Y;

[0062] Step 2: Input the heterogeneous graphs corresponding to a group of cases in the training set into the prior-driven graph neural network model to obtain the prediction result P;

[0063] Step 3: Construct the loss function L based on P and Y;

[0064] Step 4: Update all parameters of the prior-driven graph neural network through the back-propagation mechanism according to the loss function;

[0065] Step 5: Use the validation set to evaluate the network and save the model parameter weights with the best performance.

[0066] Step 6: Repeat steps 2 to 5 until the training is completed.

[0067] The cross entropy loss is used to construct the loss function L, which is expressed as:

[0068]

[0069] Among them, num neg is the number of cases with non-renal sinus invasion, num posis the number of cases of renal sinus invasion, y n is the true label of the nth case, p n is the predicted label of the nth case, and weight is the weight of the cross entropy loss.

[0070] In this embodiment, the process of predicting and evaluating the sinus invasion of renal cell carcinoma in CT images includes:

[0071] Step 1: Load the optimal model weights of the prior-driven graph neural network;

[0072] Step 2: Input the heterogeneous graphs corresponding to a group of cases in the test set into the prior-driven graph neural network model to obtain the prediction results;

[0073] Step 3: Save the prediction results;

[0074] Step 4: Repeat steps 2 to 3 until all cases are predicted;

[0075] Step 5: Convene a group of clinical experts with different experience levels and clinical backgrounds to independently diagnose renal sinus invasion using contrast-enhanced CT images without knowing the pathological findings of renal sinus invasion;

[0076] Step 6: Compare the model prediction results with the diagnosis results of human clinical experts for comprehensive evaluation.

[0077] like Figure 7 As shown, the prior-driven graph neural network can achieve accurate classification in the task of classifying sinus invasion of renal cell carcinoma in CT images with high evaluation indicators.

[0078] Compare the model predictions with the diagnoses of human clinical experts. Figure 8 As shown, senior radiologists (red) performed best among the individual groups. In contrast, the content of the present invention was close to senior radiologists in the diagnosis of sinus invasion of renal cell carcinoma, and the overall diagnostic level of clinical experts was not as good as the prior-driven graph neural network in the content of the present invention.

[0079] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for predicting sinus invasion of renal cell carcinoma based on prior information, characterized in that: include: Acquire CT images, and construct a renal cell carcinoma sinus invasion prediction dataset based on the CT images; Extract features from the data in the dataset and construct a heterogeneous graph based on the extracted feature graph; The heterogeneous graph is input into the trained prior-driven graph neural network model to obtain the prediction results of sinus invasion of renal cell carcinoma; the prediction results are evaluated.

2. The method for predicting sinus invasion of renal cell carcinoma based on prior information according to claim 1, characterized in that: The construction of a renal cell carcinoma sinus invasion prediction dataset includes: using the Unet++ model to segment the CT image; calculating the minimum bounding box of the kidney in the segmentation mask; increasing the minimum bounding box by 5px in the x and y directions to calculate the ROI; starting from the upper left corner of the ROI, using a sliding window strategy to capture a patch image of a fixed size of r×r; repeating the above process until all case patch images are captured.

3. The method for predicting sinus invasion of renal cell carcinoma based on prior information according to claim 1, characterized in that: Feature extraction of data in the dataset includes: Step 1: Obtain a dataset of renal sinus invasion and the corresponding tumor annotations as Patch labels Y patch , the patch containing the tumor is marked as 1, otherwise it is marked as 0, and divided into training set, validation set and test set; Step 2: The patch images in the training set are augmented by the diffusion perturbation strategy and then input into the patch-level feature extraction network to obtain the prediction results P patch ; Step 3: According to Y patch and P patch Construct loss function; Step 4: Update all parameters of the feature extraction network through the back-propagation mechanism according to the loss function; Step 5: Use the validation set to evaluate the network and save the model parameter weights with the best performance. Step 6: Repeat steps 2 to 5 until the training is completed; Step 7: Remove the last fully connected layer of the feature extraction network as an encoder, load the optimal model parameter weights, and transform the Patch image x∈R r×r Input encoder to get Patch-level feature representation f∈R d .

4. The method for predicting sinus invasion of renal cell carcinoma based on prior information according to claim 3, characterized in that: The data enhancement strategy through diffusion perturbation includes: marking the original Patch image as x0, according to the variance table β1,…,β T Gradually add noise to Patch, t-1 Add noise to get the Patch image x at time step t t ; Its expression is: Among them, β t is the variance table, ε~N(0,I) is the standard Gaussian noise, I is the standard deviation, and T is the diffusion time step.

5. The method for predicting sinus invasion of renal cell carcinoma based on prior information according to claim 1, characterized in that: Constructing a heterogeneous graph includes: Step 1: Define all patch images of a case as a node set V in a heterogeneous graph; Step 2: Encode the tumor annotation as the tissue discrimination prior information into the node type. The node with tumor is encoded as 1, otherwise it is encoded as 0, forming a node type set T; Step 3: Input all nodes into the trained encoder to obtain node features f∈R d , and combined with the corresponding node type to form a node feature matrix F∈R n×d ; Step 4: Define the edges and edge attributes according to the spatial position prior information of the patch in the original CT image, connect each node with its spatially adjacent nodes to form an edge set E; for each edge e∈E, calculate the Euclidean distance between the head and tail nodes as the spatial proximity prior information and introduce the edge attribute A; Step 5: Define the heterogeneous graph G = (V, E, T, A) corresponding to the case, where V, E, T are the node set, edge set and node type set respectively, and A represents the edge attribute matrix; Step 6. Repeat steps 1 to 5 until all cases have been processed.

6. The method for predicting sinus invasion of renal cell carcinoma based on prior information according to claim 1, characterized in that: The prior-driven graph neural network model includes a global perception aggregation layer, prior pooling, and a multi-layer perceptron classifier; the global perception aggregation layer is used to extract the features of each node in the heterogeneous graph; the prior pooling is used to fuse node features to generate a graph-level feature representation g; the multi-layer perceptron classifier is used to perform classification prediction on the graph-level feature representation.

7. The method for predicting sinus invasion of renal cell carcinoma based on prior information according to claim 1, characterized in that: Training the prior-driven graph neural network model includes: using the renal cell carcinoma sinus invasion prediction dataset and the corresponding renal sinus invasion label Y as the training set, and converting the data in the training set into a heterogeneous graph; inputting the heterogeneous graph corresponding to a group of cases in the training set into the prior-driven graph neural network model for training to obtain the prediction results; constructing a loss function based on the prediction results and the renal sinus invasion label Y; updating all parameters of the prior-driven graph neural network through the back propagation mechanism according to the loss function; using the validation set to evaluate the network, saving the model parameter weights with the best indicators, and repeating the above process until the training is completed.

8. The method for predicting sinus invasion of renal cell carcinoma based on prior information according to claim 7, characterized in that: The loss function expression is: Among them, num neg is the number of cases with non-renal sinus invasion, num pos is the number of cases of renal sinus invasion, y n is the true label of the nth case, p n is the predicted label of the nth case, and weight is the weight of the cross entropy loss.

9. The method for predicting sinus invasion of renal cell carcinoma based on prior information according to claim 1, characterized in that: The evaluation of the prediction results includes: loading the optimal model weights of the prior-driven graph neural network; inputting the heterogeneous graphs corresponding to all cases in the test set into the prior-driven graph neural network model to obtain the prediction results; and comparing and evaluating all prediction results.

Images