A method for predicting sinus invasion of renal cell carcinoma based on prior information

Abstract

This invention belongs to the field of image classification technology, specifically relating to a method for predicting sinus invasion of renal cell carcinoma based on prior information. The method includes: acquiring CT images; constructing a renal cell carcinoma sinus invasion prediction dataset based on the CT images; extracting features from the data in the dataset; constructing a heterogeneous graph based on the extracted feature maps; inputting the heterogeneous graph into a trained prior-driven graph neural network model to obtain the renal cell carcinoma sinus invasion prediction result; and evaluating the prediction result. This invention generates different potential appearances in a data-driven manner through a diffusion perturbation strategy and learns perturbation-invariant features, thereby correcting for the network performance degradation caused by cross-dataset applications.

Description

Technical Field

[0001] This invention belongs to the field of image classification technology, specifically relating to a method for predicting sinusoidal invasion of renal cell carcinoma based on prior information. Background Technology

[0002] Renal cell carcinoma is the third most common malignant tumor of the urinary system, and its incidence has been steadily increasing in recent years, posing a serious threat to patients' health. Surgery remains one of the primary treatment methods for renal cell carcinoma. In surgical treatment modalities for renal cell carcinoma, the presence or absence of renal sinus invasion is a key factor in determining whether partial nephrectomy is necessary. By preserving nephrons, partial nephrectomy helps improve patients' quality of life and renal function. However, in clinical practice, the diagnosis of renal sinus invasion usually relies on preoperative imaging examinations, although invasive pathological evaluation is considered the gold standard.

[0003] Computed tomography (CT) has become an important tool for assessing renal sinus invasion in renal cell carcinoma due to its widespread use in routine examinations and clinical diagnosis. However, a major limitation of CT imaging is its low soft tissue contrast, which makes it extremely difficult to accurately identify renal sinus invasion in renal cell carcinoma on CT images. Despite significant advances in modern imaging techniques, the imaging manifestations of renal sinus invasion remain complex and variable, exhibiting significant heterogeneity among different patients.

[0004] In recent years, deep learning technology has demonstrated excellent performance in cancer screening and detection in medical imaging, especially in the detection and identification of tumor regions. These methods can extract rich features when processing tumors; however, the diagnosis of renal sinus invasion still faces many challenges. This is because the invasive characteristics of renal cell carcinoma are highly complex, and due to individual differences, the appearance, density, shape, and texture of tumors may exhibit subtle variations on imaging. Around the tumor, tumors with and without renal sinus invasion differ in their boundaries, but these differences are often difficult to accurately distinguish using CT images. Furthermore, even within the tumor itself, due to its heterogeneity, tumors in the same location may present different imaging features. Tumors with renal sinus invasion often exhibit an endogenous growth tendency within their microenvironment, making models that simply rely on tumor region features unable to effectively capture relevant task features, thus limiting the accuracy of the models in practical applications. Adding to the complexity, a small number of tumors with renal sinus invasion may be very similar in location, size, and texture to tumors without renal sinus invasion. This similarity not only presents a significant challenge to radiologists, but also easily leads to confusion in diagnostic models of renal sinus invasion, thereby increasing the difficulty and error rate of the prediction task. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention proposes a method for predicting sinus invasion of renal cell carcinoma based on prior information. The method includes: acquiring CT images and constructing a renal cell carcinoma sinus invasion prediction dataset based on the CT images; extracting features from the data in the dataset and constructing a heterogeneous graph based on the extracted feature maps; inputting the heterogeneous graph into a trained prior-driven graph neural network model to obtain the renal cell carcinoma sinus invasion prediction result; and evaluating the prediction result.

[0006] The beneficial effects of this invention are:

[0007] This invention systematically integrates prior information (such as tissue differentiation, spatial proximity, and feature similarity) into a graph neural network-based classification model. It describes the imaging characteristics of tumors from multiple levels, including intratumoral, peritumoral, and carrier environments, thereby capturing discriminative features related to renal sinus invasion and achieving high-precision end-to-end patient-level classification prediction. Simultaneously, through a diffusion perturbation strategy, it generates different potential appearances in a data-driven manner and learns perturbation-invariant features, thus correcting for the performance degradation caused by cross-dataset applications. Attached Figure Description

[0008] Figure 1 This is a flowchart of the method of the present invention;

[0009] Figure 2 This is the Patch-level feature extraction diagram of the present invention;

[0010] Figure 3 This is a structural diagram of the Patch-level feature extractor of the present invention;

[0011] Figure 4 This is a diagram illustrating the construction process of the heterogeneous graph of the present invention;

[0012] Figure 5 This is a diagram of the prior-driven graph neural network structure of the present invention;

[0013] Figure 6 This is a diagram of the global perception graph aggregation layer structure of the present invention;

[0014] Figure 7 This invention serves as an evaluation indicator in the task of predicting sinusoidal invasion of renal cell carcinoma.

[0015] Figure 8 This is a comparison chart of the prediction results of the model in this invention and the diagnostic results of human clinical experts. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] In medical image classification, the introduction of prior information is of great significance, especially in the context of CT image analysis, where data is often scarce and highly dimensional. This high dimensionality makes it difficult for the sample distribution to approximate true distribution characteristics, leading to significant differences between the training and test sets. Therefore, when transferring a model from the training environment to the actual testing environment, introducing prior information can effectively reduce model variance and mitigate performance degradation. CT images contain rich prior knowledge, such as features of different tissue types. Integrating this knowledge into a deep learning framework can normalize the model during training, thereby improving its testing performance in practical applications. However, prior information also presents some challenges in application. First, prior knowledge is usually based on historical data and may not reflect newly emerging pathological features, resulting in insufficient model adaptability to new cases. Second, the selection and construction of prior information can be highly subjective; different experts may arrive at different prior assumptions, affecting model consistency. Therefore, when embedding prior information into the model, it is necessary to comprehensively consider its advantages and limitations, and select appropriate processing methods and technical strategies based on the actual situation to improve the effectiveness and application value of image classification.

[0018] In this embodiment, a method for predicting sinusoidal invasion of renal cell carcinoma based on prior information is described, such as... Figure 1 As shown, the method includes: acquiring CT images and constructing a renal cell carcinoma sinus invasion prediction dataset based on the CT images; extracting features from the data in the dataset and constructing a heterogeneous graph based on the extracted feature maps; inputting the heterogeneous graph into a trained prior-driven graph neural network model to obtain the renal cell carcinoma sinus invasion prediction result; and evaluating the prediction result.

[0019] The process of creating a dataset on sinus invasion of renal cell carcinoma includes:

[0020] Step 1: Segment the CT images using the Unet++ model;

[0021] Step 2: Calculate the minimum bounding box of the kidney in the segmentation mask;

[0022] Step 3: Increase the minimum bounding box by 5px along the x and y directions and calculate the ROI;

[0023] Step 4: Starting from the top left corner of the ROI, use a sliding window strategy to crop a patch image of fixed size r×r (r is the side length of the patch);

[0024] Step 5: Repeat steps 1 to 4 until all case patch images have been captured.

[0025] Preferably, the size of the patch image is 20×20.

[0026] Patch-level feature extraction, such as Figure 2 As shown, it includes:

[0027] Step 1: Obtain the dataset of renal sinus invasion and the corresponding tumor annotations as Patch labels Y patch Patches containing tumors are labeled as 1, and patches containing only kidneys are labeled as 0. The dataset is divided into training, validation, and test sets.

[0028] Step 2: Augment the Patch images in the training set using a diffusion perturbation strategy;

[0029] Step 3: Input the enhanced patch images from the training set into the patch-level feature extraction network to obtain the prediction result P. patch ;

[0030] Step 3: Based on Y patch and P patch Construct the loss function;

[0031] Step 4: Update all parameters of the feature extraction network using the backpropagation mechanism based on the loss function;

[0032] Step 5: Evaluate the network using the validation set and save the parameter weights of the model with the best performance metrics;

[0033] Step 6: Repeat steps 2 through 5 until training is complete;

[0034] Step 7: Remove the last fully connected layer of the feature extraction network as the encoder, load the optimal model parameter weights into the network, and apply the patch image x∈R. r×r The input encoder yields a patch-level feature representation f∈R d (d is the dimension of the feature vector).

[0035] Preferably, the diffusion perturbation strategy labels the original patch image as x0, according to the variance table β1,…,β T Noise is gradually added to the patch, for x t-1 Adding noise yields the patch image x at time step t. t Its expression is as follows:

[0036]

[0037] Where, β t It is a variance table, and ε~N(0,I) is standard Gaussian noise.

[0038] Preferably, the patch-level feature extraction network includes an encoder and a classifier, and the network structure is as follows: Figure 3 As shown, the input is a 2D patch x∈R r×r The encoder consists of six 2D convolutional blocks, one global max-pooling layer, and one multilayer perceptron; the classifier consists of one fully connected layer. Each convolutional block consists of one convolutional layer, one normalization layer, and one activation layer.

[0039] Preferably, the process of constructing heterogeneous graphs is as follows: Figure 4 As shown, it includes:

[0040] Step 1: Define all patch images of a case as a node set V in a heterogeneous graph;

[0041] Step 2: Introduce the type of the patch image (tumor or kidney) as prior information for tissue discrimination into the node type. Nodes containing tumors are encoded as 1, and nodes containing only kidneys are encoded as 0, forming a node type set T.

[0042] Step 3: Input all nodes into the pre-trained encoder to obtain node features f∈R d (d is the dimension of the feature vector), and combined with the corresponding node type to form the node feature matrix F∈R n×d (where f is the dimension of the feature vector and n is the number of nodes);

[0043] Step 4: Define edges and edge attributes based on the spatial proximity prior information of the patch in the original CT image. Connect each node to its spatially adjacent nodes to form an edge set E. For each edge e∈E, calculate the Euclidean distance between the head node and the tail node, then map these distances using a Gaussian distribution function to convert the distances into a continuous probability distribution, forming a spatial proximity matrix A'. Finally, normalize the spatial proximity matrix to obtain the edge attribute matrix A; where the spatial distance d between the head node i and the tail node j is... ij The expression is:

[0044]

[0045] Among them, (x i ,y i ,z i ) and (x j ,y j ,zj ) represent the coordinates of the top left corner of the i-th and j-th patches in the three-dimensional voxel space of the original CT image, respectively.

[0046] The expression for the edge attribute matrix A is:

[0047]

[0048] Where diag and n are the diagonal matrix and the number of nodes, respectively.

[0049] Step 5: Define the heterogeneous graph G = (V, E, T, A) corresponding to this case, where V, E, and T are the node set, edge set, and node type set, respectively, and A represents the edge attribute matrix;

[0050] Step 6: Repeat steps 1 through 5 until all cases have been processed.

[0051] In this embodiment, the prior-driven graph neural network module is as follows: Figure 5 As shown, the algorithm includes a global perception graph aggregation layer, prior pooling, and a multilayer perceptron classifier. After the heterogeneous graph is input into two global perception graph aggregation layers, node features are fused through prior pooling to generate a graph-level feature representation g. The graph-level feature representation g is then input into the multilayer perceptron classifier to produce prediction results.

[0052] Preferably, the structure of the global perception graph aggregation layer is as follows: Figure 6 As shown, for the heterogeneous graph G input to the l-th layer... l-1 , put G l-1 The node feature matrices are mapped to query and key vectors through two 1×1 convolution operations, respectively, as expressed in the following expressions:

[0053]

[0054] Among them, F l-1 It is G l-1 The node feature matrix, and Let S and V represent the parameter matrices related to the two 1×1 convolution operations in the l-th layer, respectively. The feature similarity prior information between nodes is calculated based on the dot product of the query vector and the key vector. This prior information is then normalized using softmax to obtain the feature similarity matrix S of the nodes. l Its expression is as follows:

[0055]

[0056] The node feature similarity matrix S lThe features are added to the edge attribute matrix A as weight coefficients for each neighboring node during the aggregation process. The node features are multiplied by the corresponding weight coefficients to update the features of each node. The heterogeneous graph G output by layer 1. l Updated node features F l It can be represented as:

[0057] F l =σ(( S l+A)F l-1 W l )

[0058] Wherein, σ and W l Here are the activation function and weight parameters for the linear layer. The updated graph G... l This is the output of the l-th global perception graph aggregation layer.

[0059] Prior pooling clusters node representations based on prior information about node types. For each node type t, all node features belonging to that node type t are pooled into a single vector h using max pooling. t Then, the pooling features for each node type are aggregated into the feature matrix P. Subsequently, the graph-level features g are determined by another pooling layer.

[0060] In this embodiment, the training process for the prior-driven graph neural network model includes:

[0061] Step 1: Obtain the renal cell carcinoma sinus invasion prediction dataset and the corresponding renal sinus invasion label Y;

[0062] Step 2: Input the heterogeneous graphs corresponding to a set of cases in the training set into the prior-driven graph neural network model to obtain the prediction result P;

[0063] Step 3: Construct the loss function L based on P and Y;

[0064] Step 4: Update all parameters of the prior-driven graph neural network according to the loss function through backpropagation;

[0065] Step 5: Evaluate the network using the validation set and save the parameter weights of the model with the best performance metrics;

[0066] Step 6: Repeat steps 2 through 5 until training is complete.

[0067] The loss function L is constructed using cross-entropy loss, and its expression is as follows:

[0068]

[0069] Where, num neg The number of cases without renal sinus involvement, num posThe number of cases of renal sinus invasion, y n p is the true label for the nth case. n Let be the predicted label for the nth case, and weight be the weight of the cross-entropy loss.

[0070] In this embodiment, the process for predicting and evaluating sinusoidal invasion of renal cell carcinoma in CT images includes:

[0071] Step 1: Load the optimal model weights of the prior-driven graph neural network;

[0072] Step 2: Input the heterogeneous graphs corresponding to a set of cases in the test set into the prior-driven graph neural network model to obtain the prediction results;

[0073] Step 3: Save the prediction results;

[0074] Step 4: Repeat steps 2 to 3 until all cases are predicted;

[0075] Step 5: Assemble a group of clinical experts with different experience levels and clinical backgrounds to independently diagnose renal sinus invasion by comparing enhanced CT images without knowing the pathological results of the renal sinus invasion.

[0076] Step 6: Compare the model predictions with the diagnoses of human clinical experts for a comprehensive evaluation.

[0077] like Figure 7 As shown, the prior-driven graph neural network can achieve accurate classification in the task of classifying sinusoidal invasion of renal cell carcinoma in CT images and has a high evaluation index.

[0078] Compare the model predictions with the diagnoses of human clinical experts, such as... Figure 8 As shown, senior radiologists (red) performed best in the individual group. In contrast, the present invention was close to that of senior radiologists in the diagnosis of sinusoidal invasion of renal cell carcinoma, and the overall diagnostic level of clinical experts was not as good as that of the prior-driven graph neural network in the present invention.

[0079] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting sinusoidal invasion of renal cell carcinoma based on prior information, characterized in that, include: Acquire CT images and construct a dataset for predicting sinusoidal invasion of renal cell carcinoma based on the CT images; Feature extraction is performed on the data in the dataset, and a heterogeneous graph is constructed based on the extracted feature maps. The heterogeneous graph is input into the trained prior-driven graph neural network model to obtain the prediction results of sinusoidal invasion of renal cell carcinoma. Evaluate the prediction results; The construction of the renal cell carcinoma sinus invasion prediction dataset includes: segmenting CT images using the Unet++ model; calculating the minimum bounding box of the kidney in the segmentation mask; increasing the minimum bounding box by 5px along the x and y directions to calculate the ROI; starting from the top left corner of the ROI, using a sliding window strategy to crop a fixed-size r×r patch image; repeating the above process until all case patch images have been cropped. Feature extraction from the dataset includes: Step 1: Obtain the dataset of renal sinus invasion and the corresponding tumor annotations as Patch labels Y patch Patches containing tumors are marked as 1, otherwise they are marked as 0, and the sets are divided into training set, validation set and test set; Step 2: After data augmentation using a diffusion perturbation strategy, the patch images in the training set are input into the patch-level feature extraction network to obtain the prediction result P. patch ; Data augmentation strategies using diffusion perturbation include: labeling the original patch image as x0, and then using the variance table β1,…,β... T Noise is gradually added to the patch, for x t-1 Adding noise yields the patch image x at time step t. t Its expression is: Where, β t This is a variance table, where ε~N(0,I) is standard Gaussian noise, I is the standard deviation, and T is the diffusion time step; Step 3: Based on Y patch and P patch Construct the loss function; Step 4: Update all parameters of the feature extraction network using the backpropagation mechanism based on the loss function; Step 5: Evaluate the network using the validation set and save the parameter weights of the model with the best performance metrics; Step 6: Repeat steps 2 through 5 until training is complete; Step 7: Remove the last fully connected layer of the feature extraction network as the encoder, load the optimal model parameter weights into the network, and apply the patch image x∈R. r×r The input encoder yields a patch-level feature representation f∈R d d is the dimension of the feature vector; Constructing a heterogeneous graph includes: Step 1: Define all patch images of a case as a node set V in a heterogeneous graph; Step 2: Encode the tumor annotation as a priori information for tissue discrimination into node types. Nodes with tumors are encoded as 1, and those without are encoded as 0, forming a node type set T. Step 3: Input all nodes into the trained encoder to obtain node features f∈R d And combine it with the corresponding node type to form the node feature matrix F∈R n×d where n is the number of nodes; Step 4: Define edges and edge attributes based on the prior information of the spatial location of the patch in the original CT image, and connect each node with the nodes that are spatially adjacent to it to form an edge set E; for each edge e∈E, calculate the Euclidean distance between the head and tail nodes as prior information of spatial proximity and introduce it into the edge attribute A. Step 5: Define the heterogeneous graph G = (V, E, T, A) corresponding to this case, where V, E, and T are the node set, edge set, and node type set, respectively, and A represents the edge attribute matrix. Step 6: Repeat steps 1 to 5 until all cases have been processed; The prior-driven graph neural network model includes a global perception aggregation layer, prior pooling, and a multilayer perceptron classifier. The global perception aggregation layer is used to extract features from each node in the heterogeneous graph. Prior pooling is used to fuse node features to generate a graph-level feature representation g. The multilayer perceptron classifier is used to classify and predict the graph-level feature representation.

2. The method for predicting sinusoidal invasion of renal cell carcinoma based on prior information according to claim 1, characterized in that, Training the prior-driven graph neural network model includes: using a renal cell carcinoma sinus invasion prediction dataset and the corresponding renal sinus invasion label Y as the training set; converting the data in the training set into heterogeneous graphs; inputting the heterogeneous graphs corresponding to a set of cases in the training set into the prior-driven graph neural network model for training to obtain prediction results; constructing a loss function based on the prediction results and the renal sinus invasion label Y; updating all parameters of the prior-driven graph neural network using the backpropagation mechanism based on the loss function; evaluating the network using a validation set, saving the model parameter weights with the best performance, and repeating the above process until training is complete.

3. The method for predicting sinusoidal invasion of renal cell carcinoma based on prior information according to claim 2, characterized in that, The loss function expression is: Where, num neg The number of cases without renal sinus involvement, num pos The number of cases of renal sinus invasion, y n p is the true label for the nth case. n Let be the predicted label for the nth case, and weight be the weight of the cross-entropy loss.

4. The method for predicting sinusoidal invasion of renal cell carcinoma based on prior information according to claim 1, characterized in that, The evaluation of the prediction results includes: loading the optimal model weights of the prior-driven graph neural network; inputting the heterogeneous graphs corresponding to all cases in the test set into the prior-driven graph neural network model to obtain the prediction results; and comparing and evaluating all the prediction results.

Images