Liver cancer microvessel invasion prediction method based on coordinate guidance and multi-tissue information

Abstract

A computed tomography (CT) image is used for carrying out preoperative early recognition on microvascular invasion of a liver cancer patient, important help is provided for preoperative non-invasive diagnosis of the patient and guiding individualized treatment to improve the prognosis effect, and the deep learning technology has been widely applied to the field of medical image classification. When a traditional convolutional neural network is used for transfer learning to a preoperative prediction task of liver cancer microvascular invasion, the problems of unstable feature mining and poor prediction accuracy exist. According to the method, different microvascular invasion examples are regarded as different deformations of tumors, a special coordinate guiding adaptability enhancement attention module is designed, a network is guided to position the tumors and adapt to deformation expression of the tumors, and the problem that the model cannot learn stable features is solved. Secondly, visceral fat of a patient is introduced as an auxiliary image marker, and potential information related to microvascular invasion contained in the visceral fat is excavated at the same time; and the outcome of the patient is jointly predicted in combination with tumor lesions and visceral fat, and finally excellent prediction accuracy is achieved.

Description

Technical field:

[0001] The present invention relates to deep learning feature extraction technology and dual image marker fusion prediction technology, in particular to a coordinate-guided image feature extraction technology with enhanced adaptability. The method can be specifically applied in the field of preoperative prediction and prevention of microvascular invasion of liver cancer. Background technology:

[0002] For liver computed tomography images of patients with liver cancer, some classic deep learning algorithms such as Resnet, Densenet, VGG, Mobilenet, and Shufflenet can classify and predict microvascular invasion, but the prediction effect of using these algorithms for transfer learning is not ideal. Liver tumors are reflected in different tumor sizes, shapes, locations, and enhancement patterns on computed tomography images. Unlike traditional prediction tasks, microvascular invasion of liver cancer may appear inside the tumor or around the tumor. This makes it difficult for conventional deep learning image feature extraction algorithms to learn stable features for microvascular invasion tasks, thus affecting the final prediction accuracy. Image feature extraction methods should be improved so that the model can enhance feature extraction of high-incidence areas of microvascular invasion.

[0003] In addition, existing prediction models rely solely on the imaging features of tumor lesions, ignoring information from other tissues related to disease progression, which limits the final prediction accuracy. Visceral adipose tissue in the abdomen has been shown to be associated with microvascular invasion, but a model that integrates the deep imaging features of adipose tissue and tumor lesions for joint prediction has not yet been developed. Summary of the invention:

[0004] Aiming at the problem of poor accuracy in preoperative prediction of liver cancer microvascular invasion, the present invention discloses a method for predicting liver cancer microvascular invasion based on coordinate guidance and multi-tissue information.

[0005] To this end, the present invention provides the following technical solutions:

[0006] 1. A method for predicting liver cancer microvascular invasion based on coordinate guidance and multi-tissue information, characterized in that the method comprises the following steps:

[0007] Step 1: Collect preoperative upper abdominal liver CT images of patients with liver cancer. The layer of the largest cross-section of the liver tumor and the lesser omentum layer were selected as the original images of the liver tumor and visceral fat, respectively. The liver tumor image of interest and the visceral fat image of interest (irregular area between the porta hepatis and the lesser curvature of the stomach) of 224×224 size were cropped from the original images. The image dataset was randomly divided into a training set (weight parameters of the training model), a validation set (adjusting the hyperparameters of the model), and a test set (evaluating the model performance) in a ratio of 3:1:3.

[0008] Step 2: Use CT images with dual imaging markers to capture multi-tissue complementary information through a deep learning model. For the highly heterogeneous liver tumor images of interest, a special coordinate-guided adaptive enhancement module is designed and embedded into the first two residual blocks of Resnet18, and then a coordinate-guided adaptive enhancement network is constructed to locate and mine the most relevant representations of microvascular invasion. For the visceral fat images of interest, a densely connected Densenet121 network is used to obtain deep features such as the radiodensity value, texture structure, and specific distribution changes of adipose tissue.

[0009] Step 3: Use the least absolute shrinkage and selection operator (LASSO) logistic regression algorithm to perform feature selection on the extracted tumor features and fat features respectively to eliminate redundant features and find the most discriminative features.

[0010] Step 4: The final tumor feature vector obtained after feature selection and the fat feature vector are directly concatenated together using the early fusion method to obtain the fusion features of the dual imaging markers.

[0011] Step 5: Use support vector machine as the final classifier, use the tumor features and fusion features of the training set to train the classifier, and predict the final negative or positive class label in the test set.

[0012] 2. According to the method for predicting liver cancer microvascular invasion based on coordinate guidance and multi-tissue information in claim 1, it is characterized in that in step 2, the specific algorithm steps of the coordinate guidance adaptability enhancement module are:

[0013] 1) For a given input tensor F∈R C×H×W : Average pooling is performed along the spatial direction and the horizontal direction to obtain At the same time, a 3×3 deformable convolution is used to obtain the encoded tensor F'∈R C ×H×W .

[0014] 2) Putting them together in the spatial dimension, we get

[0015] Then, through a 1×1 Conv layer (Conv, Batchnorm, Relu), the channels of the input feature map are linearly combined to obtain the mixed channel information without dimensionality reduction.

[0016] 3) Split F into two tensor representations along the spatial direction, F x ∈R C×H×1 and F y ∈R C×1×W, respectively apply Sigmoid gating to get the weight representation W x ∈R C×H×1 and W y ∈R C×1×W .

[0017] 4) In F'∈R C×H×W Perform global average pooling and global maximum pooling in the channel direction to obtain

[0018] 5) Concatenate in the channel dimension

[0019] Then the channel is remapped to 1 dimension through a 1×1 Conv layer, and we get

[0020] 6) Yes Apply the Sigmoid gating mechanism to generate the attention weight representation W z ∈R 1×H×W .

[0021] 7) Denote the weight W x , W y and W z Multiply it with the input feature F to get the output tensor F = F⊙W x ⊙W y ⊙W z .

[0022] Beneficial effects:

[0023] 1. By introducing a dedicated coordinate-guided adaptive feature extraction module, different microvascular invasion instances are regarded as certain deformations of the tumor, allowing the model to more accurately locate the liver tumor and more flexibly and adaptively capture different morphological changes of the tumor.

[0024] 2. The prediction accuracy is improved by integrating the deep features of visceral fat, an auxiliary imaging marker.

[0025] 3. A new prediction method is proposed for the preoperative diagnosis of microvascular invasion of liver cancer, which can help patients achieve initial non-invasive diagnosis before surgery. Description of the drawings:

[0026] Figure 1 The overall flow chart of the prediction method provided by the present invention.

[0027] Figure 2 A specific architecture diagram of the coordinate-guided adaptability-enhanced network provided by the present invention.

[0028] Figure 3It is a ROC diagram after the prediction method is executed in the embodiment of the present invention. Specific implementation method:

[0029] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the present invention.

[0030] Taking the people who came to a certain cancer hospital for liver cancer testing within five years as an example, a liver cancer microvascular invasion prediction method based on coordinate guidance and multi-tissue information was implemented.

[0031] The flowchart of the method for predicting liver cancer microvascular invasion based on coordinate guidance and multi-tissue information is shown in the following example. Figure 1 As shown, the following steps are included:

[0032] Step 1: Collect preoperative upper abdominal liver CT images of patients with liver cancer. The layer of the largest cross-section of the liver tumor and the lesser omentum layer were selected as the original images of the liver tumor and visceral fat, respectively. The liver tumor image of interest and the visceral fat image of interest (irregular area between the porta hepatis and the lesser curvature of the stomach) of 224×224 size were cropped from the original images. The image dataset was randomly divided into a training set (weight parameters of the training model), a validation set (adjusting the hyperparameters of the model), and a test set (evaluating the model performance) in a ratio of 3:1:3.

[0033] Step 2: The training set and validation set of liver tumor images of interest obtained in step 1 are input into Figure 2 In the coordinate-guided adaptive enhancement network shown in FIG. 1 , after the network loss on the validation set gradually and steadily converges, the model training stops, the trained model is obtained, and the feature vectors corresponding to the training set, validation set, and test set are extracted in sequence. At the same time, the visceral fat image dataset of interest is also input into Densenet121 as described above and the corresponding feature vectors are extracted.

[0034] Step 3: Use the least absolute shrinkage and selection operator (LASSO) logistic regression algorithm to select features of tumor features and fat features in the training set obtained in step 2, so as to eliminate redundant features and find the most discriminative features. The test set retains the selected features as in the training set.

[0035] Step 4: The tumor feature vector and the fat feature vector obtained in step 3 are directly concatenated together using the early fusion method to obtain the dual-image marker fusion features of the training set and the test set.

[0036] Step 5: Use support vector machine as the final classifier, use the tumor features and fusion features of the training set to train the classifier, and predict the final negative or positive class label in the test set.

[0037] The method for predicting liver cancer microvascular invasion based on coordinate guidance and multi-tissue information in the embodiment of the present invention has high accuracy in predicting microvascular invasion in liver cancer patients. The ROC result is as follows: Figure 3 shown.

[0038] The above is a detailed introduction to the implementation rules of the present invention in conjunction with the accompanying drawings. It should be pointed out that the specific implementation methods herein are only used to help understand the method of the present invention. A person skilled in the art can make several changes and modifications based on the present invention, and the above changes and modified technical solutions should all be within the scope of protection determined by the claims.

Claims

1. A method for predicting liver cancer microvascular invasion based on coordinate guidance and multi-tissue information, characterized in that: The method comprises the following steps: Step 1: Collect preoperative upper abdominal liver CT images of patients with liver cancer. The layer of the largest cross-section of the liver tumor and the lesser omentum layer were selected as the original images of the liver tumor and visceral fat, respectively. The liver tumor image of interest and the visceral fat image of interest (irregular area between the porta hepatis and the lesser curvature of the stomach) of 224×224 size were cropped from the original images. The image dataset was randomly divided into a training set (weight parameters of the training model), a validation set (adjusting the hyperparameters of the model), and a test set (evaluating the model performance) in a ratio of 3:1:

3. Step 2: Use CT images with dual imaging markers to capture multi-tissue complementary information through a deep learning model. For images of interest in liver tumors, a special coordinate-guided adaptive enhancement module is designed and embedded into the first two residual blocks of Resnet18, and then a coordinate-guided adaptive enhancement network is constructed to locate and mine the most relevant representations of microvascular invasion. For images of interest in visceral fat, a densely connected Densenet121 network is used to obtain deep features such as the radiodensity value, texture structure, and specific distribution changes of adipose tissue. Step 3: Use the least absolute shrinkage and selection operator (LASSO) logistic regression algorithm to perform feature selection on the extracted tumor features and fat features respectively to eliminate redundant features and find the most discriminative features. Step 4: The final tumor feature vector obtained after feature selection and the fat feature vector are directly concatenated together using the early fusion method to obtain the fusion features of the dual imaging markers. Step 5: Use the support vector machine as the final classifier, use the tumor features and fusion features of the training set to train the classifier, and predict the final negative or positive class label in the test set.

2. The method for predicting liver cancer microvascular invasion based on coordinate guidance and multi-tissue information according to claim 1, characterized in that: In step 2, the specific algorithm steps of the coordinate guidance adaptability enhancement module are: 1) For a given input tensor F∈R C×H×W : Average pooling is performed along the spatial direction and the horizontal direction to obtain and At the same time, a 3×3 deformable convolution is used to obtain the encoded tensor F'∈R C ×H×W . 2) and Putting them together in the spatial dimension, we get Then, through a 1×1 Conv layer (Conv, Batchnorm, Relu), the channels of the input feature map are linearly combined to obtain the mixed channel information without dimensionality reduction. 3) Split F into two tensor representations along the spatial direction, F x ∈R C×H×1 and F y ∈R C×1×W , respectively apply Sigmoid gating to get the weight representation W x ∈R C×H×1 and W y ∈R C×1×W . 4) In F'∈R C×H×W Perform global average pooling and global maximum pooling in the channel direction to obtain and 5) and Concatenate in the channel dimension Then the channel is remapped to 1 dimension through a 1×1 Conv layer, and we get 6) Yes Apply the Sigmoid gating mechanism to generate the attention weight representation W z ∈R 1×H×W . 7) Denote the weight W x , W y and W z Multiply it with the input feature F to get the output tensor F = F⊙W x ⊙W y ⊙W z .

Images