Construction method and prediction method of prediction models for VETC and MVI in hepatocellular carcinoma

Abstract

This application discloses a method for obtaining a training data set; training an initial prediction model based on the training data set to obtain a VETC status prediction model and an MVI status prediction model; constructing a prediction model for VETC and MVI of hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model. This application constructs a VETC status prediction model and an MVI status prediction model by collecting CT images of multiple imaging types, and then predicts the VETC status and the MVI status based on the constructed VETC status prediction model and MVI status prediction model. In this way, the prognosis can be predicted by combining the VETC status and the MVI status, and the value of risk stratification can be reflected in the prognosis of surgical resection and the prognosis of TACE treatment.

Description

Technical Field

[0001] This application relates to the field of biomedical engineering technology, and particularly relates to a method for constructing a prediction model and a prediction method for VETC and MVI of hepatocellular carcinoma. Background Art

[0002] Hepatocellular carcinoma (HCC) is one of the common and highly prevalent malignant tumors. Currently, surgical resection is the preferred option for radical treatment of HCC, and transcatheter arterial chemoembolization (TACE) and other non-radical treatment options are selected for non-radical treatment. Currently, clinical and basic research have shown that the recurrence after HCC surgery and the poor response to TACE treatment are closely related to the high invasiveness of HCC. Evaluating the invasiveness of HCC before treatment helps to formulate a reasonable treatment plan. Therefore, evaluating the invasiveness of HCC before treatment has important guiding significance for improving the prognosis of HCC.

[0003] Currently, the evaluation of the invasiveness of HCC before treatment is generally carried out by using imaging techniques. Imaging methods are effective means commonly used in clinical practice to reflect the invasiveness of tumors by evaluating the morphological characteristics, blood perfusion, etc. of tumors. However, the existing prediction methods using imaging techniques are generally affected by the observer's evaluation of morphological signs, and there are problems of poor consistency and strong subjectivity in the observer's evaluation of morphological signs, which will further lead to low accuracy in predicting the invasiveness of HCC before treatment.

[0004] Therefore, the existing technology still needs to be improved. Summary of the Invention [[ID=,18]]

[0005] The technical problem to be solved by this application is to provide a method for constructing a prediction model and a prediction method for VETC and MVI of hepatocellular carcinoma in view of the deficiencies of the existing technology.

[0006] To solve the above technical problem, the first aspect of this application provides a method for constructing a prediction model for VETC and MVI of hepatocellular carcinoma. Specifically, the method for constructing the prediction model for VETC and MVI of hepatocellular carcinoma includes:

[0007] Obtain a training data set, where the training data set includes a number of training data, and each training data in the number of training data includes at least two types of CT images, tumor lesion masks, labeled VETC status, and labeled MVI status;

[0008] Train an initial prediction model based on the training dataset to obtain a VETC status prediction model and an MVI status prediction model;

[0009] Construct a prediction model for VETC and MVI of hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model.

[0010] The method for constructing the prediction model for VETC and MVI of hepatocellular carcinoma, wherein the CT images of at least two imaging types include conventional CT images in the arterial phase, effective atomic number CT images, electron density CT images, iodine map CT images, and single-energy CT images with an energy of 40 keV, as well as conventional CT images, effective atomic number CT images, electron density CT images, iodine map CT images, and single-energy CT images with an energy of 40 keV in the portal phase.

[0011] The method for constructing the prediction model for VETC and MVI of hepatocellular carcinoma, wherein the VETC status prediction model and the MVI status prediction model adopt the same construction process, and the construction process specifically includes:

[0012] For the CT images of each imaging type, train the initial prediction model based on each CT image of the imaging type in the training dataset, the tumor lesion mask, and the target annotation status to obtain an intermediate prediction model corresponding to each imaging type;

[0013] Construct a target prediction model based on the training dataset and the intermediate prediction model corresponding to each imaging type. Wherein, when the target annotation status is to annotate the VETC status, the target prediction model is the VETC status prediction model, and when the target annotation status is to annotate the MVI status, the target prediction model is the MVI status prediction model.

[0014] The method for constructing the prediction model for VETC and MVI of hepatocellular carcinoma, wherein the constructing the target prediction model based on the training dataset and the intermediate prediction model corresponding to each imaging type specifically includes:

[0015] Select a candidate prediction model from the intermediate prediction models corresponding to each imaging type based on the model performance, and use the other intermediate prediction models as reference prediction models;

[0016] Determine the target prediction model based on the candidate prediction model and the reference prediction models. Wherein, determining the target prediction model based on the candidate prediction model and the reference prediction models specifically includes:

[0017] Determine the first feature map of the CT image of the corresponding imaging type through the candidate prediction model, and the second feature maps of the CT images of the corresponding imaging types through the respective reference prediction models;

[0018] Concatenate the second feature map corresponding to each reference prediction model with the first feature map determined by the candidate prediction model to obtain a number of concatenated feature maps;

[0019] Input each concatenated feature map into the classification module of the candidate prediction model respectively to obtain the target prediction state corresponding to each concatenated feature map;

[0020] Train the candidate prediction model based on each target prediction state and the target annotation state respectively, and select the target prediction model from the trained candidate prediction models based on the model performance.

[0021] The method for constructing the prediction model of VETC and MVI of hepatocellular carcinoma, wherein, after training the candidate prediction model based on each target prediction state and the target annotation state respectively, and selecting the target prediction model from the trained candidate prediction models based on the model performance, the method further includes:

[0022] Check several imaging types based on the model performance of each trained candidate prediction model to obtain several candidate imaging types;

[0023] Take the selected target prediction model as the candidate prediction model, take the intermediate prediction model corresponding to each candidate imaging type as the reference prediction model, and repeat to determine the target prediction model based on the candidate prediction model and the reference prediction model;

[0024] Repeat the step of screening the trained candidate prediction models except the selected target prediction model based on the model performance until all imaging types are selected and included or excluded, or the number of CT images combined reaches the preset quantity requirement to obtain the target prediction model.

[0025] The method for constructing the prediction model of VETC and MVI of hepatocellular carcinoma, wherein the initial prediction model includes an encoding module, a decoding module and a classification module, the encoding module and the decoding module form a U-net network structure, the encoding module is connected to the classification module, wherein the classification module includes a linear feature extraction layer and a linear classification layer, the linear feature extraction layer is connected to the encoding module, the linear classification layer is connected to the linear feature extraction layer, the linear feature extraction layer is used to expand the feature map input by the encoding module into a one-dimensional feature vector and perform feature extraction on the one-dimensional feature vector, and the linear classification layer is used to perform state prediction based on the features extracted by the linear feature extraction layer.

[0026] The second aspect of the present application provides a method for predicting VETC and MVI of hepatocellular carcinoma, using a target prediction model constructed by the method for constructing a prediction model of VETC and MVI of hepatocellular carcinoma as described above. The method for predicting VETC and MVI of hepatocellular carcinoma specifically includes:

[0027] Obtain a number of first imaging types corresponding to the VETC status prediction model in the target prediction model and a number of first imaging types of the MVI status prediction model;

[0028] Obtain the CT images corresponding to each first imaging type to obtain a number of first CT images;

[0029] Obtain the CT images corresponding to each second imaging type to obtain a number of second CT images;

[0030] Input the number of first CT images into the VETC status prediction model to obtain the predicted VETC status;

[0031] Input the number of second CT images into the MVI status prediction model to obtain the predicted MVI status.

[0032] The third aspect of the present application provides a device for constructing a prediction model of VETC and MVI of hepatocellular carcinoma. Specifically, the device for constructing a prediction model of VETC and MVI of hepatocellular carcinoma includes:

[0033] An acquisition module for acquiring a training data set, where the training data set includes a number of training data, and each training data in the number of training data includes CT images of at least two imaging types, a tumor lesion mask, a labeled VETC status, and a labeled MVI status;

[0034] A training module for training an initial prediction model based on the training data set to obtain a VETC status prediction model and an MVI status prediction model;

[0035] A construction module for constructing a prediction model of VETC and MVI of hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model.

[0036] The fourth aspect of the present application provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps in the method for constructing a prediction model of VETC and MVI of hepatocellular carcinoma as described above.

[0037] The fifth aspect of the present application provides a terminal device, which includes: a processor and a memory;

[0038] A computer-readable program that can be executed by the processor is stored in the memory;

[0039] When the processor executes the computer-readable program, the steps in the method for constructing the prediction model of VETC and MVI of hepatocellular carcinoma described above are implemented.

[0040] Beneficial effects: Compared with the prior art, the present application provides a method for constructing a prediction model of VETC and MVI of hepatocellular carcinoma and a prediction method. The method includes obtaining a training data set; training an initial prediction model based on the training data set to obtain a VETC status prediction model and an MVI status prediction model; constructing a prediction model of VETC and MVI of hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model. The present application constructs a VETC status prediction model and an MVI status prediction model by collecting CT images of multiple imaging types, and then predicts the VETC status and the MVI status based on the constructed VETC status prediction model and MVI status prediction model. In this way, prognostic prediction can be combined with the VETC status and the MVI status, and the value of risk stratification can be reflected in the prognosis of surgical resection and the prognosis of TACE treatment. Description of the Drawings

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0042] Figure 1 It is a flowchart of the method for constructing the prediction model of VETC and MVI of hepatocellular carcinoma provided by the embodiment of the present application.

[0043] Figure 2 It is an example diagram of a CT image.

[0044] Figure 3 It is a principle flowchart of the initial prediction model.

[0045] Figure 4 It is a principle flowchart of single-image features.

[0046] Figure 5 It is a principle flowchart of multi-fused image features.

[0047] Figure 6 It is a structural schematic diagram of the device for constructing the prediction model of VETC and MVI of hepatocellular carcinoma provided by the embodiment of the present application.

[0048] Figure 7This is the structural schematic diagram of the terminal device provided by the embodiment of the present application. Detailed implementation manners

[0049] The embodiment of the present application provides a method for constructing and a prediction method for a prediction model of VETC and MVI of hepatocellular carcinoma. To make the objectives, technical solutions and effects of the present application clearer and more definite, the following further describes the present application in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] Those skilled in the art of the present technology can understand that unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "including" used in the specification of the present application means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more of the associated listed items.

[0051] Those skilled in the art of the present technology can understand that unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0052] It should be understood that the sequence numbers and magnitudes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.

[0053] It has been found through research that hepatocellular carcinoma (HCC) is one of the common malignant tumors with a high incidence rate. At present, surgical resection is the preferred option for radical treatment of HCC, and transcatheter arterial chemoembolization (TACE) and other non-radical treatment options are selected for non-radical treatment. Current clinical and basic research have shown that the recurrence after HCC surgery and the poor response to TACE treatment are closely related to the high invasiveness of HCC. Evaluating the invasiveness of HCC before treatment helps to formulate a reasonable treatment plan. Therefore, evaluating the invasiveness of HCC before treatment has important guiding significance for improving the prognosis of HCC.

[0054] In current clinical work, a study based on clinical laboratory indicators has shown that 6 clinical indicators such as neutrophils can be effective predictors of preoperative VETC. However, laboratory indicators have low specificity and poor stability, requiring a relatively harsh experimental environment. Another feasible solution is to use imaging techniques for prediction. Imaging methods are effective means commonly used in clinical practice to reflect tumor invasiveness by evaluating the morphological characteristics, blood perfusion, etc. of tumors. Two MRI-based qualitative studies have shown that imaging indicators such as tumor size, heterogeneous enhancement of septal or irregular ring-like structures, necrosis, and the signal ratio of the tumor to the liver parenchyma are independent risk predictors of VETC. Another CT-based study has shown that a score composed of the combination of AFP and necrosis can effectively distinguish VETC. However, these imaging studies all have a common shortcoming, that is, the consistency of observers' evaluation of morphological signs is poor and the subjectivity is strong.

[0055] To solve the above problems, in the embodiments of the present application, a training data set is obtained; the initial prediction model is trained based on the training data set to obtain a VETC status prediction model and an MVI status prediction model; a prediction model for VETC and MVI of hepatocellular carcinoma is constructed based on the VETC status prediction model and the MVI status prediction model. In the present application, CT images of multiple imaging types are collected to construct a VETC status prediction model and an MVI status prediction model, and then the VETC status and MVI status are predicted based on the constructed VETC status prediction model and MVI status prediction model. In this way, the prognosis can be predicted by combining the VETC status and the MVI status, and the value of risk stratification can be reflected in the prognosis of surgical resection and the prognosis of TACE treatment.

[0056] The following further illustrates the application content through the description of embodiments in conjunction with the accompanying drawings.

[0057] This embodiment provides a method for constructing a prediction model for VETC and MVI of hepatocellular carcinoma, as Figure 1 shown, the method includes:

[0058] S10. Obtain a training data set.

[0059] Specifically, the training data set includes training data, and each training data includes at least two types of CT images of imaging types, a tumor lesion mask, a labeled VETC status, and a labeled MVI status. Among them, the imaging types included in each training data are the same. For example, each training data includes a CT image of imaging type A and a CT image of imaging type B. In addition, each training data can come from a hepatocellular carcinoma patient, that is to say, the CT images of each imaging type in each training data are all obtained by collecting CT images of the same hepatocellular carcinoma patient, and the training data is the CT image before the patient is treated with the treatment method corresponding to the prognosis status of hepatocellular carcinoma.

[0060] Furthermore, in addition to belonging to different imaging types, the CT images included in each training data can also be of different imaging periods. For example, the CT images in the training data can be CT images in the arterial phase or CT images in the portal vein phase. In a typical implementation, the at least two types of CT images of imaging types include CT images in the arterial phase and CT images in the portal vein phase, and both the CT images in the arterial phase and the CT images in the portal vein phase include five types of CT images, namely, conventional imaging type, effective atomic number imaging type, electron density imaging type, iodine map imaging type, and single energy imaging type with an energy of 40 keV. That is to say, the at least two types of CT images of imaging types are 10 CT images, namely, conventional CT image, effective atomic number CT image, electron density CT image, iodine map CT image, and single energy CT image with an energy of 40 keV in the arterial phase, and conventional CT image, effective atomic number CT image, electron density CT image, iodine map CT image, and single energy CT image with an energy of 40 keV in the portal vein phase. Of course, in actual applications, the at least two types of CT images of imaging types can also include CT images of other imaging types, or include some of the imaging types among the conventional imaging type, effective atomic number imaging type, electron density imaging type, iodine map imaging type, and single energy imaging type with an energy of 40 keV, etc.

[0061] The training data can be collected for hepatocellular carcinoma patients after surgical treatment to form a training data set, and then the model is trained based on this training data set to obtain a target prediction model. However, in actual use, the target prediction model obtained by training with a training data set collected for hepatocellular carcinoma patients after surgery can also be used for predicting the prognosis of TACE treatment.

[0062] For example: Collect the preoperative CT images and clinical data of liver cancer patients, which are used as the surgical cohort and the TACE cohort respectively. The surgical cohort is used to form the training dataset. The TACE cohort is used as the test set. The target prediction model is verified for the prognosis prediction of TACE treatment through this test set, that is, the target prediction model predicts the corresponding predicted VETC status and MVI status of the CT data in the TACE cohort, and then predicts the prognosis status of TACE treatment based on the predicted VETC status and MVI status, and analyzes the correlation with the TACE treatment prognosis status in the TACE cohort to verify the target prediction model.

[0063] The inclusion criteria for the surgical cohort are as follows:

[0064] Perform enhanced CT scan of the liver on IQon spectral CT; 2) Undergo surgical resection within 1 month after scanning; 3) Pathologically confirmed as HCC with CD34 immunohistochemical results; 4) Have not received other treatments such as radiofrequency or TACE before surgery; 5) Have complete SBI data in the hepatic arterial phase and portal venous phase; 6) The pathological data is relatively comprehensive. The collection and examination time points of the follow-up information of the surgical cohort are as follows: once a month in the first three months after surgery, and then re-examined every 3 - 6 months. The tumor recurrence is used as the end point during the follow-up period, and the follow-up time is 2 years.

[0065] The inclusion criteria for the TACE cohort are as follows: Imaging diagnosis of HCC or percutaneous biopsy confirmed HCC according to the EASL (European Association for the Study of the Liver) criteria or the American Liver Cancer Diagnosis Guidelines (LI-RADS v2018); 2) Receive TACE as the first-line treatment within 1 month after scanning; 3) Have complete SBI data in the hepatic arterial phase and portal venous phase; 4) The pathological data is relatively comprehensive; 5) Have no previous history of tumor treatment before CT examination. The TACE efficacy evaluation is performed 1 month after treatment by means of CT examination, and the TACE treatment efficacy is determined according to the modified Response Evaluation Criteria in Solid Tumors (mRECIST).

[0066] The tumor lesion mask includes a number of tumor lesion masks, and the number of tumor lesion masks is the same as the number of imaging periods corresponding to the CT images in the training data. That is to say, when obtaining the tumor lesion mask, the tumor lesion mask for each imaging period will be obtained. In addition, since the CT images of each imaging type in the training data are absolutely registered, when obtaining the tumor lesion mask, one of the CT images of each imaging type corresponding to each scan time can be labeled to obtain the tumor lesion mask corresponding to this imaging period. Among them, the CT image used for labeling is determined according to the clarity of the CT image. For example, Figure 2As shown, among the conventional CT image, effective atomic number CT image, electron density CT image, iodine map CT image, and monoenergetic CT image with an energy of 40 keV, the monoenergetic CT image with an energy of 40 keV has the highest image clarity. Thus, a tumor lesion mask can be marked on the monoenergetic CT image with an energy of 40 keV. Of course, in practical applications, a random selection method can also be used to determine the CT image for marking, or marking can also be performed on each type of CT image.

[0067] In the embodiment of the present application, the CT image with the highest clarity is used as the CT image for marking, which can improve the accuracy of the marked tumor lesion mask. At the same time, since all CT images are absolutely registered, marking a tumor lesion mask on one CT image can reduce the labor cost and time cost required for tumor lesion mask marking while ensuring the accuracy of the tumor lesion mask for each type of CT image.

[0068] In addition, although the CT images are absolutely registered under the same conditions, the dimensions of different CT images are different. Therefore, before marking the tumor lesion mask, preprocessing needs to be performed on each CT image. The preprocessing can include normalization and denoising processing. The denoising processing is used to remove irrelevant pixel information. Specifically, the denoising processing can perform intensity threshold processing according to the pixel value characteristics of different CT imaging: for the conventional CT image, values in the range of [-25, -175] are taken; for the effective atomic number, values within 1% to 99% are taken; for the electron density map, values in the range of [101, 109] are taken; for the iodine map, values in the range of [0, 4] are taken; and for the 40 keV monoenergetic image, values in the range of [-120, 240] are taken. Normalization is used to avoid differences in the data distribution of different patients and improve the model convergence speed. Specifically, the Min-max normalization method can be used to preprocess the patient image data. Calculate the maximum pixel value and the minimum pixel value of the patient's 3D image, and then map all pixel values to between 0 and 1. The formula is shown as follows.

[0069]

[0070] Among them, X represents the normalized pixel value, x represents the original pixel value, x min represents the minimum pixel value, x max represents the maximum pixel value.

[0071] S20. Train the initial prediction model based on the training data set to obtain a VETC state prediction model and an MVI state prediction model.

[0072] Specifically, both the VETC status prediction model and the MVI status prediction model are trained based on a training dataset. Among them, the VETC status prediction model uses the tumor lesion mask and the labeled VETC status as the labeled tags to supervise the construction process of the VETC status prediction model; the MVI status prediction model uses the tumor lesion mask and the labeled MVI status as the labeled tags to supervise the construction process of the MVI status prediction model. That is to say, the model structures of the VETC status prediction model and the MVI status prediction model are the same, but the model parameters are different. Among them, the model parameters of the VETC status prediction model are obtained by supervised training with the labeled VETC status, and the model parameters of the MVI status prediction model are obtained by supervised training with the labeled MVI status.

[0073] It should be noted that when training the VETC status prediction model and the MVI status prediction model, only different supervised data are used to learn different model parameters, and the training steps executed during the training process are the same. The VETC status prediction model and the MVI status prediction model adopt the same construction process. Therefore, here, a construction process common to the VETC status prediction model and the MVI status prediction model is used to illustrate the construction processes of the VETC status prediction model and the MVI status prediction model. In this construction process common to the VETC status prediction model and the MVI status prediction model, the target annotation status is used as the annotation information to train the target prediction model. Among them, when the target annotation status is the labeled VETC status, the target prediction model is the VETC status prediction model; when the target annotation status is the labeled MVI status, the target prediction model is the MVI status prediction model.

[0074] Exemplarily, the construction process specifically includes:

[0075] S21. For each type of CT image of the imaging type, based on each CT image of the imaging type, the tumor lesion mask, and the target annotation status in the training dataset, train the initial prediction model to obtain an intermediate prediction model corresponding to each imaging type;

[0076] S22. Based on the training dataset and the intermediate prediction model corresponding to each imaging type, construct a target prediction model. Among them, when the target annotation status is the labeled VETC status, the target prediction model is the VETC status prediction model; when the target annotation status is the labeled MVI status, the target prediction model is the MVI status prediction model.

[0077] In step S21, the initial prediction model is a multi-task learning network including a segmentation task and a classification task. Taking self-predicted segmentation as guiding knowledge, it is used to guide and improve the classification task of medical images. The initial prediction model includes an encoding module, a decoding module, and a classification module. The encoding module and the decoding module form a U-net network structure. The encoding module is connected to the classification module. The classification module includes a linear feature extraction layer and a linear classification layer. The linear feature extraction layer is connected to the encoding module, and the linear classification layer is connected to the linear feature extraction layer. The linear feature extraction layer is used to expand the feature map input by the encoding module into a one-dimensional feature vector and extract features from the one-dimensional feature vector. The linear classification layer is used to perform state prediction based on the features extracted by the linear feature extraction layer.

[0078] For example: The initial prediction model is the multi-task learning network RS 2 -net, as Figure 3 shown, the CT image is input into the encoding module, and the image depth features are extracted through the encoders E1 - E4 in the encoding module. The extracted image depth features are input into the decoders D3 - D0 in the decoding module to obtain the segmentation prediction result. The extracted image depth features are input into the linear classification layer through max pooling to obtain the binary classification prediction result. Then, a loss function is constructed based on the segmentation prediction result, the binary classification prediction result, the tumor lesion mask, and the target annotation status for backpropagation learning to obtain the intermediate prediction model corresponding to each imaging type.

[0079] In the embodiment of the present application, by using a multi-task learning network including a segmentation task and a classification task as the initial prediction model, it is possible to use self-predicted segmentation as guiding knowledge to guide and improve the classification task of medical images, learn lesion information, and thus reduce lesion-unrelated noise, enabling the model to learn the feature basis for distinguishing tumor lesions from normal tissues. At the same time, there is no need to specifically conduct a classification study on the segmented area, avoiding the problem that the performance decreases due to the omission of some key pixel information in the classification task caused by the error in the segmentation model itself.

[0080] It should be noted that the network settings of the initial network model can be preset. Specifically, the network settings can be as follows: 1) A total of 200 training rounds are set. First, in the first 50 epochs, only the segmentation loss function is optimized, and only the weights of the network layers related to segmentation are learned and updated. In the subsequent 150 epochs, both the segmentation loss function and the classification loss function are optimized, with weight coefficients of 0.1 and 1 respectively, and all network layer weights will be learned and updated. The purpose of doing this is to let the segmentation network first focus on the feature extraction of the lesion area in the first 50 epochs, and then in the subsequent 150 epochs, let the classification network be able to perform more accurate classification training by combining the semantic information pre-obtained in the segmentation network; 2) Secondly, for each CT image of each imaging type in each imaging period in the training dataset, the network framework is used for the modeling behavior. Therefore, multiple intermediate prediction models will be trained respectively for each of the two classification tasks of the VETC state and the MVI state.

[0081] As can be seen from the above, the intermediate prediction model is obtained by training the initial prediction model, and each imaging type in each period in the training dataset corresponds to an intermediate prediction model. The CT image corresponding to each imaging type in each period is the input item of its corresponding intermediate prediction model, that is, the intermediate prediction model is a deep learning model with single-image input. Then, the number of the trained intermediate prediction models is the same as the number of CT images included in the training data, and the intermediate prediction models correspond one-to-one with the CT images included in the training data. For example, if the training data includes CT images of 5 imaging types in two periods, then intermediate prediction models are constructed for the ten images respectively, and 10 intermediate prediction models will be constructed for the VETC state and 10 intermediate prediction models will be constructed for the MVI state.

[0082] In step S22, the target prediction model is constructed based on the training data set and the intermediate prediction models corresponding to each imaging type. Among them, the target prediction model can be the one with the optimal model performance among the intermediate prediction models, or it can be obtained by combining multiple intermediate prediction models by adopting a feature screening and fusion strategy for the input of the linear classification layer. Here, the combination means extracting classification features through multiple intermediate prediction models respectively, and then splicing the extracted classification features as the input items of the linear classification layer for separate training to obtain the target prediction model. That is to say, for each of the two classification tasks of the VETC state and the MVI state, different imaging types of CT images at different imaging periods are used to obtain image classification features (features of the upper layer of the final classification prediction probability output) through their respective corresponding intermediate prediction models. Then, the image classification features are screened based on the model performance, and the finally screened image features are spliced and input into a simple linear classification layer for separate training to obtain a target prediction model including multiple input images. Among them, the target prediction model can combine multiple CT images to determine the classification structure (i.e., predict the target state).

[0083] Exemplarily, constructing the target prediction model based on the training data set and the intermediate prediction models corresponding to each imaging type specifically includes:

[0084] Select a candidate prediction model from the intermediate prediction models corresponding to each imaging type based on the model performance, and use the other intermediate prediction models as reference prediction models;

[0085] Determine the target prediction model based on the candidate prediction model and the reference prediction models. Among them, determining the target prediction model based on the candidate prediction model and the reference prediction models specifically includes:

[0086] Determine the first feature map of the CT image of the corresponding imaging type through the candidate prediction model, and determine the second feature maps of the CT images of the corresponding imaging types of each reference prediction model;

[0087] Splice the second feature map corresponding to each reference prediction model with the first feature map determined by the candidate prediction model to obtain several spliced feature maps;

[0088] Input each spliced feature map into the classification module of the candidate prediction model to obtain the target prediction states corresponding to each spliced feature map;

[0089] Train the candidate prediction model based on each target prediction state and the target annotation state respectively, and select the target prediction model from the trained candidate prediction models based on the model performance.

[0090] Specifically, the model performance as the basis means selecting the one with the priority of model performance. Among them, the optimal model performance means the last in the preset performance indicators. For example, if the preset performance indicator is the AUC indicator, then select the intermediate prediction model with the highest classification AUC as the candidate prediction model. Of course, in practical applications, other indicators can also be used as the preset performance indicators. In addition, before selecting a candidate prediction model from the intermediate prediction models corresponding to each imaging type based on the model performance, the intermediate prediction models can be screened first. For example, exclude the intermediate prediction model whose classification AUC of single-image modeling has no statistically significant difference from 0.5, that is, exclude the CT image corresponding to this intermediate prediction model.

[0091] As Figure 4 shown, both the first feature map and the second feature map are image classification features. Among them, the first feature map is the image feature extracted by the classification module in the candidate prediction model, and the second feature map is the image feature extracted by the classification module in the reference prediction model. Specifically, after obtaining the candidate prediction model, use the CT image corresponding to the candidate prediction model as the first type of CT image, extract the first feature map (i.e., the image classification feature) of the first type of CT image through the candidate prediction model, and extract the second feature map (i.e., the image classification feature) of various corresponding CT images through each reference prediction model. As Figure 5 shown, after obtaining the first feature map and each second feature map, splice the first feature map with each second feature map respectively to obtain a spliced feature map, so as to obtain the image classification feature obtained by combining the first type of CT image and other CT images. Use the spliced feature map to independently train classification prediction to obtain a candidate prediction model with the combination of two types of CT images as the input item. Finally, select the target prediction model from the trained candidate prediction models based on the model performance. Finally, use the other CT image except the first type of CT image in the two CT images of the selected target prediction model as the second type of CT image. Then, when making a prediction based on the target prediction model, the first type of CT image and the second type of CT image will be used as the input items.

[0092] Furthermore, in practical applications, it is also possible to combine the target prediction model of three CT images, the target model of four CT images, etc. For this reason, after training the candidate prediction models respectively based on the respective target prediction states and target annotation states, and selecting the target prediction model from the trained candidate prediction models based on the model performance, the method further includes:

[0093] Check several imaging types based on the model performance of each trained candidate prediction model to obtain several candidate imaging types;

[0094] Select the selected target prediction model as the candidate prediction model, and use the intermediate prediction models corresponding to the respective candidate imaging types as the reference prediction models, and repeat the determination of the target prediction model based on the candidate prediction model and the reference prediction model;

[0095] Repeat the step of screening the trained candidate prediction models other than the selected target prediction model based on the model performance until all imaging types are selected and included or excluded, or the number of combined CT images reaches the preset quantity requirement to obtain the target prediction model.

[0096] Specifically, checking several imaging types based on the model performance of each trained candidate prediction model means excluding those with a prediction performance lower than that of the optimal single CT image after combining two CT images. The target prediction model is the one with the optimal prediction performance among the combinations of two CT images, so the target prediction model is used as the candidate prediction model, and the other excluded intermediate prediction models are used as the reference prediction models. Among them, the other excluded intermediate prediction models do not include the two CT images corresponding to the target prediction model and the intermediate prediction models corresponding to the excluded CT images. Then, combine the two CT images corresponding to the target prediction model with the other excluded CT images to obtain the target prediction model for the combination of three CT images. Then, the above operations can be repeated for the combination of four CT images, the combination of five CT images, until all CT images are selected and included or excluded, or the number of combined CT images reaches the preset quantity requirement.

[0097] S30. Construct a prediction model for VETC and MVI of hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model.

[0098] Specifically, after obtaining the VETC status prediction model and the MVI status prediction model, the VETC status prediction model and the MVI status prediction model are combined into a prediction model. Through the prediction model, the predicted VETC status and the predicted MVI status can be determined, where both the predicted VETC status and the predicted MVI status can be positive or negative. Microvascular invasion (MVI) is a key factor predicting postoperative recurrence and poor prognosis of HCC, and it is related to the epithelial-mesenchymal transition (EMT) process of tumors. Vessels that encapsulating tumor clusters (VETC) is a tumor metastasis mode mediated by vascular structure independent of the EMT process. By combining the VETC status and the MVI status for prognosis prediction, the value of risk stratification can be reflected in the prognosis of surgical resection and the prognosis of TACE treatment.

[0099] In summary, this embodiment provides a method for constructing a prediction model for VETC and MVI of hepatocellular carcinoma. The method includes obtaining a training data set; training an initial prediction model based on the training data set to obtain a VETC status prediction model and an MVI status prediction model; constructing a prediction model for VETC and MVI of hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model. This application constructs a VETC status prediction model and an MVI status prediction model by collecting CT images of multiple imaging types, and then predicts the VETC status and the MVI status based on the constructed VETC status prediction model and MVI status prediction model. In this way, the prognosis can be predicted by combining the VETC status and the MVI status, and the value of risk stratification can be reflected in the prognosis of surgical resection and the prognosis of TACE treatment.

[0100] Based on the above method for constructing a prediction model for VETC and MVI of hepatocellular carcinoma, the embodiment of this application also provides a prediction method for VETC and MVI of hepatocellular carcinoma. Using the prediction model constructed by the method for constructing a prediction model for VETC and MVI of hepatocellular carcinoma as described above, the prediction method for VETC and MVI of hepatocellular carcinoma specifically includes:

[0101] Obtain several first imaging types corresponding to the VETC status prediction model in the prediction model and several first imaging types of the MVI status prediction model;

[0102] Obtain the CT images corresponding to each first imaging type to obtain several first CT images;

[0103] Obtain CT images corresponding to each second imaging type to obtain a number of second CT images;

[0104] Input a number of first CT images into the VETC status prediction model to obtain the predicted VETC status;

[0105] Input a number of second CT images into the MVI status prediction model to obtain the predicted MVI status.

[0106] Based on the above method for constructing the prediction models of VETC and MVI of hepatocellular carcinoma, an embodiment of the present application further provides a device for constructing the prediction models of VETC and MVI of hepatocellular carcinoma, as Figure 6 shown. The device for constructing the prediction models of VETC and MVI of hepatocellular carcinoma specifically includes:

[0107] An acquisition module 100, configured to acquire a training data set, where the training data set includes a number of training data, and each training data in the number of training data includes CT images of at least two imaging types, a tumor lesion mask, a labeled VETC status, and a labeled MVI status;

[0108] A training module 200, configured to train an initial prediction model based on the training data set to obtain a VETC status prediction model and an MVI status prediction model;

[0109] A construction module 300, configured to construct a prediction model of VETC and MVI of hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model.

[0110] Based on the above method for constructing the prediction models of VETC and MVI of hepatocellular carcinoma, this embodiment provides a computer-readable storage medium. The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the method for constructing the prediction models of VETC and MVI of hepatocellular carcinoma as described in the above embodiments, and / or to implement the steps in the prediction method of VETC and MVI of hepatocellular carcinoma as described in the above embodiments.

[0111] Based on the above method for constructing the prediction models of VETC and MVI of hepatocellular carcinoma, the present application further provides a terminal device, as Figure 7As shown in the figure, it includes at least one processor 20; a display screen 21; and a memory 22. It may also include a communication interface 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22, and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is set to display a user guidance interface preset in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logical instructions in the memory 22 to execute the method in the above-mentioned embodiments.

[0112] In addition, when the logical instructions in the above-mentioned memory 22 can be implemented in the form of software function units and sold or used as an independent product, they can be stored in a computer-readable storage medium.

[0113] The memory 22, as a computer-readable storage medium, can be set to store software programs and computer-executable programs, such as the program instructions or modules corresponding to the method in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, that is, implements the method in the above-mentioned embodiments.

[0114] The memory 22 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs can also be transient storage media.

[0115] In addition, the specific processes of loading and executing multiple instructions by the above-mentioned storage medium and the instruction processor in the terminal device have been described in detail in the above method, and will not be repeated here one by one.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, characterized in that, The method for constructing the prediction model of VETC and MVI for hepatocellular carcinoma specifically includes: Obtain a training data set, where the training data set includes a number of training data, and each training data in the number of training data includes at least two types of CT images of imaging types, tumor lesion masks, labeled VETC status, and labeled MVI status; Train an initial prediction model based on the training data set to obtain a VETC status prediction model and an MVI status prediction model; Construct a prediction model of VETC and MVI for hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model; The VETC status prediction model and the MVI status prediction model adopt the same construction process, and the construction process specifically includes: For each type of CT image of imaging type, train the initial prediction model based on each CT image of the imaging type in the training data set, the tumor lesion mask, and the target labeled status to obtain an intermediate prediction model corresponding to each imaging type, where the intermediate prediction model is a deep learning model with single-image input, the number of intermediate prediction models is the same as the number of types of imaging types of CT images included in the training data, and the intermediate prediction models correspond one by one to the types of imaging types of CT images included in the training data; Construct a target prediction model based on the training data set and the intermediate prediction model corresponding to each imaging type, where when the target labeled status is the labeled VETC status, the target prediction model is the VETC status prediction model, and when the target labeled status is the labeled MVI status, the target prediction model is the MVI status prediction model; The constructing the target prediction model based on the training data set and the intermediate prediction model corresponding to each imaging type specifically includes: Select a candidate prediction model from the intermediate prediction models corresponding to each imaging type based on model performance, and use the other intermediate prediction models as reference prediction models; Determine the target prediction model based on the candidate prediction model and the reference prediction models; The determining the target prediction model based on the candidate prediction model and the reference prediction models specifically includes: Determine the first feature map of the CT image of the corresponding imaging type through the candidate prediction model, and the second feature maps of the CT images of the corresponding imaging types of each reference prediction model; Stitch the second feature map corresponding to each reference prediction model with the first feature map determined by the candidate prediction model to obtain a number of stitched feature maps; Input each stitched feature map into the classification module of the candidate prediction model to obtain the target prediction status corresponding to each stitched feature map; Train the candidate prediction model based on each target prediction status and the target labeled status, and select the target prediction model from the trained candidate prediction models based on model performance.

2. The method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma according to claim 1, wherein The CT images of the at least two imaging types include conventional CT images in the arterial phase, effective atomic number CT images, electron density CT images, iodine map CT images, and single-energy CT images with an energy of 40 keV, as well as conventional CT images, effective atomic number CT images, electron density CT images, iodine map CT images, and single-energy CT images with an energy of 40 keV in the portal phase.

3. The method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma according to claim 1, characterized in that, After training candidate prediction models respectively based on each target prediction state and target annotation state, and selecting a target prediction model from the trained candidate prediction models according to the model performance, the method further includes: Checking several imaging types based on the model performance of each trained candidate prediction model to obtain several candidate imaging types; Taking the selected target prediction model as the candidate prediction model, taking the intermediate prediction models corresponding to each candidate imaging type as the reference prediction models, and repeating to determine the target prediction model based on the candidate prediction model and the reference prediction model; Repeating the step of screening the trained candidate prediction models except the selected target prediction model based on the model performance until all imaging types are selected to be included or excluded, or until the number of combined CT images reaches the preset quantity requirement, so as to obtain the target prediction model.

4. The method for constructing a prediction model for VETC and MVI of hepatocellular carcinoma according to claim 1, wherein 5. A prediction method for VETC and MVI of hepatocellular carcinoma, characterized in that, The initial prediction model includes an encoding module, a decoding module, and a classification module. The encoding module and the decoding module form a U-net network structure. The encoding module is connected to the classification module. Among them, the classification module includes a linear feature extraction layer and a linear classification layer. The linear feature extraction layer is connected to the encoding module, and the linear classification layer is connected to the linear feature extraction layer. The linear feature extraction layer is used to expand the feature map input by the encoding module into a one-dimensional feature vector and extract features from the one-dimensional feature vector. The linear classification layer is used to predict the state based on the features extracted by the linear feature extraction layer. Using the prediction model construction method for VETC and MVI of hepatocellular carcinoma as described in any one of claims 1-4, the prediction method for VETC and MVI of the hepatocellular carcinoma specifically includes: Obtaining several first imaging types corresponding to the VETC state prediction model in the prediction model and several first imaging types of the MVI state prediction model; Obtaining the CT images corresponding to each first imaging type to obtain several first CT images; Obtaining the CT images corresponding to each second imaging type to obtain several second CT images; Inputting several first CT images into the VETC state prediction model to obtain the predicted VETC state; 6. An apparatus for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, characterized in that, Inputting several second CT images into the MVI state prediction model to obtain the predicted MVI state. The prediction model construction device for VETC and MVI of the hepatocellular carcinoma specifically includes: An acquisition module, configured to acquire a training data set, where the training data set includes several training data, and each training data in the several training data includes CT images of at least two imaging types, a tumor lesion mask, an annotated VETC state, and an annotated MVI state; A training module for training an initial prediction model based on the training dataset to obtain a VETC status prediction model and an MVI status prediction model; A construction module for constructing a prediction model of VETC and MVI of hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model; Wherein, the VETC status prediction model and the MVI status prediction model adopt the same construction process, and the construction process specifically includes: For the CT images of each imaging type, the initial prediction model is trained based on each CT image of the imaging type, the tumor lesion mask, and the target annotation status in the training dataset to obtain an intermediate prediction model corresponding to each imaging type; A target prediction model is constructed based on the training dataset and the intermediate prediction model corresponding to each imaging type. Wherein, when the target annotation status is to annotate the VETC status, the target prediction model is the VETC status prediction model, and when the target annotation status is to annotate the MVI status, the target prediction model is the MVI status prediction model; The constructing the target prediction model based on the training dataset and the intermediate prediction model corresponding to each imaging type specifically includes: Selecting a candidate prediction model from the intermediate prediction models corresponding to each imaging type based on the model performance, and using the other intermediate prediction models as reference prediction models; Determining the target prediction model based on the candidate prediction model and the reference prediction models; The determining the target prediction model based on the candidate prediction model and the reference prediction models specifically includes: Determining a first feature map of the CT image of the corresponding imaging type through the candidate prediction model, and second feature maps of the CT images of the corresponding imaging types of each reference prediction model; Stitching the second feature map corresponding to each reference prediction model with the first feature map determined by the candidate prediction model to obtain a plurality of stitched feature maps; Inputting each stitched feature map into the classification module of the candidate prediction model to obtain the target prediction status corresponding to each stitched feature map; Training the candidate prediction model based on each target prediction status and the target annotation status, and selecting the target prediction model from the trained candidate prediction models based on the model performance.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the method for constructing a prediction model of VETC and MVI of hepatocellular carcinoma according to any one of claims 1-4, and / or to implement the steps in the method for predicting VETC and MVI of hepatocellular carcinoma according to claim 5.

8. A terminal device, characterized in that, Including: A processor and a memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, it implements the steps in the method for constructing a prediction model of VETC and MVI of hepatocellular carcinoma according to any one of claims 1-4, and / or implements the steps in the method for predicting VETC and MVI of hepatocellular carcinoma according to claim 5.

Images