Construction method and prediction method of VETC and MVI prediction model of hepatocellular carcinoma

Abstract

The application discloses a method for acquiring a training data set; training an initial prediction model based on the training data set to obtain a VEC state prediction model and an MVI state prediction model; and constructing a VEC and MVI prediction model of the hepatocellular carcinoma based on the VEC state prediction model and the MVI state prediction model. According to the method and the device, the VEC state prediction model and the MVI state prediction model are constructed by collecting the CT images of various imaging types, and then the VEC state and the MVI state are predicted based on the construction of the VEC state prediction model and the MVI state prediction model, so that prognosis prediction can be performed by combining the VEC state and the MVI state, and the value of risk stratification can be reflected in operation resection prognosis and TACE treatment prognosis.

Description

Technical Field

[0001] The present application relates to the field of biomedical engineering technology, and in particular to a method for constructing a prediction model and a prediction method for VETC and MVI of hepatocellular carcinoma. Background Art

[0002] Hepatocellular carcinoma (HCC) is one of the most common malignant tumors with a high incidence rate. At present, surgical resection is the first choice for radical treatment of HCC, and non-radical treatment options such as transcatheter arterial chemoembolization (TACE) are also selected for non-radical treatment. Current clinical and basic research shows that postoperative recurrence of HCC and poor response to TACE treatment are closely related to the high invasiveness of HCC. Pre-treatment evaluation of HCC invasiveness is helpful to formulate a reasonable treatment plan. Therefore, pre-treatment evaluation of HCC invasiveness has important guiding significance for improving the prognosis of HCC.

[0003] At present, the invasiveness of HCC is generally predicted by imaging technology before treatment. Imaging is an effective means of reflecting tumor invasiveness by evaluating the morphological characteristics and blood perfusion of tumors. However, the existing imaging technology prediction methods are generally affected by the observer's evaluation of morphological signs, and the observer's evaluation of morphological signs has poor consistency and strong subjectivity, which leads to low accuracy in the prediction of HCC invasiveness before treatment.

[0004] Therefore the prior art still needs to be improved and enhanced. Summary of the invention

[0005] The technical problem to be solved by the present application is to provide a method for constructing a prediction model and a prediction method for VETC and MVI of hepatocellular carcinoma in view of the deficiencies of the existing technology.

[0006] In order to solve the above technical problems, the first aspect of the present application provides a method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, wherein the method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma specifically includes:

[0007] Acquire a training data set, wherein the training data set includes a plurality of training data, and each of the plurality of training data includes CT images of at least two imaging types, a tumor lesion mask, a marked VETC state, and a marked MVI state;

[0008] Training the initial prediction model based on the training data set to obtain a VETC state prediction model and an MVI state prediction model;

[0009] Based on the VETC status prediction model and the MVI status prediction model, a prediction model of VETC and MVI for hepatocellular carcinoma is constructed.

[0010] The method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, wherein the at least two imaging types of CT images include conventional CT images, effective atomic number CT images, electron density CT images, iodine map CT images and single-energy CT images with 40keV energy in the arterial phase, and conventional CT images, effective atomic number CT images, electron density CT images, iodine map CT images and single-energy CT images with 40keV energy in the portal venous phase.

[0011] The method for constructing the prediction model of VETC and MVI for hepatocellular carcinoma, wherein the VETC status prediction model and the MVI status prediction model adopt the same construction process, and the construction process specifically includes:

[0012] For each imaging type of CT image, the initial prediction model is trained based on each CT image of the imaging type, the tumor lesion mask, and the target annotation status in the training data set to obtain an intermediate prediction model corresponding to each imaging type;

[0013] A target prediction model is constructed based on the training data set and the intermediate prediction model corresponding to each imaging type, wherein when the target annotation state is the annotated VETC state, the target prediction model is the VETC state prediction model, and when the target annotation state is the annotated MVI state, the target prediction model is the MVI state prediction model.

[0014] The method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, wherein the step of constructing a target prediction model based on the training data set and the intermediate prediction model corresponding to each imaging type specifically includes:

[0015] Selecting a candidate prediction model from the intermediate prediction models corresponding to each imaging type based on model performance, and using the other intermediate prediction models as reference prediction models;

[0016] Determining a target prediction model based on the candidate prediction model and the reference prediction model, wherein determining the target prediction model based on the candidate prediction model and the reference prediction model specifically includes:

[0017] Determine a first feature map of a CT image of an imaging type corresponding to the candidate prediction model, and a second feature map of a CT image of an imaging type corresponding to each reference prediction model;

[0018] Splicing the second feature map corresponding to each reference prediction model with the first feature map determined by the candidate prediction model to obtain a plurality of spliced ​​feature maps;

[0019] Input each spliced ​​feature map into the classification module of the candidate prediction model respectively to obtain the target prediction state corresponding to each spliced ​​feature map;

[0020] The candidate prediction models are trained based on the target prediction status and target labeling status respectively, and the target prediction model is selected from the trained candidate prediction models based on the model performance.

[0021] The method for constructing the prediction model of VETC and MVI of hepatocellular carcinoma, wherein the candidate prediction models are trained based on each target prediction state and target annotation state, and after selecting the target prediction model from each trained candidate prediction model based on the model performance, the method further includes:

[0022] Screening a number of imaging types based on the model performance of each trained candidate prediction model to obtain a number of candidate imaging types;

[0023] The selected target prediction model is used as a candidate prediction model, the intermediate prediction model corresponding to each candidate imaging type is used as a reference prediction model, and the target prediction model is repeatedly determined based on the candidate prediction model and the reference prediction model;

[0024] The step of screening each trained candidate prediction model except the selected target prediction model based on model performance is repeated until all imaging types are selected for inclusion or exclusion, or the number of combined CT images reaches a preset number requirement to obtain the target prediction model.

[0025] The method for constructing a prediction model of VETC and MVI of hepatocellular carcinoma, wherein the initial prediction model includes an encoding module, a decoding module and a classification module, the encoding module and the decoding module form a U-net network structure, the encoding module is connected to the classification module, wherein the classification module includes a linear feature extraction layer and a linear classification layer, the linear feature extraction layer is connected to the encoding module, the linear classification layer is connected to the linear feature extraction layer, the linear feature extraction layer is used to expand the feature map input by the encoding module into a one-dimensional feature vector, and extract features from the one-dimensional feature vector, and the linear classification layer is used to perform state prediction based on the features extracted by the linear feature extraction layer.

[0026] In a second aspect, the present application provides a method for predicting VETC and MVI of hepatocellular carcinoma, using a target prediction model constructed based on the method for constructing a prediction model of VETC and MVI of hepatocellular carcinoma as described above, the method for predicting VETC and MVI of hepatocellular carcinoma comprising:

[0027] Acquire a plurality of first imaging types corresponding to the VETC state prediction model and a plurality of first imaging types corresponding to the MVI state prediction model in the target prediction model;

[0028] Acquire CT images corresponding to each first imaging type to obtain a plurality of first CT images;

[0029] Acquire CT images corresponding to each second imaging type to obtain a plurality of second CT images;

[0030] Inputting the plurality of first CT images into a VETC state prediction model to obtain a predicted VETC state;

[0031] The plurality of second CT images are input into the MVI state prediction model to obtain a predicted MVI state.

[0032] The third aspect of the present application provides a device for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, wherein the device for constructing a prediction model of VETC and MVI for hepatocellular carcinoma specifically comprises:

[0033] An acquisition module, configured to acquire a training data set, wherein the training data set includes a plurality of training data, each of the plurality of training data includes CT images of at least two imaging types, a tumor lesion mask, a marked VETC status, and a marked MVI status;

[0034] A training module, used for training the initial prediction model based on the training data set to obtain a VETC state prediction model and an MVI state prediction model;

[0035] A construction module is used to construct a prediction model of VETC and MVI for hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps in the method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma as described above.

[0037] A fifth aspect of the present application provides a terminal device, comprising: a processor and a memory;

[0038] The memory stores a computer-readable program executable by the processor;

[0039] When the processor executes the computer-readable program, the processor implements the steps in the method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma as described above.

[0040] Beneficial effects: Compared with the prior art, the present application provides a method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma and a prediction method, the method comprising obtaining a training data set; training an initial prediction model based on the training data set to obtain a VETC state prediction model and an MVI state prediction model; and constructing a prediction model of VETC and MVI for hepatocellular carcinoma based on the VETC state prediction model and the MVI state prediction model. The present application constructs a VETC state prediction model and an MVI state prediction model by collecting CT images of various imaging types, and then predicts the VETC state and MVI state based on the constructed VETC state prediction model and the MVI state prediction model, so that the prognosis can be predicted in combination with the VETC state and the MVI state, which can reflect the value of risk stratification in the prognosis of surgical resection and the prognosis of TACE treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 This is a flow chart of the method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma provided in the examples of the present application.

[0043] Figure 2 This is an example of a CT image.

[0044] Figure 3 This is the principle flow chart of the initial prediction model.

[0045] Figure 4 The principle flow chart of single image features.

[0046] Figure 5 The principle flow chart of multi-fusion image features.

[0047] Figure 6 This is a schematic diagram of the structure of a device for constructing a prediction model of VETC and MVI for hepatocellular carcinoma provided in an embodiment of the present application.

[0048] Figure 7A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The present application embodiment provides a method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma and a prediction method. In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0051] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0052] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0053] Studies have shown that hepatocellular carcinoma (HCC) is one of the most common malignant tumors with a high incidence rate. At present, surgical resection is the first choice for radical treatment of HCC, and non-radical treatment options such as transcatheter arterial chemoembolization (TACE) are used. Current clinical and basic research shows that postoperative recurrence of HCC and poor response to TACE treatment are closely related to the high invasiveness of HCC. Pre-treatment evaluation of HCC invasiveness is helpful in formulating a reasonable treatment plan. Therefore, pre-treatment evaluation of HCC invasiveness is of great guiding significance for improving the prognosis of HCC.

[0054] In today's clinical work, a study based on clinical laboratory indicators showed that 6 clinical indicators such as neutrophils can be used as effective predictors of preoperative VETC. However, laboratory indicators have low specificity and poor stability, which requires a more demanding experimental environment. Another feasible solution is to use imaging technology for prediction. Imaging methods are currently commonly used in clinical practice to reflect the aggressiveness of tumors by evaluating the morphological characteristics and blood perfusion of tumors. Two qualitative studies based on MRI have shown that imaging indicators such as tumor size, uneven enhancement of septate or irregular ring structures, necrosis, and tumor-to-liver parenchyma signal ratio are independent risk predictors of VETC. Another study based on CT showed that the score composed of a combination of AFP and necrosis can effectively identify VETC. However, these imaging studies all have a common shortcoming, that is, the consistency of observers' evaluation of morphological signs is poor and subjective.

[0055] In order to solve the above problems, in an embodiment of the present application, a training data set is obtained; an initial prediction model is trained based on the training data set to obtain a VETC state prediction model and an MVI state prediction model; and a prediction model of VETC and MVI for hepatocellular carcinoma is constructed based on the VETC state prediction model and the MVI state prediction model. The present application constructs a VETC state prediction model and an MVI state prediction model by collecting CT images of various imaging types, and then predicts the VETC state and MVI state based on the constructed VETC state prediction model and the MVI state prediction model. In this way, the VETC state and the MVI state can be combined for prognosis prediction, which can reflect the value of risk stratification in the prognosis of surgical resection and the prognosis of TACE treatment.

[0056] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0057] This embodiment provides a method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma. Figure 1 As shown, the method includes:

[0058] S10. Obtain a training data set.

[0059] Specifically, the training data set includes training data, each of which includes CT images of at least two imaging types, tumor lesion masks, annotated VETC states, and annotated MVI states, wherein the imaging types included in each training data are the same, for example, each training data includes CT images of imaging type A and CT images of imaging type B. In addition, each training data may come from a patient with hepatocellular carcinoma, that is, the CT images of each imaging type in each training data are obtained by collecting CT images of the same patient with hepatocellular carcinoma, and the training data are CT images of the patient before receiving a treatment method corresponding to the prognostic state of hepatocellular carcinoma.

[0060] Furthermore, the CT images included in each training data may not only belong to different imaging types, but also may be of different imaging periods. For example, the CT images in the training data may be CT images of the arterial phase and CT images of the portal phase. In a typical implementation, the CT images of the at least two imaging types include CT images of the arterial phase and CT images of the portal phase, and the CT images of the arterial phase and the CT images of the portal phase each include CT images of five imaging types, namely, conventional imaging type, effective atomic number imaging type, electron density imaging type, iodine image imaging type, and single energy imaging type of 40keV energy. That is, the CT images of at least two imaging types are 10 CT images, namely, conventional CT images of the arterial phase, effective atomic number CT images, electron density CT images, iodine image CT images, and single energy CT images of 40keV energy, and conventional CT images of the portal phase, effective atomic number CT images, electron density CT images, iodine image CT images, and single energy CT images of 40keV energy. Of course, in practical applications, at least two imaging types of CT images may also include CT images of other imaging types, or some imaging types including conventional imaging type, effective atomic number imaging type, electron density imaging type, iodine image imaging type and single energy imaging type with 40keV energy, etc.

[0061] The training data can be collected from patients with hepatocellular carcinoma after surgery to form a training data set, and then the model is trained based on the training data set to obtain a target prediction model. However, in actual use, the target prediction model obtained by training the training data set collected from patients with hepatocellular carcinoma after surgery can also be used to predict the prognosis of TACE treatment.

[0062] For example: preoperative CT images and clinical data of liver cancer patients are collected as the surgical cohort and TACE cohort respectively. The surgical cohort is used to form a training data set. The TACE cohort is used as a test set, and the target prediction model is used to verify the prognosis prediction of TACE treatment, that is, the VETC status and MVI status corresponding to the CT data in the TACE cohort are predicted by the target prediction model, and then the TACE treatment prognosis status is predicted based on the predicted VETC status and MVI status, and the correlation is analyzed with the TACE treatment prognosis status in the TACE cohort to verify the target prediction model.

[0063] Inclusion criteria for the surgical cohort were:

[0064] 1) Liver enhanced CT scan with IQon spectral CT; 2) Surgical resection within 1 month after the scan; 3) HCC confirmed by pathology and CD34 immunohistochemistry results; 4) No other treatments such as radiofrequency or TACE were received before surgery; 5) Complete hepatic arterial and portal phase SBI data; 6) Relatively comprehensive pathological data. The collection and examination time points for follow-up information of the surgical cohort were: once a month in the first three months after surgery, and then every 3-6 months. Tumor recurrence was the end point during the follow-up period, and the follow-up time was 2 years.

[0065] The inclusion criteria for the TACE cohort were: 1) HCC confirmed by imaging or biopsy according to the EASL (European Association for the Study of the Liver) criteria or the American Liver Cancer Diagnostic Guidelines (LI-RADS v2018); 2) receiving TACE as first-line treatment within 1 month after the scan; 3) having complete hepatic arterial and portal phase SBI data; 4) relatively comprehensive pathological data; 5) no history of previous tumor treatment before CT examination. The efficacy of TACE was evaluated 1 month after treatment by CT examination, and the efficacy of TACE treatment was determined according to the modified solid tumor response evaluation criteria (mRECIST).

[0066] The tumor lesion mask includes a plurality of tumor lesion masks, and the number of imaging periods corresponding to the plurality of tumor lesion masks is the same as that of the CT images in the training data. That is, when the tumor lesion mask is obtained, the tumor lesion mask of each imaging period is obtained. In addition, since the training data includes CT images of each imaging type that are absolutely registered, when the tumor lesion mask is obtained, one imaging type of CT image among all imaging types of CT images corresponding to each scanning time can be annotated to obtain the tumor lesion mask corresponding to the imaging period. The CT image used for annotation is determined according to the clarity of the CT image, for example, Figure 2As shown in the figure, among the conventional CT images, the effective atomic number CT images, the electron density CT images, the iodine map CT images and the 40keV single energy CT images, the 40keV single energy CT images have the highest image clarity, so that the tumor lesion mask can be annotated on the 40keV single energy CT images. Of course, in practical applications, a random selection method can also be used to determine the CT images for annotation, or annotation can be performed on each CT image.

[0067] The embodiment of the present application uses the CT image with the highest definition as the CT image to be annotated, which can improve the accuracy of the annotated tumor lesion mask. At the same time, since all CT images are absolutely registered, annotating a tumor lesion mask on a CT image can reduce the manpower cost and time cost required for annotating the tumor lesion mask while ensuring the accuracy of the tumor lesion mask of each CT image.

[0068] In addition, although each CT image is absolutely registered under the same conditions, the dimensions of different CT images need to be preprocessed before marking the tumor lesion mask. The preprocessing may include normalization and denoising. The denoising is used to remove irrelevant pixel information. Specifically, the denoising can perform intensity threshold processing according to the pixel value characteristics of different CT images: conventional CT images take values ​​in the range of [-25, -175], effective atomic numbers take values ​​in the range of 1% to 99%, electron density maps take values ​​in the range of [101, 109], iodine maps take values ​​in the range of [0, 4], and 40ev single energy maps take values ​​in the range of [-120, 240]. Normalization is used to avoid differences in data distribution of different patients and improve the convergence speed of the model. Specifically, normalization can use the Min-max normalization method to preprocess the patient image data. Calculate the maximum and minimum pixel values ​​of the patient's 3D image, and then map all pixel values ​​to between 0 and 1. The formula is as follows.

[0069]

[0070] Among them, X represents the normalized pixel value, x represents the original pixel value, and x min Indicates the minimum pixel value, x max Indicates the maximum pixel value.

[0071] S20. Training the initial prediction model based on the training data set to obtain a VETC state prediction model and an MVI state prediction model.

[0072] Specifically, the VETC state prediction model and the MVI state prediction model are both obtained through training based on the training data set, wherein the VETC state prediction model uses the tumor lesion mask and the annotated VETC state as the annotation labels to supervise the construction process of the VETC state prediction model; the MVI state prediction model uses the tumor lesion mask and the annotated MVI state as the annotation labels to supervise the construction process of the MVI state prediction model. In other words, the VETC state prediction model and the MVI state prediction model have the same model structure but different model parameters, wherein the model parameters of the VETC state prediction model are obtained through supervised training with the annotated VETC state, and the model parameters of the MVI state prediction model are obtained through supervised training with the annotated MVI state.

[0073] It should be noted that when training the VETC state prediction model and the MVI state prediction model, only different supervision data are used to learn different model parameters, and the training steps performed during the training process are the same, and the VETC state prediction model and the MVI state prediction model use the same construction process. For this reason, here a construction process that is common to the VETC state prediction model and the MVI state prediction model is used to illustrate the construction process of the VETC state prediction model and the MVI state prediction model. In the construction process that is common to the VETC state prediction model and the MVI state prediction model, the target annotation state is used as the annotation information to train the target prediction model, wherein when the target annotation state is the labeled VETC state, the target prediction model is the VETC state prediction model, and when the target annotation state is the labeled MVI state, the target prediction model is the MVI state prediction model.

[0074] Exemplarily, the construction process specifically includes:

[0075] S21, for each imaging type of CT image, training the initial prediction model based on each imaging type of CT image, tumor lesion mask, and target annotation status in the training data set to obtain an intermediate prediction model corresponding to each imaging type;

[0076] S22. Construct a target prediction model based on the training data set and the intermediate prediction model corresponding to each imaging type, wherein when the target annotation state is the annotated VETC state, the target prediction model is the VETC state prediction model, and when the target annotation state is the annotated MVI state, the target prediction model is the MVI state prediction model.

[0077] In step S21, the initial prediction model is a multi-task learning network including segmentation tasks and classification tasks, and uses self-prediction segmentation as guiding knowledge to guide the improvement of medical image classification tasks. The initial prediction model includes an encoding module, a decoding module and a classification module, the encoding module and the decoding module form a U-net network structure, the encoding module is connected to the classification module, wherein the classification module includes a linear feature extraction layer and a linear classification layer, the linear feature extraction layer is connected to the encoding module, the linear classification layer is connected to the linear feature extraction layer, the linear feature extraction layer is used to expand the feature map input by the encoding module into a one-dimensional feature vector, and extract features from the one-dimensional feature vector, and the linear classification layer is used to perform state prediction based on the features extracted by the linear feature extraction layer.

[0078] For example: the initial prediction model is a multi-task learning network RS 2 -net, such as Figure 3 As shown, the CT image is input into the encoding module, and the image depth features are extracted through the encoders E1-E4 in the encoding module. The extracted image depth features are input into the decoders D3-D0 in the decoding module to obtain the segmentation prediction results. The extracted image depth features are input into the linear classification layer after maximum pooling to obtain the binary classification prediction results. Then, based on the segmentation prediction results, the binary classification prediction results, the tumor lesion mask and the target annotation status, a loss function is constructed for reverse learning to obtain the intermediate prediction model corresponding to each imaging type.

[0079] The embodiment of the present application adopts a multi-task learning network including segmentation tasks and classification tasks as the initial prediction model. It can use self-predicted segmentation as guidance knowledge to guide the improvement of medical image classification tasks, learn lesion information, and then reduce lesion-irrelevant noise, so that the model can learn the characteristic basis for distinguishing tumor lesions from normal tissues. At the same time, there is no need to conduct a special classification study on the segmented area, avoiding the problem of performance degradation caused by the classification task missing some key pixel information due to errors in the segmentation model itself.

[0080] It should be noted that the network settings of the initial network model can be pre-set. Specifically, the network settings can be 1) set to a total of 200 training rounds. First, the first 50 epochs only optimize the segmentation loss function, and only learn to update the network layer weights related to segmentation. The next 150 epochs optimize the segmentation loss function and the classification loss function at the same time, with weight coefficients of 0.1 and 1 respectively, and all network layer weights will be learned and updated. The purpose of this is to allow the segmentation network to focus on the feature extraction of the lesion area in the first 50 epochs, and secondly, the classification network can combine the semantic information pre-acquired in the segmentation network for more accurate classification training in the next 150 epochs; 2) Secondly, each imaging type of CT image in each imaging period in the training data set must use the network framework for modeling behavior, so multiple intermediate prediction models will be trained for each of the two classification tasks of VETC status and MVI status.

[0081] From the above, it can be seen that the intermediate prediction model is obtained by training the initial prediction model, and each imaging type in each period in the training data set corresponds to an intermediate prediction model, and the CT image corresponding to each imaging type in each period is the input item of the corresponding intermediate prediction model, that is, the intermediate prediction model is a deep learning model with single image input. Then, the number of intermediate prediction models obtained by training is the same as the number of CT images included in the training data, and the intermediate prediction models correspond one-to-one to the CT images included in the training data. For example, the training data includes CT images of 5 imaging types in two periods, then intermediate prediction models are constructed for each of the ten images, and 10 intermediate prediction models are constructed for the VETC state and 10 intermediate prediction models are constructed for the MVI state.

[0082] In step S22, the target prediction model is constructed based on the training data set and the intermediate prediction model corresponding to each imaging type, wherein the target prediction model can be the one with the best model performance among the intermediate prediction models, or can be obtained by combining multiple intermediate prediction models using a feature screening and fusion strategy at the input of the linear classification layer, wherein the combination refers to extracting classification features through multiple intermediate prediction models respectively, and then splicing the extracted classification features as input items of the linear classification layer for separate training to obtain the target prediction model. That is to say, for each of the two classification tasks of VETC state and MVI state, the image classification features of CT images of different imaging types at different imaging periods are obtained through the corresponding intermediate prediction models (the features of the upper layer of the final classification prediction probability output), and then the image classification features are screened based on the model performance, and the image features finally screened are spliced ​​and input into a simple linear classification layer for separate training to obtain a target prediction model containing multiple input images, wherein the target prediction model can combine multiple CT images to determine the classification structure (i.e., predict the target state).

[0083] Exemplarily, the constructing of the target prediction model based on the training data set and the intermediate prediction model corresponding to each imaging type specifically includes:

[0084] Selecting a candidate prediction model from the intermediate prediction models corresponding to each imaging type based on model performance, and using the other intermediate prediction models as reference prediction models;

[0085] Determining a target prediction model based on the candidate prediction model and the reference prediction model, wherein determining the target prediction model based on the candidate prediction model and the reference prediction model specifically includes:

[0086] Determine a first feature map of a CT image of an imaging type corresponding to the candidate prediction model, and a second feature map of a CT image of an imaging type corresponding to each reference prediction model;

[0087] Splicing the second feature map corresponding to each reference prediction model with the first feature map determined by the candidate prediction model to obtain a plurality of spliced ​​feature maps;

[0088] Input each spliced ​​feature map into the classification module of the candidate prediction model respectively to obtain the target prediction state corresponding to each spliced ​​feature map;

[0089] The candidate prediction models are trained based on the target prediction status and target labeling status respectively, and the target prediction model is selected from the trained candidate prediction models based on the model performance.

[0090] Specifically, model performance as the basis refers to selecting a model with priority on model performance, wherein the best model performance refers to the model performance being the last in the preset performance indicator. For example, if the preset performance indicator is the AUC indicator, then the intermediate prediction model with the highest classification AUC is selected as the candidate prediction model. Of course, in practical applications, other indicators can also be used as the preset performance indicators. In addition, before selecting a candidate prediction model from the intermediate prediction models corresponding to each imaging type based on model performance, the intermediate prediction models can be screened first, for example, excluding the intermediate prediction models whose single image modeling classification AUC is not statistically significantly different from 0.5, that is, excluding the CT images corresponding to the intermediate prediction models.

[0091] like Figure 4 As shown, the first feature map and the second feature map are both image classification features, wherein the first feature map is the image feature extracted by the classification module in the candidate prediction model, and the second feature map is the image feature extracted by the classification module in the reference prediction model. Specifically, after the candidate prediction model is acquired, the CT image corresponding to the candidate prediction model is used as the first type of CT image, the first feature map (i.e., image classification feature) of the first type of CT image is extracted by the candidate prediction model, and the second feature maps (i.e., image classification feature) of various corresponding CT images are extracted by each reference prediction model. Figure 5 As shown, after obtaining the first feature map and each second feature map, the first feature map is spliced ​​with each second feature map to obtain a spliced ​​feature map to obtain an image classification feature obtained by combining the first CT image and other CT images. The spliced ​​feature map is used to independently train classification prediction to obtain a candidate prediction model with the combination of two CT images as input items. Finally, based on the model performance, a target prediction model is selected from the trained candidate prediction models. Finally, the other CT image except the first CT image of the two CT images of the selected target prediction model is used as the second CT image. Then, when predicting based on the target prediction model, the first CT image and the second CT image are used as input items.

[0092] Furthermore, in practical applications, target prediction models of three CT images, target models of four CT images, etc. can also be combined. To this end, the candidate prediction models are trained based on each target prediction state and target annotation state, and after the target prediction model is selected from the trained candidate prediction models based on the model performance, the method further includes:

[0093] Screening a number of imaging types based on the model performance of each trained candidate prediction model to obtain a number of candidate imaging types;

[0094] The selected target prediction model is used as a candidate prediction model, the intermediate prediction model corresponding to each candidate imaging type is used as a reference prediction model, and the target prediction model is repeatedly determined based on the candidate prediction model and the reference prediction model;

[0095] The step of screening each trained candidate prediction model except the selected target prediction model based on model performance is repeated until all imaging types are selected for inclusion or exclusion, or the number of combined CT images reaches a preset number requirement to obtain the target prediction model.

[0096] Specifically, screening several imaging types based on the model performance of each trained candidate prediction model refers to excluding the prediction performance of two CT images after the combination is lower than the prediction performance of the best single CT image. The target prediction model is the one with the best prediction performance in the combination of the two CT images, so the target prediction model is used as the candidate prediction model, and the other excluded intermediate prediction models are used as the reference prediction models, wherein the other excluded intermediate prediction models do not include the two CT images corresponding to the target prediction model and the intermediate prediction models corresponding to the excluded CT images. Then, the two CT images corresponding to the target prediction model are combined with the other excluded CT images to obtain the target prediction model combining three CT images. Then, the above operation can be repeated to combine four CT images, five CT images, until all CT images are selected for inclusion or exclusion, or the number of combined CT images reaches the preset number requirement.

[0097] S30. Constructing prediction models of VETC and MVI for hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model.

[0098] Specifically, after obtaining the VETC status prediction model and the MVI status prediction model, the VETC status prediction model and the MVI status prediction model are combined into a prediction model, and the prediction model can determine the predicted VETC status and the predicted MVI status, wherein the predicted VETC status and the predicted MVI status can both be positive or negative. Microvascular invasion (MVI) is a key factor in predicting postoperative recurrence and poor prognosis of HCC, which is related to the epithelial–mesenchymal transition (EMT) process of the tumor. Vessels that encapsu l at ing tumor clusters (VETC) are a tumor metastasis mode mediated by vascular structures independent of the EMT process. By combining VETC status and MVI status for prognosis prediction, the value of risk stratification can be reflected in the prognosis of surgical resection and TACE treatment.

[0099] In summary, this embodiment provides a method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, the method comprising obtaining a training data set; training an initial prediction model based on the training data set to obtain a VETC state prediction model and an MVI state prediction model; and constructing a prediction model of VETC and MVI for hepatocellular carcinoma based on the VETC state prediction model and the MVI state prediction model. This application constructs a VETC state prediction model and an MVI state prediction model by collecting CT images of various imaging types, and then predicts the VETC state and MVI state based on the constructed VETC state prediction model and the MVI state prediction model, so that the prognosis can be predicted in combination with the VETC state and the MVI state, which can reflect the value of risk stratification in the prognosis of surgical resection and the prognosis of TACE treatment.

[0100] Based on the above-mentioned method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, the present application embodiment further provides a method for predicting VETC and MVI for hepatocellular carcinoma, using a prediction model constructed based on the above-mentioned method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, the method for predicting VETC and MVI for hepatocellular carcinoma comprises:

[0101] Acquire a number of first imaging types corresponding to the VETC state prediction model and a number of first imaging types corresponding to the MVI state prediction model in the prediction model;

[0102] Acquire CT images corresponding to each first imaging type to obtain a plurality of first CT images;

[0103] Acquire CT images corresponding to each second imaging type to obtain a plurality of second CT images;

[0104] Inputting the plurality of first CT images into a VETC state prediction model to obtain a predicted VETC state;

[0105] The plurality of second CT images are input into the MVI state prediction model to obtain a predicted MVI state.

[0106] Based on the above-mentioned method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, the present application embodiment also provides a device for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, such as Figure 6 As shown, the device for constructing the prediction model of VETC and MVI of hepatocellular carcinoma specifically includes:

[0107] An acquisition module 100 is used to acquire a training data set, wherein the training data set includes a plurality of training data, each of the plurality of training data includes a CT image of at least two imaging types, a tumor lesion mask, a marked VETC state, and a marked MVI state;

[0108] A training module 200, for training the initial prediction model based on the training data set to obtain a VETC state prediction model and an MVI state prediction model;

[0109] The construction module 300 is used to construct a prediction model of VETC and MVI for hepatocellular carcinoma based on the VETC state prediction model and the MVI state prediction model.

[0110] Based on the above-mentioned method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, this embodiment provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma as described in the above-mentioned embodiment, and / or to implement the steps in the method for predicting VETC and MVI for hepatocellular carcinoma as described in the above-mentioned embodiment.

[0111] Based on the above-mentioned method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, the present application also provides a terminal device, such as Figure 7As shown, it includes at least one processor 20, a display screen 21, and a memory 22, and may also include a communication interface 23 and a bus 24. The processor 20, the display screen 21, the memory 22, and the communication interface 23 may communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 may transmit information. The processor 20 may call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0112] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0113] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0114] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0115] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, characterized in that: The method for constructing the prediction model of VETC and MVI for hepatocellular carcinoma specifically includes: Acquire a training data set, wherein the training data set includes a plurality of training data, and each of the plurality of training data includes CT images of at least two imaging types, a tumor lesion mask, a marked VETC state, and a marked MVI state; Training the initial prediction model based on the training data set to obtain a VETC state prediction model and an MVI state prediction model; Based on the VETC status prediction model and the MVI status prediction model, a prediction model of VETC and MVI for hepatocellular carcinoma is constructed.

2. The method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma according to claim 1, characterized in that: The at least two imaging types of CT images include conventional CT images, effective atomic number CT images, electron density CT images, iodine map CT images and single-energy CT images with 40keV energy in the arterial phase, and conventional CT images, effective atomic number CT images, electron density CT images, iodine map CT images and single-energy CT images with 40keV energy in the portal venous phase.

3. The method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma according to claim 1 or 2, characterized in that: The VETC state prediction model and the MVI state prediction model adopt the same construction process, and the construction process specifically includes: For each imaging type of CT image, the initial prediction model is trained based on each CT image of the imaging type, the tumor lesion mask, and the target annotation status in the training data set to obtain an intermediate prediction model corresponding to each imaging type; A target prediction model is constructed based on the training data set and the intermediate prediction model corresponding to each imaging type, wherein when the target annotation state is the annotated VETC state, the target prediction model is the VETC state prediction model, and when the target annotation state is the annotated MVI state, the target prediction model is the MVI state prediction model.

4. The method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma according to claim 3, characterized in that: The said constructing the target prediction model based on the training data set and the intermediate prediction model corresponding to each imaging type specifically includes: Selecting a candidate prediction model from the intermediate prediction models corresponding to each imaging type based on model performance, and using the other intermediate prediction models as reference prediction models; Determining a target prediction model based on the candidate prediction model and the reference prediction model, wherein determining the target prediction model based on the candidate prediction model and the reference prediction model specifically includes: Determine a first feature map of a CT image of an imaging type corresponding to the candidate prediction model, and a second feature map of a CT image of an imaging type corresponding to each reference prediction model; Splicing the second feature map corresponding to each reference prediction model with the first feature map determined by the candidate prediction model to obtain a plurality of spliced ​​feature maps; Input each spliced ​​feature map into the classification module of the candidate prediction model respectively to obtain the target prediction state corresponding to each spliced ​​feature map; The candidate prediction models are trained based on the target prediction status and target labeling status respectively, and the target prediction model is selected from the trained candidate prediction models based on the model performance.

5. The method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma according to claim 4, characterized in that: After respectively training candidate prediction models based on each target prediction state and target labeling state, and selecting a target prediction model from each trained candidate prediction model based on model performance, the method further includes: Screening a number of imaging types based on the model performance of each trained candidate prediction model to obtain a number of candidate imaging types; The selected target prediction model is used as a candidate prediction model, the intermediate prediction model corresponding to each candidate imaging type is used as a reference prediction model, and the target prediction model is repeatedly determined based on the candidate prediction model and the reference prediction model; The step of screening each trained candidate prediction model except the selected target prediction model based on model performance is repeated until all imaging types are selected for inclusion or exclusion, or the number of combined CT images reaches a preset number requirement to obtain the target prediction model.

6. The method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma according to claim 1, characterized in that: The initial prediction model includes an encoding module, a decoding module and a classification module. The encoding module and the decoding module form a U-net network structure. The encoding module is connected to the classification module, wherein the classification module includes a linear feature extraction layer and a linear classification layer. The linear feature extraction layer is connected to the encoding module, and the linear classification layer is connected to the linear feature extraction layer. The linear feature extraction layer is used to expand the feature map input by the encoding module into a one-dimensional feature vector and perform feature extraction on the one-dimensional feature vector. The linear classification layer is used to perform state prediction based on the features extracted by the linear feature extraction layer.

7. A method for predicting VETC and MVI of hepatocellular carcinoma, characterized in that: A prediction model constructed based on the method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma according to any one of claims 1 to 6, wherein the prediction method for VETC and MVI for hepatocellular carcinoma comprises: Acquire a number of first imaging types corresponding to the VETC state prediction model and a number of first imaging types corresponding to the MVI state prediction model in the prediction model; Acquire CT images corresponding to each first imaging type to obtain a plurality of first CT images; Acquire CT images corresponding to each second imaging type to obtain a plurality of second CT images; inputting the plurality of first CT images into a VETC state prediction model to obtain a predicted VETC state; The plurality of second CT images are input into the MVI state prediction model to obtain a predicted MVI state.

8. A device for constructing a prediction model of VETC and MVI for hepatocellular carcinoma, characterized in that: The device for constructing the prediction model of VETC and MVI of hepatocellular carcinoma specifically includes: An acquisition module, configured to acquire a training data set, wherein the training data set includes a plurality of training data, each of the plurality of training data includes CT images of at least two imaging types, a tumor lesion mask, a marked VETC status, and a marked MVI status; A training module, used for training the initial prediction model based on the training data set to obtain a VETC state prediction model and an MVI state prediction model; A construction module is used to construct a prediction model of VETC and MVI for hepatocellular carcinoma based on the VETC status prediction model and the MVI status prediction model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma as described in any one of claims 1-6, and / or to implement the steps in the method for predicting VETC and MVI for hepatocellular carcinoma as described in claim 7.

10. A terminal device, characterized in that: include: Processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, it implements the steps in the method for constructing a prediction model of VETC and MVI for hepatocellular carcinoma as described in any one of claims 1-6, and / or implements the steps in the method for predicting VETC and MVI for hepatocellular carcinoma as described in claim 7.

Images