An adversarial generative network-based post-hepatectomy effect prediction tool for liver tumors

Abstract

The application provides a liver tumor resection effect prediction tool based on an adversarial generative network, comprising a trained deep learning model, wherein the deep learning model comprises a first model and a second model; the first model is used to output postoperative enhanced MRI data according to preoperative enhanced MRI data and a liver resection range; and the second model is used to output a recurrence rate evaluation result by taking preoperative serum tumor marker data and the postoperative enhanced MRI data output by the first model as input. The application can give a relatively accurate postoperative prediction result based on existing data (preoperative liver enhanced MRI data, preoperative serum tumor marker data) and a surgical plan (liver resection range) before operation, and can also provide visualized prediction of images, which is of great significance for the prognosis of treatment and the determination of the liver resection range before operation.

Description

Technical Field

[0001] This invention belongs to the field of postoperative outcome prediction technology, and in particular relates to a tool for predicting the outcome of liver tumor resection based on adversarial generative networks. Background Technology

[0002] To date, liver resection remains the first-line treatment for liver cancer. With the continuous advancement of precision medicine and the tireless efforts of Chinese medical professionals over many years, my country has achieved significant progress in liver resection techniques, currently holding a high level globally and continuing to develop rapidly. While liver resection has a good therapeutic effect on liver tumors, there is still a certain probability of liver dysfunction or even acute liver failure, and complications such as gastrointestinal bleeding and hepatic encephalopathy may also occur, which can be life-threatening in severe cases.

[0003] Currently, there are no visual predictive methods for postoperative outcomes of hepatectomy patients, especially in terms of imaging. Accurate prognostic assessment is crucial for improving surgical results and validating the role of adjuvant therapies. Particularly for hepatectomy, the extent of resection can significantly impact postoperative outcomes. Predicting postoperative outcomes, including imaging-based prognoses and recurrence rates, would be extremely important for both patient treatment and determining the extent of preoperative hepatectomy. Summary of the Invention

[0004] The purpose of this invention is to address the above-mentioned problems by providing a method and system for evaluating the post-interventional response to liver tumors based on generative adversarial networks.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions:

[0006] A tool for predicting the outcome of liver tumor resection based on generative adversarial networks includes a trained deep learning model. The deep learning model includes a first model and a second model. The first model is used to output postoperative enhanced MRI data based on preoperative enhanced MRI data and the extent of liver resection. The second model is used to output a recurrence rate assessment result based on preoperative serum tumor marker data and the postoperative enhanced MRI data output by the first model.

[0007] In the aforementioned liver tumor resection outcome prediction tool based on adversarial generative networks, the first model is trained using preoperative enhanced MRI data, liver resection extent, and postoperative enhanced MRI data as sample data.

[0008] In the aforementioned liver tumor resection outcome prediction tool based on adversarial generative networks, the second model is trained using postoperative enhanced MRI data, preoperative serum tumor marker data, and recurrence data as sample data.

[0009] In the aforementioned liver tumor resection outcome prediction tool based on adversarial generative networks, the output of the deep learning model includes postoperative enhanced MRI data output by the first model and recurrence rate assessment results output by the second model.

[0010] In the aforementioned liver tumor resection outcome prediction tool based on adversarial generative networks, the first model employs an adversarial generative network that includes a generator and a discriminator.

[0011] In the aforementioned liver tumor resection outcome prediction tool based on adversarial generative networks, the generator includes a mapping network and a synthesis network.

[0012] In the aforementioned liver tumor resection outcome prediction tool based on adversarial generative networks, the mapping network comprises eight fully connected layers and is used to perform the following steps:

[0013] S1. Transform the hidden variable z to obtain the intermediate variable w;

[0014] S2. Transform the intermediate variable w into a style control vector A and input it into the synthesis network.

[0015] In the aforementioned liver tumor resection outcome prediction tool based on adversarial generative networks, the integrated network, combined with the AdaIN style transformation method, aligns the mean and standard deviation of the target image with the result image to apply style to the spatial feature map. The AdaIN style transformation method is as follows:

[0016] x and y are the feature maps after encoding the content image and style image, respectively, and σ and μ are the mean and standard deviation, respectively.

[0017] In the aforementioned liver tumor resection effect prediction tool based on adversarial generative networks, each layer of the integrated network contains two inputs: a control vector A and random noise B, and the random noise is noise after scaling transformation.

[0018] In the aforementioned liver tumor resection outcome prediction tool based on adversarial generative networks, the second model employs an SVM model.

[0019] The advantages of this invention are:

[0020] This paper proposes a deep learning network consisting of a Generative Adversarial Network (GAN) and a Semi-Vision Machine (SVM). The GAN can predict postoperative enhanced MRI data using preoperative enhanced MRI data of the liver and the extent of liver resection in patients undergoing hepatectomy. The SVM model can predict postoperative recurrence rate using preoperative serum tumor marker data and postoperative enhanced MRI data. Through the cooperation of the two network models, relatively accurate postoperative prediction results can be provided preoperatively based on existing data (preoperative enhanced MRI data of the liver and preoperative serum tumor marker data) and surgical plan (the extent of liver resection). At the same time, it can also provide imaging visualization predictions, which is of great significance for both patient prognosis and treatment and for determining the extent of liver resection before surgery. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the training process of the adversarial generative network in the liver tumor resection effect prediction tool based on adversarial generative networks of this invention.

[0022] Figure 2 This is a flowchart of the prediction process of the adversarial generative network in the liver tumor resection effect prediction tool based on adversarial generative networks of the present invention.

[0023] Figure 3 This is a flowchart of the training process of the SVM model in the liver tumor resection effect prediction tool based on adversarial generative networks of this invention.

[0024] Figure 4 This is a flowchart of the prediction process of the SVM model in the liver tumor resection effect prediction tool based on adversarial generative networks of this invention.

[0025] Figure 5 This is a flowchart of the prediction process of the deep learning model consisting of an adversarial generative network and an SVM model in the liver tumor resection effect prediction tool based on adversarial generative networks of the present invention.

[0026] Figure 6 This is a network structure diagram of the adversarial generative network in the liver tumor resection effect prediction tool based on adversarial generative network of the present invention.

[0027] Figure 7 ROC curves are used to evaluate the 6-month recurrence rate predicted after surgery using the liver tumor resection outcome prediction tool based on adversarial generative networks of this invention.

[0028] Figure 8a , Figure 8b , Figure 8c The images shown are T2-weighted, in-phase, and out-phase images of a patient before right hepatectomy.

[0029] Figure 9a , Figure 9b , Figure 9c The images shown are T2-weighted, inphase, and outphase images of enhanced MRI images of a simulated patient after right hepatectomy.

[0030] Figure 10a , Figure 10b , Figure 10c The images shown are T2-weighted, in-phase, and out-phase images of a patient after right hepatectomy. Detailed Implementation

[0031] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0032] This solution addresses the current inability to visually predict surgical resection outcomes by proposing a liver tumor resection efficacy prediction tool based on generative adversarial networks (GANs). This tool utilizes popular deep learning techniques, including deep learning models. Specifically, the deep learning models in this solution include a first model and a second model. In this embodiment, the first model employs a generative adversarial network containing a generator and a discriminator, while the second model uses an SVM model.

[0033] like Figure 1 As shown, an adversarial generative network was trained using preoperative enhanced MRI data, the extent of liver resection, and postoperative enhanced MRI data as sample data. The specific training method is not detailed here. Figure 2 As shown, the trained adversarial generative network outputs postoperative enhanced MRI data based on preoperative enhanced MRI data and the extent of liver resection. It can generate predicted postoperative enhanced MRI data based on preoperative enhanced MRI data and the extent of liver resection in the surgical resection plan after the surgeon has formulated the surgical resection plan, thereby providing visualized postoperative prediction.

[0034] like Figure 3 As shown, the SVM model is trained using postoperative enhanced MRI data, preoperative serum tumor marker data, and recurrence data as sample data. Recurrence data refers to information on recurrence rates at three months, two years, and five years post-surgery for the patients used in the sample. Specific training methods are not detailed here. Figure 4 As shown, the trained SVM model uses enhanced MRI data and preoperative serum tumor marker data as inputs and outputs to predict the recurrence rate assessment results. It can predict postoperative prognosis even with both postoperative enhanced MRI data and preoperative serum tumor marker data. Furthermore, by combining it with an adversarial generative network, it can obtain postoperative enhanced MRI data from preoperative enhanced MRI data. Therefore, as... Figure 5As shown, a deep neural network composed of a generative adversarial network and an SVM model can obtain predicted postoperative enhanced MRI data and recurrence rate assessment results with only preoperative enhanced MRI data, liver resection range, and preoperative serum tumor marker data. This enables the provision of relatively accurate postoperative prediction results based on existing data and surgical plans before surgery, and can also provide imaging visualization of prognostic predictions. This is of great significance for both patient prognosis and treatment and for determining the preoperative liver resection range.

[0035] Furthermore, such as Figure 6 As shown, the preferred adversarial generative network in this scheme is the StyleGAN network, which consists of two parts: a mapping network and a synthesis network. The mapping network comprises eight fully connected layers, which transform the hidden variable z into an intermediate variable w through a series of affine transformations. Then, the intermediate variable w is transformed into a style control vector A, which is input into the synthesis network. The transformed w will act as style information on the spatial data. The process of generating the intermediate hidden variable w from the hidden variable z is what controls the style of the generated image.

[0036] The AdaIN style transformation method, combined with a comprehensive network, aligns the mean and standard deviation of the target image with the result image to apply the style to the spatial feature map. The AdaIN style transformation method is as follows:

[0037] x and y are the feature maps after encoding the content image and style image, respectively, and σ and μ are the mean and standard deviation, respectively. StyleGAN calculates the style value pair (y(s,i), y(b,i)) of each layer as the scaling and bias values ​​of w, thereby applying the style to the spatial feature map i.

[0038] Furthermore, each layer of the integrated network contains two inputs: a control vector A and random noise B. The random noise B is noise that has been scaled and transformed, meaning that each convolutional layer can adjust its "style" according to the input A, while the random noise B is used to enrich the details of the generated image.

[0039] Unlike traditional GAN ​​networks, StyleGAN's input is not a random variable or volume variable z. Instead, z is transformed into w using a mapping network, w is transformed into a style control vector A, and then fed into each layer of the synthesis network. Scaled noise is added to each layer to improve the network's generation performance.

[0040] To verify the feasibility and predictive effectiveness of this scheme, this embodiment conducted numerous experiments, and some of the experimental data are provided here:

[0041] like Figure 7 The figure shows the ROC curve for predicting recurrence in patients 6 months post-resection, as predicted by this invention. As can be seen, the area under the ROC curve is 0.8673, which is close to 1, indicating that this method can provide relatively accurate prognostic results and has good predictive value.

[0042] in addition, Figures 8a-8c This is an enhanced MRI image of a patient before a right hepatectomy; Figures 9a-9c These are enhanced MRI images of patients who underwent right hepatectomy using this protocol. Figures 10a-10c This is the actual enhanced MRI image of a patient after right hepatectomy. The area enclosed by the rectangle in the image is the tumor region. It can be seen that the tumor region in the generated image is consistent with the tumor region in the real image, indicating that this method can predict postoperative enhanced MRI images relatively accurately.

[0043] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method for predicting the outcome of liver tumor resection based on generative adversarial networks, characterized in that, The deep learning model includes a trained deep learning model, which includes a first model and a second model. The first model employs an adversarial generative network that includes a generator and a discriminator. The second model mentioned above uses the SVM model; The first model was trained using preoperative enhanced MRI data, liver resection extent, and postoperative enhanced MRI data as sample data. The second model was trained using postoperative enhanced MRI data, preoperative serum tumor marker data, and recurrence data as sample data. The first model is used to output postoperative enhanced MRI data based on preoperative enhanced MRI data and the extent of liver resection. The second model is used to input and output recurrence rate assessment results using preoperative serum tumor marker data and postoperative enhanced MRI data output by the first model.

2. The method for predicting the effect of liver tumor resection based on generative adversarial networks according to claim 1, characterized in that, The output of the deep learning model includes postoperative enhanced MRI data output by the first model and recurrence rate assessment results output by the second model.

3. The method for predicting the effect of liver tumor resection based on generative adversarial networks according to claim 2, characterized in that, The generator includes a mapping network and a synthesis network.

4. The method for predicting the effect of liver tumor resection based on generative adversarial networks according to claim 3, characterized in that, The mapping network comprises eight fully connected layers and is used to perform the following steps: S1. Transform the hidden variable z to obtain the intermediate variable w; S2. Transform the intermediate variable w into a style control vector A and input it into the synthesis network.

5. The method for predicting the effect of liver tumor resection based on generative adversarial networks according to claim 4, characterized in that, The integrated network, combined with the AdaIN style transformation method, aligns the mean and standard deviation of the target image with the result image to apply the style to the spatial feature map. The AdaIN style transformation method is as follows: , and These are the feature maps after encoding the content image and the style image, where σ and μ are the mean and standard deviation, respectively.

6. The method for predicting the effect of liver tumor resection based on generative adversarial networks according to claim 5, characterized in that, Each layer of the integrated network contains two inputs: a control vector A and random noise B, and the random noise is noise that has been scaled and transformed.

Images