Liver transplant rejection prediction device based on feature separation and recombination

Abstract

The present invention discloses a device for predicting liver transplant rejection based on feature separation and recombination, which belongs to the technical field of medical image processing. The device comprises the following steps: extracting liver tissue CT image features, portal vein CT image features and abdominal plain scan CT image features, and using a multi-head attention mechanism to perform a key feature fusion operation. The liver tissue CT image features and abdominal plain scan CT image features are fused to obtain liver tissue key features, and the portal vein CT image features and abdominal plain scan CT image features are fused to obtain portal vein key features. The liver tissue key features and the portal vein key features are then dimensionality reduced and fused to obtain fused features. The fused features are used to predict liver transplant rejection, thereby improving prediction efficiency and accuracy, and helping doctors to predict rejection reactions of liver transplant recipients more accurately and efficiently.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and in particular relates to a device for predicting liver transplant rejection based on feature separation and recombination. Background Art

[0002] In recent years, with the increase in computing power, the application of deep learning technology in engineering has rapidly developed. As a subfield of deep learning, medical artificial intelligence leverages deep learning to assist experts in testing and avoid redundant and complex work. Deep learning is a crucial field in the application of computer science and technology. Through the depth of the network, it can uncover subtle features, thus holding broad application prospects in medical assistance. With advances in surgical techniques and the maturation of organ donation procedures, liver transplants are becoming increasingly common. Liver transplants can extend a patient's lifespan by over five years, but the procedure requires a healthy, matching liver donation for success. Because liver transplants are allogeneic, meaning the transplanted liver and the recipient come from different individuals, rejection is common after liver transplants.

[0003] Despite immunosuppressive therapy, approximately 60% of liver transplant recipients experience at least one episode of acute rejection within 1-6 weeks after surgery, or 2-3 days or 6 weeks later. Early clinical manifestations of rejection include fever, fatigue, lethargy, loss of appetite, liver tenderness, and increased ascites. Biliary drainage reveals thinner, paler, and less abundant bile. Blood biochemical tests reveal elevated bilirubin, transaminases, and alkaline phosphatase, as well as increased eosinophils and lymphocytes in the peripheral blood and transplanted liver. Serum levels of neopterin, sIL-2 receptor, guanine dehydrogenase, amyloid A protein, and α-microglobulin are elevated. Bilirubin is the most sensitive marker. Percutaneous liver biopsy can confirm the diagnosis, characterized by histological features of portal tract inflammatory cell infiltration, abnormal interlobar bile duct epithelium, and portal and / or central venous endothelial inflammation.

[0004] From an imaging perspective, the possibility of rejection can be indirectly determined by observing the patient's abdominal CT scan. Because the portal vein is poorly visible on plain CT scans, it is generally necessary to inject contrast agent into the patient and observe their venous CT scan to characterize the portal vein. However, there is a risk of contrast agent allergy.

[0005] Therefore, noninvasively preventing rejection through imaging data is a very meaningful aspect of postoperative liver transplantation observation. Furthermore, to avoid the side effects of contrast agents on patients, whether it is possible to use only plain CT images to determine rejection is also a question worth exploring.

[0006] With the development of artificial intelligence (AI), convolutional neural networks (CNNs), represented by ResNet, have achieved remarkable results in image classification. However, the recent emergence of the Transformer architecture has led to a gradual shift in feature fusion modules from CNNs to the Transformer architecture. As a result, hybrid models, where CNNs are used as feature extraction frameworks and Transformers are used for fusion, are now performing even better in image classification. Summary of the Invention

[0007] In view of the above, the purpose of the present invention is to provide a liver transplant rejection prediction device based on feature separation and recombination, so as to achieve efficient, accurate and non-invasive prediction of liver transplant rejection in patients.

[0008] To achieve the above-mentioned object of the invention, the present invention provides a device for predicting liver transplant rejection based on feature separation and recombination, comprising a memory and a processor, wherein the memory is used to store a computer executable program for predicting liver transplant rejection based on feature separation and recombination, and the processor is communicatively connected to the memory and configured to execute the computer executable program stored in the memory. When the processor executes the computer executable program, the following steps are implemented:

[0009] Perform semantic segmentation and annotation of the liver and portal vein on the abdominal plain scan CT images to form labeled samples;

[0010] Construct a liver transplant rejection prediction model with feature separation and recombination, including a tissue image separation module, a feature extraction module, a key feature fusion module, a feature fusion module and a rejection prediction module, wherein the tissue image separation module is used to perform image separation on the abdominal plain scan CT image and the corresponding labeled sample, and separate the liver tissue CT image and the portal vein CT image from the abdominal plain scan CT image; the feature extraction module is used to extract the corresponding liver tissue CT image features, abdominal plain scan CT image features and portal vein CT image features from the liver tissue CT image, the abdominal plain scan CT image and the portal vein CT image in their respective feature extraction networks; the key feature fusion module is used to fuse the liver tissue CT image features and the portal vein CT image features with the abdominal plain scan CT image features into corresponding liver tissue key features and portal vein key features; the feature fusion module is used to fuse the liver tissue key features and the portal vein key features with the abdominal plain scan CT image features and then compress them, and use the fused features for rejection prediction; the rejection prediction module is used to predict rejection based on the fused features;

[0011] All plain abdominal CT images and their corresponding annotated samples were fed into a liver transplant rejection prediction model for supervised training on the true value label of rejection, and the model parameters were continuously optimized.

[0012] A liver transplant rejection prediction model with optimized parameters was used to predict rejection.

[0013] Preferably, in the tissue image separation module, the liver area and the portal vein area are expanded outward by pixels based on the labeled samples and then segmented from the abdominal plain scan CT image, and the remaining area is filled with 0 vectors to supplement the segmented labeled samples to the size of the original CT image, and the CT image without the portal vein segmentation label is replaced with all 0 vectors.

[0014] Preferably, the feature extraction module adopts a convolutional neural network, including three feature extraction networks: a liver tissue CT image feature extraction network for extracting liver tissue CT image features, an abdominal plain scan CT image feature extraction network for extracting abdominal plain scan CT image features, and a portal vein CT image feature extraction network for extracting portal vein CT image features.

[0015] Preferably, the key feature fusion module includes a liver tissue key feature fusion module and a portal vein key feature fusion module. The liver tissue key feature fusion module is used to fuse liver tissue CT image features and abdominal plain CT image features into liver tissue key features. The portal vein key feature fusion module is used to fuse portal vein CT image features and abdominal plain CT image features into portal vein key features. Both the liver tissue key feature fusion module and the portal vein key feature fusion module use a multi-head attention mechanism for feature fusion, specifically including:

[0016] The feature F of size C×H×W l Input to the spatial multi-head attention Spatial-MHA, through the pooling operation Pool() and deformation operation into a (H×W)×C two-dimensional vector Sp(F l ), which can be expressed as:

[0017] Sp(F l )=MHA(Pool(F l ),F l )

[0018] F l Indicates the extracted liver tissue CT image features or portal vein CT image features;

[0019] Two-dimensional vector Sp(F l ) Calculate the key feature F' through channel multi-head attention Channel-MHA l , which can be expressed as:

[0020] F′ l =MHA(Conv1(F t ),Sp(F l ))

[0021] F t Represents the abdominal plain scan CT image features, Conv1() represents the convolution operation with the output channel being 1, and F' l Indicates key features of liver tissue or key features of hepatic portal vein;

[0022] In the above formula, MHA() represents the attention mechanism, which is expressed as follows:

[0023]

[0024] Where W Q , W K , W V is a C×C weight matrix, dk is used to represent the feature vector dimension and here dk is equal to C, f x and f y Represent two variables respectively, in the calculation of Sp(F l ), f x and f y The values ​​are Pool(F l ) and F l , in calculating F' l When f x and f y The values ​​are Conv1(F t ) and Sp(F l ).

[0025] Preferably, in the feature fusion module, the fusion feature F' is calculated using the following formula: t :

[0026] F' t =Conv Fusion (Concat[F t ,Conv d (F' l1 ),Conv d (F' l2 )])

[0027] F' l1 Indicates the key features of liver tissue, F' l2 Represents the key features of the portal vein, Conv d () represents the convolution layer that can achieve feature dimensionality reduction and compression, Concat() is used to connect the features after dimensionality reduction, Conv Fusion () is to restore the connected tensor to its original size, that is, to obtain the fusion feature F' t .

[0028] Preferably, the rejection reaction prediction module is used to predict rejection reaction using a logistic regression algorithm based on fusion features.

[0029] Preferably, the liver transplant rejection prediction model further comprises a plurality of sets of alternately connected feature extraction modules, key feature fusion modules and feature fusion modules;

[0030] The feature extraction module is used to extract the liver tissue key features and portal vein key features output by the previous group of key feature fusion modules, and the fusion features output by the previous group of feature fusion modules, respectively, in their respective extraction networks to obtain the corresponding high-order liver tissue features, high-order portal vein features, and high-order abdominal plain CT image features;

[0031] The key feature fusion module includes a liver tissue key feature fusion module and a portal vein key feature fusion module. The liver tissue key feature fusion module is used to fuse the extracted high-order liver tissue features with the high-order abdominal plain scan CT image features to obtain high-order liver tissue key features. The portal vein key feature fusion module is used to fuse the extracted high-order portal vein features with the high-order abdominal plain scan CT image features to obtain high-order portal vein key features.

[0032] The feature fusion module is used to perform dimensionality reduction operations on high-order liver tissue key features and high-order portal vein key features, and then fuse them with high-order abdominal plain scan CT image features to obtain high-order fusion features. Among them, the high-order liver tissue key features, high-order portal vein key features and high-order fusion features serve as inputs to the next set of deep feature extraction modules, and the high-order fusion features participate in the final rejection reaction prediction.

[0033] Preferably, a cross entropy function is used during training to calculate the loss between the liver transplant rejection result and the annotated true value label and to update the parameter model for continuous optimization.

[0034] To achieve the above-mentioned purpose, the present invention also provides a device for predicting liver transplant rejection based on feature separation and recombination, comprising a data acquisition unit, a model building unit, a training unit, and an application unit.

[0035] The data acquisition unit is used to perform semantic segmentation and annotation of the liver and portal vein on the abdominal plain scan CT image to form annotated samples;

[0036] The model construction unit is used for a liver transplant rejection prediction model with feature separation and recombination, and includes a tissue image separation module, a feature extraction module, a key feature fusion module, a feature fusion module and a rejection prediction module, wherein the tissue image separation module is used to perform image separation on the abdominal plain scan CT image and the corresponding labeled sample, and separate the liver tissue CT image and the portal vein CT image from the abdominal plain scan CT image; the feature extraction module is used to extract the corresponding liver tissue CT image features, abdominal plain scan CT image features and portal vein CT image features from the liver tissue CT image, the abdominal plain scan CT image and the portal vein CT image in their respective feature extraction networks; the key feature fusion module is used to fuse the liver tissue CT image features and the portal vein CT image features with the abdominal plain scan CT image features into corresponding liver tissue key features and portal vein key features; the feature fusion module is used to fuse the liver tissue key features and the portal vein key features with the abdominal plain scan CT image features and then compress them, and use the fused features for rejection prediction; the rejection prediction module is used to predict rejection based on the fused features;

[0037] The training unit is used to send all abdominal plain scan CT images and corresponding annotated samples into the liver transplant rejection prediction model to perform supervised training on whether there is a true value label of rejection reaction, and continuously optimize by updating model parameters;

[0038] The application unit is used to predict rejection reaction by using the liver transplant rejection prediction model with optimized parameters.

[0039] Compared with the prior art, the technical effects of the present invention include at least:

[0040] This invention, based on CT imaging technology, avoids the invasive procedures required by traditional methods and is more beneficial to patients' postoperative recovery. Using plain CT scans, it eliminates the need for contrast agents, thus avoiding potential allergic reactions and nephrotoxicity. This invention utilizes a parameter-optimized liver transplant rejection prediction model, enabling automated prediction and improving the accuracy and efficiency of rejection prediction, helping liver transplant physicians more accurately manage patients' postoperative conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1This is a flow chart of a method for predicting liver transplant rejection based on feature separation and recombination provided by an embodiment of the present invention;

[0043] Figure 2 This is a structural diagram of a liver transplant rejection prediction model based on feature separation and recombination provided by an embodiment of the present invention;

[0044] Figure 3 It is a structural diagram of a liver transplant rejection prediction device based on feature separation and recombination provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] To make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0046] In order to solve the problems in the prior art of not being able to non-invasively determine whether a patient has a rejection reaction after liver transplantation and the low efficiency and accuracy of manual detection, the embodiment provides a method and device for predicting liver transplant rejection reaction based on feature separation and recombination.

[0047] like Figure 1 As shown in the flowchart of the embodiment, a method for predicting liver transplant rejection based on feature separation and recombination is provided, which includes the following steps:

[0048] S110 , semantic segmentation and annotation of the liver and portal vein are performed on the abdominal plain scan CT image to form annotated samples.

[0049] In this embodiment, a postoperative plain CT image of the patient is obtained using computed tomography (CT). Since a plain CT scan scans the patient's entire torso, the entire CT image includes multiple internal organs. To improve prediction accuracy, after the patient completes the plain CT scan, the physician needs to segment the CT image, annotating the entire liver region and labeling the portal vein within the liver. Because these are plain abdominal CT images, the physician cannot label the portal vein. In these images, a tensor of all zeros is used to replace the portal vein in the subsequent tissue segmentation step.

[0050] After labeling the liver and portal vein, doctors will classify each patient's clinical records into binary true values ​​based on whether or not rejection occurred, with label 1 indicating rejection occurred after surgery and label 0 indicating no rejection.

[0051] After labeling, the patient's CT images were stratified and sampled, with 20% of the total data used as the test set and the remaining 80% used as the training set. The data was randomly assigned to the training and test sets in equal proportions of positive and negative data.

[0052] In this embodiment, batch data augmentation is performed on the CT images and segmentation annotations of a single sample within a certain range, including: scaling and cropping, random horizontal flipping, random vertical flipping, and random angle rotation. Because the data of the same sample must be at the same angle, the above data augmentation requires unified processing for the CT images and segmentation annotations of a single sample. That is, the same operation is performed on the CT images and segmentation annotation images in a given sample to ensure the consistency of the annotations and ensure that the relative positions of the features in the image are fixed within the same sample. In this embodiment, only batch normalization and batch center cropping are performed on the test set.

[0053] After sampling and data enhancement, a single plain scan CT image and the corresponding segmentation annotation are used as input data, and whether the patient has postoperative rejection reaction is used as the true value label to form a single sample.

[0054] S120, constructing a liver transplant rejection prediction model based on feature separation and recombination, including a tissue image separation module, a feature extraction module, a key feature fusion module, a feature fusion module, and a rejection prediction module.

[0055] In the embodiment, the prediction model structure is as follows Figure 2 As shown, the liver transplant rejection prediction model constructed in the embodiment includes a tissue image separation module, a feature extraction module, a feature fusion module and a liver transplant rejection prediction module. Among them, the tissue image separation module is to expand the liver area and the portal vein area outward by N pixels based on the segmentation and annotation image, and then segment them from the abdominal plain scan CT image, and use 0 (black pixel) vectors to fill the empty area to supplement the segmented image to the original size, and the image samples without portal vein segmentation annotation are replaced with all 0 vectors. Among them, N has a value of 2 or 5, 2 is used for the expansion of the slice layer, and 5 is used for the expansion of the height and width in space.

[0056] The feature extraction module is used to extract the corresponding liver tissue CT image features, abdominal plain scan CT image features, and portal vein CT image features from the liver tissue CT image, the abdominal plain scan CT image, and the portal vein CT image in their respective feature extraction networks. Specifically, the feature extraction module adopts a convolutional neural network, wherein the convolutional neural network adopts at least the first 5 layers of the ResNet-50 network, and the first 5 layers are adopted in the embodiment, wherein the first 5 layers are 5 convolutional layers, the first convolutional layer is a single-layer convolutional layer (Convolutional Layer), and then 2-4 convolutional layers are connected in series to form a residual module with an increasing number of channels by connecting multiple bottleneck layers, and each convolutional layer is activated by a ReLU function.

[0057] Specifically, the feature extraction module includes three feature extraction networks, wherein each feature extraction network adopts the above-mentioned convolutional neural network, the liver tissue CT image feature extraction network is used to extract liver tissue CT image features, the abdominal plain scan CT image feature extraction network is used to extract abdominal plain scan CT image features, and the portal vein CT image feature extraction network is used to extract portal vein CT image features.

[0058] The key feature fusion module is used to fuse the liver tissue CT image features and the portal vein CT image features with the abdominal plain scan CT image features into corresponding liver tissue key features and portal vein key features. Specifically, the key feature fusion module includes a liver tissue key feature fusion module and a portal vein key feature fusion module, wherein the liver tissue key feature fusion module and the portal vein key feature fusion module are constructed based on the multi-head attention module (MHA), and the key feature fusion is realized only by performing feature fusion in the last two layers of the entire model.

[0059] The feature F of size C×H×W l Input to the spatial multi-head attention Spatial-MHA, through the pooling operation Pool() and deformation operation into a (H×W)×C two-dimensional vector Sp(F l ), H represents the height of the feature, W represents the width, and C represents the number of channels, which can be expressed as follows:

[0060] Sp(F l )=MHA(Pool(F l ),F l )

[0061] F l Indicates the extracted liver tissue CT image features or portal vein CT image features;

[0062] Two-dimensional vector Sp(F l ) Calculate the key feature F' through channel multi-head attention Channel-MHA l , which can be expressed as:

[0063] F l =MHA(Conv1(F t ),Sp(F l ))

[0064] F t Represents the abdominal plain scan CT image features, Conv1() represents the convolution operation with the output channel being 1, and F' l Indicates key features of liver tissue or key features of hepatic portal vein;

[0065] In the above formula, MHA() represents the attention mechanism, which is expressed as follows:

[0066]

[0067] Where W Q , W K , W V is a C×C weight matrix, dk is used to represent the feature vector dimension and here dk is equal to C, f x and f y Represent two variables respectively, in the calculation of Sp(F l ), f x and f y The values ​​are Pool(F l ) and F l , in calculating F' l When f x and f y The values ​​are Conv1(F t ) and Sp(F l ).

[0068] The feature fusion module is used to compress the key features of liver tissue and portal vein to focus on a small number of key features. The compressed features are fused with the features of the abdominal plain scan CT image to obtain the fused features, specifically:

[0069] F' t =Conv Fusion (Concat[F t ,Conv d (F' l1 ),Conv d (F' l2 )])

[0070] F' l1 Indicates the key features of liver tissue, F' l2 Represents the key features of the portal vein, Conv d () represents the convolution layer that can achieve feature dimensionality reduction and compression, Concat() is used to connect the features after dimensionality reduction, Conv Fusion () is to restore the connected tensor to its original size, that is, to obtain the fusion feature F' t , the fusion feature F' t Used for the prediction of rejection reactions.

[0071] The rejection reaction prediction module is used to predict rejection reactions based on fusion features. Specifically, a logistic regression algorithm is used to predict rejection reactions.

[0072] In the embodiment, Figure 2 As shown, the liver transplant rejection prediction model also includes multiple sets of alternately connected feature extraction modules, key feature fusion modules and feature fusion modules;

[0073] The feature extraction module is used to extract the liver tissue key features and portal vein key features output by the previous group of key feature fusion modules, and the fusion features output by the previous group of feature fusion modules, respectively, in their respective extraction networks to obtain the corresponding high-order liver tissue features, high-order portal vein features, and high-order abdominal plain CT image features;

[0074] The key feature fusion module includes a liver tissue key feature fusion module and a portal vein key feature fusion module. The liver tissue key feature fusion module is used to fuse the extracted high-order liver tissue features with the high-order abdominal plain scan CT image features to obtain high-order liver tissue key features. The portal vein key feature fusion module is used to fuse the extracted high-order portal vein features with the high-order abdominal plain scan CT image features to obtain high-order portal vein key features.

[0075] The feature fusion module is used to perform dimensionality reduction operations on high-order liver tissue key features and high-order portal vein key features, and then fuse them with high-order abdominal plain scan CT image features to obtain high-order fusion features. Among them, the high-order liver tissue key features, high-order portal vein key features and high-order fusion features serve as inputs to the next set of deep feature extraction modules, and the high-order fusion features participate in the final rejection reaction prediction.

[0076] S130: All abdominal plain scan CT images and corresponding annotated samples are sent to the liver transplant rejection prediction model for supervised training on whether there is a true value label of rejection reaction, and the model parameters are continuously optimized by updating.

[0077] In the embodiment, the constructed loss function L=L cls , where “cls” represents the classification task, L cls The classification loss constructed based on the predicted results of liver transplant rejection and the true value label is the cross entropy loss, which is:

[0078] L cls = -log(S(X y ))

[0079]

[0080] Among them, S represents the Softmax normalization function, whose purpose is to convert the output result into a probability value between 0 and 1, and the exp() function is used to y Perform exponential transformation to convert them into exponential form. And the sum of the probability values ​​of all categories is 1. X represents the binary vector predicted by the liver transplant rejection prediction unit, y represents its corresponding true label, 1 represents the presence of rejection, 0 represents no rejection after surgery, and X yIt represents the value of the yth element of vector X. The loss function reaches its minimum value when X and y completely correspond, so it can be used to optimize the parameters of the classification model.

[0081] During training, labeled samples and abdominal plain CT images are used as input for the liver transplant rejection prediction model. A loss function is used to supervise the learning of the liver transplant rejection prediction model under the supervision of true value labels. Specifically, the labeled samples and abdominal plain CT images are first separated into three parts through the tissue imaging module: liver tissue CT images, portal vein tissue CT images, and abdominal plain CT images. These three CT images are then input into their respective feature extraction networks to obtain corresponding feature vectors. After a specific feature extraction module, the key features of the liver tissue and portal vein are compressed and fused into the abdominal plain CT image features through a feature fusion unit to obtain fused features. Finally, the fused features are input into the liver transplant rejection prediction module to obtain the liver transplant rejection prediction result. This prediction result is used as the final prediction result, and the classification loss value is calculated. Backpropagation is used to help the network perform more accurate classification to optimize the model parameters.

[0082] In this embodiment, the number of iterations is set to 75. In each round, data is input according to a batch size of 3, the aforementioned loss is calculated and back-propagated, and the model parameters are updated until training is completed. The model with the best validation performance is saved during each training round. By modifying hyperparameters, including the learning rate (LR) and the LR decrease rate (K% decrease every N rounds), the loss value, accuracy, recall rate, and other indicators on the validation set are optimized, resulting in better generalization performance of the model.

[0083] S140, predicting rejection reaction of liver transplantation using a parameter-optimized liver transplant rejection prediction model.

[0084] In an embodiment, a parameter-optimized liver transplant rejection prediction model is used to predict rejection, including: inputting the abdominal plain scan CT image to be detected and the corresponding semantic segmentation annotations of the liver and portal vein into the liver transplant rejection prediction model, and obtaining the rejection prediction result through calculation.

[0085] For patients undergoing liver transplantation, invasive observation or contrast agent injection is required to determine rejection reactions. However, some patients may be allergic to contrast agents. In the liver transplantation rejection prediction model provided in the embodiment, only plain scan CT images are used to predict postoperative rejection. This can avoid the impact of the above two situations on the patient's postoperative recovery.

[0086] Based on the same inventive concept, an embodiment further provides a device for predicting liver transplant rejection based on feature separation and recombination, comprising a memory and a processor, wherein the memory is used to store a computer executable program for predicting liver transplant rejection based on feature separation and recombination, and the processor is communicatively connected to the memory and configured to execute the computer executable program stored in the memory. The processor executes the above-mentioned method for predicting liver transplant rejection based on feature separation and recombination, which specifically includes the following steps:

[0087] S110, performing semantic segmentation and annotation of the liver and portal vein on the abdominal plain scan CT image to form annotated samples;

[0088] S120, constructing a liver transplant rejection prediction model based on feature separation and recombination, including a tissue image separation module, a feature extraction module, a key feature fusion module, a feature fusion module, and a rejection prediction module;

[0089] S130, all abdominal plain scan CT images and corresponding annotated samples are sent to the liver transplant rejection prediction model for supervised training on the true value label of whether there is rejection, and the model parameters are continuously optimized;

[0090] S140, predicting rejection reaction of liver transplantation using a parameter-optimized liver transplant rejection prediction model.

[0091] In an embodiment, the memory can be a proximal volatile memory such as RAM, or a non-volatile memory such as ROM, FLASH, a floppy disk, a mechanical hard disk, or a remote storage device. The processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), and these processors can be used to implement the steps of predicting liver transplant rejection based on feature separation and recombination.

[0092] Based on the same inventive concept, the embodiment further provides a liver transplant rejection prediction device 300 based on feature separation and recombination, comprising a data acquisition unit 310, a model construction unit 320, a training unit 330, and an application unit 340.

[0093] The data acquisition unit 310 is used to perform semantic segmentation and annotation of the liver and portal vein on the abdominal plain scan CT image to form annotated samples;

[0094] The model construction unit 320 is used to construct a liver transplant rejection prediction model with feature separation and recombination, including a tissue image separation module, a feature extraction module, a key feature fusion module, a feature fusion module and a rejection prediction module, wherein the tissue image separation module is used to perform image separation on the abdominal plain scan CT image and the corresponding labeled sample, and separate the liver tissue CT image and the portal vein CT image from the abdominal plain scan CT image; the feature extraction module is used to extract the corresponding liver tissue CT image features, abdominal plain scan CT image features and portal vein CT image features from the liver tissue CT image, the abdominal plain scan CT image and the portal vein CT image in their respective feature extraction networks; the key feature fusion module is used to fuse the liver tissue CT image features and the portal vein CT image features with the abdominal plain scan CT image features into corresponding liver tissue key features and portal vein key features; the feature fusion module is used to fuse the liver tissue key features and the portal vein key features with the abdominal plain scan CT image features and then compress them, and use the fused features for rejection prediction; the rejection prediction module is used to predict rejection based on the fused features;

[0095] The training unit 330 is used to send all the abdominal plain scan CT images and the corresponding annotated samples into the liver transplant rejection prediction model for supervised training on whether there is a true value label of rejection, and continuously optimize by updating the model parameters;

[0096] The application unit 340 is used to predict rejection reaction using the liver transplant rejection prediction model with optimized parameters.

[0097] It should be noted that the liver transplant rejection prediction device based on feature separation and recombination provided in the above embodiment should be illustrated by the division of the above functional units when performing rejection prediction. The above functions are assigned to different functional units as needed, that is, the internal structure of the terminal or server is divided into different functional units to complete all or part of the functions described above. In addition, the liver transplant rejection prediction device based on feature separation and recombination provided in the above embodiment and the embodiment of the liver transplant rejection prediction device based on feature separation and recombination are of the same concept. The specific implementation process is detailed in the embodiment of the liver transplant rejection prediction device based on feature separation and recombination, and will not be repeated here.

[0098] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent interchanges made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A device for predicting liver transplant rejection based on feature separation and recombination, comprising a memory and a processor, wherein the memory is configured to store a computer executable program for predicting liver transplant rejection based on feature separation and recombination, and the processor is communicatively connected to the memory and configured to execute the computer executable program stored in the memory. When the processor executes the computer executable program, the following steps are implemented: Perform semantic segmentation and annotation of the liver and portal vein on the abdominal plain scan CT images to form labeled samples; Construct a liver transplant rejection prediction model with feature separation and recombination, including a tissue image separation module, a feature extraction module, a key feature fusion module, a feature fusion module and a rejection prediction module, wherein the tissue image separation module is used to perform image separation on the abdominal plain scan CT image and the corresponding labeled sample, and separate the liver tissue CT image and the portal vein CT image from the abdominal plain scan CT image; the feature extraction module is used to extract the corresponding liver tissue CT image features, abdominal plain scan CT image features and portal vein CT image features from the liver tissue CT image, the abdominal plain scan CT image and the portal vein CT image in their respective feature extraction networks; the key feature fusion module is used to fuse the liver tissue CT image features and the portal vein CT image features with the abdominal plain scan CT image features into corresponding liver tissue key features and portal vein key features; the feature fusion module is used to fuse the liver tissue key features and the portal vein key features with the abdominal plain scan CT image features and then compress them; the fused features are used for rejection prediction; and the rejection prediction module is used to predict rejection based on the fused features; All plain abdominal CT images and their corresponding annotated samples were fed into a liver transplant rejection prediction model for supervised training on the true value label of rejection, and the model parameters were continuously optimized. Predicting rejection of liver transplantation using a parameter-optimized rejection prediction model; The key feature fusion module includes a liver tissue key feature fusion module and a portal vein key feature fusion module. The liver tissue key feature fusion module is used to fuse the liver tissue CT image features and the abdominal plain CT image features into the liver tissue key features. The portal vein key feature fusion module is used to fuse the portal vein CT image features and the abdominal plain CT image features into the portal vein key features. Both the liver tissue key feature fusion module and the portal vein key feature fusion module use a multi-head attention mechanism for feature fusion, specifically including: The feature F of size C×H×W l Input to the spatial multi-head attention Spatial-MHA, through the pooling operation Pool() and deformation operation into a (H×W)×C two-dimensional vector Sp(F l ), H represents the height of the feature, W represents the width, and C represents the number of channels, which can be expressed as follows: Sp(F l )=MHA(Pool(F l ),F l ) F l Indicates the extracted liver tissue CT image features or hepatic portal vein CT image features; Two-dimensional vector Sp(F l ) Calculate the key feature F′ through channel multi-head attention Channel-MHA l , which can be expressed as: F′ l =MHA(Conv1(F t ),Sp(F l )) F t Represents the features of abdominal plain scan CT images, Conv1() represents the convolution operation with an output channel of 1 and a convolution kernel size of 1, F′ l Indicates key features of liver tissue or key features of hepatic portal vein; In the above formula, MHA() represents the attention mechanism, which is expressed as follows: Where W Q , W K , W V is a C×C weight matrix, dk is used to represent the feature vector dimension and here dk is equal to C, f x and f y Represent two variables respectively, in the calculation of Sp(F l ), f x and f y The values ​​are Pool(F l ) and F l , in calculating F′ l When f x and f y The values ​​are Conv1(F t ) and Sp(F l ).

2. The device for predicting liver transplant rejection based on feature separation and recombination according to claim 1, characterized in that: In the tissue image separation module, the liver region and the portal vein region are expanded outward by pixels based on the labeled samples and then segmented from the abdominal plain scan CT image. The remaining area is filled with 0 vectors to supplement the segmented labeled samples to the size of the original CT image. The CT image without the portal vein segmentation label is replaced with all 0 vectors.

3. The device for predicting liver transplant rejection based on feature separation and recombination according to claim 1, characterized in that: The feature extraction module adopts a convolutional neural network, including three feature extraction networks: a liver tissue CT image feature extraction network for extracting liver tissue CT image features, an abdominal plain scan CT image feature extraction network for extracting abdominal plain scan CT image features, and a portal vein CT image feature extraction network for extracting portal vein CT image features.

4. The device for predicting liver transplant rejection based on feature separation and recombination according to claim 1, characterized in that: In the feature fusion module, the fusion feature F′ is calculated using the following formula: t : F′ t =Conv Fusion (Concat[F t ,Conv d (F′ l1 ),Conv d (F′ l2 )]) F′ l1 Represents the key features of liver tissue, F′ l2 Represents the key features of the portal vein, Conv d () represents the convolution layer that can achieve feature dimensionality reduction and compression, Concat() is used to connect the features after dimensionality reduction, Conv Fusion () is to restore the connected tensor to its original size, that is, to obtain the fusion feature F′ t .

5. The device for predicting liver transplant rejection based on feature separation and recombination according to claim 1, characterized in that: The rejection reaction prediction module is used to predict rejection reaction using a logistic regression algorithm based on fusion features.

6. The device for predicting liver transplant rejection based on feature separation and recombination according to any one of claims 1 to 5, characterized in that: The liver transplant rejection prediction model also includes multiple sets of alternately connected feature extraction modules, key feature fusion modules and feature fusion modules; The feature extraction module is used to extract the liver tissue key features and portal vein key features output by the previous group of key feature fusion modules, and the fusion features output by the previous group of feature fusion modules, respectively, in their respective extraction networks to obtain the corresponding high-order liver tissue features, high-order portal vein features, and high-order abdominal plain CT image features; The key feature fusion module includes a liver tissue key feature fusion module and a portal vein key feature fusion module. The liver tissue key feature fusion module is used to fuse the extracted high-order liver tissue features with the high-order abdominal plain scan CT image features to obtain high-order liver tissue key features. The portal vein key feature fusion module is used to fuse the extracted high-order portal vein features with the high-order abdominal plain scan CT image features to obtain high-order portal vein key features. The feature fusion module is used to perform dimensionality reduction operations on high-order liver tissue key features and high-order portal vein key features, and then fuse them with high-order abdominal plain scan CT image features to obtain high-order fusion features, among which high-order liver tissue key features, high-order portal vein key features and high-order fusion features serve as inputs to the next set of deep feature extraction modules; the high-order fusion features participate in the final rejection reaction prediction.

7. The device for predicting liver transplant rejection based on feature separation and recombination according to any one of claims 1 to 5, characterized in that: During training, the cross-entropy function is used to calculate the loss between the liver transplant rejection reaction results and the annotated true value labels, and the parameter model is updated for continuous optimization.

8. A device for predicting liver transplant rejection based on feature separation and recombination, characterized in that: Including data acquisition unit, model building unit, training unit, application unit, The data acquisition unit is used to perform semantic segmentation and annotation of the liver and portal vein on the abdominal plain scan CT image to form annotated samples; The model construction unit is used for a liver transplant rejection prediction model with feature separation and recombination, and includes a tissue image separation module, a feature extraction module, a key feature fusion module, a feature fusion module and a rejection prediction module, wherein the tissue image separation module is used to perform image separation on the abdominal plain scan CT image and the corresponding labeled sample, and separate the liver tissue CT image and the portal vein CT image from the abdominal plain scan CT image; the feature extraction module is used to extract the corresponding liver tissue CT image features, abdominal plain scan CT image features and portal vein CT image features from the liver tissue CT image, the abdominal plain scan CT image and the portal vein CT image in their respective feature extraction networks; the key feature fusion module is used to fuse the liver tissue CT image features and the portal vein CT image features with the abdominal plain scan CT image features into corresponding liver tissue key features and portal vein key features; the feature fusion module is used to fuse the liver tissue key features and the portal vein key features with the abdominal plain scan CT image features and then compress them; the fused features are used for rejection prediction; and the rejection prediction module is used to predict rejection based on the fused features; The key feature fusion module includes a liver tissue key feature fusion module and a portal vein key feature fusion module. The liver tissue key feature fusion module is used to fuse the liver tissue CT image features and the abdominal plain CT image features into the liver tissue key features. The portal vein key feature fusion module is used to fuse the portal vein CT image features and the abdominal plain CT image features into the portal vein key features. Both the liver tissue key feature fusion module and the portal vein key feature fusion module use a multi-head attention mechanism for feature fusion, specifically including: The feature F of size C×H×W l Input to the spatial multi-head attention Spatial-MHA, through the pooling operation Pool() and deformation operation into a (H×W)×C two-dimensional vector Sp(F l ), H represents the height of the feature, W represents the width, and C represents the number of channels, which can be expressed as follows: Sp(F l )=MHA(Pool(F l ),F l ) F l Indicates the extracted liver tissue CT image features or hepatic portal vein CT image features; Two-dimensional vector Sp(F l ) Calculate the key feature F′ through channel multi-head attention Channel-MHA l , which can be expressed as: F′ l =MHA(Conv1(F t ),Sp(F l )) F t Represents the features of abdominal plain scan CT images, Conv1() represents the convolution operation with an output channel of 1 and a convolution kernel size of 1, F′ l Indicates key features of liver tissue or key features of hepatic portal vein; In the above formula, MHA() represents the attention mechanism, which is expressed as follows: Where W Q , W K , W V is a C×C weight matrix, dk is used to represent the feature vector dimension and here dk is equal to C, f x and f y Represent two variables respectively, in the calculation of Sp(F l ), f x and f y The values ​​are Pool(F l ) and F l , in calculating F′ l When f x and f y The values ​​are Conv1(F t ) and Sp(F l ); The training unit is used to send all abdominal plain scan CT images and corresponding annotated samples into the liver transplant rejection prediction model to perform supervised training on whether there is a true value label of rejection reaction, and continuously optimize by updating model parameters; The application unit is used to predict rejection reaction by using the liver transplant rejection prediction model with optimized parameters.

Images