18F-FDG PET / CT liver cancer kinetic parameter estimation method of near-end strategy optimization algorithm combined with clinical priori reward

By combining proximal strategy optimization algorithm with clinical prior rewards, the data sparse problem in the estimation of dynamic parameters of 18F-FDG PET/CT liver cancer in the prior art was solved, and the perfusion and metabolic parameters of hepatocellular carcinoma and normal liver tissues were realized. Accurate estimation provides more accurate diagnostic support for hepatocellular carcinoma.

CN119048480AActive Publication Date: 2024-11-29KUNMING UNIV OF SCI & TECH

Patent Information

Application Number
CN202411195976.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-29
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

The existing reinforcement learning algorithm has data sparse problems in the estimation of dynamic parameters of liver cancer in 18F-FDG PET/CT, which leads to limited exploration capabilities of agents and has not been effectively used for accurate estimation of dynamic parameters of liver cancer.

Method used

Using a proximal strategy optimization algorithm combining clinical prior rewards, the reward function is designed by building a reversible dual-input three-room model and an agent interaction environment, and the overall framework of the proximal strategy optimization algorithm is built, including a policy network and a value function network. and generalized advantage estimation algorithm to carry out perfusion and metabolic parameters estimation of hepatocellular carcinoma tissue and normal liver tissue.

Benefits of technology

Improve the accuracy of parameter estimation and capture more pharmacokinetic physiological information. The estimated pharmacokinetic parameters have statistical differences in hepatocellular carcinoma tissue and normal liver tissue, effectively assessing the 18F-FDG trace. The transmission and phosphorylation rates of agents in different tissues provide technical support for the auxiliary diagnosis of hepatocellular carcinoma.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048480A_ABST
    Figure CN119048480A_ABST
Patent Text Reader

Abstract

The invention relates to an 18F-FDG PET / CT liver cancer kinetic parameter estimation method of a near-end strategy optimization algorithm combined with clinical priori awards, and belongs to the field of deep reinforcement learning and liver cancer feature evaluation. According to the method, a deep reinforcement learning algorithm is adopted, and perfusion and metabolism parameters are estimated on a double-input three-atrioventricular reversible model; by utilizing the characteristics of a near-end strategy optimization algorithm and combining clinical priori rewards, the invention provides the method, the intelligent agent can be guided to obtain pharmacokinetic parameters which conform to clinical expectation and have smaller fitting errors, and statistical differences are achieved. According to the method, the pharmacokinetic parameters corresponding to the hepatocellular carcinoma tissue and the normal hepatocellular carcinoma tissue are effectively estimated on the reversible double-input three-atrioventricular model by using a near-end strategy optimization algorithm combined with clinical priori awards, and statistical difference is achieved; according to the kinetic parameters, different transport rates of the tracer serving as the protein analogue in hepatocellular carcinoma tissues and normal liver tissues can be effectively evaluated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention provides a proximal strategy optimization algorithm combined with clinical prior rewards 18 The F-FDG PET / CT liver cancer kinetic parameter estimation method belongs to the field of deep reinforcement learning and pharmacokinetic parameter estimation. Background Art

[0002] Liver cancer is a malignant tumor that originates from hepatocytes and hepatobiliary cells and is divided into primary and secondary types. Among them, hepatocellular carcinoma (HCC) is the main primary intrahepatic malignant tumor. The symptoms of early liver cancer are often not obvious, while mid- and late-stage liver cancer has a variety of manifestations, such as pain in the liver area, abdominal distension, loss of appetite, fatigue, weight loss, progressive liver enlargement or upper abdominal mass, etc.; some patients may experience symptoms such as low fever, jaundice, diarrhea and upper gastrointestinal bleeding. Liver cancer is predicted to be the sixth most common cancer in the world.

[0003] Recent studies have shown that reinforcement learning has shown potential in pharmacokinetic parameter estimation. Existing studies have used temporal difference learning (TD-learning) and state-action-reward-state-action (SARSA) algorithms to estimate the parameters of kinetic models. However, these reinforcement learning algorithms need to maintain a Q table to store the value of each state-action pair. When the state space or action space is large, the size of the Q table becomes difficult to manage and update, resulting in data sparsity problems, which limits the exploration ability of the intelligent agent to a certain extent.

[0004] In terms of technology, the Proximal Policy Optimization (PPO) algorithm is a deep reinforcement learning algorithm proposed in recent years and has been verified in some other parameter estimation fields such as industrial parameter estimation and optimization. This method uses a neural network to approximate the Q value and other policy functions and value functions, which can effectively alleviate the data sparsity problem of the reinforcement learning algorithm based on the Q table. However, there is no research on applying the proximal policy optimization algorithm to 18 F-FDG PET / CT liver cancer dynamics parameter estimation; Therefore, the present invention proposes to use a proximal strategy optimization algorithm to perform 18 The paper proposes to estimate the parameters of F-FDG PET / CT liver cancer dynamics and proposes to combine medical prior knowledge to set the reward function, so that the parameter estimation of the agent is more accurate and in line with clinical expectations, and the optimization direction of the agent is more accurately guided. 18 F-FDG stands for 18F-fluorodeoxyglucose, PET / CT is positron emission tomography / computed tomography. Summary of the invention

[0005] In view of the above defects or deficiencies in the prior art and the above research gaps on liver cancer, the present invention proposes a proximal strategy optimization algorithm combined with clinical prior rewards. 18 F-FDG PET / CT liver cancer dynamic parameter estimation method, realize the proximal strategy optimization algorithm combined with clinical prior rewards 18 The estimation of perfusion and metabolic parameters corresponding to F-FDG PET / CT hepatocellular carcinoma tissue and normal liver tissue showed statistically significant differences and met clinical expectations, which effectively supported the evaluation of the characteristics of liver cancer tissue.

[0006] The technical solution of the present invention is: a proximal strategy optimization algorithm combined with clinical prior rewards 18 The F-FDGPET / CT liver cancer dynamic parameter estimation method includes the following:

[0007] Step 1: Obtain clinical liver PET / CT images, including a 5-minute dynamic PET / CT scan of the liver and a 60-minute conventional static PET / CT scan, to capture the liver dynamics of HCC patients. 18 Time course of F-FDG tracer uptake.

[0008] Step 2: Manually outline the region of interest (ROI), including hepatocellular carcinoma, normal liver tissue, artery and portal vein, adjust the ROI layer by layer, and obtain the time-concentration activity curve (TAC) and its data composed of the maximum SUV (SUVmax) of each frame.

[0009] Step 3: Divide the TAC data obtained in Step 2 into a data set, with the training set accounting for 29% and the test set accounting for 71%;

[0010] Step 4: Construct a mathematical model of the reversible double-input three-compartment model, including 18 The rate constant K1 of F-FDG uptake from blood into liver tissue cells is 18 The rate constant k2 (1 / min) of F-FDG transfer from liver tissue cells back to blood is 18 F-FDG is phosphorylated by hexokinase to 18 The rate constant k3 for F-FDG 6-phosphate, the rate constant k4 for dephosphorylation, the aortic blood fraction fa, and the proportion of the measured blood volume vb.

[0011] Step 5. Construct an intelligent agent interaction environment combined with a reversible dual-input three-compartment model, including defining the dynamic parameter state, action distribution and action space, designing a reward function, etc.; the reward function part is based on clinical prior knowledge of parameters, including the upper and lower limits of parameters obtained from previous studies and the expected parameter range of individual parameters, to improve the reward function in the designed intelligent agent interaction environment.

[0012] Step 6. Build the overall framework of the proximal policy optimization (PPO) algorithm, including the actor network, the critic network, the generalized advantage estimation (GAE) algorithm for calculating the advantage function, and hyperparameters such as the clipping ratio of the new and old strategies.

[0013] Step 7. Use the proximal strategy optimization algorithm combined with medical prior rewards to estimate the parameters of the compartment model and perform descriptive analysis, statistical analysis and other clinical analysis on the perfusion and metabolic parameters of hepatocellular carcinoma tissue and normal liver tissue.

[0014] Specifically, the specific steps of Step 1 are as follows:

[0015] Step 1.1. Perform a low-dose CT scan of the liver (120 kV, 100 mA) at one bed; then quickly manually apply 2 mL of 0.9% saline. 18 F-FDG (5.5 MBq / kg) was flushed with 20 mL of 0.9% saline at a flow rate of 2 mL / s to allow the tracer to enter the cardiopulmonary circulation more quickly. After the injection, a 5-minute liver PET scan was performed. 18 Approximately 60 minutes after the F-FDG bolus injection, a whole-body CT scan (120 kV, 200 mA) was performed on a total of 11 beds from the vertex of the skull to the proximal thigh, followed by a 1-minute PET scan on each bed.

[0016] Step 1.2. After scanning, a short-term PET acquisition protocol was used to reconstruct PET images using the standard ordered subset expectation maximization (OSEM) algorithm. The first 1 minute of data was reconstructed into 12 frames with an interval of 5 seconds, and the next four minutes of data were reconstructed into 4 frames with an interval of 60 seconds. One frame of liver static PET image was selected at the 60th minute. A total of 17 frames of PET images were obtained and fused and registered with the CT images.

[0017] Specifically, in Step 2, if the patient has a lesion 18 If F-FDG uptake was not obvious, the ROI was drawn relative to the conventional CT image. The ROIs of the abdominal aorta and portal vein were located at approximately two-thirds of the cross-section of the blood vessels. In order to compare the HCC tumor with the surrounding tumor-free liver tissue, the corresponding ROI was drawn in the tumor-free liver tissue. The interference of the internal blood vessels of the liver on the SUV was avoided as much as possible during the delineation process. The ROI was copied to the PET / CT fusion image to obtain the time-concentration activity curve (TAC) consisting of the maximum SUV (SUVmax) of each frame.

[0018] Specifically, in Step 3, 5 cases of clinical liver cancer data are randomly selected as a training set, and the remaining 12 cases of clinical liver cancer data are used as a test set.

[0019] Specifically, the mathematical form of the reversible two-input three-compartment model in Step 4 can be described by a set of ordinary differential equations:

[0020]

[0021] Where t is the time variable, C(t) is the tissue concentration function, and the calculation is based on the formula C(t) = C f (t)+C p (t), its expression is [c(t1),c(t2),…,c(t k )] T , k is the total number of frames in the PET scanning protocol. The matrix form of the model system can be expressed as:

[0022]

[0023] Solving this system of ordinary differential equations yields:

[0024]

[0025] For the target area in the PET image, the model not only considers the tracer activity in the tissue within the area, but also the proportion of the tracer concentration in the blood. Therefore, the parameter blood volume factor v b Used to calculate tissue concentration C(t) and blood input concentration C i (t) Do the weighted sum and finally get the total output concentration C of the target area t (t):

[0026] C t (t) = v b ×C i (t)+(1-v b )×C(t)

[0027] Specifically, in the agent interaction environment in Step 5, the parameter state is defined as:

[0028] s=[K1,k2,k3,k4,f a ,v b ]

[0029] The distribution of intelligent physical actions is defined as a 12-dimensional discrete space:

[0030] set a =[K1+,K1-,k2+,k2-,k3+,k3-,k4+,k4-,f a +,f a -,v b +,v b -]

[0031] In the setting of the reward function, a reward function combining clinical prior knowledge of kinetic parameters is proposed, and its calculation formula is:

[0032]

[0033]

[0034] r t =λ1r1+λ2r2+λ3r3+λ4r4

[0035] Among them, r1~r4 are multiple reward signals, λ1~λ4 are the weights of reward signals, and r t is the reward value obtained at that moment, and t represents the time.

[0036] Specifically, the proximal strategy optimization algorithm based on Clip is adopted in Step 6. First, the generalized advantage estimation (GAE) algorithm is used to calculate the advantage value, and the formula is as follows:

[0037]

[0038] Among them, γ is the reward discount coefficient, λ is a constant between 0 and 1, is the calculation method of GAE advantage function, r t+l is the total reward of a single step at the t+lth moment, l is the number of steps experienced, V represents the state value function, s t Represents the state at time t. Then PPO updates the Actor and Critic networks multiple times. For the Critic network, the loss function can be calculated as:

[0039]

[0040] In the formula, r t′ represents the reward at time t′, t represents the current time, V represents the state value function, φ represents the parameters of the value function network, and s t Represents the state at time t. For the Actor network, the objective function can be calculated as:

[0041]

[0042] Among them, θ represents the model parameters, represents the expectation of the objective function, ε represents the clipping coefficient, π θ (a t |s t ) represents the new policy function, represents the old policy function, Represents the ratio of new and old probability strategies.

[0043] Specifically, the descriptive analysis and statistical analysis performed in Step 7 include, but are not limited to, root mean square error (RMSE), area under the curve (AUC), and T test (P<0.05 is considered statistically significant). In addition, the differences in kinetic parameters K1, k2, k3, k4, fa, and vb between tumor and normal liver tissues are analyzed to evaluate their clinical significance.

[0044] The beneficial effects of the present invention are: the present invention combines the proximal strategy optimization algorithm of clinical prior rewards 18 The F-FDGPET / CT liver cancer kinetic parameter estimation method improves the accuracy of parameter estimation. At the same time, the proposed reward function combined with prior medical knowledge can better guide the intelligent agent to perform parameter estimation and parameter optimization, capture more physiological information in pharmacokinetics, and the estimated pharmacokinetic parameters have statistical differences between hepatocellular carcinoma tissue and normal liver tissue, thereby effectively evaluating the 18 The F-FDG tracer has significantly different transmission and phosphorylation rates in different tissues, providing technical support for auxiliary diagnosis of hepatocytes. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of the present invention;

[0046] Figure 2 ROIs in PET, CT, PET / CT images and dynamic PET / CT imaging used in the present invention; A: ROI in PET image; B: ROI in CT image; C: ROI in PET / CT image; D: ROI in dynamic PET / CT imaging;

[0047] Figure 3 A time-concentration activity curve (TAC) consisting of the maximum SUV (SUVmax) of each frame obtained from the ROI region of the PET / CT image used in the present invention;

[0048] Figure 4 It is a double-input reversible three-compartment model used in the present invention;

[0049] Figure 5 This is a model structure diagram of the 18F-FDG PET / CT liver cancer dynamic parameter estimation method of the proximal strategy optimization algorithm combined with clinical prior rewards proposed in the present invention;

[0050] Figure 6 The ROC curve and AUC value results of the parameter results obtained by the algorithm model proposed in the present invention. Among them, PRPPO represents the method of the present invention; NLLS represents the nonlinear least squares method. DETAILED DESCRIPTION

[0051] Example: Figure 1-Figure 6 As shown in Figure 2, a proximal policy optimization algorithm combining clinical prior rewards 18 The F-FDGPET / CT liver cancer dynamic parameter estimation method includes the following:

[0052] Experimental materials of the present invention: 14 patients underwent PET / CT scan and metabolic PET / CT scan; patient data were provided by the Southern Pet Center of the Southern Hospital of Southern Medical University, and written informed consent was obtained from all patients; this study was approved by the School Review Committee of the First People's Hospital of Yunnan Province, number KHLL2022-KY189. In this embodiment, there were 14 patients (13 males and 1 female), with an age range of 31 to 78 years old; 17 cases were diagnosed with liver cancer, and the longest axis diameter of the tumor was 6.5±3.6cm (1.9 to 15.0cm).

[0053] First PET / CT scan: After the patient fasted for more than 6 hours and his blood sugar was stable, a liver CT scan (120 kV, 100 mAs) was performed; after intravenous injection of 5.5 MBq / kg 18After F-FDG, a 5-minute dynamic PET scan was performed in the liver region, supplemented by a 1-minute static PET (60 minutes). The dynamic PET data were segmented into 17 frames: 12 frames at 5 seconds, 4 frames at 60 seconds, and 1 frame at 60 minutes, and reconstructed using the standard ordered subset expectation maximization algorithm. Regions of interest (ROIs) in the CT images were manually delineated for each patient, including hepatocellular carcinoma, normal liver tissue, aorta, and portal vein (especially the extrahepatic part). After image fusion, the regions of interest were copied to the PET / CT images, time activity curves (TACs) were generated, and the maximum standard uptake values ​​(SUVmax) were extracted from each frame. Figure 2 Shown are the ROIs outlined at corresponding positions in a CT image and a frame of PET / CT image, where the black outlined area is the outline of hepatocellular carcinoma, the yellow outlined area is the location of the aorta, the red area is the location of the portal vein, and the green outlined area is normal liver tissue.

[0054] A proximal strategy optimization algorithm in this embodiment combined with clinical prior rewards 18 The specific steps of the F-FDG PET / CT liver cancer dynamic parameter estimation method are as follows:

[0055] Step 1: Obtain clinical liver PET / CT images, including a 5-minute dynamic PET / CT scan of the liver and a 60-minute conventional static PET / CT scan, to capture the liver dynamics of HCC patients. 18 Time course of F-FDG tracer uptake.

[0056] Step 2: Manually outline the region of interest (ROI), including hepatocellular carcinoma, normal liver tissue, artery and portal vein, adjust the ROI layer by layer, and obtain the time-concentration activity curve (TAC) and its data composed of the maximum SUV (SUVmax) of each frame.

[0057] Step 3: Divide the TAC data obtained in Step 2 into a data set, with the training set accounting for 29% and the test set accounting for 71%;

[0058] Step 4: Construct a mathematical model of the reversible double-input three-compartment model, including 18 The rate constant K1 of F-FDG uptake from blood into liver tissue cells is 18 The rate constant k2 (1 / min) of F-FDG transfer from liver tissue cells back to blood is 18 F-FDG is phosphorylated by hexokinase to 18The rate constant k3 for F-FDG 6-phosphate, the rate constant k4 for dephosphorylation, the aortic blood fraction fa, and the proportion of the measured blood volume vb.

[0059] Step 5. Construct an intelligent agent interaction environment combined with a reversible dual-input three-compartment model, including defining the dynamic parameter state, action distribution and action space, designing a reward function, etc.; the reward function part is based on clinical prior knowledge of parameters, including the upper and lower limits of parameters obtained from previous studies and the expected parameter range of individual parameters, to improve the reward function in the designed intelligent agent interaction environment.

[0060] Step 6. Build the overall framework of the proximal policy optimization (PPO) algorithm, including the actor network, the critic network, the generalized advantage estimation (GAE) algorithm for calculating the advantage function, and hyperparameters such as the clipping ratio of the new and old strategies.

[0061] Step 7. Use the proximal strategy optimization algorithm combined with medical prior rewards to estimate the parameters of the compartment model and perform descriptive analysis, statistical analysis and other clinical analysis on the perfusion and metabolic parameters of hepatocellular carcinoma tissue and normal liver tissue.

[0062] Specifically, the specific steps of Step 1 are as follows:

[0063] Step 1.1. Perform a low-dose CT scan of the liver (120 kV, 100 mA) at one bed; then quickly manually apply 2 mL of 0.9% saline. 18 F-FDG (5.5 MBq / kg) was flushed with 20 mL of 0.9% saline at a flow rate of 2 mL / s to allow the tracer to enter the cardiopulmonary circulation more quickly. After the injection, a 5-minute liver PET scan was performed. 18 Approximately 60 minutes after the F-FDG bolus injection, a whole-body CT scan (120 kV, 200 mA) was performed on a total of 11 beds from the vertex of the skull to the proximal thigh, followed by a 1-minute PET scan on each bed.

[0064] Step 1.2. After scanning, a short-term PET acquisition protocol was used to reconstruct PET images using the standard ordered subset expectation maximization (OSEM) algorithm. The first 1 minute of data was reconstructed into 12 frames with an interval of 5 seconds, and the next four minutes of data were reconstructed into 4 frames with an interval of 60 seconds. One frame of liver static PET image was selected at the 60th minute. A total of 17 frames of PET images were obtained and fused and registered with the CT images.

[0065] Specifically, in Step 2, if the patient has a lesion 18 If F-FDG uptake is not obvious, the ROI is drawn relative to the conventional CT image. The ROIs of the abdominal aorta and portal vein are located at approximately two-thirds of the cross-section of the blood vessels. In order to compare the HCC tumor with the surrounding tumor-free liver tissue, the corresponding ROI is drawn in the tumor-free liver tissue. During the delineation process, the interference of the internal blood vessels of the liver on the SUV is avoided as much as possible. The ROI is copied to the PET / CT fusion image to obtain the time-concentration activity curve (TAC) composed of the maximum SUV (SUVmax) of each frame. The results are shown in Figure 2. Figure 3 shown.

[0066] Specifically, in Step 3, 5 cases of clinical liver cancer data are randomly selected as a training set, and the remaining 12 cases of clinical liver cancer data are used as a test set.

[0067] Specifically, Figure 4 As shown, the mathematical form of the reversible two-input three-compartment model in Step 4 can be described by a set of ordinary differential equations:

[0068]

[0069] Where t is the time variable, C(t) is the tissue concentration function, and the calculation is based on the formula C(t) = C f (t)+C p (t), its expression is [c(t1),c(t2),…,c(t k )] T , k is the total number of frames in the PET scanning protocol. The matrix form of the model system can be expressed as:

[0070]

[0071] Solving this system of ordinary differential equations yields:

[0072]

[0073] For the target area in the PET image, the model not only considers the tracer activity in the tissue within the area, but also the proportion of the tracer concentration in the blood. Therefore, the parameter blood volume factor v b Used to calculate tissue concentration C(t) and blood input concentration C i (t) Do the weighted sum and finally get the total output concentration C of the target area t (t):

[0074] C t (t) = v b ×C i (t)+(1-v b )×C(t)

[0075] Specifically, in the agent interaction environment in Step 5, the parameter state is defined as:

[0076] s=[K1,k2,k3,k4,f a ,v b ]

[0077] The distribution of intelligent physical actions is defined as a 12-dimensional discrete space:

[0078] set a =[K1+,K1-,k2+,k2-,k3+,k3-,k4+,k4-,f a +,f a -,v b +,v b -]

[0079] In the setting of the reward function, a reward function combining clinical prior knowledge of kinetic parameters is proposed, and its calculation formula is:

[0080]

[0081] r t =λ1r1+λ2r2+λ3r3+λ4r4

[0082] Among them, r1~r4 are multiple reward signals, λ1~λ4 are the weights of reward signals, and r t is the reward value obtained at that moment, and t represents the time.

[0083] Specifically, the proximal strategy optimization algorithm based on Clip is adopted in Step 6. First, the generalized advantage estimation (GAE) algorithm is used to calculate the advantage value, and the formula is as follows:

[0084]

[0085] Among them, γ is the reward discount coefficient, λ is a constant between 0 and 1, is the calculation method of GAE advantage function, r t+l is the total reward of a single step at the t+lth moment, l is the number of steps experienced, V represents the state value function, s t Represents the state at time t. Then PPO updates the Actor and Critic networks multiple times. For the Critic network, the loss function can be calculated as:

[0086]

[0087] In the formula, r t′ represents the reward at time t′, t represents the current time, V represents the state value function, φ represents the parameters of the value function network, and s t Represents the state at time t. For the Actor network, the objective function can be calculated as:

[0088]

[0089] Among them, θ represents the model parameters, represents the expectation of the objective function, ε represents the clipping coefficient, π θ (a t |s t ) represents the new policy function, represents the old policy function, Represents the ratio of new and old probability strategies.

[0090] Specifically, the descriptive analysis and statistical analysis performed in Step 7 include, but are not limited to, root mean square error (RMSE), area under the curve (AUC), and T test (P<0.05 is considered statistically significant). In addition, the differences in kinetic parameters K1, k2, k3, k4, fa, and vb between tumor and normal liver tissues are analyzed to evaluate their clinical significance.

[0091] Experiment 1:

[0092] Comparative experiment with traditional algorithms. The traditional nonlinear least square (NLLS) experiment was performed on the data of 12 patients with hepatocellular carcinoma. The obtained parameter estimation results and experimental evaluation index comparison are shown in Table 1:

[0093] Table 1 Statistical analysis of parameter estimation results of NLLS method

[0094]

[0095] *Indicates that the average comparison between normal liver tissue and HCC does not conform to physiological expectations

[0096] As shown in Table 1, in the NLLS optimization method, for hepatocellular carcinoma, the obtained kinetic parameters of K1 (1.4899±0.0003 vs. 1.3925±0.1914, P<0.05) and fa (0.9279±0.1340 vs. 0.3095±0.2378, P<0.001) were significantly higher than those of normal liver tissue. k2 (0.8486±0.4080 vs. 1.2169±0.3561, P=0.0125) was significantly lower than that of normal liver tissue, but did not meet physiological expectations. For k3 (0.0736±0.0874 vs. 0.0198±0.0431, P=0.2025) and vb (0.0999±0.0001 vs. 0.0929±0.0201, P=0.0948), although HCC was higher than normal liver tissue, there was no significant difference. For k4 (0.0669±0.0347 vs. 0.0860±0.0001, P=0.1340), HCC was lower than normal liver tissue, but there was no significant difference. In general, the traditional NLLS optimization method can only effectively distinguish HCC from normal liver tissue through parameters K1 and fa.

[0097] Experiment 2:

[0098] According to the experimental steps, the test was conducted on the same batch of 12 HCC patient data. The specific parameter estimation results and experimental evaluation index comparison are shown in Table 2:

[0099] Table 2 Experimental results of the present invention

[0100]

[0101] As shown in Table 2, the proximal strategy optimization algorithm combined with clinical prior rewards in the present invention 18In the F-FDG PET / CT liver cancer kinetic parameter estimation method, the kinetic parameters of hepatocellular carcinoma obtained, including K1 (1.1491±0.1620 vs. 0.9658±0.0849, P<0.001), k2 (1.4342±0.0888 vs. 1.2763±0.0846, P<0.001), k3 (0.1875±0.0040 vs. 0.1817±0.0026, P<0.001), fa (0.8251±0.2035 vs. 0.4046±0.1789, P<0.001), and vb (0.0487±0.0045 vs. 0.0425±0.0031, P<0.001), were significantly higher than those of normal liver tissue. The significant increase of K1 and K3 indicated that the activities of glucose transporter and hexokinase were increased in HCC, while K2 indicated that the two regions of HCC and normal liver tissue were different. 18 The clearance rate of F-FDG transported back to the blood is different. fa is significantly increased, which is consistent with the assumption that dual input is the actual blood supply mode of most HCC tumors. vb is significantly increased, indicating that the blood supply intensity of hepatocellular carcinoma is significantly higher than that of normal liver tissue. The k4 parameter of hepatocellular carcinoma is significantly lower than that of normal liver tissue, indicating that G6P activity is increased. All parameter results obtained by the algorithm model of the present invention are consistent with the expected clinical performance.

[0102] In summary, the proximal strategy optimization algorithm combined with clinical prior rewards 18 The pharmacokinetic parameter results obtained by the F-FDG PET / CT liver cancer kinetic parameter estimation method are more accurate and more consistent with physiological characteristics and clinical performance expectations than the traditional NLLS algorithm. At the same time, a higher overall AUC value and a lower fitting error RMSE value are obtained. The results are as follows: Figure 6 As shown, it can be seen that the method of the present invention can be effectively carried out 18 F-FDG PET / CT liver cancer dynamics parameter estimation and better assessment of hepatocellular carcinoma characteristics.

[0103] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A proximal strategy optimization algorithm combined with clinical prior rewards 18 F-FDG PET / CT liver cancer dynamic parameter estimation method, It is characterized in that The specific steps include: Step 1: Obtain clinical liver PET / CT images to capture HCC patients 18 Time course of F-FDG tracer uptake; Step 2, manually outline the region of interest (ROI), adjust the ROI layer by layer, and obtain the time-concentration activity curve (TAC) and its data composed of the maximum SUV in each frame; Step 3, divide the TAC data obtained in Step 2 into data sets; Step 4, construct a mathematical model of the reversible double-input three-compartment model; Step 5, construct an intelligent agent interaction environment combined with a reversible dual-input three-compartment model, including defining the dynamic parameter state, action distribution and action space, and designing a reward function; the reward function is improved based on the clinical prior knowledge of the parameters, including the upper and lower limits of the parameters obtained from previous studies and the expected parameter range of individual parameters. Step 6. Build the overall framework of the proximal policy optimization PPO algorithm, including the policy network, value function network, generalized advantage estimation algorithm for calculating advantage function, and hyperparameters such as the clipping ratio of new and old policies; Step 7. Use the proximal strategy optimization algorithm combined with medical prior rewards to estimate the parameters of the compartment model and perform descriptive analysis, statistical analysis and other clinical analysis on the perfusion and metabolic parameters of hepatocellular carcinoma tissue and normal liver tissue.

2. The proximal strategy optimization algorithm combined with clinical prior rewards according to claim 1 18 F-FDG PET / CT liver cancer dynamic parameter estimation method, Features: The specific steps of Step 1 are as follows: Step 1.

1. Perform a low-dose CT scan of the liver at one bed position; then quickly manually apply 2 mL of 0.9% saline. 18 F-FDG, in order to make the tracer enter the cardiopulmonary circulation faster, it was flushed with 20 mL of 0.9% saline at a flow rate of 2 mL / s; after the injection, a 5-minute liver PET scan was performed. Conventional static PET / CT scans 18 Approximately 60 minutes after the F-FDG bolus injection, a whole-body CT scan was first performed on a total of 11 beds from the vertex of the skull to the proximal thigh, followed by a 1-minute PET scan on each bed. Step 1.

2. After scanning, a short-term PET acquisition protocol was used, and the PET image was reconstructed using the standard ordered subset expectation maximization (OSEM) algorithm. The first 1 minute of data was reconstructed into 12 frames with an interval of 5 seconds, and the next four minutes of data were reconstructed into 4 frames with an interval of 60 seconds. A static PET image of the liver was selected at the 60th minute. A total of 17 frames of PET images were obtained and fused and registered with the CT images.

3. The proximal strategy optimization algorithm combined with clinical prior rewards according to claim 1 18 F-FDG PET / CT liver cancer dynamic parameter estimation method, Features: In Step 2, if the patient has a lesion 18 If F-FDG uptake was not obvious, the ROI was drawn relative to the conventional CT image; the ROIs of the abdominal aorta and portal vein were located at approximately two-thirds of the vascular cross-section. In order to compare the HCC tumor with the surrounding tumor-free liver tissue, the corresponding ROI was drawn in the tumor-free liver tissue, and the interference of the internal blood vessels of the liver on the SUV was avoided as much as possible during the outlining process; the ROI was copied to the PET / CT fusion image to obtain the time-concentration activity curve TAC composed of the maximum SUV in each frame.

4. The proximal strategy optimization algorithm combined with clinical prior rewards according to claim 1 18 F-FDG PET / CT liver cancer dynamic parameter estimation method, Features: In Step 2, the regions of interest including hepatocellular carcinoma, normal liver tissue, artery and portal vein were manually outlined.

5. The proximal strategy optimization algorithm combined with clinical prior rewards according to claim 1 18 F-FDG PET / CT liver cancer dynamic parameter estimation method, Features: The mathematical form of the reversible two-input three-compartment model in Step 4 is described by a set of ordinary differential equations: Where t is the time variable, C(t) is the tissue concentration function, and the calculation is based on the formula C(t) = C f (t)+C p (t), its expression is [c(t 1 ),c(t 2 ),…,c(t k )] T , k is the total number of frames in the PET scanning protocol, and the matrix form of the model system is expressed as: Solving this system of ordinary differential equations yields: For the target region in the PET image, the model not only considers the tracer activity in the tissue within the region, but also the ratio of the tracer concentration in the blood; therefore, the parameter blood volume factor v b Used to calculate tissue concentration C(t) and blood input concentration C i (t) Do the weighted sum and finally get the total output concentration C of the target area t (t): C t (t)=v b ×C i (t)+(1-v b )×C(t)。 6. The proximal strategy optimization algorithm combined with clinical prior rewards according to claim 1 18 F-FDG PET / CT liver cancer dynamic parameter estimation method, Features: In the agent interaction environment in Step 5, the parameter state is defined as: s=[K 1 ,k 2 ,k 3 ,k 4 ,f a ,v b ] The distribution of intelligent physical actions is defined as a 12-dimensional discrete space: set a =[K 1 +,K 1 -,k 2 +,k 2 -,k 3 +,k 3 -,k 4 +,k 4 -,f a +,f a -,v b +,v b -] In the setting of the reward function, a reward function combining clinical prior knowledge of kinetic parameters is proposed, and its calculation formula is: r t =λ 1 r 1 +λ 2 r 2 +λ 3 r 3 +λ 4 r 4 Among them, r 1 ~r 4 are multiple reward signals, λ 1 ~λ 4 is the weight of the reward signal, r t is the reward value obtained at that moment, and t represents the time.

7. The proximal strategy optimization algorithm combined with clinical prior rewards according to claim 1 18 F-FDG PET / CT liver cancer dynamic parameter estimation method, Features: The proximal strategy optimization algorithm based on cutting is adopted in Step 6. First, the advantage value is calculated using the generalized advantage estimation algorithm, and the formula is as follows: Among them, γ is the reward discount coefficient, λ is a constant between 0 and 1, is the calculation method of GAE advantage function, r t+l is the total reward of a single step at the t+lth moment, l is the number of steps experienced, V represents the state value function, s t represents the state at time t; Then PPO updates the Actor and Critic networks multiple times. For the Critic network, the loss function is calculated as: In the formula, r t′ represents the reward at time t′, t represents the current time, V represents the state value function, φ represents the parameters of the value function network, and s t represents the state at time t; For the Actor network, the objective function is calculated as: Among them, γ represents the model parameter, represents the expectation of the objective function, ε represents the clipping coefficient, π θ (a t |s t ) represents the new policy function, represents the old policy function, Represents the ratio of new and old probability strategies.

8. The proximal strategy optimization algorithm combined with clinical prior rewards according to claim 1 18 F-FDG PET / CT liver cancer dynamic parameter estimation method, Features: The descriptive analysis and statistical analysis performed in Step 7 include root mean square error RMSE, area under the curve AUC and T test, and P < 0.05 is considered to be statistically significant; in addition, it also includes analysis of the differences in kinetic parameters K1, k2, k3, k4, fa, and vb between tumor and normal liver tissues to evaluate their clinical significance.

Citation Information

Patent Citations

  • Quadruped robot path planning method based on DRL

    CN114859911A

  • Medical image segmentation method for introducing priori knowledge based on reward function

    CN115187571A

  • Human-machine cooperation layered reinforcement learning method and framework for online inquiry

    CN117711641A

  • Unmanned system searching and planning strategy optimization method and device for dynamic scene, medium and product

    CN118036469A

  • 18F-FDG PET / CT liver cancer kinetic parameter estimation method of Advantageous Actor-Critic algorithm

    CN118378663A

Cited By

  • Radiotherapy lattice parameter optimization method and system based on space segmentation technology

    CN120047469A