An early diagnosis method for pancreatic cancer based on deep reinforcement learning

Abstract

The present invention discloses a method for early diagnosis of pancreatic cancer based on deep reinforcement learning, including S1, collecting multimodal medical image datasets of pancreatic cancer patients and healthy individuals respectively; S2, preprocessing the multimodal medical image datasets respectively; S3, using a deep neural network to extract features from the preprocessed multimodal medical image datasets of pancreatic cancer patients and healthy individuals; S4, using the result of deep learning feature extraction as input, and guiding the learning of the reinforcement learning model by defining a reward function; S5, optimizing the diagnostic strategy through continuous trial and error and model self-adjustment; S6, testing and evaluating the model through a validation set; S7, deploying the optimized reinforcement learning model in the actual medical diagnosis process to provide doctors with auxiliary decision support for early diagnosis of pancreatic cancer. The present invention improves the accuracy of early diagnosis of pancreatic cancer by integrating deep learning and reinforcement learning technologies, while optimizing the diagnostic process and improving diagnostic efficiency.

Description

Technical Field

[0001] The present invention relates to the field of biomedical information technology, and in particular to an early diagnosis method for pancreatic cancer based on deep reinforcement learning. Background Art

[0002] Pancreatic cancer is considered one of the most deadly cancers because of its unclear early symptoms and difficulty in detection. Early detection and diagnosis are crucial to improving patient survival rates. However, existing diagnostic methods face many challenges and limitations. First, traditional image processing and analysis methods rely on the experience and subjective judgment of radiologists, which is not only inefficient, but the accuracy of the diagnostic results is limited by the doctor's experience level. In addition, early pancreatic cancer may appear similar to normal tissue in medical images, making it difficult to distinguish even for experienced radiologists. These factors together lead to a low accuracy rate in the early diagnosis of pancreatic cancer and a high risk of misdiagnosis and missed diagnosis.

[0003] In addition, with the rapid increase in the amount of medical image data, doctors are facing tremendous work pressure and analysis challenges, and need to spend a lot of time manually checking images, which limits the efficiency of the diagnostic process. Although computer-aided diagnosis (CAD) systems have made some progress in improving diagnostic efficiency, many existing systems lack sufficient flexibility and adaptability to efficiently handle complex and changing diagnostic scenarios. Therefore, how to provide an early diagnosis method for pancreatic cancer based on deep reinforcement learning is an urgent problem that technicians in this field need to solve. Summary of the invention

[0004] One purpose of the present invention is to propose a method for early diagnosis of pancreatic cancer based on deep reinforcement learning. The present invention aims to improve the accuracy of early diagnosis of pancreatic cancer by integrating deep learning and reinforcement learning techniques, while optimizing the diagnostic process and improving diagnostic efficiency.

[0005] A method for early diagnosis of pancreatic cancer based on deep reinforcement learning according to an embodiment of the present invention comprises the following steps:

[0006] S1. Collect multimodal medical image datasets of pancreatic cancer patients and healthy individuals respectively, wherein the multimodal medical image datasets include CT, MRI, ultrasound images, photoacoustic imaging and microfluidic chip image data, gene expression data extracted from blood samples, and protein marker data;

[0007] S2. Perform image normalization, denoising and enhancement on multimodal medical image datasets of pancreatic cancer patients and healthy individuals.

[0008] S3. Using deep neural networks, we extract features from preprocessed multimodal medical image datasets of pancreatic cancer patients and healthy individuals, and learn complex features and patterns in two multimodal medical image datasets to distinguish pancreatic cancer images from normal images.

[0009] S4. Establish a reinforcement learning model, use the results of deep learning feature extraction as input, and guide the learning of the reinforcement learning model by defining a reward function;

[0010] S5. Set goals in the reinforcement learning model to maximize diagnostic accuracy and minimize diagnostic time, and optimize the diagnostic strategy through continuous trial and error and model self-adjustment;

[0011] S6. Test and evaluate the model using the validation set, and use diagnostic accuracy, recall, and F1 score indicators to evaluate model performance;

[0012] S7. Deploy the optimized reinforcement learning model in the actual medical diagnosis process to provide doctors with auxiliary decision support for early diagnosis of pancreatic cancer.

[0013] Optionally, the S1 specifically includes:

[0014] Collect multimodal medical image data of N pancreatic cancer patients and construct a multimodal medical image dataset D P,i :

[0015]

[0016] Collect multimodal medical image data of M healthy individuals and construct a multimodal medical image dataset D H,j :

[0017]

[0018] in, and represents the computed tomography image dataset of the i-th pancreatic cancer patient and the j-th healthy individual, and represent the MRI image datasets of the i-th pancreatic cancer patient and the j-th healthy individual, respectively. and represent the ultrasound image datasets of the i-th pancreatic cancer patient and the j-th healthy individual, respectively. and Represent the photoacoustic imaging image datasets of the i-th pancreatic cancer patient and the j-th healthy individual, and represents the microfluidic chip image dataset of the i-th pancreatic cancer patient and the j-th healthy individual, and Represent the gene expression datasets extracted from blood samples of the i-th pancreatic cancer patient and the j-th healthy individual, and Represent the protein marker datasets of the i-th pancreatic cancer patient and the j-th healthy individual, respectively.

[0019] Optionally, the S3 specifically includes:

[0020] S31. Configure multiple deep convolutional neural network models M DNN , each model is specialized for D P,i and D H,j The feature extraction is performed on one modality data in each dataset, and each dataset includes multiple modalities such as CT, MRI, ultrasound, photoacoustic imaging, microfluidic chips, gene expression and protein markers;

[0021] S32. For each modality of medical image, including and The corresponding deep convolutional neural network model is applied for feature extraction to obtain feature vectors. This step is repeated for all modalities, including CT, MRI, ultrasound, photoacoustic imaging and microfluidic chips, gene expression and protein markers, to generate feature vectors. and

[0022]

[0023] Among them, Modality∈{CT,MRI,US,PAI,MFC,GE,PM},(i=1,2,...,N),(j=1,2,...,M),σis the activation function,w k and b k is a model parameter, k represents the level or feature dimension;

[0024] S33, for each pancreatic cancer patient i and each healthy individual j, the feature vectors of all their modalities and Integrate to form a comprehensive feature vector F P,i and F H,j ,The feature integration method uses feature fusion technology to capture complementary information across modalities:

[0025]

[0026] in, represents the complex fusion operation of different modal features, φ is the nonlinear transformation function, and Θ represents the parameters in the transformation;

[0027] S34. Establish a deep learning model to process each comprehensive feature vector F P,i and FH,j , extract high-level abstract features and obtain the final feature representation F High-P,i and F High-H,j :

[0028]

[0029] in, and represents the features after the lth layer, represents the complex combination or fusion mechanism between layers, θ DNN are the parameters of the deep network;

[0030] S35, using F High-P,i and F High-H,j Train a classifier C to distinguish pancreatic cancer images from normal images, and assign a prediction probability to each sample through a classification model based on the softmax function, indicating the possibility that it belongs to the pancreatic cancer category:

[0031]

[0032] Among them, P(L P,i =1|F High-P,i ) is a given high-level eigenvector F High-P,i The probability that sample i is classified as pancreatic cancer is P(L H,j =0|F High-H,j ) is a given high-level eigenvector F High-H,j When , the probability that sample j is classified as normal, W cancer and W normal are model parameters corresponding to the weights of pancreatic cancer and normal categories, respectively, and b cancer and b normal are bias terms, corresponding to pancreatic cancer and normal classes, respectively;

[0033] The classification decision is made by choosing the class with the highest probability:

[0034] If P(L P,i =1|F High-P,i )>P(L H,j =0|F High-H,j ), then sample i is classified as pancreatic cancer;

[0035] On the contrary, if P(L H,j =0|F High-H,j )>P(L P,i =1|F High-P,i ), then sample j is classified as normal.

[0036] Optionally, the S4 specifically includes:

[0037] S41. Initialize the reinforcement learning model M RL , accepted by the deep learning model M DNN The extracted and integrated high-level feature vector F High-P,i and F High-H,j As state input;

[0038] S42, define action set A = {a1, a2, ..., a m}, where each action a k represents a possible manipulation of the diagnostic decision process, including increasing the weight of a particular modality or adjusting the classification threshold;

[0039] S43. Set a reward function R(s,a) to evaluate the effect of the new state reached after performing action a in state s, where s is the current state, represented as a high-level feature vector, and a is the action performed:

[0040] R(s,a)=α·Accuracy(s,a)-β·Time(s,a);

[0041] Among them, Accuracy(s,a) represents the improvement of diagnostic accuracy after executing action a, Time(s,a) represents the time required to execute the action, and α and β are weight coefficients to balance accuracy and time efficiency;

[0042] S44. Use Q-learning to train the reinforcement learning model M RL , continuously update the strategy π(a|s) through an iterative process to maximize the expected reward:

[0043] π * (a|s)=argmax a ∑ s′ P(s′|s,α)(R(s,α)+γV * (s′));

[0044] Among them, π * (a|s) is the optimal strategy for selecting action a in state s, P(s′|s,a) is the probability of transitioning from state s to state s′ by executing action a, and V * (s′) is the maximum expected reward that can be obtained in state s′, and γ is a discount factor used to balance immediate rewards and future rewards.

[0045] Optionally, the S5 specifically includes:

[0046] S51. In the reinforcement learning model M RL The comprehensive objective function O is set in the paper to maximize the diagnostic accuracy and minimize the diagnostic time:

[0047]

[0048] Among them, y i is the true label of the i-th sample, is the label predicted by the model, II(·) is the indicator function, which takes the value of 1 when the prediction is correct and 0 otherwise, t i is the time required to process the i-th sample, and N is the total number of samples;

[0049] S52. Use Q-learning to reinforce the learning model M RL Perform training to learn how to select action a to optimize the objective function O:

[0050]

[0051] Among them, s' is the new state reached after executing action a, α is the learning rate, and γ is the discount factor used to adjust the weights of immediate rewards and future rewards;

[0052] S53, through the trial and error learning mechanism, strengthen the learning model M RL In each iteration, based on the current state s and the set of possible actions A, the expected reward of each action is predicted, and the action that maximizes the expected reward of the objective function O is selected for execution. The action selection adopts a greedy strategy:

[0053]

[0054] Combined with the influence of objective function O, the adjustment of O on Q value update is introduced;

[0055] S54. After each round of learning, the reinforcement learning model M RL According to the actual results of the action, including the change in diagnostic accuracy and the time required, adjust the parameters of the action selection strategy π(a|s) and the reward function R(s,a):

[0056] R(s,a)←(1-η)R(s,a)+η[λ·ΔAccuracy(s,a)-(1-λ)·ΔTime(s,a)];

[0057]

[0058] Among them, η and ξ are adjustment coefficients used to control the adjustment speed of the reward function and strategy during the learning process. ΔAccuracy(s,a) and ΔTime(s,a) represent the changes in diagnostic accuracy and time after executing action a, respectively.

[0059] S55, repeat steps S53 and S54 until the reinforcement learning model M RL The comprehensive objective function O on the validation set no longer improves.

[0060] Optionally, the S6 specifically includes

[0061] S61. Use an independent validation set:

[0062] V={(x v1 ,y v1 ),(x v2 ,y v2 ),...,(x pN ,y vN )};

[0063] Among them, x vi is the feature vector of the i-th sample in the validation set, y vi is the corresponding true label, N is the number of samples in the validation set;

[0064] S62. For each sample in the validation set, use the trained deep reinforcement learning model M RL Perform diagnostic prediction and obtain prediction labels

[0065] S63. Calculate the diagnostic accuracy, recall rate and F1 score of the model on the validation set:

[0066]

[0067] Among them, (TP) (TruePositives) is the number of true positives, and FN (FalseNegatives) is the number of false negatives;

[0068] S64, based on the indicators in step S63, comprehensively evaluate the model M RL performance in early diagnosis of pancreatic cancer;

[0069] S65. According to the performance evaluation results, make necessary adjustments and optimizations to the model, readjust the parameters in the reinforcement learning model, optimize the feature extraction strategy, or adjust the reward function.

[0070] The beneficial effects of the present invention are:

[0071] (1) By combining multimodal medical image datasets with deep learning technology, the present invention can effectively identify and extract small and difficult-to-distinguish features of early pancreatic cancer, significantly improving the accuracy of early diagnosis.

[0072] (2) The present invention uses reinforcement learning technology to provide high-accuracy diagnostic results initially while optimizing itself based on real-time feedback, thereby continuously improving the efficiency of the diagnostic process. The reinforcement learning model can self-learn and adjust based on the continuously collected diagnostic feedback, continuously optimizing the diagnostic strategy, so that the diagnostic system can adapt to various complex and changing diagnostic scenarios, and maintain its advancement and effectiveness in the rapidly developing medical field. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0074] Figure 1 This is a flow chart of a method for early diagnosis of pancreatic cancer based on deep reinforcement learning proposed in the present invention. DETAILED DESCRIPTION

[0075] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0076] refer to Figure 1 , an early diagnosis method for pancreatic cancer based on deep reinforcement learning, comprising the following steps:

[0077] S1. Collect multimodal medical image datasets of pancreatic cancer patients and healthy individuals. The multimodal medical image datasets include CT, MRI, ultrasound images, photoacoustic imaging and microfluidic chip image data, gene expression data extracted from blood samples, and protein marker data.

[0078] S2. Perform image normalization, denoising and enhancement on multimodal medical image datasets of pancreatic cancer patients and healthy individuals.

[0079] S3. Using deep neural networks, we extract features from preprocessed multimodal medical image datasets of pancreatic cancer patients and healthy individuals, and learn complex features and patterns in two multimodal medical image datasets to distinguish pancreatic cancer images from normal images.

[0080] S4. Establish a reinforcement learning model, use the results of deep learning feature extraction as input, and guide the learning of the reinforcement learning model by defining a reward function;

[0081] S5. Set goals in the reinforcement learning model to maximize diagnostic accuracy and minimize diagnostic time, and optimize the diagnostic strategy through continuous trial and error and model self-adjustment;

[0082] S6. Test and evaluate the model using the validation set, and use diagnostic accuracy, recall, and F1 score indicators to evaluate model performance;

[0083] S7. Deploy the optimized reinforcement learning model in the actual medical diagnosis process to provide doctors with auxiliary decision support for early diagnosis of pancreatic cancer.

[0084] In this implementation, S1 specifically includes:

[0085] Collect multimodal medical image data of N pancreatic cancer patients and construct a multimodal medical image dataset D P,i :

[0086]

[0087] Collect multimodal medical image data of M healthy individuals and construct a multimodal medical image dataset D H,j :

[0088]

[0089] in, and represents the computed tomography image dataset of the i-th pancreatic cancer patient and the j-th healthy individual, and represent the MRI image datasets of the i-th pancreatic cancer patient and the j-th healthy individual, respectively. and represent the ultrasound image datasets of the i-th pancreatic cancer patient and the j-th healthy individual, respectively. and Represent the photoacoustic imaging image datasets of the i-th pancreatic cancer patient and the j-th healthy individual, and represents the microfluidic chip image dataset of the i-th pancreatic cancer patient and the j-th healthy individual, and Represent the gene expression datasets extracted from blood samples of the i-th pancreatic cancer patient and the j-th healthy individual, and Represent the protein marker datasets of the i-th pancreatic cancer patient and the j-th healthy individual, respectively.

[0090] In this implementation, S3 specifically includes:

[0091] S31. Configure multiple deep convolutional neural network models M DNN , each model is specialized for D P,i and D H,jThe feature extraction is performed on one modality data in each dataset, and each dataset includes multiple modalities such as CT, MRI, ultrasound, photoacoustic imaging, microfluidic chips, gene expression and protein markers;

[0092] S32. For each modality of medical image, including and The corresponding deep convolutional neural network model is applied for feature extraction to obtain feature vectors. This step is repeated for all modalities, including CT, MRI, ultrasound, photoacoustic imaging and microfluidic chips, gene expression and protein markers, to generate feature vectors. and

[0093]

[0094] Among them, Modality∈{CT,MRI,US,PAI,MFC,GE,PM},(i=1,2,...,N),(j=1,2,...,M),σis the activation function,w k and b k is a model parameter, k represents the level or feature dimension;

[0095] S33, for each pancreatic cancer patient i and each healthy individual j, the feature vectors of all their modalities and Integrate to form a comprehensive feature vector F P,i and F H,j ,The feature integration method uses feature fusion technology to capture complementary information across modalities:

[0096]

[0097] in, represents the complex fusion operation of different modal features, φ is the nonlinear transformation function, and Θ represents the parameters in the transformation;

[0098] S34. Establish a deep learning model to process each comprehensive feature vector F P,i and F H,j , extract high-level abstract features and obtain the final feature representation F High-P,i and F High-H,j :

[0099]

[0100] in, and represents the features after the lth layer, represents the complex combination or fusion mechanism between layers, θ DNN are the parameters of the deep network;

[0101] S35, using F High-P,i and F High-H,j Train a classifier C to distinguish pancreatic cancer images from normal images, and assign a prediction probability to each sample through a classification model based on the softmax function, indicating the possibility that it belongs to the pancreatic cancer category:

[0102]

[0103] Among them, P(L P,i =1|F High-P,i ) is a given high-level eigenvector F High-P,i The probability that sample i is classified as pancreatic cancer is P(L H,j =0|F High-H,j ) is a given high-level eigenvector F High-H,j When , the probability that sample j is classified as normal, W cancer and W normal are model parameters corresponding to the weights of pancreatic cancer and normal categories, respectively, and b cancer and b normal are bias terms, corresponding to pancreatic cancer and normal classes, respectively;

[0104] The classification decision is made by choosing the class with the highest probability:

[0105] If P(L P,i =1|F High-P,i )>P(L H,j =0|F High-H,j ), then sample i is classified as pancreatic cancer;

[0106] On the contrary, if P(L H,j =0|F High-H,j )>P(L P,i =1|F High-P,i ), then sample j is classified as normal.

[0107] In this implementation, S4 specifically includes:

[0108] S41. Initialize the reinforcement learning model M RL , accepted by the deep learning model M DNN The extracted and integrated high-level feature vector F High-P,i and F High-H,j As state input;

[0109] S42, define action set A = {a1, a2, ..., a m}, where each action a k represents a possible manipulation of the diagnostic decision process, including increasing the weight of a particular modality or adjusting the classification threshold;

[0110] S43. Set a reward function R(s,a) to evaluate the effect of the new state reached after performing action a in state s, where s is the current state, represented as a high-level feature vector, and a is the action performed:

[0111] R(s,a)=α·Accuracy(s,a)-β·Time(s,a);

[0112] Among them, Accuracy(s,a) represents the improvement of diagnostic accuracy after executing action a, Time(s,a) represents the time required to execute the action, and α and β are weight coefficients to balance accuracy and time efficiency;

[0113] S44. Use Q-learning to train the reinforcement learning model M RL , continuously update the strategy π(a|s) through an iterative process to maximize the expected reward:

[0114] π * (a|s)=argmax a ∑ s′ P(s′|s,a)(R(s,a)+γV * (s′));

[0115] Among them, π * (a|s) is the optimal strategy for selecting action a in state s, P(s′|s,a) is the probability of transitioning from state s to state s′ by executing action a, and V * (s′) is the maximum expected reward that can be obtained in state s′, and γ is a discount factor used to balance immediate rewards and future rewards.

[0116] In this implementation, S5 specifically includes:

[0117] S51. In the reinforcement learning model M RL The comprehensive objective function O is set in the paper to maximize the diagnostic accuracy and minimize the diagnostic time:

[0118]

[0119] Among them, y i is the true label of the i-th sample, is the label predicted by the model, II(·) is the indicator function, which takes the value of 1 when the prediction is correct and 0 otherwise, t i is the time required to process the i-th sample, and N is the total number of samples;

[0120] S52. Use Q-learning to reinforce the learning model M RL Perform training to learn how to select action a to optimize the objective function O:

[0121]

[0122] Among them, s′ is the new state reached after executing action a, α is the learning rate, and γ is the discount factor used to adjust the weights of immediate rewards and future rewards;

[0123] S53, through the trial and error learning mechanism, strengthen the learning model M RL In each iteration, based on the current state s and the set of possible actions A, the expected reward of each action is predicted, and the action that maximizes the expected reward of the objective function O is selected for execution. The action selection adopts a greedy strategy:

[0124]

[0125] Combined with the influence of objective function O, the adjustment of O on Q value update is introduced;

[0126] S54. After each round of learning, the reinforcement learning model M RL According to the actual results of the action, including the change in diagnostic accuracy and the time required, adjust the parameters of the action selection strategy π(a|s) and the reward function R(s,a):

[0127] R(s,a)←(1-η)R(s,a)+η[λ·ΔAccuracy(s,a)-(1-λ)·ΔTime(s,a)];

[0128]

[0129] Among them, η and ξ are adjustment coefficients used to control the adjustment speed of the reward function and strategy during the learning process. ΔAccuracy(s,a) and ΔTime(s,a) represent the changes in diagnostic accuracy and time after executing action a, respectively.

[0130] S55, repeat steps S53 and S54 until the reinforcement learning model M RL The comprehensive objective function O on the validation set no longer improves.

[0131] In this implementation, S6 specifically includes

[0132] S61. Use an independent validation set:

[0133] V={(x v1 ,y v1 ),(x v2 ,y v2 ),...,(x vN ,y vN )};

[0134] Among them, x viis the feature vector of the i-th sample in the validation set, y vi is the corresponding true label, N is the number of samples in the validation set;

[0135] S62. For each sample in the validation set, use the trained deep reinforcement learning model M RL Perform diagnostic prediction and obtain prediction labels

[0136] S63. Calculate the diagnostic accuracy, recall rate and F1 score of the model on the validation set:

[0137]

[0138] Among them, (TP) (TruePositives) is the number of true positives, and FN (FalseNegatives) is the number of false negatives;

[0139] S64, based on the indicators in step S63, comprehensively evaluate the model M RL performance in early diagnosis of pancreatic cancer;

[0140] S65. According to the performance evaluation results, make necessary adjustments and optimizations to the model, readjust the parameters in the reinforcement learning model, optimize the feature extraction strategy, or adjust the reward function.

[0141] Embodiment 1:

[0142] In a top hospital in Beijing, doctors are facing an increasingly severe challenge: early diagnosis of pancreatic cancer. Pancreatic cancer is difficult to be accurately diagnosed in the early stages because of its strong concealment and unclear early symptoms. Early detection is crucial to improving the survival rate of patients. Traditional diagnostic methods rely on radiologists to interpret medical images such as CT and MRI, but since early tumors are small in size and difficult to distinguish from normal tissues in images, this greatly increases the difficulty and uncertainty of diagnosis. In order to solve this problem, the hospital decided to adopt the method of the present invention.

[0143] The scenario of Example 1 takes place in 2023, and the hospital collects a large number of multimodal medical image data sets, including pancreatic cancer patients and healthy individuals, such as CT, MRI, ultrasound, as well as gene expression data and protein marker data. After rigorous preprocessing, these data are used to train deep learning models. In particular, the data of each modality is input into a deep convolutional neural network designed for it, and feature extraction is performed, and the information of different modalities is integrated through feature fusion technology to extract richer and more accurate features for diagnosis.

[0144] Subsequently, a reinforcement learning model was established, using these high-level features as input, and a reward function was defined to guide the model to learn how to improve the accuracy of diagnosis while shortening the time required for diagnosis. After a lot of trial and error and self-adjustment, the model finally formed an optimized diagnostic strategy.

[0145] In the second half of 2023, the method was applied in clinical trials in hospitals. Compared with traditional diagnostic methods, this new method based on deep reinforcement learning has significantly improved the accuracy of early pancreatic cancer diagnosis. In a validation set of 500 patients suspected of early pancreatic cancer, the new method had a diagnostic accuracy of 95.36%, a significant improvement over the 75.14% of the traditional method. At the same time, the new method reduced the average diagnostic time from 30 minutes to 10 minutes, greatly improving diagnostic efficiency.

[0146] In addition, the performance of this method has been further improved by continuously collecting clinical feedback data to iteratively optimize the model. By the beginning of 2024, the hospital's recall rate and F1 score for early diagnosis of pancreatic cancer using this method have also been significantly improved, with the recall rate increasing from 85.3% to 93.48% and the F1 score increasing from 80.87% to 92.63%. This means that this method can not only accurately diagnose pancreatic cancer, but also greatly reduce the possibility of missed diagnosis.

[0147] By combining multimodal medical image datasets with deep learning technology, the present invention can effectively identify and extract tiny and indistinguishable features of early pancreatic cancer, significantly improving the accuracy of early diagnosis.

[0148] The present invention uses reinforcement learning technology to provide high-accuracy diagnostic results initially while optimizing itself based on real-time feedback, thereby continuously improving the efficiency of the diagnostic process. The reinforcement learning model can self-learn and adjust based on the continuously collected diagnostic feedback, continuously optimizing the diagnostic strategy, so that the diagnostic system can adapt to various complex and changing diagnostic scenarios, and maintain its advancement and effectiveness in the rapidly developing medical field.

[0149] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for early diagnosis of pancreatic cancer based on deep reinforcement learning, characterized in that: The following steps are involved: S1. Collect multimodal medical image datasets of pancreatic cancer patients and healthy individuals respectively, wherein the multimodal medical image datasets include CT, MRI, ultrasound images, photoacoustic imaging and microfluidic chip image data, gene expression data extracted from blood samples, and protein marker data; S2. Perform image normalization, denoising and enhancement on multimodal medical image datasets of pancreatic cancer patients and healthy individuals. S3. Using deep neural networks, we extract features from preprocessed multimodal medical image datasets of pancreatic cancer patients and healthy individuals, and learn complex features and patterns in two multimodal medical image datasets to distinguish pancreatic cancer images from normal images. S4. Establish a reinforcement learning model, use the results of deep learning feature extraction as input, and guide the learning of the reinforcement learning model by defining a reward function; S5. Set goals in the reinforcement learning model to maximize diagnostic accuracy and minimize diagnostic time, and optimize the diagnostic strategy through continuous trial and error and model self-adjustment; S6. Test and evaluate the reinforcement learning model through the validation set, and use the diagnostic accuracy, recall rate and F1 score indicators to evaluate the model performance; S7. Deploy the optimized reinforcement learning model in the actual medical diagnosis process to provide doctors with auxiliary decision support for early diagnosis of pancreatic cancer; The S1 specifically includes: Collect multimodal medical image data of N pancreatic cancer patients and construct a multimodal medical image dataset D P,i : D P,i ={D PCT,i ,D PMRI,i ,D PUS,i ,D PPAI,i ,D PMFC,i ,D PGE,i ,D PPM,i },i=1,2,...,N; Collect multimodal medical image data of M healthy individuals and construct a multimodal medical image dataset D H,j : D H,j ={D HCT,j ,D HMRI,j ,D HUS,j ,D HPAI,j ,D HMFC,j ,D HGE,j ,D HPM,j },j=1,2,...,M; Among them, D PCT,i and D HCT,j represents the computed tomography image dataset of the i-th pancreatic cancer patient and the j-th healthy individual, respectively. PMRI,i and D HMRI,j represents the MRI image dataset of the i-th pancreatic cancer patient and the j-th healthy individual, respectively. PUS,i and D HUS,j represents the ultrasound image dataset of the i-th pancreatic cancer patient and the j-th healthy individual, respectively. PPAI,i and D HPAI,j Represent the photoacoustic imaging image datasets of the i-th pancreatic cancer patient and the j-th healthy individual, and represents the microfluidic chip image dataset of the i-th pancreatic cancer patient and the j-th healthy individual, and Represent the gene expression datasets extracted from blood samples of the i-th pancreatic cancer patient and the j-th healthy individual, and Represent the protein marker datasets of the i-th pancreatic cancer patient and the j-th healthy individual respectively; The S3 specifically includes: S31. Configure multiple deep neural network models M DNN , each model is specialized for D P,i and D H,j The feature extraction is performed on one modality data in each dataset, and each dataset includes multiple modalities such as CT, MRI, ultrasound, photoacoustic imaging, microfluidic chips, gene expression and protein markers; S32. For each modality of medical image, including and Apply the corresponding deep neural network model for feature extraction to obtain the feature vector. This step is repeated for all modalities, including CT, MRI, ultrasound, photoacoustic imaging and microfluidic chips, gene expression and protein markers, to generate the feature vector and Among them, Modality∈{CT,MRI,US,PAI,MFC,GE,PM},(i=1,2,...,N),(j=1,2,...,M),σis the activation function,w k and b k is a model parameter, k represents the level or feature dimension; S33, for each pancreatic cancer patient i and each healthy individual j, the feature vectors of all their modalities and Integrate to form a comprehensive feature vector F P,i and F H,j ,The feature integration method uses feature fusion technology to capture complementary information across modalities: in, represents the complex fusion operation of different modal features, φ is the nonlinear transformation function, and Θ represents the parameters in the transformation; S34. Establish a deep neural network model to process each comprehensive feature vector F P,i and F H,j , extract high-level abstract features and obtain the final feature representation F High-P,i and F High-H,j : in, and represents the features after the lth layer, represents the complex combination or fusion mechanism between layers, θ DNN are the parameters of the deep neural network; S35, using F High-P,i and F High-H,j Train a classifier C to distinguish pancreatic cancer images from normal images, and assign a prediction probability to each sample through a classification model based on the softmax function, indicating the possibility that it belongs to the pancreatic cancer category: Among them, P(L P,i =1|F High-P,i ) is a given high-level eigenvector F High-P,i The probability that sample i is classified as pancreatic cancer is P(L H,j =0|F High-H,j ) is a given high-level eigenvector F High-H,j When , the probability that sample j is classified as normal, W cancer and W normal are model parameters corresponding to the weights of pancreatic cancer and normal categories, respectively, and b cancer and b normal are bias terms, corresponding to pancreatic cancer and normal classes, respectively; The classification decision is made by choosing the class with the highest probability: If P(L P,i =1|F High-P,i )>P(L H,j =0|F High-H,j ), then sample i is classified as pancreatic cancer; On the contrary, if P(L H,j =0|F High-H,j )>P(L P,i =1|F High-P,i ), then sample j is classified as normal; The S4 specifically includes: S41. Initialize the reinforcement learning model M RL , accept the deep neural network model M DNN The extracted and integrated high-level feature vector F High-P,i and F High-H,j As state input; S42, define action set A = {a1, a2, ..., a m }, where each action a k represents a possible manipulation of the diagnostic decision process, including increasing the weight of a particular modality or adjusting the classification threshold; S43. Set a reward function R(s,a) to evaluate the effect of the new state reached after performing action a in state s, where s is the current state, represented as a high-level feature vector, and a is the action performed: R(s,a)=α·Accuracy(s,a)-β·Time(s,a); Among them, Accuracy(s,a) represents the improvement of diagnostic accuracy after executing action a, Time(s,a) represents the time required to execute the action, and α and β are weight coefficients to balance accuracy and time efficiency; S44. Use Q-learning to train the reinforcement learning model M RL , continuously update the strategy π(a|s) through an iterative process to maximize the expected reward: π * (a|s)=argmax a ∑ s' P(s′|s,a)(R(s,a)+γV * (s′)); Among them, π * (a|s) is the optimal strategy for selecting action a in state s, P(s'|s,a) is the probability of transitioning from state s to state s' by executing action a, and V * (s') is the maximum expected reward that can be obtained in state s', γ is a discount factor used to balance immediate rewards and future rewards; The S5 specifically includes: S51. In the reinforcement learning model M RL The comprehensive objective function O is set in the paper to maximize the diagnostic accuracy and minimize the diagnostic time: Among them, y i is the true label of the i-th sample, is the label predicted by the model, II(·) is the indicator function, which takes the value of 1 when the prediction is correct and 0 otherwise, t i is the time required to process the i-th sample, and N is the total number of samples; S52. Use Q-learning to reinforce the learning model M RL Perform training to learn how to select action a to optimize the objective function O: Among them, s' is the new state reached after executing action a, α is the learning rate, and γ is the discount factor used to adjust the weights of immediate rewards and future rewards; S53, through the trial and error learning mechanism, strengthen the learning model M RL In each iteration, based on the current state s and the set of possible actions A, the expected reward of each action is predicted, and the action that maximizes the expected reward of the objective function O is selected for execution. The action selection adopts a greedy strategy: Combined with the influence of objective function O, the adjustment of O on Q value update is introduced; S54. After each round of learning, the reinforcement learning model M RL According to the actual results of the action, including the change in diagnostic accuracy and the time required, adjust the parameters of the action selection strategy π(a|s) and the reward function R(s,a): R(s,a)←(1-η)R(s,a)+η[λ·ΔAccuracy(s,a)-(1-λ)·ΔTime(s,a)]; Among them, η and ξ are adjustment coefficients used to control the adjustment speed of the reward function and strategy during the learning process. ΔAccuracy(s,a) and ΔTime(s,a) represent the changes in diagnostic accuracy and time after executing action a, respectively. S55, repeat steps S53 and S54 until the reinforcement learning model M RL The comprehensive objective function O on the validation set no longer improves.

2. The method for early diagnosis of pancreatic cancer based on deep reinforcement learning according to claim 1, characterized in that: The S6 specifically includes S61. Use an independent validation set: V={(x v1 ,and v1 ),(x v2 ,and v2 ),...,(x vN ,and vN )}; Among them, x vi is the feature vector of the i-th sample in the validation set, y vi is the corresponding true label, N is the number of samples in the validation set; S62. For each sample in the validation set, use the trained deep reinforcement learning model M RL Perform diagnostic prediction and obtain prediction labels S63. Calculate the diagnostic accuracy, recall rate and F1 score of the model on the validation set: Among them, (TP) (TruePositives) is the number of true positives, and FN (FalseNegatives) is the number of false negatives; S64, based on the indicators in step S63, comprehensively evaluate the model M RL performance in early diagnosis of pancreatic cancer; S65. According to the performance evaluation results, make necessary adjustments and optimizations to the model, readjust the parameters in the reinforcement learning model, optimize the feature extraction strategy, or adjust the reward function.

Images