Radiology department image quality optimization method and system based on deep reinforcement learning

Abstract

The invention relates to the technical field of medical images, in particular to a radiology department image quality optimization method and system based on deep reinforcement learning. The method comprises the steps of obtaining multi-modal medical image data of a patient and basic clinical information of the patient; based on the multi-modal medical image data and the basic clinical information of the patient, optimizing the image quality of a radiographic image by using a deep reinforcement learning algorithm; according to an optimization result, selecting an optimal parameter combination to carry out image examination on the patient; and outputting the optimized radiographic image, and by integrating the multi-modal medical image data and the basic clinical information of the patient, the individual difference of the patient can be comprehensively considered, and real personalized optimization is realized. Therefore, not only is the image quality improved, but also more comprehensive and accurate diagnosis information can be provided for doctors; and the adopted deep reinforcement learning algorithm can continuously adjust the optimization strategy according to real-time feedback. Due to the dynamic adaptive capacity, the system can quickly respond to the change of the state of the patient, and the optimal image quality is always kept.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a method and system for optimizing radiological image quality based on deep reinforcement learning. Background Art

[0002] With the rapid development of medical imaging technology, radiological images play an increasingly important role in disease diagnosis and treatment. However, the optimization of image quality has always been a major challenge in this field. Traditional radiological image quality optimization methods mainly rely on manual experience and fixed parameter settings, which have many limitations.

[0003] First, traditional methods are difficult to adapt to the individual differences of different patients. Each patient has a different body structure and condition, and fixed parameter settings are difficult to meet the needs of personalized diagnosis. Secondly, traditional methods cannot be dynamically adjusted based on real-time feedback. During the image acquisition process, the patient's condition may change, and fixed parameters are difficult to respond to these changes in a timely manner. Furthermore, traditional methods find it difficult to balance the relationship between image quality and radiation dose. Although excessive radiation dose can provide clearer images, it also increases the health risks of patients.

[0004] In recent years, machine learning technology has been widely used in the field of medical imaging, and some researchers have tried to use deep learning methods to optimize image quality. However, these methods still have some problems. For example, most deep learning methods require a large amount of labeled data for training, and in the field of medical imaging, high-quality labeled data is often difficult to obtain. In addition, these methods are usually trained offline and are difficult to adapt to real-time changing clinical environments.

[0005] Recently, some researchers have begun to try to apply reinforcement learning to medical image quality optimization. Reinforcement learning has the characteristics of online learning and dynamic adaptation, which can theoretically overcome the above problems. However, existing reinforcement learning-based methods still have some limitations. First, they usually only consider single-modality imaging data and ignore the potential of multimodal data fusion. Second, these methods often do not make full use of historical data and individual patient information, resulting in less than ideal optimization results. Finally, existing methods often face problems such as slow convergence and unstable performance when dealing with high-dimensional action spaces. Summary of the invention

[0006] The present invention aims to solve the above technical problems and proposes a method and system for optimizing radiology image quality based on deep reinforcement learning. The method can comprehensively utilize multimodal medical imaging data and basic clinical information of patients, and realize real-time optimization and personalized adjustment of image quality through deep reinforcement learning algorithms.

[0007] The present invention proposes a method for optimizing radiology image quality based on deep reinforcement learning, including:

[0008] The acquisition steps include:

[0009] Obtain the patient's multimodal medical imaging data and basic clinical information;

[0010] Processing steps include:

[0011] Based on the multimodal medical imaging data and the patient's basic clinical information, using a deep reinforcement learning algorithm to optimize the image quality of radiological images;

[0012] According to the optimization results, the optimal parameter combination is selected to perform imaging examination on the patient;

[0013] Output steps include:

[0014] Output optimized radiographic images.

[0015] Preferably, the processing steps specifically include:

[0016] Extract image features using medical image quality optimization network;

[0017] Use the fusion module to fuse different features;

[0018] Optimizing radiological image quality using deep reinforcement learning algorithms.

[0019] As a preference, it also includes:

[0020] Construct a comparative learning module to find case information of similar patients from the historical database.

[0021] As a preference, it also includes:

[0022] Use transfer learning to achieve personalized optimization. By modeling historical data, the similarities between different patients are evaluated to guide the adjustment of parameters in the network.

[0023] Preferably, the deep reinforcement learning algorithm adopts a deep Q network method, including:

[0024] Use the Q network to evaluate the value of each parameter adjustment strategy;

[0025] Update the network by minimizing the loss function.

[0026] As a preference, it also includes:

[0027] A similarity judgment model is constructed by using a comparative learning method of similar historical images;

[0028] According to the clinical information of the case to be tested, the trained similarity judgment model is used to retrieve the case data with the highest similarity in the historical case database.

[0029] As a preference, it also includes:

[0030] Establish patient analysis models, including building patient status representation models;

[0031] Based on the patient's clinical information, a model to characterize the patient's clinical status is established.

[0032] Preferably, the multimodal medical image data in the acquiring step includes:

[0033] DICOM images of the patient;

[0034] The DICOM image is converted into a CT image.

[0035] Preferably, the processing step further comprises:

[0036] Normalize the optimization parameters;

[0037] The normalized parameters are input into the transfer learning neural network.

[0038] A radiology image quality optimization system based on deep reinforcement learning for executing the method comprises:

[0039] Database module for collecting, managing and preprocessing medical imaging data;

[0040] A reinforcement learning module for optimizing patient image quality using a medical image quality optimization network and automatically adjusting device control parameters;

[0041] The contrastive learning module is used to integrate the medical image quality optimization network into the contrastive learning network, and to assist in generating strategies through similarity comparison of historical data and parameter recommendation;

[0042] The parameter optimization module is used to recommend system-generated strategies based on the personalized data characteristics of historical data, fusion results, and patient analysis models, accelerate strategy generation, and improve system efficiency.

[0043] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0044] First, by integrating multimodal medical imaging data and basic clinical information of patients, this method can fully consider the individual differences of patients and achieve true personalized optimization. This not only improves the image quality, but also provides doctors with more comprehensive and accurate diagnostic information.

[0045] Secondly, the deep reinforcement learning algorithm used in the present invention can continuously adjust the optimization strategy based on real-time feedback. This dynamic adaptability allows the system to quickly respond to changes in the patient's condition and always maintain the best image quality. At the same time, through continuous learning and accumulation of experience, the performance of the system will continue to improve as the usage time increases.

[0046] Furthermore, the method of the present invention not only optimizes the image quality, but also takes into account the control of radiation dose. By accurately adjusting the parameters, the system can minimize unnecessary radiation exposure while ensuring the quality of diagnosis, thereby reducing the health risks of patients.

[0047] In addition, the present invention introduces a comparative learning module, which can find similar cases from historical data to provide reference for current optimization. This method not only improves the optimization efficiency, but also provides doctors with diagnostic experience of similar cases to assist in medical decision-making.

[0048] Finally, the method of the present invention has good scalability and versatility. Although initially developed for radiology imaging, its core algorithm and system architecture can be easily extended to other medical imaging fields, such as ultrasound, endoscopy, etc. This versatility makes the present invention have broad application prospects.

[0049] In summary, the method proposed in this paper effectively solves many problems faced by traditional methods and existing machine learning methods by innovatively combining deep reinforcement learning, multimodal data fusion, contrastive learning and other technologies. It can not only significantly improve the quality of radiology images, but also improve diagnostic efficiency, reduce patient risks, and provide strong support for the development of precision medicine. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The figure is a flow chart of the overall method of the present invention.

[0051] Figure 2 It is a detailed logic block diagram of the comparative learning module of the present invention.

[0052] Figure 3 This is a logical block diagram of the transfer learning and patient analysis model of the present invention.

[0053] Figure 4 It is a logic block diagram of the system of the present invention. DETAILED DESCRIPTION

[0054] Please refer to Figure 1-4 The present invention provides a method and system for optimizing radiology image quality based on deep reinforcement learning. The method can effectively improve the quality of radiology images and provide clearer and more accurate image data for doctors' diagnosis. The present invention will be described in detail below in conjunction with specific implementation methods.

[0055] First, the method of the present invention includes an acquisition step, a processing step and an output step. In the acquisition step, the system acquires the multimodal medical image data of the patient and the basic clinical information of the patient. These data are the basis for subsequent processing. The multimodal medical image data may include different types of images such as CT, MRI, and X-ray, while the basic clinical information of the patient may include age, gender, medical history, etc.

[0056] In the processing step, the method uses a deep reinforcement learning algorithm to optimize the quality of radiological images. This is the core step of the present invention. The deep reinforcement learning algorithm can gradually find the optimal image processing parameters through continuous trial and learning. Preferably, the present invention uses a deep Q network (DQN) as a specific implementation of reinforcement learning. The mathematical expression of DQN is as follows:

[0057] Q(s,a)=E[R t+1 +γmax a′ Q(s t+1 ,a′)|s t =s,a t =a]

[0058] Among them, Q(s, a) is the value function of taking action a in state s, R t +1 is the immediate reward, γ is the discount factor, s t +1 is the next state, and a′ is the possible next action.

[0059] In one embodiment of the present invention, the state s may be the current image quality parameters, such as contrast, brightness, etc.; the action a may be the adjustment of these parameters; and the reward R may be the degree of improvement in the image quality after the adjustment. Through continuous iteration, the system can learn the optimal parameter adjustment strategy.

[0060] Based on the optimization results, the method selects the optimal parameter combination to perform imaging examination on the patient. These parameters may include X-ray dose, scanning speed, image reconstruction algorithm, etc. For example, for an adult male patient weighing 80 kg, the system may select a tube voltage of 120 kV and a tube current of 200 mA as the optimal parameter combination.

[0061] In the output step, the system outputs optimized radiological images, which may have significant improvements in clarity, contrast, signal-to-noise ratio, etc. compared to the original images.

[0062] Furthermore, the processing step of the present invention also includes extracting image features using a medical image quality optimization network. This network can be a deep convolutional neural network, such as ResNet or DenseNet. The mathematical expression of the network can be summarized as:

[0063] f(x)=σ(W*x+b)

[0064] Among them, f(x) is the network output, x is the input image, W is the convolution kernel weight, b is the bias term, σ is the activation function, and * represents the convolution operation.

[0065] The present invention also uses a fusion module to fuse different features. Fusion can be achieved by simple concatenation or by more complex attention mechanisms. For example, the mathematical expression using the self-attention mechanism is as follows:

[0066]

[0067] Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the dimension of the key.

[0068] In addition, the present invention also constructs a contrastive learning module for searching case information of similar patients from the historical database. This module can help the system better understand the current patient's situation and make more accurate optimization decisions. The loss function of contrastive learning can be expressed as:

[0069]

[0070] Among them, z i and z j is the feature representation of the positive sample pair, sin is the similarity function, τ is the temperature parameter, and N is the batch size.

[0071] Through the above steps, the method of the present invention can effectively improve the quality of radiology images, reduce radiation dose, and improve the accuracy of diagnosis. This method based on deep reinforcement learning has the characteristics of strong adaptability and sustainable optimization, and can continuously improve performance as data accumulates. The method of the present invention also uses transfer learning technology to achieve personalized optimization. By modeling historical data, the system can evaluate the similarity between different patients, thereby guiding the adjustment of parameters in the network. This method can make full use of existing knowledge, speed up learning, and improve optimization effects.

[0072] In a preferred embodiment of the present invention, transfer learning can be implemented using a fine-tuning strategy. Specifically, the system first pre-trains a deep neural network model on a large-scale general radiology image dataset. Then, for the data of a specific patient, the system only fine-tunes the parameters of the last few layers of the network. This method can be expressed mathematically as follows:

[0073]

[0074] Among them, θ ncware the fine-tuned model parameters, θ prc are the parameters of the pre-trained model, α is the learning rate, is the loss function, is the data of the target domain (i.e., a specific patient).

[0075] In the method of the present invention, the deep reinforcement learning algorithm preferably adopts the deep Q network (DQN) method. The core idea of ​​the DQN method is to use a neural network to approximate the Q function, thereby solving the limitations of traditional Q-learning when facing a high-dimensional state space. In the specific implementation of the present invention, the Q network is used to evaluate the value of each parameter adjustment strategy.

[0076] For example, suppose the system needs to adjust three parameters: X-ray dose, scanning speed, and reconstruction algorithm. Each parameter may have multiple optional values, forming a high-dimensional action space. The input of the Q network is the current image state (which may be a vector of a series of image quality indicators), and the output is the Q value of each possible action. The system will select the action with the highest Q value to execute.

[0077] The update of the Q network is achieved by minimizing the loss function. The loss function is usually defined as:

[0078]

[0079] Where r is the immediate reward, γ is the discount factor, s′ is the next state, θ and θ -- are the parameters of the current network and the target network respectively.

[0080] Preferably, the present invention uses a double Q network (Double DQN) to reduce the over-estimation problem of Q value. In this method, the selection and evaluation of actions are completed by two different networks:

[0081]

[0082]

[0083] Among them, a * is the action chosen by the current network, but its Q-value is calculated by the target network.

[0084] The present invention also adopts a comparative learning method of similar historical images to construct a similarity judgment model. This method can find cases similar to the current patient from historical data, providing more reference information for optimization. In one embodiment of the present invention, the similarity judgment model adopts a Siamese Network structure. The loss function of the network can be defined as Contrastive Loss:

[0085]

[0086] Among them, x1 and x2 are two input samples, y is the label (1 means similarity, 0 means dissimilarity), d is the Euclidean distance between the two samples in the feature space, and margin is a preset threshold.

[0087] Based on the clinical information of the case to be tested, the system uses the trained similarity judgment model to retrieve the case data with the highest similarity in the historical case database. This process can be implemented through the nearest neighbor search algorithm, such as the k-nearestneighbors (k-NN) algorithm. In practice, in order to improve the search efficiency, an approximate nearest neighbor search algorithm can be used, such as Locality-Sensitive Hashing (LSH).

[0088] Through the above method, the present invention can make full use of historical data to provide a more personalized and accurate image quality optimization solution for current patients. This can not only improve image quality, but also reduce unnecessary radiation exposure, providing patients with safer and more efficient diagnostic services. The method of the present invention also includes establishing a patient analysis model, an important part of which is to construct a patient status characterization model. This model can establish a patient's clinical status characterization based on the patient's clinical information, providing an important basis for subsequent image quality optimization.

[0089] In a preferred embodiment of the present invention, the patient status representation model adopts a multimodal deep learning architecture. The model can simultaneously process structured clinical data (such as age, gender, medical history, etc.) and unstructured text data (such as doctor's diagnosis report). For structured data, the model uses a fully connected neural network for processing; for text data, a long short-term memory network (LSTM) or transformer architecture is used for processing. Finally, the model fuses the features of these different modalities to obtain a comprehensive patient status representation.

[0090] Mathematically, this process can be expressed as:

[0091] h struct =f Fc (x struct )

[0092] h text =f LSIM (x text )

[0093] h patient =f fusion ([h struct ,h text ])

[0094] Among them, fFC 、f LSTM and f fusion Represent the fully connected network, LSTM network and feature fusion function respectively, x struct and x text are the input of structured data and text data respectively, h patient It is the final representation of the patient's status.

[0095] In the acquisition step of the present invention, the multimodal medical imaging data includes DICOM images of the patient. DICOM (Digital Imaging and Communications in Medicine) is an international standard format in the field of medical imaging, which contains rich metadata information. In this method, the DICOM images need to be converted into CT images first for subsequent processing.

[0096] This conversion process usually includes the following steps:

[0097] 1. Read the metadata of the DICOM file to obtain information such as the storage method and bit depth of the pixel data.

[0098] 2. Extract the pixel data, usually in 16-bit integer format.

[0099] 3. Apply the window width and window level adjustment to map the CT value (Hounsfield Units, HU) to a grayscale range suitable for display.

[0100] 4. If necessary, perform image reconstruction, such as reconstructing coronal or sagittal images from axial slices.

[0101] Preferably, the present invention uses Python's pydicom library to process DICOM files, uses the numpy library for numerical calculations, and uses OpenCV or PIL library for image processing and storage.

[0102] In the processing step, the method also includes normalizing the optimization parameters. Normalization is a commonly used data preprocessing technology in machine learning, which can convert features of different scales to the same scale, helping to improve the training effect and convergence speed of the model.

[0103] In the embodiment of the present invention, the Z-score standardization method is used for normalization processing. Its mathematical expression is:

[0104]

[0105] Among them, x is the original value, μ is the mean of the feature, σ is the standard deviation of the feature, and x normis the normalized value.

[0106] The normalized parameters are then fed into a transfer learning neural network. The structure of this network may vary depending on the specific task, but it usually includes a pre-trained base network (such as ResNet or DenseNet) and some additional layers for specific tasks.

[0107] Finally, the present invention also provides a radiology image quality optimization system based on deep reinforcement learning. The system includes a database module, a reinforcement learning module, a comparative learning module and a parameter optimization module.

[0108] The database module is responsible for collecting, managing and preprocessing medical imaging data. In practical applications, this module may be a distributed storage system that can efficiently store and retrieve large amounts of medical imaging data.

[0109] The reinforcement learning module uses the medical image quality optimization network to optimize the patient image quality and automatically adjust the control parameters of the device. This module implements the deep Q-network algorithm described above.

[0110] The contrastive learning module integrates the medical image quality optimization network into the contrastive learning network, and performs auxiliary generation strategies through similarity comparison and parameter recommendation of historical data. This module implements the similarity judgment model described above.

[0111] The parameter optimization module recommends the strategies generated by the system based on the personalized data features of historical data, the fusion results and the patient analysis model, accelerates strategy generation and improves system efficiency. This module integrates the various technologies described above, such as transfer learning and patient status representation.

[0112] Through the collaborative work of these modules, the system of the present invention can provide personalized, high-quality radiology imaging services for different patients, greatly improving the accuracy and efficiency of diagnosis.

[0113] In order to verify the superiority of the present invention, a series of comparative tests were conducted. This article will introduce in detail the specific implementation methods of the embodiments and comparative examples, as well as the corresponding test results and analysis.

[0114] Example 1 adopts the complete method of the present invention, including multimodal data fusion, deep reinforcement learning optimization, contrastive learning module and patient analysis model. Comparative Example 1 adopts the traditional fixed parameter setting method, and experienced radiologists manually adjust the scanning parameters according to the basic information of the patient.

[0115] 100 patients of different ages, genders and body types were selected for chest CT scans. In order to comprehensively evaluate the performance of the present invention, the following key indicators were selected for testing:

[0116] 1. Image quality rating: Three senior radiologists rated the image quality using a 5-point scale, mainly considering aspects such as image clarity, contrast, and detail presentation.

[0117] 2. Radiation dose: Dose-area product (DAP) is used as the measurement standard, with the unit of mGy cm 2 .

[0118] 3. Scan time: The total time from the patient's positioning to the end of the scan, in minutes.

[0119] 4. Diagnostic accuracy: Three other radiologists made diagnoses based on the scan results, and compared them with the final clinical diagnosis results to calculate the accuracy.

[0120] 5. Number of parameter adjustments: The number of times the system automatically adjusts the parameters during a complete scan (applicable only to the method of the present invention).

[0121] The following is a detailed comparison table of the test results:

[0122] index Example 1 Comparative Example 1 Image quality rating (1-5 points) 4.7±0.3 4.1±0.5 <![CDATA[Radiation dose (mGy·cm 2 )]]> 245±30 320±45 Scan time (minutes) 8.5±1.2 12.3±2.5 Diagnostic accuracy (%) 97.5％ 92.0％ Number of parameter adjustments 4.2±1.1 N / A

[0123] It can be seen from the test results that the method of the present invention shows obvious advantages in all aspects:

[0124] 1. Image quality: The average score of the method of the present invention reached 4.7 points, which is significantly higher than the 4.1 points of the traditional method. This shows that the present invention can provide clearer and more diagnostically valuable images.

[0125] 2. Radiation dose: The average radiation dose of the method of the present invention is reduced by about 23.4% compared with the traditional method, which is of great significance for reducing the radiation exposure risk of patients.

[0126] 3. Scanning time: The method of the present invention shortens the average scanning time by about 30.9%, which not only improves the efficiency of equipment use, but also improves the patient experience.

[0127] 4. Diagnostic accuracy: The diagnostic accuracy of the method of the present invention was improved by 5.5 percentage points, and this improvement has a significant impact on the accuracy of clinical diagnosis.

[0128] 5. Number of parameter adjustments: The average of 4.2 times of automatic parameter adjustment indicates that the method of the present invention can dynamically optimize scanning parameters according to real-time data, which is impossible to achieve with traditional methods.

[0129] These results fully demonstrate the superiority of the method of the present invention. It is particularly noteworthy that the present invention significantly reduces the radiation dose while improving image quality. This effect of having the best of both worlds is precisely because the deep reinforcement learning algorithm can find the optimal parameter combination. In addition, the improvement in diagnostic accuracy shows that the present invention not only improves in technical indicators, but also shows practical value in clinical applications.

[0130] The unique indicator of the number of parameter adjustments reflects the adaptive ability of the method of the present invention. An average of 4.2 adjustments means that the system can continuously optimize parameter settings based on individual differences of patients and real-time feedback during the scanning process. This dynamic adjustment capability is not available in traditional methods, and is also the key to the present invention's ability to comprehensively surpass traditional methods in various indicators.

[0131] In summary, the radiology image quality optimization method based on deep reinforcement learning and the system thereof of the present invention have shown significant advantages in improving image quality, reducing radiation dose, shortening scanning time and improving diagnostic accuracy. These improvements can not only improve the efficiency and accuracy of medical diagnosis, but also reduce the health risks of patients, and have important clinical application value.

[0132] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A radiology image quality optimization method based on deep reinforcement learning, characterized in that: include: The acquisition steps include: Obtain the patient's multimodal medical imaging data and basic clinical information; Processing steps include: Based on the multimodal medical imaging data and the patient's basic clinical information, using a deep reinforcement learning algorithm to optimize the image quality of radiological images; According to the optimization results, the optimal parameter combination is selected to perform imaging examination on the patient; Output steps include: Output optimized radiographic images.

2. The method according to claim 1, characterized in that The processing steps specifically include: Extract image features using medical image quality optimization network; Use the fusion module to fuse different features; Optimizing radiological image quality using deep reinforcement learning algorithms.

3. The method according to claim 1, characterized in that Also includes: Construct a comparative learning module to find case information of similar patients from the historical database.

4. The method according to claim 1, characterized in that: Also includes: Use transfer learning to achieve personalized optimization. By modeling historical data, the similarities between different patients are evaluated to guide the adjustment of parameters in the network.

5. The method according to claim 1, characterized in that The deep reinforcement learning algorithm adopts a deep Q network method, including: Use the Q network to evaluate the value of each parameter adjustment strategy; Update the network by minimizing the loss function.

6. The method according to claim 1, characterized in that Also includes: A similarity judgment model is constructed by using a comparative learning method of similar historical images; According to the clinical information of the case to be tested, the trained similarity judgment model is used to retrieve the case data with the highest similarity in the historical case database.

7. The method according to claim 1, characterized in that Also includes: Establish patient analysis models, including building patient status representation models; Based on the patient's clinical information, a model to characterize the patient's clinical status is established.

8. The method according to claim 1, characterized in that The multimodal medical image data in the acquisition step includes: DICOM images of the patient; The DICOM image is converted into a CT image.

9. The method according to claim 1, characterized in that: The processing steps also include: Normalize the optimization parameters; The normalized parameters are input into the transfer learning neural network.

10. A radiology image quality optimization system based on deep reinforcement learning that implements the method according to any one of claims 1 to 9, characterized in that: include: Database module for collecting, managing and preprocessing medical imaging data; A reinforcement learning module for optimizing patient image quality using a medical image quality optimization network and automatically adjusting device control parameters; The contrastive learning module is used to integrate the medical image quality optimization network into the contrastive learning network, and to assist in generating strategies through similarity comparison of historical data and parameter recommendation; The parameter optimization module is used to recommend system-generated strategies based on the personalized data characteristics of historical data, fusion results, and patient analysis models, accelerate strategy generation, and improve system efficiency.

Images