Medical image intelligent diagnosis method and system based on deep learning

Abstract

The invention discloses a medical image intelligent diagnosis method and system based on deep learning, and the method comprises the steps: obtaining multi-modal medical image data through a medical image collection device, carrying out the data preprocessing, and forming a standardized image data set; constructing a deep learning diagnosis model based on multi-scale feature fusion, inputting the standardized image data set into the deep learning diagnosis model for transfer learning training, and optimizing model parameters by adopting a dynamic weight adjustment strategy; verifying the trained deep learning diagnosis model through an integrated learning framework, generating a diagnosis confidence score, and performing probability calibration on a diagnosis result in combination with a Bayesian optimization algorithm to obtain an optimized diagnosis model; and inputting medical image data to be diagnosed, and outputting a pathological classification result. The problems that in the prior art, multi-modal medical image data processing is insufficient, model optimization strategies are insufficient, and diagnosis confidence coefficient calibration methods are insufficient are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of cross-integration of artificial intelligence and medical imaging, and in particular to a medical imaging intelligent diagnosis method and system based on deep learning. Background Art

[0002] Traditional medical imaging diagnostic methods mainly rely on the experience of clinicians for image recognition and diagnosis, which is limited by strong subjectivity, high misdiagnosis rate and relatively low diagnostic efficiency. In recent years, deep learning technology represented by convolutional neural networks (CNN) has been widely used in automatic analysis of medical images. Through effective feature extraction and pattern recognition capabilities, it has improved the accuracy of lesion identification and diagnostic efficiency, and partially replaced traditional manual diagnosis. However, existing technologies are mostly focused on the processing of single-modality images, lack of deep fusion analysis of multi-modal medical imaging data, and existing deep learning diagnostic models mostly have poor model generalization ability, difficulty in parameter optimization and insufficient calibration of diagnostic result confidence, which seriously restricts the reliability of clinical applications.

[0003] At present, traditional deep learning medical imaging diagnosis methods usually ignore the multimodal characteristics of imaging data and fail to fully utilize the complementary information between different imaging modalities, thus limiting the improvement of diagnostic performance. In addition, in the training and optimization process of diagnostic models, there is a lack of effective transfer learning mechanisms and dynamic weight adjustment strategies, resulting in poor model generalization performance and difficulty in coping with the diversity and complexity of clinical images. At the same time, the verification of existing models and the calibration of diagnostic results are relatively weak, failing to fully integrate the diagnostic advantages of multiple models and failing to fully utilize Bayesian optimization probability calibration technology, thereby reducing the credibility of diagnostic results and the reliability of clinical decision support.

[0004] Therefore, how to build an efficient and reliable deep learning medical imaging diagnosis method and improve the generalization ability and diagnostic accuracy of the model has become an important issue that needs to be urgently addressed in the current field of medical imaging artificial intelligence.

[0005] In summary, the existing medical imaging diagnosis technology has the problems of limited diagnostic accuracy, insufficient generalization performance and low diagnostic credibility. The medical imaging intelligent diagnosis method based on deep learning proposed in the present invention aims to solve the problems of insufficient processing of multimodal medical imaging data, insufficient model optimization strategy and lack of diagnostic confidence calibration method in the existing technology. Summary of the invention

[0006] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract of the specification and the title of the invention of this application to avoid blurring the purpose of this section, the abstract of the specification and the title of the invention, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0007] In view of the existing problems described above, the present invention is proposed.

[0008] To solve the above technical problems, the present invention provides the following technical solutions: acquiring multi-modal medical image data through a medical image acquisition device, and performing data preprocessing to form a standardized image data set;

[0009] Constructing a deep learning diagnosis model based on multi-scale feature fusion, inputting the standardized image data set into the deep learning diagnosis model for transfer learning training, and optimizing the model parameters by adopting a dynamic weight adjustment strategy;

[0010] Verifying the trained deep learning diagnosis model through an ensemble learning framework, generating a diagnosis confidence score, and calibrating the probability of the diagnosis result by combining with the Bayesian optimization algorithm to obtain an optimized diagnosis model;

[0011] Inputting the medical image data to be diagnosed into the optimized diagnosis model for diagnosis analysis, and outputting a pathological classification result.

[0012] As a preferred solution of the intelligent medical image diagnosis method based on deep learning of the present invention, the multi-modal medical image data is image data related to the same lesion acquired by at least two different imaging methods such as X-ray imaging, CT imaging, MRI imaging, ultrasonic imaging, PET-CT imaging, digital imaging of pathological sections, i.e., infrared thermal imaging;

[0013] Performing quality inspection on the acquired multi-modal image data, and removing images with serious artifacts, defocusing, insufficient exposure or overexposure that affect the diagnosis result;

[0014] Storing the multi-modal image data in the DICOM image file format and assigning a unified lesion identifier.

[0015] As a preferred solution of the intelligent medical image diagnosis method based on deep learning of the present invention, performing data preprocessing on the multi-modal medical image data to form a standardized image data set, including:

[0016] For different-modal medical image data, adopting a registration algorithm based on similarity measurement to achieve spatial alignment to ensure accurate correspondence of each modal image in anatomical structure;

[0017] Removing random noise from the registered multi-modal medical image data, and improving the clarity of the lesion site through histogram equalization and contrast enhancement;

[0018] Performing unified normalization processing on the multi-modal medical image data in terms of gray level, resolution and pixel size to obtain standardized image data;

[0019] Perform flipping, rotation, cropping, noise perturbation, and color offset processing on the standardized image data to form a standardized image dataset.

[0020] As a preferred solution of the deep learning-based medical image intelligent diagnosis method of the present invention, constructing a deep learning diagnosis model based on multi-scale feature fusion includes:

[0021] Set convolutional kernels of different sizes at different levels of the convolutional neural network to obtain local features and global features within different field-of-view ranges;

[0022] Perform weighted fusion on the features output from each scale convolutional layer, and highlight key lesion areas through an attention mechanism to suppress redundant background information;

[0023] Pass the fused multi-scale features through several fully connected layers to form a high-level semantic representation of the target pathological region;

[0024] Configure a classification layer for determining the pathological type at the end of the convolutional neural network, and output the lesion category probability through the Softmax function.

[0025] As a preferred solution of the deep learning-based medical image intelligent diagnosis method of the present invention, input the standardized image dataset into the deep learning diagnosis model for transfer learning training, and adopt a dynamic weight adjustment strategy to optimize model parameters, including:

[0026] Use the network parameters pre-trained on a large-scale medical image dataset as the initial parameters of the base model;

[0027] For the current lesion type, perform fine-tuning training on all the parameters of the base model;

[0028] Define an objective function based on cross-entropy:

[0029]

[0030] where N is the number of training samples, yi is the true label of the i-th sample, is the predicted probability output by the model;

[0031] Assign a greater weight to samples with higher losses or prone to misjudgment, and the weight iteration update rule is expressed as:

[0032]

[0033] where, represents the weight value of the i-th sample after the t-th iteration, δ iIt represents the relative measure of the loss gradient of the i-th sample, and α is the balance factor;

[0034] The network parameters are continuously updated using the stochastic gradient descent optimization algorithm until the preset maximum number of training epochs is reached.

[0035] As a preferred solution of the medical image intelligent diagnosis method based on deep learning described in the present invention, the trained deep learning diagnosis model is verified through an ensemble learning framework to generate a diagnostic confidence score, including:

[0036] Using the Bagging method, multiple sub-models are constructed;

[0037] The prediction results of the multiple sub-models are weighted and averaged to obtain a fused comprehensive diagnostic result;

[0038] According to the fused comprehensive diagnostic result, the predicted probability distribution of each sample to be tested on each pathological category is calculated, and the confidence of the model in the prediction result is statistically analyzed to generate a diagnostic confidence score.

[0039] As a preferred solution of the medical image intelligent diagnosis method based on deep learning described in the present invention, the Bayesian optimization algorithm is combined to calibrate the probability of the diagnostic result to obtain an optimized diagnostic model, including:

[0040] For the key hyperparameters of the deep learning diagnosis model, corresponding search ranges are set;

[0041] A Gaussian process regression model is established to estimate the diagnostic performance distribution that can be achieved by different combinations of hyperparameters on the validation set;

[0042] The next set of optimal hyperparameter combinations is selected by maximizing the expected improvement value;

[0043] The finally determined hyperparameters are applied to the deep learning diagnosis model, and the posterior probability distribution is output for each pathological category to calibrate the diagnostic result of the model to obtain an optimized diagnostic model;

[0044] Among them, the key hyperparameters include the learning rate, regularization coefficient, number of convolutional kernels, and network depth.

[0045] As a preferred solution of the medical image intelligent diagnosis method based on deep learning described in the present invention, the medical image data to be diagnosed is input into the optimized diagnostic model for diagnostic analysis, and the pathological classification result is output, including:

[0046] The medical image data to be diagnosed is preprocessed in the same way as in the training stage, including registration, denoising, normalization, and size resampling;

[0047] Input the preprocessed medical images to be diagnosed into a diagnostic model calibrated by Bayesian optimization and ensemble learning, and output the predicted probabilities corresponding to the pathological categories;

[0048] Define the predicted probabilities of each category output by the diagnostic model as p1, p2, …, p k , where k is the total number of pathological categories;

[0049] If there exists a predicted probability p of a certain category c c ≥ threshold T high , then it is determined as this category;

[0050] If the predicted probabilities of all categories are less than the threshold T high but there is at least one category c that satisfies p c ≥ T high , then it is determined as suspected of this category and further examination is recommended;

[0051] If the predicted probabilities of all categories are lower than the threshold T high , then a tentative determination is made according to the category with the highest probability, but it needs to be reviewed in combination with the doctor's clinical experience.

[0052] As a preferred embodiment of the intelligent medical image diagnosis system based on deep learning according to the present invention, it includes: one or more processors;

[0053] A memory that stores instructions that can be operated, and when the instructions are executed by the one or more processors, the one or more processors are caused to perform operations, and the operations include the processes of the intelligent medical image diagnosis method based on deep learning as described above.

[0054] As a preferred embodiment of a computer-readable medium storing software according to the present invention, the software includes instructions that can be executed by one or more computers, and when the instructions are executed in this way, the one or more computers are caused to perform operations, and the operations include the processes of the intelligent medical image diagnosis method based on deep learning as described above.

[0055] Advantages of the present invention:

[0056] 1. By performing data standardization on medical image data, the spatial and intensity calibration and unification between different modality medical image data are realized, the diagnostic interference caused by modality differences is eliminated, a high-quality and unified data source is provided for the subsequent diagnostic model, the heterogeneity of input data devices and data consistency are reduced, and the stability is improved;

[0057] 2. Multi-scale feature extraction based on multi-modal medical images, and making full use of transfer learning to improve the transfer mechanism learning of the model for multi-modal medical images, enabling the model to obtain good generalization ability in a short time. A dynamic weight adjustment strategy is assigned to training samples of different difficulty levels, enhancing the model's attention to confusing or difficult cases and improving the generalization performance and overall accuracy of the diagnostic model.

[0058] 3. The fusion process utilizes the diagnostic advantages of multiple sub-models to reduce the risk of a single model, and further optimizes the diagnostic probability with the help of Bayesian optimization, enhancing the credibility of the model for diagnostic results, thereby improving the reliability of diagnostic decisions and reducing the credibility of misdiagnosis risks.

[0059] 4. Input the medical image data to be diagnosed into the optimized diagnostic model for diagnostic analysis, and efficiently output the pathological classification results, achieving the effective effects of assisting clinicians in classification, accurately completing pathological diagnosis and rapid pathological diagnosis, and improving medical diagnostic efficiency and medical service quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings. Among them:

[0061] Figure 1 It is a schematic flowchart of a medical image intelligent diagnosis method based on deep learning shown in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the specific embodiments of the present invention in detail with reference to the drawings of the specification. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention.

[0063] Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0064] In the following description, many specific details are set forth to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0065] According to the embodiments of the present invention, in combination with Figure 1The flowchart shown, an intelligent medical image diagnosis method based on deep learning, specifically includes the following steps:

[0066] S1. Obtain multi-modal medical image data through a medical image acquisition device and perform data preprocessing to form a standardized image dataset. Here, it should be noted in this step that:

[0067] Use at least two different imaging methods, such as X-ray imaging, CT imaging, MRI imaging, ultrasound imaging, PET-CT imaging, digital imaging of pathological sections (infrared thermal imaging), to image the same lesion and obtain the corresponding multi-modal medical image data;

[0068] Perform quality inspection on the acquired multi-modal image data, and eliminate the images that have serious artifacts, defocus, insufficient exposure or overexposure and affect the diagnosis results;

[0069] Uniformly store the multi-modal image data that has passed the quality inspection in DICOM format, and assign a unique lesion identifier to each lesion for subsequent analysis and call of the model;

[0070] To ensure the accurate correspondence of different modal images in the anatomical structure, use an image registration algorithm based on similarity measurement to perform spatial alignment on each modal image. Denote the registered j-th modal image as

[0071] For the multi-modal images that have completed spatial alignment Perform random noise removal, preferably using a filtering denoising method, to ensure reducing noise interference on the premise of retaining the lesion features to the greatest extent;

[0072] Further highlight the lesion area through histogram equalization and contrast enhancement, so as to improve the visibility and resolution of the lesion site in each modal image;

[0073] Perform unified normalization processing on the registered and denoised multi-modal images in terms of gray level, resolution and pixel size;

[0074] Let I ′ represent the j-th modal image after denoising and contrast enhancement, and let μ ′ and σ ′ respectively represent the average gray value and standard deviation of this image, then the following normalization processing formula can be constructed:

[0075]

[0076] Among them, is the normalized gray value at pixel (x, y) of the j-th modal image, and I ′(x, y) is the gray value at pixel (x, y) after denoising and enhancement, μ′ j is the average gray value of the j-th modality image, σ′ j is the standard deviation of the gray value of the j-th modality image;

[0077] Through the above normalization process, the consistency of different modality images in gray value distribution and spatial resolution is ensured, laying a unified data foundation for the input of subsequent deep learning models.

[0078] Furthermore, after completing the standardization of multi-modal images, to improve the robustness of the model to various deformations and noises, data augmentation operations are performed on the standardized image data: including horizontal flipping, angular rotation, random cropping, noise perturbation, and color shift;

[0079] All the images after the above processing are uniformly encapsulated into a standardized image dataset, which is represented as:

[0080]

[0081] where D std represents the multi-modal image dataset after standardization and data augmentation, n represents the total number of all images finally included in this dataset, represents the i-th image after normalization, registration, denoising, and augmentation processing;

[0082] Through this standardized image dataset, the unity and authenticity of data input during subsequent network training and model diagnosis are ensured.

[0083] S2. Construct a deep learning diagnosis model based on multi-scale feature fusion, input the standardized image dataset into the deep learning diagnosis model for transfer learning training, and adopt a dynamic weight adjustment strategy to optimize the model parameters. Among them, it should be noted in this step that:

[0084] Set convolution kernels of different sizes at different levels of the convolutional neural network to obtain local features and global features within different field-of-view ranges;

[0085] This convolutional neural network contains several convolutional layers, pooling layers, and fully connected layers. Different-sized convolutional kernels are set in each convolutional layer, which are used to capture local features and global features of the lesion area respectively;

[0086] The feature maps output by each convolutional layer are respectively denoted as F1, F2, …, F m , where m is the number of multi-scale convolutional layers;

[0087] F1, F2, …, F are fused through weighted fusion operation m on the feature dimension to obtain the comprehensive feature Ffusion ;

[0088] During the fusion process, an attention mechanism is added to assign higher weights to the features related to the lesion area and suppress background noise and irrelevant regions;

[0089] The fused feature F fusion is input into several fully connected layers for feature extraction to generate a high-level semantic representation of the target pathological area;

[0090] A classification layer is set at the end of the network, and the probability of the lesion category is output through the Softmax function. Denote the overall output of this network as:

[0091]

[0092] where z represents the output vector of the last fully connected layer, and W and b are the learnable parameters corresponding to the classification layer respectively;

[0093] Denote the overall above network as:

[0094]

[0095] where θ represents all the learnable parameters of each layer of this convolutional neural network, X is the input normalized image data, is the prediction output (lesion category probability distribution);

[0096] Before this model undergoes transfer learning training and parameter optimization, it is regarded as a deep learning diagnostic model that is constructed but not yet converged;

[0097] Denote the network parameters that have been pre-trained on a large-scale medical image dataset as θ pre , which are used as the initial parameters of this model;

[0098] For the target task required to identify the current lesion type, all the learnable parameters in θ pre are used as the basis for fine-tuning training;

[0099] During the training process, in order to measure the prediction error of the model for each training sample, the cross-entropy loss L is defined as the main optimization objective:

[0100]

[0101] where N is the number of training samples, y i is the true label of the i-th sample, is the probability value corresponding to the true category in the predicted probability distribution of the model for the i-th sample;

[0102] Greater weights are assigned to samples with higher losses or those that are prone to misjudgment to accelerate the model's learning of key samples;

[0103] Let the weight value of the \(i\)-th sample after the \(t\)-th iteration be Its update rule is expressed as:

[0104]

[0105] where represents the weight value of the \(i\)-th sample after the \(t\)-th iteration, and \(\delta\) i represents the relative measure of the loss gradient of the \(i\)-th sample, and \(\alpha\) is the balance factor;

[0106] The random gradient descent algorithm is used to continuously iterate and update the network parameter \(\theta\), and the iteration direction is determined by the gradient information of the current batch of samples;

[0107] When the preset maximum number of training epochs is reached, the training ends;

[0108] Let \(\theta\) diag be the optimal parameter set when the training converges, then the model is denoted as:

[0109]

[0110] That is, the diagnostic model obtained after transfer learning training and parameter optimization.

[0111] S3. Validate the trained deep learning diagnostic model through an ensemble learning framework, generate a diagnostic confidence score, and calibrate the probability of the diagnostic result by combining the Bayesian optimization algorithm to obtain an optimized diagnostic model. Among them, it should be noted in this step that:

[0112] After completing the training of the deep learning diagnostic model in step S2, construct multiple sub-models \(\{M_1, M_2, \ldots, M\) Q \} in the form of Bagging (bootstrap aggregation);

[0113] Each sub-model is independently trained on different sub-datasets randomly sampled from the original training set to ensure the difference between sub-models;

[0114] The predicted probability vectors of each sub-model on the validation sample \(x\) i are respectively denoted as where \(q = 1, 2, \ldots, Q\);

[0115] Exemplarily, in this embodiment, the adaptive weights \(\{w_1, w_2, \ldots, w\) Q \} are used. Multiply the predicted probability of each sub-model by its corresponding weight and then sum them to obtain the fused comprehensive diagnostic probability distribution

[0116] where

[0117] For the fused comprehensive diagnosis result In the set of pathological categories, record the predicted probability of each category as c represents a certain pathological category;

[0118] To quantify the diagnostic confidence of the model for sample x i Define the diagnostic confidence score C i It can take the following form:

[0119]

[0120] That is, take the maximum category prediction probability value in the probability distribution as the confidence;

[0121] Thus, C can be obtained for each sample to be tested i , that is, the diagnostic confidence score, to measure the reliability of the integrated model's diagnostic output for this sample;

[0122] Furthermore, while keeping the main parameters θ of the trained model Diag unchanged, set a reasonable search range for the key hyperparameters (such as learning rate, regularization coefficient, number of convolutional kernels, and network depth) that affect the diagnostic performance;

[0123] Denote the set of key hyperparameters as λ = (λ1, λ2,..., λ k ), and specify their respective value intervals;

[0124] By experimenting with different hyperparameter combinations λ on the validation set, regard the model diagnostic performance (such as accuracy, F1 score, ROC - AUC) obtained for each combination as the objective function f(λ);

[0125] Establish a Gaussian process regression model to estimate f(λ) and predict the diagnostic performance of hyperparameter combinations not yet tried on the validation set;

[0126] Use the expected improvement criterion in Bayesian optimization to select the next set of hyperparameter combinations from the current Gaussian process posterior distribution that can bring the maximum expected performance improvement;

[0127] Repeat this process until performance convergence is met and then stop;

[0128] Denote the optimal hyperparameter combination determined in the above process as λ * , and apply it to the deep - learning diagnostic model to re - fine - tune and calibrate the output;

[0129] Under the Bayesian optimization result, output the posterior probability distribution for each pathological category to make the model more stable in terms of confidence and have higher diagnostic reliability;

[0130] Exemplarily, the model obtained at this time is denoted as:

[0131]

[0132] where θ Diag represents the main body parameters of the model after training in step S2, and λ * is the optimal hyperparameter selected through Bayesian optimization. is the class prediction result output after probability calibration for the input image X, that is, the optimized diagnostic model.

[0133] S4. Input the medical image data to be diagnosed into the optimized diagnostic model for diagnostic analysis, and output the pathological classification result. Here, it should be noted in this step that:

[0134] Adopt the same method as in step S1 to perform spatial alignment on the image to be diagnosed in terms of structure to ensure consistency with the registration method in the training stage;

[0135] If the image comes from a multi-modal data source, perform the registration algorithm of similarity measurement for each modality one by one, and ensure that the images of each modality accurately correspond to the position of the lesion area;

[0136] Perform random noise removal operation on the registered image to be diagnosed, and perform unified normalization processing in terms of gray level, resolution and pixel size, so that the numerical distribution of the image to be diagnosed is consistent with the standardized image data set;

[0137] Crop the image to be diagnosed according to the input size required by the training model to ensure that the image to be diagnosed is consistent with the training stage in terms of input resolution and pixel arrangement method, and reduce the risk of misjudgment caused by size deviation of the model;

[0138] Let X * represent the medical image data to be diagnosed that has completed preprocessing;

[0139] Input X * into the optimized diagnostic model (denoted as M Opt ) obtained in step S3, and this model outputs the predicted probability distribution corresponding to the pathological category:

[0140]

[0141] where k is the total number of pathological categories;

[0142] If there exists a certain category c such that is greater than the set decision threshold, it can be directly determined as this category;

[0143] If all All are below the threshold, but if at least one category c meets the secondary suspicious criteria, it is determined to be suspected of that category, and further medical examinations are recommended;

[0144] If the predicted probabilities of all categories are below the established threshold and there are no suspicious categories that meet the criteria, a tentative determination is made based on the category with the highest probability, and a final review is conducted in combination with the experience of clinicians;

[0145] For the input imaging X to be diagnosed * , the output of the optimized diagnostic model is denoted as:

[0146]

[0147] That is, the category with the highest predicted probability is selected. In the case of recommending a reexamination, the probability distribution output by the model and the suspicious category information are provided to the doctor for reference;

[0148] Furthermore, the output pathological classification results include:

[0149] Final pathological type: corresponding to one or more categories with the highest probability;

[0150] Confidence information: showing as the basic confidence of this diagnostic result;

[0151] Abnormality reminder: When the probabilities of all categories are relatively low or there is no confirmed category, it is prompted that other examinations need to be combined for auxiliary judgment;

[0152] Suggested plan: If it is determined to be a suspected category, corresponding medical examination suggestions (such as further imaging, reexamination of pathological sections) are given to improve the safety and accuracy of the diagnosis.

[0153] It should be noted that, in order to better understand the application of the diagnostic method proposed in this embodiment, through the intelligent recognition and classification of a certain hospital based on multi-modal imaging data of lung CT and X-ray for patient A with suspected lung nodules / tumors, the same preprocessing operations are performed on it through the aforementioned step S1 to form a standardized imaging data set, and it is input into the optimized diagnostic model in step S3 to output the final pathological classification results; among them:

[0154] Perform the same registration, denoising, normalization, and size resampling operations on the lung CT and X-ray data of patient A as in the training stage to ensure that the data feature distribution input into the model is the same as that in the previous training / verification;

[0155] Input the preprocessed imaging X * into the model M that has completed probability calibration in step S3 Opt , and output the predicted probability distribution for each pathological category, for example:

[0156] Benign nodules: 0.05;

[0157] Inflammatory lesions: 0.10;

[0158] Early-stage malignant tumors: 0.70;

[0159] Advanced-stage malignant tumors: 0.15;

[0160] The model detects that the probability of the early-stage malignant tumor category is the highest (0.70). If this probability is higher than the set threshold (e.g., 0.65), it can be directly determined that the nodule of patient A is an early-stage malignant tumor;

[0161] If the predicted probabilities of all categories are lower than the threshold, but a certain category still exceeds the suspicious standard (such as 0.50), it is determined as suspected of this category, indicating that further examination is needed;

[0162] If all probabilities are low, a tentative determination is made according to the category with the highest probability, and it is recommended that the doctor conduct an in-depth review in combination with other clinical examination results;

[0163] At this time, the final output result is expressed as: And the corresponding confidence information (0.70) is attached.

[0164] The data preprocessing method for obtaining the imaging data described above can be carried out by means and methods in the existing technology and will not be elaborated in this example.

[0165] Under the application of the above embodiments, some other aspects disclosed in the embodiments of the present invention also propose an intelligent medical imaging diagnosis system based on deep learning, including: one or more processors and a memory.

[0166] The memory is used to store executable instructions. When these instructions are executed by the one or more processors, the one or more processors perform operations, and these operations include the processes of the intelligent medical imaging diagnosis method based on deep learning in the foregoing embodiments, especially Figure 1 the processes of the method shown.

[0167] Some other aspects disclosed in the embodiments of the present invention also propose a computer-readable medium storing software, and these software include instructions that can be executed by one or more computers. When these instructions are executed in this way, the one or more computers perform operations, and these operations include the processes of the intelligent medical imaging diagnosis method based on deep learning in the foregoing embodiments, especially Figure 1 the processes of the method shown.

[0168] It should be recognized that the embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or computer instructions stored in a non-transitory computer-readable memory.

[0169] The method can be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with the computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner.

[0170] Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language.

[0171] In any case, the language can be a compiled or interpreted language.

[0172] In addition, for this purpose the program is capable of running on a programmed application-specific integrated circuit.

[0173] The processes described herein (or variations and / or combinations thereof) can be executed under the control of one or more computer systems configured with executable instructions and can be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executed jointly on one or more processors, by hardware, or by a combination thereof. The computer program includes a plurality of instructions executable by one or more processors.

[0174] Further, the method can be implemented in any type of computing platform operably connected to a suitable one, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a stand-alone or integrated computer platform, or in communication with a charged particle tool or other imaging device.

[0175] Aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, RAM, ROM, etc., such that it can be read by a programmable computer and can be used to configure and operate the computer to execute the processes described herein when the storage medium or device is read by the computer.

[0176] In addition, the machine-readable code, or portions thereof, can be transmitted via a wired or wireless network.

[0177] When such media includes instructions or programs that implement the above-described steps in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media.

[0178] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and all of them should be covered by the scope of the claims of the present invention.

Claims

1. An intelligent medical image diagnosis method based on deep learning, characterized in that, Including: Obtain multi-modal medical image data through a medical image acquisition device, and perform data preprocessing to form a standardized image data set; Construct a deep learning diagnosis model based on multi-scale feature fusion, input the standardized image data set into the deep learning diagnosis model for transfer learning training, and adopt a dynamic weight adjustment strategy to optimize model parameters; Verify the trained deep learning diagnosis model through an ensemble learning framework, generate a diagnosis confidence score, and calibrate the probability of the diagnosis result in combination with the Bayesian optimization algorithm to obtain an optimized diagnosis model; Input the medical image data to be diagnosed into the optimized diagnosis model for diagnostic analysis, and output the pathological classification result.

2. The intelligent medical image diagnosis method based on deep learning according to claim 1, wherein, The multi-modal medical image data is image data related to the same lesion obtained through at least two different imaging methods, namely X-ray imaging, CT imaging, MRI imaging, ultrasound imaging, PET-CT imaging, digital imaging of pathological sections, and infrared thermal imaging; Perform quality inspection on the obtained multi-modal image data, and eliminate images with serious artifacts, defocus, insufficient exposure or overexposure that affect the diagnosis result; Store the multi-modal image data in the DICOM image file format and assign a unified lesion identifier.

3. The intelligent medical image diagnosis method based on deep learning according to claim 1 or 2, characterized in that Perform data preprocessing on the multi-modal medical image data to form a standardized image data set, including: For medical image data of different modalities, use a registration algorithm based on similarity measurement to achieve spatial alignment to ensure accurate correspondence of each modality image in the anatomical structure; Remove random noise from the registered multi-modal medical image data, and improve the clarity of the lesion area through histogram equalization and contrast enhancement; Perform unified normalization processing on the multi-modal medical image data in terms of gray level, resolution and pixel size to obtain standardized image data; Perform flipping, rotation, cropping, noise perturbation and color offset processing on the standardized image data to form a standardized image data set.

4. The intelligent medical image diagnosis method based on deep learning according to claim 1, characterized in that The construction of the deep learning diagnosis model based on multi-scale feature fusion includes: Set convolution kernels of different sizes at different levels of the convolutional neural network to obtain local features and global features within different field of views; Perform weighted fusion on the features output from each scale convolutional layer, and highlight the key lesion areas through an attention mechanism to suppress redundant background information; Pass the fused multi-scale features through several fully connected layers to form a high-level semantic representation of the target pathological area; Configure a classification layer for determining the pathological type at the end of the convolutional neural network, and output the lesion category probability through the Softmax function.

5. The intelligent medical image diagnosis method based on deep learning according to claim 1 or 4, characterized in that Input the standardized image data set into the deep learning diagnosis model for transfer learning training, and adopt a dynamic weight adjustment strategy to optimize model parameters, including: Use the network parameters pre-trained on a large-scale medical image data set as the initial parameters of the basic model; For the current lesion type, perform fine-tuning training on all parameters of the basic model; Define an objective function based on cross-entropy: where N is the number of training samples, and y i is the true label of the i-th sample, and is the predicted probability output by the model; Assign greater weights to samples with higher losses or prone to misjudgment, and the weight iteration update rule is expressed as: Among them, represents the weight value of the i-th sample after the t-th iteration, and δ i represents the relative measure of the loss gradient of the i-th sample, and α is the balance factor; The network parameters are continuously updated using the stochastic gradient descent optimization algorithm until the preset maximum number of training epochs is reached and then the process ends.

6. The intelligent medical image diagnosis method based on deep learning according to claim 5, characterized in that The trained deep learning diagnostic model is verified through an ensemble learning framework to generate a diagnostic confidence score, including: Adopting the Bagging method to construct multiple sub-models; Performing weighted average fusion on the prediction results of the multiple sub-models to obtain a fused comprehensive diagnostic result; According to the fused comprehensive diagnostic result, calculating the predicted probability distribution of each sample to be tested on each pathological category, and statistically analyzing the confidence of the model in the prediction result to generate a diagnostic confidence score.

7. The intelligent medical image diagnosis method based on deep learning according to claim 6, wherein And combining the Bayesian optimization algorithm to perform probability calibration on the diagnostic result to obtain an optimized diagnostic model, including: Setting corresponding search ranges for the key hyperparameters of the deep learning diagnostic model; Establishing a Gaussian process regression model to estimate the diagnostic performance distribution that can be achieved by different combinations of hyperparameters on the validation set; Selecting the next set of optimal hyperparameter combinations by maximizing the expected improvement value; Applying the finally determined hyperparameters to the deep learning diagnostic model and outputting the posterior probability distribution for each pathological category, so that the model calibrates the diagnostic result to obtain an optimized diagnostic model; Among them, the key hyperparameters include the learning rate, regularization coefficient, number of convolutional kernels, and network depth.

8. The intelligent medical image diagnosis method based on deep learning according to claim 7, characterized in that Inputting the medical image data to be diagnosed into the optimized diagnostic model for diagnostic analysis and outputting a pathological classification result, including: Performing preprocessing operations consistent with the training stage on the medical image data to be diagnosed, including registration, denoising, normalization, and size resampling; Inputting the preprocessed image to be diagnosed into the diagnostic model calibrated by Bayesian optimization and ensemble learning, and outputting the predicted probability corresponding to the pathological category; Define the predicted probabilities of each category output by the diagnostic model as \(p_1, p_2, \ldots, p_k\), k where \(k\) is the total number of pathological categories; If there exists a predicted probability p of a certain category c c ≥ threshold T high , then it is determined to be that category; If the predicted probabilities for all categories are less than the threshold T high but there is at least one category c such that p c ≥ T high , then it is determined to be suspected of this category and further examination is recommended; If the predicted probabilities of all categories are lower than the threshold T high , a tentative determination is made according to the category with the highest probability, but it needs to be reviewed in combination with the doctor's clinical experience.

9. An intelligent medical image diagnosis system based on deep learning, characterized in that, Including: One or more processors; A memory storing operable instructions, the instructions when executed by the one or more processors cause the one or more processors to perform operations, the operations including the process of the deep learning-based medical image intelligent diagnosis method according to any one of claims 1 to 8.

10. A computer-readable medium storing software, characterized in that: The software includes instructions executable by one or more computers, the instructions when executed by such computers cause the one or more computers to perform operations, the operations including the process of the deep learning-based medical image intelligent diagnosis method according to any one of claims 1 to 8.

Images