Deep learning-based pulmonary nodule medical image intelligent diagnosis method and device

Abstract

The invention provides a pulmonary nodule medical image intelligent diagnosis method and device based on deep learning. The problems that in the prior art, the cost of a detection model is large, the accuracy rate of model detection is poor, and the generalization ability is poor are solved. Through a CNN model based on a U-net network architecture or a K-net network architecture, segmentation processing is carried out on pulmonary nodule image data, feature extraction is carried out on an image obtained after segmentation processing, classification is carried out according to features obtained through extraction, a classification result of pulmonary nodules is formed, and efficient detection of pulmonary nodules is realized. And meanwhile, effective prediction of the development trend of the pulmonary nodules with risks is realized based on a partial differential equation.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a method and device for intelligent diagnosis of pulmonary nodules based on deep learning. Background Art

[0002] In the field of medical image analysis, the detection and diagnosis of lung nodules has always been a focus and difficulty in research and clinical practice. As an early manifestation of lung disease, especially an early sign of lung cancer, accurate identification of lung nodules plays a vital role in improving cure rates and reducing mortality. However, due to the diverse appearance of lung nodules on images, large differences in size, shape, density and other characteristics, coupled with the complex anatomical structure of the lungs, the detection of lung nodules has become an extremely challenging task.

[0003] Traditional lung nodule detection methods mainly rely on radiologists' manual observation and analysis of CT images. Doctors need to rely on their rich clinical experience and expertise to observe small changes in CT images to determine whether there are lung nodules and further evaluate their nature. In this method, doctors need to identify lung nodules through naked eye observation and empirical judgment, which is not only time-consuming and labor-intensive, but also easily affected by individual experience and subjective judgment, resulting in inconsistent diagnostic results and high rates of missed diagnosis and misdiagnosis.

[0004] In addition, with the surge in the amount of medical imaging data, it is becoming increasingly unrealistic to rely on manual large-scale image analysis. Therefore, automated lung nodule detection and diagnosis technology has become a research hotspot in the field of medical imaging. However, the existing deep learning models for lung nodule detection require a large amount of labeled data for training, which has high labor costs. At the same time, the trained deep learning models have poor detection accuracy for lung nodules and poor generalization ability of the model. Summary of the invention

[0005] The present invention provides a deep learning-based intelligent medical imaging diagnosis method and device for lung nodules to solve the problems in the prior art of high cost of detection models, poor accuracy of model detection and poor generalization ability of models.

[0006] In a first aspect, the present invention provides a method for intelligent diagnosis of pulmonary nodules based on deep learning medical imaging, which specifically comprises the following steps:

[0007] Step S1, obtaining CT image data DATA1;

[0008] Step S2, segmenting the image data DATA1 through a neural network model to form segmented image data DATA2;

[0009] Step S3, inputting DATA2 into the feature extraction network model, extracting features of the image data DATA2 through the feature extraction network model, and forming feature data DATA3;

[0010] Step S4, outputting lung nodule classification data DATA4 according to the feature data DATA3;

[0011] Step S5, generating a diagnosis report according to the data DATA1-DATA4 in steps S1-S4.

[0012] Preferably, the deep learning-based intelligent medical imaging diagnosis method for pulmonary nodules described in the present invention further includes:

[0013] Step S6, monitoring the malignant risk data RD of the pulmonary nodules with malignant risk in the diagnosis report, and predicting the growth trend of the pulmonary nodules with malignant risk according to the RD;

[0014] The malignant risk data RD of pulmonary nodules include the concentration of macrophages Mc, the concentration of bacteria Bc, the moving speed of cells v, the mortality rate μ1 of macrophages killed by bacteria, the mortality rate μ2 of bacteria phagocytosed by macrophages, the natural mortality rate α of macrophages, the natural reproduction rate λ of bacteria, the pressure between cells p, the diffusion coefficient of macrophages set to 1, the diffusion coefficient of bacteria 1+δ, the regional position Ω of pulmonary nodules, and the boundary position of pulmonary nodule regions. The boundary curvature κ of the lung nodule and the moving speed V of the lung nodule boundary n ;

[0015] Among them, the regional position Ω of the lung nodule and the boundary position of the lung nodule region All change with time t.

[0016] Preferably, in step S1, DATA1 may also be preprocessed. Preferably, the preprocessing may include operations such as denoising and contrast enhancement to improve the quality of image data.

[0017] Preferably, in step S2, the neural network model may include an activation function, a loss function, a convolution layer, a pooling layer and a deconvolution layer.

[0018] More preferably, the activation function may be one of the functions including Sigmoid function, ReLU function, Softmax function and the like.

[0019] More preferably, the optimization of the neural network model is implemented by minimizing a loss function; wherein the method of minimizing the loss function includes a back propagation algorithm and a gradient descent method.

[0020] More preferably, the loss function is selected from a cross entropy loss function.

[0021] Preferably, the optimization of the neural network model is also implemented by cross-validation and hyperparameter tuning.

[0022] More preferably, the cross-validation includes k-fold cross-validation to evaluate the generalization ability of the model.

[0023] More preferably, the hyperparameter tuning adopts one or both of grid search and random search to find the optimal hyperparameter combination.

[0024] Preferably, in step S2, the neural network model includes a convolutional neural network model (U-CNN) based on a U-net network architecture and a convolutional neural network model (K-CNN) based on a K-net network architecture.

[0025] Preferably, the neural network model is also trained; preferably, the training of the U-CNN model specifically includes the following steps:

[0026] Step S201a, obtaining CT image data D1 for model training;

[0027] Step S201b, preprocessing the image data D1 to form preprocessed image data D2;

[0028] Step S201c, inputting the image data D2 into the U-CNN model to form image data New-D2-1 obtained after U-CNN prediction;

[0029] Step S201d, calculating the difference between D2 and New-D2-1 by using the loss function to generate loss 1;

[0030] Step S201e, calculating the gradient of loss 1 relative to the U-CNN model parameters through the back propagation algorithm, and updating the U-CNN model parameters according to the optimization algorithm and the gradient;

[0031] Step S201f, repeat steps S201a to S201e to form a trained U-CNN model.

[0032] Preferably, in step S201e, the optimization algorithm includes a stochastic gradient descent method; the U-CNN model parameters include U-CNN model parameters such as the size of a model convolution kernel and matrix elements.

[0033] Preferably, the training of the K-CNN model specifically includes the following steps:

[0034] Step S202a, obtaining CT image data D3 for model training;

[0035] Step S202b, preprocessing the image data D3 to form preprocessed image data D4;

[0036] Step S202c, inputting the image data D4 into the K-CNN model to form image data New-D4-1 obtained after K-CNN prediction;

[0037] Step S202d, calculating the difference between D4 and New-D4-1 by using the loss function to generate loss 2;

[0038] Step S202e, calculating the gradient of loss 2 relative to the K-CNN model parameters through the back propagation algorithm, and updating the K-CNN model parameters according to the optimization algorithm and the gradient;

[0039] Step S202f, repeat steps S202a to S202e to form a trained K-CNN model.

[0040] Preferably, in step S202e, the optimization algorithm includes a stochastic gradient descent method; and the K-CNN model parameters include K-CNN model parameters such as the size of a model convolution kernel and matrix elements.

[0041] Preferably, in step S3, the feature extraction network model sequentially connects an input layer, a convolution layer 1, a pooling layer 1, a convolution layer 2, a pooling layer 2, a Flatten layer, a fully connected layer and an output layer to form a link.

[0042] Preferably, in step S3, the features include size, shape, edge clarity, texture and other features of the pulmonary nodules.

[0043] Preferably, in step S4, the feature data DATA3 may be classified by using a non-parallel support vector machine (NPSVM) to classify lung nodules; wherein the lung nodules include benign lung nodules and malignant lung nodules.

[0044] Preferably, in step S4, the classification process further includes screening the feature data DATA3 to exclude areas that meet the following conditions, which specifically include the following:

[0045] 1) If rectangularity C = B / (E·F) ≥ 1;

[0046] 2) If slenderness G = (min(E·F)) / (max(E·F))≤0.4;

[0047] 3) If flatness H = I / J ≥ 4;

[0048] 4) If the average CT value (i.e., the average value of pixel values ​​obtained after pixelation of the CT image) is greater than 1000;

[0049] Among them, E represents the width of the region; F represents the height of the region; B represents the number of pixels in the region; I represents the major axis of the equivalent ellipse of the region; and J represents the minor axis of the equivalent ellipse of the region.

[0050] Preferably, in step S5, the diagnosis report includes information such as patient ID, detection date, three-dimensional image of lung nodules, number of lung nodules, location, shape, malignancy probability of each lung nodule, and follow-up medical advice.

[0051] Preferably, in step S6, the malignant risk data RD of the pulmonary nodules with malignant risk in the diagnosis report is monitored, and the growth trend of the pulmonary nodules with malignant risk is predicted according to RD, which specifically includes the following steps:

[0052] Step S601, constructing a partial differential equation according to a biomedical mechanism;

[0053] Step S602: Obtain malignant risk data of lung nodules with malignant risk at equal intervals of time T+1 to form RD={rd0,rd1,…,rd T};

[0054] Step S603: construct a 3D-PDE-Net network model, input RD into the 3D-PDE-Net network model, and form

[0055] Step S604: construct a loss function, and optimize the 3D-PDE-Net network model by using the loss function to form an optimized 3D-PDE-Net network model;

[0056] Step S605: input the malignancy risk data of lung nodules with malignancy risk in the diagnosis report at any time into the optimized 3D-PDE-Net network model, and predict the growth trend of lung nodules through the optimized 3D-PDE-Net network model.

[0057] Wherein, T is a positive integer.

[0058] Preferably, in step S601, the partial differential equation is specifically expressed as follows:

[0059]

[0060] Among them, (x, y, z) represents the three-dimensional spatial position of the lung nodule; t represents the time variable; Ω(t) represents the regional position of the lung nodule at time t; Mc+Bc≡1, through Darcy's law The following formula can be obtained:

[0061]

[0062] Among them, f(Bc)=(λ-μ1-μ2+α)×Bc+(μ1+μ2)×Bc 2 , and the boundary meets the following conditions:

[0063]

[0064] in, represents the derivative of the unit normal; β>0; γ>0; κ represents the curvature of the lung nodule boundary Ω; represents the boundary position of the lung nodule at time t;

[0065] Among them, from Mc+Bc≡1, we can get:

[0066]

[0067] Among them, the moving speed V of the lung nodule boundary n The following conditions must be met:

[0068]

[0069] Among them, n represents the unit external normal; at the same time, the initialization condition of the above partial differential equation is:

[0070] Mc(x,y,z,0)=Mc0(x,y,z),(x,y,z)∈Ω(0)

[0071] v(x,y,z,0)=v0(x,y,z),(x,y,z)∈Ω(0)

[0072] Among them, Ω(0) represents the location of the lung nodule area monitored for the first time (i.e., the location of the lung nodule area at t=0), M0 represents the macrophage concentration monitored for the first time (i.e., the macrophage concentration at t=0); v0 represents the bacterial concentration monitored for the first time (i.e., the bacterial concentration at t=0).

[0073] Preferably, in step S604, the loss function is specifically expressed as follows:

[0074]

[0075] Where time is an integer, time∈[0,T]; loss(·) represents the loss function, which is used to evaluate the difference between the predicted output and the actual label and guide the update of the model parameters; argmin(loss(W m ,b m )) Returns the objective function loss(W m ,b m ) when the independent variable (W) reaches the minimum valuem ,b m ) value; Indicates the parameter value when the loss function reaches the minimum value.

[0076] In a second aspect, the present invention also provides a deep learning-based intelligent medical imaging diagnostic device for pulmonary nodules, which specifically includes the following modules:

[0077] An image data acquisition module, used for acquiring CT image data DATA1;

[0078] An image data segmentation processing module, used for segmenting the image data DATA1 through a neural network model to form segmented image data DATA2;

[0079] An image data feature extraction module is used to input DATA2 into a feature extraction network model, extract features of the image data DATA2 through the feature extraction network model, and form feature data DATA3;

[0080] An image data classification processing module is used to output lung nodule classification data DATA4 according to the feature data DATA3;

[0081] The report generating module is used to generate a diagnosis report according to the data DATA1-DATA4.

[0082] Preferably, the deep learning-based intelligent medical imaging diagnostic device for pulmonary nodules described in the present invention further includes the following modules:

[0083] A pulmonary nodule growth prediction module is used to monitor the malignant risk data RD of pulmonary nodules with malignant risks in the diagnosis report, and predict the growth trend of pulmonary nodules with malignant risks according to RD;

[0084] The malignant risk data RD of pulmonary nodules include the concentration of macrophages Mc, the concentration of bacteria Bc, the moving speed of cells v, the mortality rate μ1 of macrophages killed by bacteria, the mortality rate μ2 of bacteria phagocytosed by macrophages, the natural mortality rate α of macrophages, the natural reproduction rate λ of bacteria, the pressure between cells p, the diffusion coefficient of macrophages set to 1, the diffusion coefficient of bacteria 1+δ, the regional position Ω of pulmonary nodules, and the boundary position of pulmonary nodule regions. The boundary curvature κ of the lung nodule and the moving speed V of the lung nodule boundary n ;

[0085] Among them, the regional position Ω of the lung nodule and the boundary position of the lung nodule region All change with time t.

[0086] Preferably, in the image data acquisition module, DATA1 may also be preprocessed. Preferably, the preprocessing may include operations such as denoising and contrast enhancement to improve the quality of the image data.

[0087] Preferably, in the image data segmentation processing module, the neural network model may include an activation function, a loss function, a convolution layer, a pooling layer and a deconvolution layer.

[0088] More preferably, the activation function may be one of the functions including Sigmoid function, ReLU function, Softmax function and the like.

[0089] More preferably, the optimization of the neural network model is implemented by minimizing a loss function; wherein the method of minimizing the loss function includes a back propagation algorithm and a gradient descent method.

[0090] More preferably, the loss function is selected from a cross entropy loss function.

[0091] Preferably, the optimization of the neural network model is also implemented by cross-validation and hyperparameter tuning.

[0092] More preferably, the cross-validation includes k-fold cross-validation to evaluate the generalization ability of the model.

[0093] More preferably, the hyperparameter tuning adopts one or both of grid search and random search to find the optimal hyperparameter combination.

[0094] Preferably, in the image data segmentation processing module, the neural network model includes a convolutional neural network model (U-CNN) based on a U-net network architecture and a convolutional neural network model (K-CNN) based on a K-net network architecture.

[0095] Preferably, the neural network model is also trained; preferably, when the U-CNN model is trained, the model construction and training module specifically includes the following submodules:

[0096] A first training submodule is used to obtain CT image data D1 for model training;

[0097] A second training submodule, used for preprocessing the image data D1 to form preprocessed image data D2;

[0098] A third training submodule is used to input the image data D2 into a U-CNN model to form image data New-D2-1 obtained after U-CNN prediction;

[0099] The fourth training submodule is used to calculate the difference between D2 and New-D2-1 through the loss function to generate a loss of 1;

[0100] The fifth training submodule is used to calculate the gradient of loss 1 relative to the U-CNN model parameters through the back propagation algorithm, and update the U-CNN model parameters according to the optimization algorithm and the gradient;

[0101] The sixth training submodule is used to repeatedly execute the first training submodule, the second training submodule, the third training submodule, the fourth training submodule and the fifth training submodule to form a trained U-CNN model.

[0102] Preferably, in the fifth training submodule, the optimization algorithm includes a stochastic gradient descent method; and the U-CNN model parameters include U-CNN model parameters such as the size of a model convolution kernel and matrix elements.

[0103] Preferably, when the K-CNN model is trained, the model construction and training module specifically includes the following submodules:

[0104] A seventh training submodule, used for acquiring CT image data D3 for model training;

[0105] an eighth training submodule, configured to preprocess the image data D3 to form preprocessed image data D4;

[0106] A ninth training submodule, used for inputting the image data D4 into the K-CNN model to form image data New-D4-1 obtained after K-CNN prediction;

[0107] The tenth training submodule is used to calculate the difference between D4 and New-D4-1 through the loss function to generate loss 2;

[0108] The eleventh training submodule is used to calculate the gradient of loss 2 relative to the K-CNN model parameters through the back propagation algorithm, and update the K-CNN model parameters according to the optimization algorithm and the gradient;

[0109] The twelfth training submodule is used to repeatedly execute the seventh training submodule, the eighth training submodule, the ninth training submodule, the tenth training submodule and the eleventh training submodule to form a trained K-CNN model.

[0110] Preferably, in the eleventh training submodule, the optimization algorithm includes a stochastic gradient descent method; the K-CNN model parameters include K-CNN model parameters such as the size of the model convolution kernel and matrix elements.

[0111] Preferably, in the image data feature extraction module, the feature extraction network model is connected in sequence to include an input layer, a convolution layer 1, a pooling layer 1, a convolution layer 2, a pooling layer 2, a Flatten layer, a fully connected layer and an output layer to form a link.

[0112] Preferably, in the image data feature extraction module, the features include size, shape, edge clarity, texture and other features of the lung nodules.

[0113] Preferably, in the image data classification processing module, the feature data DATA3 may be classified by using a non-parallel support vector machine (NPSVM) to classify lung nodules; wherein the lung nodules include benign lung nodules and malignant lung nodules.

[0114] Preferably, in the image data classification processing module, the feature data DATA3 is screened before the classification processing to exclude areas that meet the following conditions, which specifically include the following:

[0115] 1) If rectangularity C = B / (E·F) ≥ 1;

[0116] 2) If slenderness G = (min(E·F)) / (max(E·F))≤0.4;

[0117] 3) If flatness H = I / J ≥ 4;

[0118] 4) If the average CT value is greater than 1000;

[0119] Among them, E represents the width of the region; F represents the height of the region; B represents the number of pixels in the region; I represents the major axis of the equivalent ellipse of the region; and J represents the minor axis of the equivalent ellipse of the region.

[0120] Preferably, in the report generation module, the diagnosis report includes information such as patient ID, detection date, three-dimensional image of lung nodules, number of lung nodules, location, shape, malignancy probability of each lung nodule, and follow-up medical advice.

[0121] Preferably, the pulmonary nodule growth prediction module specifically includes the following submodules:

[0122] The first submodule of growth prediction is used to construct partial differential equations based on biomedical mechanisms;

[0123] The second submodule of growth prediction is used to obtain the malignant risk data of pulmonary nodules with malignant risk at equal intervals of time T+1, forming RD={rd0,rd1,…,rd T};

[0124] The third submodule of growth prediction is used to construct a 3D-PDE-Net network model, input RD into the 3D-PDE-Net network model, and form

[0125] The fourth submodule of growth prediction is used to construct a loss function, and optimize the 3D-PDE-Net network model through the loss function to form an optimized 3D-PDE-Net network model;

[0126] The fifth submodule of growth prediction is used to input the malignant risk data of lung nodules with malignant risks in the diagnosis report at any time into the optimized 3D-PDE-Net network model, and predict the growth trend of lung nodules through the optimized 3D-PDE-Net network model.

[0127] Wherein, T is a positive integer.

[0128] Preferably, in the second submodule of growth prediction, the partial differential equation is specifically expressed as follows:

[0129]

[0130] Among them, (x, y, z) represents the three-dimensional spatial position of the lung nodule; t represents the time variable; Ω(t) represents the regional position of the lung nodule at time t; Mc+Bc≡1, through Darcy's law The following formula can be obtained:

[0131]

[0132] Among them, f(Bc)=-α+(λ-μ1-μ2+α)×Bc+(μ1+μ2)×Bc 2 , and the boundary meets the following conditions:

[0133]

[0134] in, represents the derivative of the unit normal; β>0; γ>0; κ represents the curvature of the lung nodule boundary Ω; represents the boundary position of the lung nodule at time t;

[0135] Among them, from Mc+Bc≡1, we can get:

[0136]

[0137] Among them, the moving speed V of the lung nodule boundary n The following conditions must be met:

[0138]

[0139] Among them, n represents the unit external normal; at the same time, the initialization condition of the above partial differential equation is:

[0140] Mc(x,y,z,0)=Mc0(x,y,z),(x,y,z)∈Ω(0)

[0141] v(x,y,z,0)=v0(x,y,z),(x,y,z)∈Ω(0)

[0142] Among them, Ω(0) represents the location of the lung nodule area monitored for the first time (i.e., the location of the lung nodule area at t=0), M0 represents the macrophage concentration monitored for the first time (i.e., the macrophage concentration at t=0); v0 represents the bacterial concentration monitored for the first time (i.e., the bacterial concentration at t=0).

[0143] Preferably, in the fourth submodule of growth prediction, the loss function is specifically expressed as follows:

[0144]

[0145] Where time is an integer, time∈[0,T]; loss(·) represents the loss function, which is used to evaluate the difference between the predicted output and the actual label and guide the update of the model parameters; argmin(loss(W m ,b m )) Returns the objective function loss(W m ,b m ) when the independent variable (W) reaches the minimum value m ,b m ) value; Indicates the parameter value when the loss function reaches the minimum value.

[0146] In a third aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a deep learning-based intelligent medical imaging diagnosis method for pulmonary nodules as described in any one of the first aspects of the present application.

[0147] In a fourth aspect, the present invention also provides an electronic device, comprising: a memory storing a computer program; and a processor communicating with the memory, for executing a deep learning-based intelligent medical imaging diagnosis method for pulmonary nodules as described in any one of the first aspects of the present application when the computer program is called.

[0148] Compared with the prior art, the present invention has the following obvious outstanding substantive features and significant advantages:

[0149] The present invention provides a method and device for intelligent diagnosis of medical images of lung nodules based on deep learning, which solves the problems of high cost, poor accuracy of model detection, and poor generalization ability of the existing technology. Through the CNN model based on the U-net network architecture or the K-net network architecture, the lung nodule image data is segmented, and the image obtained after the segmentation process is subjected to feature extraction, and the image is classified according to the extracted features to form a classification result of the lung nodules, thereby realizing efficient detection of the lung nodules. At the same time, based on the partial differential equation, an effective prediction of the development trend of lung nodules with risks is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0150] The accompanying drawings, which constitute part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention. In the accompanying drawings:

[0151] Figure 1 This is a flow chart of a method for intelligent diagnosis of pulmonary nodules based on deep learning in medical imaging according to a preferred embodiment of the present invention.

[0152] Figure 2 It is a schematic diagram of the U-net network architecture in the preferred embodiment of the present invention.

[0153] Figure 3 It is a schematic diagram of the K-net network architecture in a preferred embodiment of the present invention.

[0154] Figure 4 It is a schematic diagram of the kernel update header structure in the K-net network architecture in the preferred embodiment of the present invention.

[0155] Figure 5 It is a structural schematic diagram of a lung nodule feature extraction model in a preferred embodiment of the present invention.

[0156] Figure 6 It is a schematic diagram of the major and minor axes of an equivalent ellipse of a lung nodule area in a preferred embodiment of the present invention.

[0157] Figure 7 It is a schematic diagram of the positional relationship between voxels and isosurfaces in a preferred embodiment of the present invention.

[0158] Figure 8 It is a schematic diagram of the regional position and boundary position of a lung nodule in a preferred embodiment of the present invention.

[0159] Fig. 9 It is a schematic diagram of the 3D-PDE-Net network model structure in the preferred embodiment of the present invention.

[0160] Fig.10It is a structural schematic diagram of a deep learning-based intelligent medical imaging diagnostic device for lung nodules according to a preferred embodiment of the present invention.

[0161] Fig.11 It is a schematic diagram of the user interface of the software in the preferred embodiment of the present invention.

[0162] Fig.12 It is a schematic diagram of the running results of the software in the preferred embodiment of the present invention.

[0163] Fig.13 It is a schematic diagram of the operating results displayed by the software in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0164] The present invention provides a method and device for intelligent diagnosis of pulmonary nodules based on deep learning. In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0165] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0166] Embodiment 1:

[0167] like Figure 1-Figure 9 As shown, the deep learning-based intelligent diagnosis method for medical imaging of pulmonary nodules described in this embodiment specifically includes the following steps:

[0168] Step S1, acquiring CT image data DATA1; wherein the CT image data DATA1 includes one or at least two lung nodules.

[0169] Step S2, preprocessing the image data DATA1 to form preprocessed image data DATA2; wherein the preprocessing includes operations such as denoising and contrast enhancement to improve the quality of the image data.

[0170] Step S3: construct and train a neural network model to form a trained neural network model.

[0171] The neural network model includes an activation function, a loss function, a convolution layer, a pooling layer, and a deconvolution layer; wherein the activation function usually transforms a linear classification problem into a nonlinear one, so that the neural network can better solve more complex problems. The activation function in this embodiment selects the ReLU function.

[0172] The optimization of the neural network model is achieved by minimizing the loss function; the method of minimizing the loss function includes a back propagation algorithm and a gradient descent method; the loss function is selected from a cross entropy loss function.

[0173] Among them, the optimization of the neural network model is also implemented by cross-validation and hyperparameter tuning; in this embodiment, the cross-validation is implemented by k-fold cross-validation; the hyperparameter tuning is implemented by grid search or random search.

[0174] Optionally, the convolutional neural network model in step S3 includes a convolutional neural network model (U-CNN) based on a U-net network architecture and a convolutional neural network model (K-CNN) based on a K-net network architecture.

[0175] Among them, the U-net network architecture is as follows Figure 2 shown. Figure 2 The U-net network shown has four layers, which downsample the image 4 times (i.e., deconvolve the image through the deconvolution layer) and upsample the image 4 times (i.e., convolve the image through the convolution layer). Starting from the left, the input is a 572×572×1 image, which is then convolved with 64 3×3 convolution kernels. The convolution result is used as the input of the ReLU activation function, that is, the positive elements in the convolution result remain unchanged, and the negative elements become 0, resulting in 64 570×570×1 feature channels. The 570×570×64 result is then convolved with 64 3×3 convolution kernels, and after passing through the ReLU function, 64 568×568×1 feature extraction results are obtained, which is the processing result of the first layer.

[0176] The processing result of the first layer is a 568×568×64 feature image. Through a 2×2 pooling kernel, the image is downsampled to half of its original size, that is, 284×284×64, and then 128 convolution kernels are used to further extract image features. Similarly, each layer will undergo two convolutions to extract image features; each downsampling layer will reduce the image size by half and double the number of convolution kernels. The final result of the downsampling part is 28×28×1024, that is, there are a total of 1024 feature layers, and the feature size of each layer is 28×28.

[0177] The right part is a 4-time upsampling process from bottom to top. Starting from the bottom right corner, the 28×28×1024 feature matrix is ​​deconvolved through 512 2×2 convolution kernels, and the matrix is ​​expanded to 56×56×512 (512 feature channels in the blue part of the right half). Since deconvolution can only expand the image but not restore it, in order to collect the high-pixel feature information retained in the feature analysis and reduce data loss so that the image can be better synthesized, the downsampled image is cropped into the same size and directly spliced ​​together to increase the feature layer (512 feature channels in the white part of the left half), and then convolution is performed to extract features. Since each deconvolution will make the result smaller, the image in the downsampling process on the left needs to be cropped before each splicing. After the matrices are concatenated, the entire new feature matrix becomes 56×56×1024, and then passes through 512 convolution kernels, and after two convolutions, a 52×52×512 feature matrix is ​​obtained, and upsampling is performed again, and the above process is repeated. Each layer will perform two convolutions to extract features. Each upsampling layer will double the image size and reduce the number of convolution kernels by half. The final upsampling result is 388×388×64, that is, there are 64 feature layers in total, and the feature size of each layer is 388×388. In the last step, two 1×1 convolution kernels are selected to turn the 64 feature channels into 2, that is, the final 388×388×2, and the image is divided into two categories: background and target.

[0178] The training of the U-CNN model includes the following steps:

[0179] Step S301a: Acquire CT image data D1 for model training.

[0180] Step S301b: preprocess the image data D1 to form preprocessed image data D2.

[0181] Step S301c, input the image data D2 into the U-CNN model to form image data New-D2-1 obtained after U-CNN prediction.

[0182] Step S301d, calculate the difference between D2 and New-D2-1 through the loss function to generate loss 1.

[0183] Step S301e, calculating the gradient of loss 1 relative to the U-CNN model parameters through the back propagation algorithm, and updating the U-CNN model parameters according to the optimization algorithm and the gradient; wherein the optimization algorithm includes the stochastic gradient descent method; the U-CNN model parameters include U-CNN model parameters such as the size of the model convolution kernel and matrix elements.

[0184] Step S301f, repeat steps S301a to S301e to form a trained U-CNN model.

[0185] Among them, the K-net network architecture is as follows Figure 3 As shown. K-net is a segmentation model based on dynamic kernels. Assuming the number of groups is N1 (N1 is a positive integer), K-net divides the image into N1 groups by using N1 kernels, and each kernel is responsible for finding the pixels belonging to its corresponding group. Specifically, given the input feature map of B1 image Only N1 cores are needed To perform the convolution:

[0186] M=σ1(K*F1)

[0187] To obtain the corresponding prediction results Where C1, H1 and W1 represent the number of channels, height and width of the feature map respectively, and σ1 represents the activation function (in this embodiment, σ1 is the softmax function); at the same time, during the execution of K-net, the kernel and mask prediction need to be dynamically updated. The specific process is as follows ( Figure 7 ) as shown:

[0188] Step b1, use a set of learnable kernels K0 to first convolve with the feature map F1 to generate the prediction mask M0;

[0189] Step b2: Predict M based on mask i (where i represents a positive integer), kernel K i And the feature map F1, using the kernel to update the head f i+1 Generate class predictions, dynamic kernel K i+1 And the mask prediction M i+1 , which can be expressed as:

[0190] K i+1 ,M i+1 =f i+1 (M i ,K i ,F1)

[0191] Where i∈[0,S], and i is an integer.

[0192] Step b3, the generated mask prediction, dynamic kernel and feature map are sent to the next kernel update header. The kernel update header is the structure of K-net as shown in Figure 4 As shown, it includes the following three key steps:

[0193] Stepc1, group feature assembly: use mask to predict M i-1 Multiply with feature map F1 as new assembly feature Right now:

[0194]

[0195] Step 2, adaptive function update: using A new set of kernels is calculated. In order to eliminate the influence of noise caused by misclassification, an adaptive kernel update strategy is adopted here. First, and K i-1 Perform element-wise multiplication between:

[0196]

[0197] Among them, ψ1 and ψ2 represent linear transformations; then two gates are calculated and The calculation formula is as follows:

[0198]

[0199] Stepc3, these two gates and Compute a new set of kernels The calculation formula is as follows:

[0200]

[0201] Among them, ψ ii (ii=1,…,4) all represent fully connected layers. In this embodiment, σ2 represents the Sigmoid function.

[0202] Kernel interaction: based on Use the Multi-Head Attention mechanism and the Feed-Forward Neural Network to get a new K i , and through M i =g i (K i )*F1 generates new mask predictions.

[0203] By iteratively executing the above process and gradually refining the kernel and mask predictions, we can balance the processing of details and global information, dynamically predict and accurately segment the image.

[0204] The training of the K-CNN model includes the following steps:

[0205] Step S302a: Acquire CT image data D3 for model training.

[0206] Step S302b: pre-process the image data D3 to form pre-processed image data D4.

[0207] Step S302c, input the image data D4 into the K-CNN model to form image data New-D4-1 obtained after K-CNN prediction.

[0208] Step S302d, calculate the difference between D4 and New-D4-1 through the loss function to generate loss 2.

[0209] Step S302e, calculating the gradient of loss 2 relative to the K-CNN model parameters through the back propagation algorithm, and updating the K-CNN model parameters according to the optimization algorithm and the gradient; wherein the optimization algorithm includes the stochastic gradient descent method; the K-CNN model parameters include K-CNN model parameters such as the size of the model convolution kernel and matrix elements.

[0210] Step S302f, repeat steps S302a to S302e to form a trained K-CNN model.

[0211] Step S4: segment the image data DATA2 using the trained neural network model to form segmented image data DATA3.

[0212] Step S5, input DATA3 into the feature extraction network model, extract the features of the image data DATA3 through the feature extraction network model to form feature data DATA4; wherein the feature extraction network model sequentially connects an input layer, a convolution layer 1, a pooling layer 1, a convolution layer 2, a pooling layer 2, a Flatten layer, a fully connected layer and an output layer to form a link, such as Figure 5 As shown; the features include information such as size, shape, edge clarity, and texture features of the lung nodules.

[0213] Step S6, classify the lung nodules according to the characteristic data DATA4 to form classification result data DATA5.

[0214] Optionally, the classification process further includes screening the feature data DATA4 to exclude areas that meet the following conditions, which specifically include the following:

[0215] 1) If rectangularity C = B / (E·F) ≥ 1;

[0216] 2) If slenderness G = (min(E·F)) / (max(E·F))≤0.4;

[0217] 3) If flatness H = I / J ≥ 4;

[0218] 4) If the average CT value is greater than 1000;

[0219] Among them, E represents the width of the region; F represents the height of the region; B represents the number of pixels in the region; I and J represent the major axis and minor axis of the equivalent ellipse of the region, respectively. Figure 6 shown.

[0220] Optionally, the feature data DATA4 is classified and processed by a non-parallel support vector machine (NPSVM) to achieve the classification of lung nodules; wherein the lung nodules include benign lung nodules and malignant lung nodules.

[0221] Optionally, in addition to step S1 to step S6, the method further includes: step S7, generating a diagnosis report according to data DATA1 to DATA5 in step S1 to step S6.

[0222] The diagnostic report includes information such as patient ID, detection date, three-dimensional image of lung nodules, number of lung nodules, location, shape, malignancy probability of each lung nodule, and follow-up medical advice.

[0223] The 3D image of the lung nodule is used to generate a 3D model of the lung nodule using the open source library vtk. position ,y position ,z position ) and its gray value I in CT image g (x positIoN ,y POsition ,z position ), you can build a 3D model by following these steps:

[0224] Step 1, discretize the CT image into a three-dimensional grid;

[0225] Step 2, determine whether it is a nodule based on the gray value of the pixel;

[0226] Step 3, draw the 3D image of the nodule using the Marching Cubes algorithm;

[0227] Among them, the 3D image of the nodule is drawn by the Marching Cubes algorithm to form the following Figure 7 The schematic diagram of the relationship between the voxel and the isosurface position shown in FIG. 1 specifically includes the following steps:

[0228] Step 1, determine the volume element containing the isosurface;

[0229] Step 2, determine the intersection of the isosurface and the voxel boundary;

[0230] Stepe3. Find the normal vector of the isosurface.

[0231] Stepe4: Use Gaussian smoothing to smooth the three-dimensional image.

[0232] In this embodiment, the detection results obtained by the lung nodule detection method described in this embodiment are compared with the diagnosis results of radiologists to complete the evaluation of the lung nodule detection method described in this embodiment. The performance of the lung nodule detection method described in this embodiment is measured by calculating indicators such as sensitivity, specificity, and accuracy.

[0233] Optionally, in addition to step S1 to step S7, the method further includes: step S8, monitoring the malignant risk data RD of the pulmonary nodules with malignant risk in the diagnosis report, and predicting the growth trend of the pulmonary nodules with malignant risk according to the RD;

[0234] The malignant risk data RD of pulmonary nodules include the concentration of macrophages Mc, the concentration of bacteria Bc, the moving speed of cells v, the mortality rate μ1 of macrophages killed by bacteria, the mortality rate μ2 of bacteria phagocytosed by macrophages, the natural mortality rate α of macrophages, the natural reproduction rate λ of bacteria, the pressure between cells p, the diffusion coefficient of macrophages set to 1, the diffusion coefficient of bacteria 1+δ, the regional position Ω of pulmonary nodules, and the boundary position of pulmonary nodule regions. (like Figure 8 The figure shows the regional position and boundary position of a pulmonary nodule, where the circle represents the regional position Ω of the pulmonary nodule, and the outer boundary of the region represents the boundary position of the pulmonary nodule. ), the boundary curvature κ of the lung nodule and the moving speed V of the lung nodule boundary n ; Among them, the regional position of the lung nodule Ω and the boundary position of the lung nodule area All change with time t.

[0235] Optionally, step S6 specifically includes the following steps:

[0236] Step S601: construct a partial differential equation according to the data RD.

[0237] The partial differential equation is specifically expressed as follows:

[0238]

[0239] Among them, (x, y, z) represents the three-dimensional spatial position of the lung nodule; t represents the time variable; Ω(t) represents the regional position of the lung nodule at time t; Mc+Bc≡1, through Darcy's law The following formula can be obtained:

[0240]

[0241] Among them, f(Bc)=-α+(λ-μ1-μ2+α)×Bc+(μ1+μ2)×Bc 2 , and the boundary meets the following conditions:

[0242]

[0243] in, represents the derivative of the unit normal; β>0; γ>0; κ represents the curvature of the lung nodule boundary Ω; represents the boundary position of the lung nodule at time t;

[0244] Among them, from Mc+Bc≡1, we can get:

[0245]

[0246] Among them, the moving speed V of the lung nodule boundary n The following conditions must be met:

[0247]

[0248] Among them, n represents the unit external normal; at the same time, the initialization condition of the above partial differential equation is:

[0249] Mc(x,y,z,0)=Mc0(x,y,z),(x,y,z)∈Ω(0)

[0250] v(x,y,z,0)=v0(x,y,z),(x,y,z)∈Ω(0)

[0251] Among them, Ω(0) represents the location of the lung nodule area at t=0, Mc0 represents the macrophage concentration at t=0; v0 represents the bacterial concentration at t=0.

[0252] Step S602: Obtain malignant risk data of lung nodules with malignant risk at equal intervals of time T+1 to form RD={rd0,rd1,…,rd T}; where T is a positive integer.

[0253] Step S603: construct a 3D-PDE-Net network model (such as Fig. 9 As shown in FIG. 1 , which is a 3D-PDE-Net model in an embodiment of the present invention), RD is input into the 3D-PDE-Net network model to form

[0254] Step S604: construct a loss function, and optimize the 3D-PDE-Net network model through the loss function to form an optimized 3D-PDE-Net network model.

[0255] The loss function is specifically expressed as follows:

[0256]

[0257]

[0258] Where time is an integer, time∈[0,T]; loss(·) represents the loss function, which is used to evaluate the difference between the predicted output and the actual label and guide the update of the model parameters; argmin(loss(W m ,b m )) Returns the objective function loss(W m ,b m ) when the independent variable (W) reaches the minimum value m ,b m ) value; Indicates the parameter value when the loss function reaches the minimum value.

[0259] For a filter q with dimension R×R×R, its matrix is

[0260]

[0261] Among them, s1, s2, s3 represent the index of the matrix Filter(q); k1, k2, k3 represent the index in the three-dimensional discrete space.

[0262] The Haar wavelet frame transform corresponding to the filter q is defined as:

[0263]

[0264] in, Represents the convolution kernel.

[0265] Thus, the corresponding differential operator u can be approximated using the convolution kernel under the constraint x ,u y ,u z ,u xx Etc. Based on the partial differential equation, we define a mapping of the following form

[0266]

[0267] Through convolution approximation, we can get:

[0268]

[0269] Among them, the operator D0 and They correspond to q0 and Haar wavelet frame transform. On this basis, the adaptive activation function is introduced Construct a 3D-PDE-Net; wherein n≥1 is a predefined scaling factor; parameter A represents the slope of the activation function (also referred to as a learnable parameter); in this embodiment, σ uses the Tanh activation function.

[0270] Step S605: input the malignancy risk data of lung nodules with malignancy risk in the diagnosis report at any time into the optimized 3D-PDE-Net network model, and predict the growth trend of lung nodules through the optimized 3D-PDE-Net network model.

[0271] Embodiment 2:

[0272] like Fig.10 As shown, the deep learning-based intelligent medical imaging diagnostic device for pulmonary nodules described in this embodiment specifically includes the following modules:

[0273] The image data acquisition module is used to acquire CT image data DATA1; wherein the CT image data DATA1 includes one or at least two lung nodules.

[0274] The image data preprocessing module is used to preprocess the image data DATA1 to form preprocessed image data DATA2; wherein the preprocessing includes operations such as denoising and contrast enhancement to improve the quality of the image data.

[0275] The model building and training module is used to build and train the neural network model to form a trained neural network model.

[0276] The neural network model includes an activation function, a loss function, a convolution layer, a pooling layer and a deconvolution layer; the activation function includes one of the following functions: a Sigmoid function, a ReLU function, a Softmax function and the like.

[0277] The optimization of the neural network model is achieved by minimizing the loss function.

[0278] Optionally, the method of minimizing the loss function includes a back propagation algorithm and a gradient descent method; wherein the loss function is selected from a cross entropy loss function.

[0279] Among them, the optimization of the neural network model is also implemented by cross-validation and hyperparameter tuning; in this embodiment, the cross-validation is implemented by k-fold cross-validation; the hyperparameter tuning is implemented by grid search or random search.

[0280] Among them, the neural network model includes a convolutional neural network model (U-CNN) based on the U-net network architecture and a convolutional neural network model (K-CNN) based on the K-net network architecture.

[0281] When training the U-CNN model, the model building and training module specifically includes the following submodules:

[0282] The first training submodule is used to obtain CT image data D1 for model training.

[0283] The second training submodule is used to preprocess the image data D1 to form preprocessed image data D2.

[0284] The third training submodule is used to input the image data D2 into the U-CNN model to form the image data New-D2-1 obtained after U-CNN prediction.

[0285] The fourth training submodule is used to calculate the difference between D2 and New-D2-1 through the loss function to generate loss 1.

[0286] The fifth training submodule is used to calculate the gradient of loss 1 relative to the U-CNN model parameters through the back propagation algorithm, and update the U-CNN model parameters according to the optimization algorithm and the gradient; wherein the optimization algorithm includes the stochastic gradient descent method; the U-CNN model parameters include the size of the model convolution kernel and the matrix elements and other U-CNN model parameters.

[0287] The sixth training submodule is used to repeatedly execute the first training submodule, the second training submodule, the third training submodule, the fourth training submodule and the fifth training submodule to form a trained U-CNN model.

[0288] When the K-CNN model is trained, the model construction and training module specifically includes the following submodules:

[0289] The seventh training submodule is used to obtain CT image data D3 for model training.

[0290] The eighth training submodule is used to preprocess the image data D3 to form preprocessed image data D4.

[0291] The ninth training submodule is used to input the image data D4 into the K-CNN model to form the image data New-D4-1 obtained after K-CNN prediction.

[0292] The tenth training submodule is used to calculate the difference between D4 and New-D4-1 through the loss function to generate loss 2.

[0293] The eleventh training submodule is used to calculate the gradient of loss 2 relative to the K-CNN model parameters through the back propagation algorithm, and update the K-CNN model parameters according to the optimization algorithm and the gradient; wherein the optimization algorithm includes the stochastic gradient descent method; the K-CNN model parameters include K-CNN model parameters such as the size of the model convolution kernel and matrix elements.

[0294] The twelfth training submodule is used to repeatedly execute the seventh training submodule, the eighth training submodule, the ninth training submodule, the tenth training submodule and the eleventh training submodule to form a trained K-CNN model.

[0295] The image data segmentation processing module is used to segment the image data DATA2 through the trained neural network model to form segmented image data DATA3.

[0296] The image data feature extraction module is used to input DATA3 into the feature extraction network model, and extract the features of the image data DATA3 through the feature extraction network model to form feature data DATA4; wherein the feature extraction network model is sequentially connected to include an input layer, a convolution layer 1, a pooling layer 1, a convolution layer 2, a pooling layer 2, a Flatten layer, a fully connected layer and an output layer to form a link; the features include the size, shape, edge clarity, texture and other features of the lung nodules.

[0297] The image data classification processing module is used to classify the lung nodules according to the feature data DATA4 to form the classification result data DATA5; wherein, the classification processing of the feature data DATA4 includes using a non-parallel support vector machine (NPSVM) to realize the classification of the lung nodules; wherein, the lung nodules include benign lung nodules and malignant lung nodules.

[0298] Optionally, the classification process further includes screening the feature data DATA4 to exclude areas that meet the following conditions, which specifically include the following:

[0299] 1) If rectangularity C = B / (E·F) ≥ 1;

[0300] 2) If slenderness G = (min(E·F)) / (max(E·F))≤0.4;

[0301] 3) If flatness H = I / J ≥ 4;

[0302] 4) If the average CT value is greater than 1000;

[0303] Among them, E represents the width of the region; F represents the height of the region; B represents the number of pixels in the region; I represents the major axis of the equivalent ellipse of the region; and J represents the minor axis of the equivalent ellipse of the region.

[0304] Optionally, the deep learning-based intelligent medical imaging diagnostic device for lung nodules described in the present invention further includes a report generation module for generating a diagnostic report based on the data DATA1-DATA5; wherein the diagnostic report includes information such as patient ID, detection date, three-dimensional image of lung nodules, number of lung nodules, location, shape, malignancy probability of each lung nodule, and follow-up medical treatment recommendations.

[0305] Optionally, the deep learning-based intelligent medical imaging diagnostic device for lung nodules described in the present invention further includes a lung nodule growth prediction module, which is used to monitor the malignant risk data RD of lung nodules with malignant risks in the diagnosis report, and predict the growth trend of lung nodules with malignant risks based on RD.

[0306] The malignant risk data RD of pulmonary nodules include the concentration of macrophages Mc, the concentration of bacteria Bc, the moving speed of cells v, the mortality rate μ1 of macrophages killed by bacteria, the mortality rate μ2 of bacteria phagocytosed by macrophages, the natural mortality rate α of macrophages, the natural reproduction rate λ of bacteria, the pressure between cells p, the diffusion coefficient of macrophages set to 1, the diffusion coefficient of bacteria 1+δ, the regional position Ω of pulmonary nodules, and the boundary position of pulmonary nodule regions. The boundary curvature κ of the lung nodule and the moving speed V of the lung nodule boundary n ; Among them, the regional position of the lung nodule Ω and the boundary position of the lung nodule area All change with time t.

[0307] Among them, the pulmonary nodule growth prediction module specifically includes the following submodules:

[0308] The first submodule of growth prediction is used to construct partial differential equations based on biomedical mechanisms.

[0309] The partial differential equation is specifically expressed as follows:

[0310]

[0311] Among them, (x, y, z) represents the three-dimensional spatial position of the lung nodule; t represents the time variable; Ω(t) represents the regional position of the lung nodule at time t; Mc+Bc≡1, through Darcy's law The following formula can be obtained:

[0312]

[0313] Among them, f(Bc)=-α+(λ-μ1-μ2+α)×Bc+(μ1+μ2)×Bc 2 , and the boundary meets the following conditions:

[0314]

[0315] in, represents the derivative of the unit normal; β>0; γ>0; κ represents the curvature of the lung nodule boundary Ω; represents the boundary position of the lung nodule at time t;

[0316] Among them, from Mc+Bc≡1, we can get:

[0317]

[0318] Among them, the moving speed V of the lung nodule boundary n The following conditions must be met:

[0319]

[0320] Among them, n represents the unit external normal; at the same time, the initialization condition of the above partial differential equation is:

[0321] Mc(x,y,z,0)=Mc0(x,y,z),(x,y,z)∈Ω(0)

[0322] v(x,y,z,0)=v0(x,y,z),(x,y,z)∈Ω(0)

[0323] Among them, Ω(0) represents the location of the lung nodule area at t=0, Mc0 represents the macrophage concentration at t=0; v0 represents the bacterial concentration at t=0.

[0324] The second submodule of growth prediction is used to obtain the malignant risk data of pulmonary nodules with malignant risk at equal intervals of time T+1, forming RD={rd0,rd1,…,rd T}; where T is a positive integer.

[0325] The third submodule of growth prediction is used to construct a 3D-PDE-Net network model, input RD into the 3D-PDE-Net network model, and form

[0326] The fourth submodule of growth prediction is used to construct a loss function, and optimize the 3D-PDE-Net network model through the loss function to form an optimized 3D-PDE-Net network model.

[0327] The loss function is specifically expressed as follows:

[0328]

[0329]

[0330] Where time is an integer, time∈[0,T]; loss(·) represents the loss function, which is used to evaluate the difference between the predicted output and the actual label and guide the update of the model parameters; argmin(loss(W m ,b m )) Returns the objective function loss(W m ,b m ) when the independent variable (W) reaches the minimum value m ,b m ) value; Indicates the parameter value when the loss function reaches the minimum value.

[0331] The fifth submodule of growth prediction is used to input the malignant risk data of lung nodules with malignant risks in the diagnosis report at any time into the optimized 3D-PDE-Net network model, and predict the growth trend of lung nodules through the optimized 3D-PDE-Net network model.

[0332] Embodiment 3:

[0333] like Figure 11-13 As shown, according to the lung nodule detection method described in the present invention, a lung nodule detection software is also developed (in this embodiment, the underlying model of the software is a K-CNN model).

[0334] The operation flow of the software is as follows:

[0335] Stepf1. After logging in, the user enters the main interface and can start uploading the ZIP compressed package containing CT images.

[0336] Stepf2. ​​After uploading, the system starts to automatically analyze the image, identify nodules and extract features.

[0337] Step 3. After the analysis is completed, the user can view the nodule detection results on the interface, including the location, size, and characteristics of the nodule. Fig.11 , Fig.12 shown.

[0338] Step 4. Users can also view the 3D nodule model and perform interactive operations such as rotation and scaling. Fig.13 shown.

[0339] In addition, the user can also generate and export a diagnostic report and print it through the software. An example of a diagnostic report is shown in Table 1:

[0340] Table 1 Diagnostic report example

[0341]

[0342] Through the diagnostic report shown in Table 1, doctors can quickly understand the patient's condition and make corresponding clinical decisions.

[0343] This software was also compared with other software. The results of the comparison experiments are shown in Table 2 and Table 3:

[0344] Table 2 Comparison of functions and performance of different software

[0345] product Software of the present invention ASoftware BSoftware C Software Detection speed Average 64 seconds About 5 minutes - - Pulmonary nodule contour analysis √ × × √ 3D Reconstruction √ √ × × Diagnostic Report √ √ √ √

[0346] Table 3 Comparison of the accuracy of different models for lung nodule recognition

[0347] product Software of the present invention ASoftware BSoftware C Software The underlying model K-CNN DenseSharp SDTL XimaSharp Accuracy 93.32％ 88％ 88.3％ 89.1％

[0348] As shown in Table 2 and Table 3, the lung nodule detection software designed according to the method of the present invention has significantly improved detection speed and detection accuracy compared with other software.

[0349] In addition, a comparative experiment was conducted between the lung nodule detection software designed according to the method of the present invention and the traditional manual identification. The experimental results are shown in Table 4:

[0350] Table 4 Accuracy of the software of the present invention in classifying lung nodules

[0351] Number of samples The accuracy of the software identification Traditional manual identification accuracy 153 98.96％ 90% (data from professional literature)

[0352] As can be seen from Table 4, the accuracy of lung nodule detection by the software of the present invention is significantly improved compared with the accuracy of traditional manual identification.

[0353] The specific embodiments of the present invention are described in detail above, but they are only examples, and the present invention is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications and substitutions made to the present invention are also within the scope of the present invention. Therefore, the equalization changes and modifications made without departing from the spirit and scope of the present invention should be included in the scope of the present invention.

Claims

1. A deep learning-based intelligent diagnosis method for medical images of pulmonary nodules, characterized in that: The specific steps include: Step S1, obtaining CT image data DATA1; Step S2, segmenting the image data DATA1 through a neural network model to form segmented image data DATA2; Step S3, inputting DATA2 into the feature extraction network model, extracting features of the image data DATA2 through the feature extraction network model, and forming feature data DATA3; Step S4, outputting lung nodule classification data DATA4 according to the feature data DATA3; Step S5, generating a diagnosis report according to the data DATA1-DATA4 in steps S1-S4.

2. According to claim 1, a deep learning-based intelligent diagnosis method for medical images of pulmonary nodules, characterized in that: Also includes: Step S6, monitoring the malignant risk data RD of the pulmonary nodules with malignant risk in the diagnosis report, and predicting the growth trend of the pulmonary nodules with malignant risk according to the RD; The malignant risk data RD of pulmonary nodules include the concentration of macrophages Mc, the concentration of bacteria Bc, the cell movement speed v, the mortality rate μ1 of macrophages killed by bacteria, the mortality rate μ2 of bacteria phagocytosed by macrophages, the natural mortality rate α of macrophages, the natural reproduction rate λ of bacteria, the pressure between cells p, the diffusion coefficient of macrophages set to 1, the diffusion coefficient of bacteria 1+δ, the regional position Ω of pulmonary nodules, and the boundary position of pulmonary nodule regions. The boundary curvature κ of the lung nodule and the moving speed V of the lung nodule boundary n ; Among them, the regional position Ω of the lung nodule and the boundary position of the lung nodule region All change with time t; Preferably, the optimization of the CNN model is implemented by minimizing the loss function; wherein the method of minimizing the loss function includes a back propagation algorithm and a gradient descent method; Wherein, the loss function is selected from a cross entropy loss function.

3. According to the deep learning-based intelligent diagnosis method for pulmonary nodules in medical images of claim 1, it is characterized in that: In step S2, the neural network model is further trained; the neural network model includes a convolutional neural network model U-CNN based on a U-net network architecture and a convolutional neural network model K-CNN based on a K-net network architecture; The training of the U-CNN model specifically includes the following steps: Step S201a, obtaining CT image data D1 for model training; Step S201b, preprocessing the image data D1 to form preprocessed image data D2; Step S201c, inputting the image data D2 into the U-CNN model to form image data New-D2-1 obtained after U-CNN prediction; Step S201d, calculating the difference between D2 and New-D2-1 by using the loss function to generate loss 1; Step S201e, calculating the gradient of loss 1 relative to the U-CNN model parameters through the back propagation algorithm, and updating the U-CNN model parameters according to the optimization algorithm and the gradient; Step S201f, repeat steps S201a to S201e to form a trained U-CNN model; Preferably, in step S201e, the optimization algorithm includes a stochastic gradient descent method; the model parameters include the size of the model convolution kernel and the matrix element U-CNN model parameters; The training of the K-CNN model specifically includes the following steps: Step S202a, obtaining CT image data D3 for model training; Step S202b, preprocessing the image data D3 to form preprocessed image data D4; Step S202c, inputting the image data D4 into the K-CNN model to form image data New-D4-1 obtained after K-CNN prediction; Step S202d, calculating the difference between D4 and New-D4-1 by using the loss function to generate loss 2; Step S202e, calculating the gradient of loss 2 relative to the K-CNN model parameters through the back propagation algorithm, and updating the K-CNN model parameters according to the optimization algorithm and the gradient; Step S202f, repeating steps S202a to S202e to form a trained K-CNN model; Among them, in step S202e, the optimization algorithm includes the stochastic gradient descent method; the model parameters include the size of the model convolution kernel and the matrix element K-CNN model parameters.

4. The method for intelligent diagnosis of pulmonary nodules based on deep learning according to claim 1, characterized in that: In step S4, the feature data DATA3 is classified and processed, including using a non-parallel support vector machine to classify lung nodules; wherein the lung nodules include benign lung nodules and malignant lung nodules; The classification process also includes screening the feature data DATA3 before classification to exclude areas that meet the following conditions, which are as follows: a. If rectangularity C = B / (E·F) ≥ 1; b. If slenderness G = (min(E·F)) / (max(E·F))≤0.4; c. If the flatness H=I / J≥4; d. If the average CT value is greater than 1000; Among them, E represents the width of the region; F represents the height of the region; B represents the number of pixels in the region; I represents the major axis of the equivalent ellipse of the region; and J represents the minor axis of the equivalent ellipse of the region.

5. The method for intelligent diagnosis of pulmonary nodules based on deep learning according to claim 2, characterized in that: In step S6, the malignant risk data RD of the pulmonary nodules with malignant risk in the diagnosis report is monitored, and the growth trend of the pulmonary nodules with malignant risk is predicted according to RD, which specifically includes the following steps: Step S601, constructing a partial differential equation according to a biomedical mechanism; Step S602: Obtain malignant risk data of lung nodules with malignant risk at equal intervals of time T+1 to form RD={rd0,rd1,…,rd T }; Step S603: construct a 3D-PDE-Net network model, input RD into the 3D-PDE-Net network model, and form Step S604: construct a loss function, and optimize the 3D-PDE-Net network model by using the loss function to form an optimized 3D-PDE-Net network model; Step S605, inputting the malignant risk data of pulmonary nodules with malignant risk in the diagnosis report at any time into the optimized 3D-PDE-Net network model, and predicting the growth trend of pulmonary nodules through the optimized 3D-PDE-Net network model; Wherein, T is a positive integer.

6. The method for intelligent diagnosis of pulmonary nodules based on deep learning according to claim 5, characterized in that: In step S601, the partial differential equation is specifically expressed as follows: Among them, (x, y, z) represents the three-dimensional spatial position of the lung nodule; t represents the time variable; Ω(t) represents the regional position of the lung nodule at time t; Mc+Bc≡1, through Darcy's law The following formula can be obtained: Among them, f(Bc)=-α+(λ-μ1-μ2+α)×Bc+(μ1+μ2)×Bc 2 , and the boundary meets the following conditions: in, represents the derivative of the unit normal; β>0; γ>0; κ represents the curvature of the lung nodule boundary Ω; represents the boundary position of the lung nodule at time t; Among them, from Mc+Bc≡1, we can get: Among them, the moving speed V of the lung nodule boundary n The following conditions must be met: Among them, n represents the unit external normal; at the same time, the initialization condition of the above partial differential equation is: Mc(x,y,z,0)=Mc0(x,y,z),(x,y,z)∈Ω(0) v(x,y,z,0)=v0(x,y,z),(x,y,z)∈Ω(0) Among them, Ω(0) represents the location of the lung nodule area at t=0, M0 represents the macrophage concentration at t=0, and v0 represents the bacterial concentration at t=0.

7. The method for intelligent diagnosis of pulmonary nodules based on deep learning according to claim 5, characterized in that: In step S604, the loss function is specifically expressed as follows: Among them, time is an integer, time∈[0,T]; loss represents the loss function, which is used to evaluate the difference between the predicted output and the actual label and guide the update of the model parameters.

8. A deep learning-based intelligent medical imaging diagnostic device for pulmonary nodules, characterized in that: The modules include: An image data acquisition module, used for acquiring CT image data DATA1; An image data segmentation processing module, used for segmenting the image data DATA1 through a neural network model to form segmented image data DATA2; An image data feature extraction module is used to input DATA2 into a feature extraction network model, extract features of the image data DATA2 through the feature extraction network model, and form feature data DATA3; An image data classification processing module is used to output lung nodule classification data DATA4 according to the feature data DATA3; A report generating module, used for generating a diagnosis report according to the data DATA1-DATA4; A pulmonary nodule growth prediction module is used to monitor the malignant risk data RD of pulmonary nodules with malignant risks in the diagnosis report, and predict the growth trend of pulmonary nodules with malignant risks according to RD; The malignant risk data RD of pulmonary nodules include the concentration of macrophages Mc, the concentration of bacteria Bc, the moving speed of cells v, the mortality rate μ1 of macrophages killed by bacteria, the mortality rate μ2 of bacteria phagocytosed by macrophages, the natural mortality rate α of macrophages, the natural reproduction rate λ of bacteria, the pressure between cells p, the diffusion coefficient of macrophages set to 1, the diffusion coefficient of bacteria 1+δ, the regional position Ω of pulmonary nodules, and the boundary position of pulmonary nodule regions. The boundary curvature κ of the lung nodule and the moving speed V of the lung nodule boundary n ; Among them, the regional position Ω of the lung nodule and the boundary position of the lung nodule region All change with time t.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the deep learning-based medical imaging intelligent diagnosis method for pulmonary nodules as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a deep learning-based medical imaging intelligent diagnosis method for pulmonary nodules as described in any one of claims 1 to 7.

Images