Lung cancer targeted drug medication effect prediction method and prediction device based on convolutional neural network, and computing equipment

Abstract

The invention belongs to the technical field of medical health care information, and provides a lung cancer targeted drug use effect prediction method and prediction device based on a convolutional neural network, and a computing device, and the method comprises the steps: obtaining a CT image containing a lung tumor region, and drawing a corresponding lung tumor region segmentation annotation image, obtaining a plurality of tumor imaging features on the lung CT image; constructing a lung tumor targeted drug administration effect prediction model based on dense convolution sparse coding; and inputting the tumor iconography characteristic values based on the lung tumor targeted drug medication effect prediction model, and adjusting model training parameters to obtain a prediction result of the lung tumor targeted drug medication effect. According to the method, tumor iconography feature extraction is carried out by means of tumor iconography feature categories, features of lung tumors on all levels can be comprehensively reflected, the prediction model is constructed by combining dense connection and a multi-layer iteration soft threshold algorithm, the model interpretability is improved, and the model calculation efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the field of medical health information technology, and in particular to a method, a prediction device and a computing device for predicting the medication effect of lung cancer targeted drugs based on a convolutional neural network. Background Art

[0002] Lung cancer is a malignant tumor that poses a major threat to human health. Not only does the disease have a high morbidity and mortality rate, but it also has high treatment costs and a complicated cure process, which has caused significant economic and psychological pressure on patients and their families. Therefore, there is an urgent need for a more sophisticated and efficient diagnosis and treatment method to meet the challenges of lung cancer. Cancer has different manifestations at different stages. In the early stage of lung cancer, lung cancer cells are tightly present in the lungs, while in the later stage of lung cancer, lung cancer cells spread to other organs. Therefore, the best treatment for lung cancer at present is early detection and early treatment, which requires sufficient understanding of the formation of lung cancer, the development of lung cancer, and the treatment effect of lung cancer. Targeted drug therapy, as a more effective treatment method, is widely used in the treatment of lung cancer, especially for advanced cases. In the medical field, applying deep learning technology to the detection, diagnosis, and treatment of diseases has become a hot research topic.

[0003] In the clinical research and practice of lung cancer, accurate segmentation and diagnosis of lesions are the crucial first step. At present, researchers have made significant progress, especially in the use of deep learning technology to assist in the analysis of CT scan images, which can more accurately identify and delineate lung tumors, and play an important role in improving the accuracy and efficiency of diagnosis. The analysis and prediction of treatment mainly focuses on immunotherapy, while there is a lack of research on the efficacy of targeted drugs, and the treatment prognosis research of non-small cell lung cancer is also very limited. Treatment prognosis involves the prediction of the long-term survival rate and quality of life of the case after treatment, which is crucial for formulating treatment plans and conducting case consultations. At present, research in this area relies more on traditional clinical indicators, such as tumor size, stage, age and overall health of the case. However, with the development of intelligent medical care, more and more evidence shows that the morphological characteristics of the tumor may have an important impact on the treatment prognosis. Therefore, in order to adapt to the above research trends and solve the current practical problems encountered in clinical practice, it is urgent and necessary to seek a method for predicting the effect of lung cancer targeted drugs based on convolutional neural networks to achieve the prediction of the effect of drugs for patients who have not yet used drugs for lung cancer treatment. Summary of the invention

[0004] In view of the defects in the above-mentioned prior art, the present invention proposes a method for predicting the drug effect of lung cancer targeted drugs based on convolutional neural networks. The method includes obtaining a CT image containing a lung tumor area and drawing a corresponding segmentation and annotation image of the lung tumor area; based on the segmentation and annotation image of the lung tumor area, obtaining a number of tumor imaging features on the lung CT image; constructing a lung tumor targeted drug effect prediction model based on dense convolutional sparse coding; based on the lung tumor targeted drug effect prediction model, inputting the tumor imaging feature value, adjusting the model training parameters, and obtaining the prediction result of the lung tumor targeted drug effect. The present invention extracts tumor imaging features with the help of tumor imaging feature categories, which can comprehensively reflect the features of lung tumors at all levels. The prediction model constructed by combining dense connection and multi-layer iterative soft threshold algorithm ML-ISTA increases the model interpretability and improves the model calculation efficiency.

[0005] The present invention provides a method for predicting the efficacy of lung cancer targeted drugs based on a convolutional neural network, which comprises the following steps:

[0006] S1. Acquire a CT image containing a lung tumor region and draw a corresponding segmentation and annotation image of the lung tumor region: acquire a plurality of lung CT images containing a lung tumor region, and draw a segmentation and annotation image of the lung tumor region on the lung CT image based on medical image processing software;

[0007] S2. Based on the segmented and labeled image of the lung tumor region, a number of tumor imaging features on the lung CT image are obtained: based on the segmented and labeled image of the lung tumor region, feature extraction is performed on the corresponding lung CT image to obtain a number of tumor imaging features of the lung CT image;

[0008] S3. Construct a lung tumor targeted drug effect prediction model based on dense convolutional sparse coding: By introducing a random inactivation Dropout layer, a dense convolutional sparse coding module and an attention module into the convolutional neural network, a lung tumor targeted drug effect prediction model is constructed;

[0009] S31. Build a basic structure of a convolutional neural network, wherein the basic structure includes an initial convolutional layer, a maximum pooling layer, and a fully connected layer;

[0010] S32, introducing a random dropout layer: introducing a random dropout layer after the initial convolution layer and the fully connected layer of the basic structure;

[0011] S33, introducing a dense convolution sparse coding module: combining dense connections and a multi-layer iterative soft threshold algorithm ML-ISTA, introducing a dense convolution sparse coding module after the maximum pooling layer of the basic structure, wherein the dense convolution sparse coding module performs a sparse representation of the network features in the basic structure by the multi-layer iterative soft threshold algorithm ML-ISTA; the sparse representation is expressed as:

[0012]

[0013] Where b1 and b2 represent the bias terms of the first and second layers respectively; ReLU represents the linear rectification nonlinear function;

[0014] S34, combining the attention mechanism, introducing an attention module: introducing an attention module after the dense convolutional sparse coding module; the attention module selects a compression excitation SE module;

[0015] S35, adjusting the connection relationship between the modules according to the result of feature extraction of the lung tumor targeted drug effect prediction model, and adjusting part of the set network feature parameters;

[0016] S4, based on the lung tumor targeted drug effect prediction model, input the tumor imaging feature value, adjust the model training parameters, and obtain the prediction result of the lung tumor targeted drug effect;

[0017] The prediction result is a binary classification output. If the lung tumor targeted drug treatment is effective, the prediction result is 1, otherwise it is 0; the judgment condition for the effectiveness of the lung tumor targeted drug treatment is that the area of ​​the lung tumor region in the lung tumor region segmentation and annotation image before and after the use of the lung tumor targeted drug is reduced.

[0018] Further, the step S2 specifically includes the following steps:

[0019] S21. Setting feature parameters for extracting tumor imaging features according to lung tumor segmentation requirements and lung CT image characteristics;

[0020] S22, determining the image type of the lung CT image used for tumor imaging feature extraction;

[0021] S23, extracting several tumor imaging features of the lung CT image based on the image type and in combination with the tumor imaging feature category;

[0022] S24, respectively calculating a number of tumor imaging features of the lung CT image for each tumor imaging feature category to obtain a number of tumor imaging feature values ​​of the lung CT image;

[0023] S25, preprocessing the tumor imaging characteristic value;

[0024] S26. Perform feature selection and optimization on the tumor imaging feature values.

[0025] Preferably, the sparse representation of the network features in the basic structure by the multi-layer iterative soft threshold algorithm ML-ISTA in step S33 specifically includes the following steps:

[0026] S331. According to the sparse coding principle, the output feature map y of the dense convolutional sparse coding module is set to:

[0027] y=D1x1 (1)

[0028] Wherein, x1 represents the first sparse data; D1 represents the first dictionary;

[0029] S332, setting the feature map in the model training process to:

[0030] F(x)=f(x)+g(x) (2)

[0031] Among them, f(x) represents a convex smooth function; g(x) represents a convex non-smooth function; x represents the input feature map;

[0032] Then the gradient mapping during model training Expressed as:

[0033]

[0034] Where L represents the Lipschitz constant; represents the first approximate mapping function; represents the gradient;

[0035] S333, based on the iterative soft threshold algorithm ISTA, update with the help of the neighboring operator:

[0036]

[0037] Among them, x k+1 represents the k+1th layer feature map; x k Represents the k-th layer feature map;

[0038] S334. The optimization problem of multi-layer basis pursuit is formulated as:

[0039]

[0040] Wherein, Y represents the image to be expressed; D2 represents the second dictionary; represents the second norm; ||||1 represents the first norm; λ1 and λ2 represent the first threshold and the second threshold respectively;

[0041] This is equivalent to minimizing F(x) = f(D2x) + g1(D2x) + g2(x);

[0042] S335, approximation x1=D2x2, where x2 represents the second sparse data, and formula (4) is expressed as:

[0043]

[0044] in, Represents the second sparse data of the k+1th layer; represents the second sparse data of the kth layer; t represents the first learnable parameter; prox tg2 represents the second approximate mapping function; μ represents the second learnable parameter;

[0045] S336, when {D i} is known, assuming x 0 = 0 and 0, equation (6) is converted to:

[0046]

[0047] in, and denote the first thresholding operator and the second thresholding operator respectively; T denotes transposition;

[0048] S337, adopting the non-negativity assumption on the representation coefficients, thresholding operator Converted into a non-negative projection with offset λ, equation (7) is equivalent to equation (8).

[0049] Preferably, the expression of the compressed excitation SE module in step S34 specifically includes the following steps:

[0050] S341, given the i-th input feature map x of the compressed excitation SE module i :

[0051]

[0052] Among them, C represents the number of channels; W represents the length of the feature map; represents a real matrix;

[0053] S342. Calculate the channel attention map A with the help of two fully connected layers SE :

[0054]

[0055] Where δ represents the linear rectification ReLU nonlinear function; σ represents the hierarchical S-type Hsigmoid function; W1 and W2 represent the weights of the first fully connected layer and the second fully connected layer, respectively; z represents the statistic of the global average pooling and And the cth element z of z c It is expressed as:

[0056]

[0057] in, represents the full channel average pooling function; x i,c represents the cth channel of the ith input feature map; x i,c (j) represents the j-th pixel value of the c-th channel; W represents the number of pixels;

[0058] S343. To avoid the loss of channel information caused by dimensionality reduction, only one fully connected layer is retained to obtain the improved channel attention map A. eSE :

[0059] A eSE =σ(W3z) (12)

[0060] Among them, W3 represents the weight of the fully connected layer and

[0061] S344, the improved channel attention map A eSE The i-th input feature map x applied to the squeeze-excitation SE module i , and use channel-by-channel multiplication to obtain the i-th output feature map of the compressed excitation SE module

[0062]

[0063] Among them, A eSE,c represents the attention of the cth channel; x i,c Represents the input feature map of the c-th channel.

[0064] Preferably, the step S1 specifically includes the following steps:

[0065] S11, screening a number of lung CT images containing lung tumor areas: selecting lung CT images that are relatively clear and that meet certain resolution and quantity requirements;

[0066] S12, determining the basic range of the lung tumor area on the lung CT image: determining the basic range of the lung tumor area on the lung CT image according to the lung CT image and the diagnosis instructions of the professional physician, and verifying the accuracy of the basic range of the lung tumor area with the professional physician;

[0067] S13, segmenting and annotating the lung tumor region layer by layer on the cross section of the lung CT image to obtain a lung tumor region segmentation image corresponding to the lung CT image;

[0068] S14, saving the obtained lung tumor region segmentation image and submitting it to a professional physician for review, and modifying the lung tumor region segmentation image again according to the professional physician's opinion to obtain a lung tumor region segmentation annotated image.

[0069] Preferably, the feature parameters in step S21 include the width of the histogram bin, the Gaussian filter standard deviation sigma, the interpolator type, the pixel spacing and the voxel array offset; the image type in step S22 includes the original image, the image after Laplace-Gaussian filtering and the image after wavelet transformation; the tumor imaging feature categories in step S23 include first-order features, shape features and high-order texture features, and the high-order texture features are extracted through the gray-level co-occurrence matrix GLCM, the gray-level dependency matrix GLDM and the gray-level run length matrix GLRLM; the preprocessing in step S25 includes normalization and denoising; the feature selection and optimization in step S26 include feature dimensionality reduction and selection of the most informative tumor imaging features.

[0070] Preferably, in the process of layer-by-layer segmentation and annotation in step S13, the connection between each layer of the lung CT image needs to be considered so that a smooth transition is maintained between each layer of the obtained lung tumor region segmentation image.

[0071] Preferably, a flattening layer is provided after the attention module in step S34.

[0072] In a second aspect of the present invention, a drug effect prediction device is provided, which is used to predict the drug effect of the lung cancer targeted drug of the aforementioned lung cancer targeted drug effect prediction method based on convolutional neural network, comprising:

[0073] An image acquisition module, used for acquiring a number of lung CT images including lung tumor areas;

[0074] A segmentation and annotation module, used for drawing a segmentation and annotation image of the lung tumor region on the lung CT image based on medical image processing software;

[0075] A feature extraction module, used to extract features of the corresponding lung CT image based on the segmentation and annotation image of the lung tumor region, and obtain several tumor imaging features of the lung CT image;

[0076] The model prediction module is used to input the tumor imaging feature value based on the lung tumor targeted drug effect prediction model, adjust the model training parameters, and obtain the prediction result of the lung tumor targeted drug effect.

[0077] Another aspect of the present invention provides a computing device, comprising:

[0078] a memory configured to store computer-executable instructions;

[0079] A processor is configured to perform any of the above methods when the computer executable instructions are executed by the processor.

[0080] Compared with the prior art, the technical effects of the present invention are:

[0081] 1. The present invention designs a method and device for predicting the efficacy of targeted drugs for lung cancer based on a convolutional neural network. Based on the segmentation and annotation images of the lung tumor region and the corresponding lung CT images, tumor imaging features are extracted with the help of tumor imaging feature categories. The extracted tumor imaging features are highly representative and general, and can comprehensively reflect the characteristics of lung tumors at all levels, thereby providing rich information for the prediction model of the efficacy of targeted drugs for lung cancer. By combining dense connections and the multi-layer iterative soft threshold algorithm ML-ISTA, a prediction model for the efficacy of targeted drugs for lung tumors is constructed, which has a clear feature extraction goal, which not only increases the interpretability of the model, but also effectively improves the calculation efficiency of the model.

[0082] 2. The present invention designs a method and device for predicting the medication effect of targeted lung cancer drugs based on convolutional neural networks. Based on the specific network structure of convolutional neural networks, it provides a reference research method and idea for exploring the medication rules of targeted lung cancer drugs, which can effectively improve the computing efficiency and provide reference information on the medication of lung cancer as a medical auxiliary means. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0084] Figure 1 It is a flow chart of a method for predicting the efficacy of lung cancer targeted drugs based on a convolutional neural network according to the present invention;

[0085] Figure 2 It is a schematic diagram of a segmented and labeled image of a lung tumor region drawn on a lung CT image according to the present invention;

[0086] Figure 3 It is a schematic diagram of the dense connection principle in the dense convolutional sparse coding module of the present invention;

[0087] Figure 4 It is a schematic diagram of the network structure of the lung tumor targeted drug effect prediction model based on dense convolutional sparse coding of the present invention;

[0088] Figure 5 is a confusion matrix result diagram obtained when a prediction model is processed in a specific embodiment of the present invention;

[0089] Figure 6 It is a ROC curve diagram of the operating characteristics obtained when the prediction model is processed in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0090] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0091] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0092] Figure 1 The present invention shows a method for predicting the medication effect of lung cancer targeted drugs based on a convolutional neural network. The method is also a method for training a medication effect prediction model, which includes the following steps:

[0093] S1. Obtain a CT image containing a lung tumor area and draw a corresponding segmentation and annotation image of the lung tumor area: obtain several lung CT images containing a lung tumor area, and draw a segmentation and annotation image of the lung tumor area on the lung CT image based on the medical image processing software ITK-SNAP.

[0094] S11. Screening a number of lung CT images containing lung tumor areas: Selecting lung CT images that are relatively clear and that meet certain resolution and quantity requirements.

[0095] S12. Determine the basic range of the lung tumor area on the lung CT image: Determine the basic range of the lung tumor area on the lung CT image based on the lung CT image and the professional physician's diagnosis instructions, and verify the accuracy of the basic range of the lung tumor area with the professional physician.

[0096] S13. Segment and annotate the lung tumor region layer by layer on the cross-section of the lung CT image to obtain a lung tumor region segmentation image corresponding to the lung CT image. In the process of segmentation and annotating layer by layer, the connection between each layer of the lung CT image needs to be considered to ensure a smooth transition between each layer of the obtained lung tumor region segmentation image.

[0097] S14, saving the obtained lung tumor region segmentation image and submitting it to a professional physician for review, modifying the lung tumor region segmentation image again according to the professional physician's opinion, and obtaining a lung tumor region segmentation annotated image.

[0098] In a specific embodiment, a schematic diagram of a segmented and labeled image of a lung tumor region drawn on a lung CT image is shown in FIG. Figure 2 shown.

[0099] S2. Based on the segmentation and annotation of the lung tumor region image, several tumor imaging features on the lung CT image are obtained: Based on the segmentation and annotation of the lung tumor region image, the feature extraction of the corresponding lung CT image is performed with the help of the python third-party pyradiomics package library to obtain several tumor imaging features of the lung CT image. By enabling multiple image types and optional features, a series of tumor imaging features are obtained.

[0100] S21. According to the requirements of lung tumor segmentation and the characteristics of lung CT images, characteristic parameters for extracting tumor imaging features are set. The characteristic parameters include the width of the histogram bin, the Gaussian filter standard deviation sigma, the interpolator type, the pixel spacing, and the voxel array offset.

[0101] S22. Determine the image type of the lung CT image used for tumor imaging feature extraction, where the image type includes the original image, the image after Laplace-Gaussian filtering, and the image after wavelet transformation.

[0102] S23. Based on the image type and combined with the tumor imaging feature categories, several tumor imaging features of lung CT images are extracted; the tumor imaging feature categories include first-order features, shape features and high-order texture features. The high-order texture features are extracted through the gray-level co-occurrence matrix GLCM, the gray-level dependency matrix GLDM and the gray-level run length matrix GLRLM.

[0103] S24, calculating a number of tumor imaging features of the lung CT image for each tumor imaging feature category to obtain a number of tumor imaging feature values ​​of the lung CT image;

[0104] In a specific embodiment, three image types are selected: original image, Laplace-Gaussian filtered image, and wavelet transformed image, all tumor imaging feature categories are enabled, and 1042 tumor imaging features are extracted. In the field of medical diagnosis, a single biomarker is often difficult to provide enough information to ensure accurate diagnosis of the disease. However, when multiple biomarkers are considered comprehensively, their combined effect can significantly improve the accuracy of diagnosis.

[0105] S25. Preprocess the tumor imaging feature values, including normalization and denoising.

[0106] S26. Feature selection and optimization of tumor imaging feature values: Feature selection and optimization include feature dimensionality reduction and selection of the most informative tumor imaging features to optimize model performance and reduce computational complexity.

[0107] S3. Construct a lung tumor targeted drug effect prediction model based on dense convolutional sparse coding: By introducing the random inactivation Dropout layer, dense convolutional sparse coding module and attention module in the convolutional neural network, a lung tumor targeted drug effect prediction model is constructed. The lung tumor targeted drug effect prediction model integrates a variety of modules to improve the network effect, which can focus on important features and ensure the prediction accuracy of the lung tumor targeted drug effect prediction model.

[0108] S31. Build the basic structure of a convolutional neural network, which includes the initial convolution layer, the maximum pooling layer, and the fully connected layer.

[0109] S32. Introduce random dropout layer: A random dropout layer is introduced after the initial convolution layer and the fully connected layer of the basic structure to randomly shut down some neurons to prevent overfitting of the model.

[0110] S33, introduce dense convolution sparse coding module: Combine dense connection and multi-layer iterative soft threshold algorithm ML-ISTA, introduce dense convolution sparse coding module after the maximum pooling layer of the basic structure, and use multi-layer iterative soft threshold algorithm ML-ISTA to perform sparse representation of network features in the basic structure in the dense convolution sparse coding module, thereby enhancing the depth and representation ability of the network. Figure 3 As shown, F1, F2 and F3 respectively represent the first feature, the second feature and the third feature obtained by the dense convolutional sparse coding module, and y represents the input feature of the dense convolutional sparse coding module.

[0111] In the forward propagation of the lung tumor targeted drug effect prediction model, dense connections are combined with the multi-layer iterative soft threshold algorithm ML-ISTA to enhance the interpretability and feature extraction capabilities of the multi-layer convolutional sparse coding model. This method allows the network to increase the depth of the network by increasing the number of unfolding times while maintaining interpretability.

[0112] The sparse representation of network features in the basic structure through the multi-layer iterative soft threshold algorithm ML-ISTA specifically includes the following steps:

[0113] S331. According to the sparse coding principle, the dense convolution sparse coding module output feature map y is set to:

[0114] y=D1x1 (1)

[0115] Wherein, x1 represents the first sparse data; D1 represents the first dictionary.

[0116] S332, setting the feature map in the model training process to:

[0117] F(x)=f(x)+g(x) (2)

[0118] Among them, f(x) represents a convex smooth function; g(x) represents a convex non-smooth function; x represents the input feature map.

[0119] Then the gradient mapping during model training Expressed as:

[0120]

[0121] Where L represents the Lipschitz constant; represents the first approximate mapping function; Represents the gradient.

[0122] S333, based on the iterative soft threshold algorithm ISTA, update with the help of the neighboring operator:

[0123]

[0124] Among them, x k+1 represents the k+1th layer feature map; x k Represents the k-th layer feature map.

[0125] S334. The optimization problem of multi-layer basis pursuit is formulated as:

[0126]

[0127] Wherein, Y represents the image to be expressed; D2 represents the second dictionary; represents the second norm; || ||1 represents the first norm; λ1 and λ2 represent the first threshold and the second threshold respectively.

[0128] This is equivalent to minimizing F(x)=f(D2x)+g1(D2x)+g2(x).

[0129] However, when g(·) is a sum of composite terms, the above method cannot be applied in general, because prox g (x)Not directly divisible.

[0130] S335, adopt the approximation x1=D2x2 to circumvent the problem, x2 represents the second sparse data; D2 represents the second dictionary; then formula (4) is expressed as:

[0131]

[0132] in, Represents the second sparse data of the k+1th layer; represents the second sparse data of the kth layer; t represents the first learnable parameter; prox tg2represents the second approximate mapping function; μ represents the second learnable parameter;

[0133] S336, when {D i} is known, assuming x 0 =0 and 0, equation (6) is converted to:

[0134]

[0135] in, and denote the first thresholding operator and the second thresholding operator respectively; T denotes transpose.

[0136] S337, adopting the non-negativity assumption on the representation coefficients, thresholding operator Converted to a non-negative projection with offset λ, equation (7) is equivalent to:

[0137]

[0138] Where b1 and b2 represent the bias terms of the first and second layers, respectively. Accordingly, using the multi-layer iterative soft threshold algorithm ML-ISTA to solve the multi-layer densely connected convolutional model is equivalent to the forward pass of the convolutional neural network CNN. Sparse representation x i Corresponds to the i-th feature map in the convolutional neural network CNN. The feature extraction process of the network corresponds to the iterative process of the iterative soft threshold algorithm ISTA, and the algorithm is derived for solving the model. Therefore, the neural network constructed by the unfolding algorithm has a clear feature extraction goal. In addition to increasing interpretability, the computational efficiency can also be improved by unfolding.

[0139] S34. Combine the attention mechanism and introduce the attention module: The attention module is introduced after the dense convolution sparse coding module. The attention module selects the Squeeze-Excitation SE (Squeeze-Excitation) module to enhance the feature representation ability of the network and highlight important features. A flattening layer is set after the attention module.

[0140] The expression of the compressed excitation SE module in step S34 specifically includes the following steps:

[0141] In the squeeze-excitation SE module, the inter-channel dependencies of convolutional features are explicitly modeled, which enables the network to exploit global information to enhance those useful and informative features, thereby improving the quality of feature representation.

[0142] S341, given the i-th input feature map x of the compressed excitation SE module i :

[0143]

[0144] Among them, C represents the number of channels; W represents the length of the feature map; represents a real matrix.

[0145] S342. Calculate the channel attention map A with the help of two fully connected layers SE :

[0146]

[0147] Where δ represents the linear rectification ReLU nonlinear function; σ represents the hierarchical S-type Hsigmoid function; W1 and W2 represent the weights of the first fully connected layer and the second fully connected layer, respectively; z represents the statistic of the global average pooling and And the cth element z of z c It is expressed as:

[0148]

[0149] in, represents the full channel average pooling function; x i,c represents the cth channel of the ith input feature map; x i,c (j) represents the j-th pixel value of the c-th channel; W represents the number of pixels.

[0150] In this case, the global information is compressed into one channel descriptor.

[0151] S343. To avoid the loss of channel information caused by dimensionality reduction, only one fully connected layer is retained to obtain the improved channel attention map A. eSE :

[0152] A eSE =σ(W3z) (12)

[0153] Among them, W3 represents the weight of the fully connected layer and

[0154] S344. In order to obtain more effective information, the improved channel attention map A eSE The i-th input feature map x applied to the squeeze-excitation SE module i , and use channel-by-channel multiplication to obtain the i-th output feature map of the compressed excitation SE module

[0155]

[0156] Among them, A eSE,c represents the attention of the cth channel; x i,c Represents the input feature map of the c-th channel.

[0157] The channel attention map generated by the compression-excitation SE module is multiplied by each channel of the original feature map to modulate the feature response of the channel, so that the network can pay more attention to those feature channels that are more important to the current task.

[0158] S35. Adjust the connection relationship between the modules according to the result of feature extraction of the lung tumor targeted drug effect prediction model, and adjust some of the set network feature parameters.

[0159] In a specific embodiment, the network structure of the lung tumor targeted drug effect prediction model based on dense convolutional sparse coding is as follows: Figure 4 shown.

[0160] S4. Based on the lung tumor targeted drug effect prediction model, the tumor imaging feature values ​​are input, and the model training parameters are adjusted to obtain the prediction results of the lung tumor targeted drug effect; the prediction result is a binary classification output, if the lung tumor targeted drug treatment is effective, the prediction result is 1, otherwise it is 0; the judgment condition for the effectiveness of lung tumor targeted drug treatment is that the area of ​​the lung tumor region in the lung tumor region segmentation and annotation image before and after the use of lung tumor targeted drugs is reduced.

[0161] In a specific embodiment, the data used are all real medical clinical data with reliable sources, including basic information descriptions of the cases and corresponding lung CT images. All cases are non-small cell lung cancer NSCLC, with a total of 45 cases, and a total of 101 sets of lung CT images obtained. The targeted drugs used are osimertinib, icotinib and gefitinib.

[0162] In a specific embodiment, the lung CT images before and after the effective targeted drug treatment are marked as 1, and the two lung CT images of the ineffective cases are marked as 0, and finally 62 1 marks and 39 0 marks are obtained. 80% of the lung CT images and the corresponding prediction results are used as training sets, and 20% are used as test sets.

[0163] When training the lung tumor targeted drug effect prediction model, the confusion matrix is ​​introduced, and the training parameters are continuously adjusted to make the training results stable. The confusion matrix results are as follows: Figure 5 shown.

[0164] Use accuracy, sensitivity, specificity, and ROC curves (such as Figure 6The prediction model of the drug effect prediction of lung tumor targeted drugs is evaluated by using evaluation indicators including the following: (shown in Table 1). In a specific embodiment, the accuracy is 0.85, the sensitivity and specificity are both over 0.8, and the area under the ROC curve of the operating characteristic and the coordinate axis AUC value reaches 0.84, showing a higher accuracy than the traditional machine learning model. The test results obtained when the traditional machine learning model processes the instance are shown in Table 1. The proposed prediction model shows good prediction performance, and the proposed prediction model provides an efficient and reliable new solution for drug effect prediction.

[0165]

[0166] Table 1

[0167] The present invention also provides a drug effect prediction device, which is used to predict the drug effect of lung cancer targeted drugs in the above-mentioned lung cancer targeted drug drug effect prediction method based on convolutional neural network, including: an image acquisition module, used to acquire a number of lung CT images containing lung tumor areas; a segmentation and annotation module, used to draw on the lung CT image based on medical image processing software to obtain a segmentation and annotation image of the lung tumor area; a feature extraction module, used to extract features of the corresponding lung CT image based on the segmentation and annotation image of the lung tumor area, and obtain a number of tumor imaging features of the lung CT image;

[0168] The model prediction module is used to input the tumor imaging feature value based on the lung tumor targeted drug effect prediction model, adjust the model training parameters, and obtain the prediction result of the lung tumor targeted drug effect.

[0169] Another aspect of the present invention provides a computing device, comprising: a memory configured to store computer-executable instructions; and a processor configured to perform any one of the above methods when the computer-executable instructions are executed by the processor.

[0170] The present invention proposes a method, a prediction device and a computing device for predicting the effect of targeted drugs for lung cancer based on a convolutional neural network. Based on the segmentation and annotation images of the lung tumor region and the corresponding lung CT images, tumor imaging features are extracted with the help of tumor imaging feature categories. The extracted tumor imaging features are highly representative and general, and can comprehensively reflect the characteristics of lung tumors at all levels, thereby providing rich information for the prediction model of the effect of targeted drugs for lung cancer. By combining dense connection and multi-layer iterative soft threshold algorithm ML-ISTA, a prediction model for the effect of targeted drugs for lung tumors is constructed, which has a clear feature extraction goal, not only increases the interpretability of the model, but also effectively improves the calculation efficiency of the model. Based on the specific network structure of the convolutional neural network, a research method and idea for reference is provided for exploring the medication rules of targeted drugs for lung cancer, and the calculation efficiency can be effectively improved.

[0171] Finally, it should be noted that the above embodiments are only intended to illustrate rather than limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the present invention can still be modified or replaced by equivalents. Any modification or partial replacement that does not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for predicting the efficacy of lung cancer targeted drugs based on convolutional neural networks, characterized in that: It includes the following steps: S1. Acquire a CT image containing a lung tumor region and draw a corresponding segmentation and annotation image of the lung tumor region: acquire a plurality of lung CT images containing a lung tumor region, and draw a segmentation and annotation image of the lung tumor region on the lung CT image based on medical image processing software; S2. Based on the segmented and labeled image of the lung tumor region, a number of tumor imaging features on the lung CT image are obtained: based on the segmented and labeled image of the lung tumor region, feature extraction is performed on the corresponding lung CT image to obtain a number of tumor imaging features of the lung CT image; S3. Construct a lung tumor targeted drug effect prediction model based on dense convolutional sparse coding: By introducing a random inactivation Dropout layer, a dense convolutional sparse coding module and an attention module into the convolutional neural network, a lung tumor targeted drug effect prediction model is constructed; S31. Build a basic structure of a convolutional neural network, wherein the basic structure includes an initial convolutional layer, a maximum pooling layer, and a fully connected layer; S32, introducing a random dropout layer: introducing a random dropout layer after the initial convolution layer and the fully connected layer of the basic structure; S33, introducing a dense convolution sparse coding module: combining dense connections and a multi-layer iterative soft threshold algorithm ML-ISTA, introducing a dense convolution sparse coding module after the maximum pooling layer of the basic structure, wherein the dense convolution sparse coding module performs a sparse representation of the network features in the basic structure by the multi-layer iterative soft threshold algorithm ML-ISTA; the sparse representation is expressed as: Where b1 and b2 represent the bias terms of the first and second layers respectively; ReLU represents the linear rectification nonlinear function; S34, combining the attention mechanism, introducing an attention module: introducing an attention module after the dense convolutional sparse coding module; the attention module selects a compression excitation SE module; S35, adjusting the connection relationship between the modules according to the result of feature extraction of the lung tumor targeted drug effect prediction model, and adjusting part of the set network feature parameters; S4, based on the lung tumor targeted drug effect prediction model, input the tumor imaging feature value, adjust the model training parameters, and obtain the prediction result of the lung tumor targeted drug effect; The prediction result is a binary classification output. If the lung tumor targeted drug treatment is effective, the prediction result is 1, otherwise it is 0; the judgment condition for the effectiveness of the lung tumor targeted drug treatment is that the area of ​​the lung tumor region in the lung tumor region segmentation and annotation image before and after the use of the lung tumor targeted drug is reduced.

2. The method for predicting the efficacy of targeted drugs for lung cancer based on convolutional neural networks according to claim 1, characterized in that: The step S2 specifically includes the following steps: S21. Setting feature parameters for extracting tumor imaging features according to lung tumor segmentation requirements and lung CT image characteristics; S22, determining the image type of the lung CT image used for tumor imaging feature extraction; S23, extracting several tumor imaging features of the lung CT image based on the image type and in combination with the tumor imaging feature category; S24, respectively calculating a number of tumor imaging features of the lung CT image for each tumor imaging feature category to obtain a number of tumor imaging feature values ​​of the lung CT image; S25, preprocessing the tumor imaging characteristic value; S26. Perform feature selection and optimization on the tumor imaging feature values.

3. The method for predicting the efficacy of targeted drugs for lung cancer based on convolutional neural networks according to claim 1, characterized in that: In step S33, the sparse representation of the network features in the basic structure is performed by the multi-layer iterative soft threshold algorithm ML-ISTA, which specifically includes the following steps: S331. According to the sparse coding principle, the output feature map y of the dense convolutional sparse coding module is set to: y=D1x1 (1) Wherein, x1 represents the first sparse data; D1 represents the first dictionary; S332, setting the feature map in the model training process to: F(x)=f(x)+g(x) (2) Among them, f(x) represents a convex smooth function; g(x) represents a convex non-smooth function; x represents the input feature map; Then the gradient mapping during model training Expressed as: Where L represents the Lipschitz constant; represents the first approximate mapping function; represents the gradient; S333, based on the iterative soft threshold algorithm ISTA, update with the help of the neighboring operator: Among them, x k+1 represents the k+1th layer feature map; x k Represents the k-th layer feature map; S334. The optimization problem of multi-layer basis pursuit is formulated as: Wherein, Y represents the image to be expressed; D2 represents the second dictionary; represents the second norm; || ||1 represents the first norm; λ1 and λ2 represent the first threshold and the second threshold respectively; This is equivalent to minimizing F(x) = f(D2x) + g1(D2x) + g2(x); S335, approximation x1=D2x2, where x2 represents the second sparse data, and formula (4) is expressed as: in, Represents the second sparse data of the k+1th layer; represents the second sparse data of the kth layer; t represents the first learnable parameter; prox tg2 represents the second approximate mapping function; μ represents the second learnable parameter; S336, when {D i } is known, assuming x 0 =0 and 0, equation (6) is transformed into equation (7): in, and denote the first thresholding operator and the second thresholding operator respectively; T denotes transposition; S337, adopting the non-negativity assumption on the representation coefficients, thresholding operator Converted into a non-negative projection with offset λ, equation (7) is equivalent to equation (8).

4. The method for predicting the efficacy of targeted drugs for lung cancer based on convolutional neural networks according to claim 1, characterized in that: The expression of the compressed excitation SE module in step S34 specifically includes the following steps: S341, given the i-th input feature map x of the compressed excitation SE module i : Among them, C represents the number of channels; W represents the length of the feature map; represents a real matrix; S342. Calculate the channel attention map A with the help of two fully connected layers SE : Where δ represents the linear rectification ReLU nonlinear function; σ represents the hierarchical S-type Hsigmoid function; W1 and W2 represent the weights of the first fully connected layer and the second fully connected layer, respectively; z represents the statistic of the global average pooling and And the cth element z of z c It is expressed as: in, represents the full channel average pooling function; x i,c represents the cth channel of the ith input feature map; x i,c (j) represents the j-th pixel value of the c-th channel; W represents the number of pixels; S343. To avoid the loss of channel information caused by dimensionality reduction, only one fully connected layer is retained to obtain the improved channel attention map A. eSE : A eSE =σ(W3z) (12) Among them, W3 represents the weight of the fully connected layer and S344, the improved channel attention map A eSE The i-th input feature map x applied to the squeeze-excitation SE module i , and use channel-by-channel multiplication to obtain the i-th output feature map of the compressed excitation SE module Among them, A eSE,c represents the attention of the cth channel; x i,c Represents the input feature map of the c-th channel.

5. The method for predicting the efficacy of lung cancer targeted drugs based on convolutional neural networks according to claim 1, characterized in that: The step S1 specifically includes the following steps: S11, screening a number of lung CT images containing lung tumor areas: selecting lung CT images that are relatively clear and that meet certain resolution and quantity requirements; S12, determining the basic range of the lung tumor area on the lung CT image: determining the basic range of the lung tumor area on the lung CT image according to the lung CT image and the diagnosis instructions of the professional physician, and verifying the accuracy of the basic range of the lung tumor area with the professional physician; S13, segmenting and annotating the lung tumor region layer by layer on the cross section of the lung CT image to obtain a lung tumor region segmentation image corresponding to the lung CT image; S14, saving the obtained lung tumor region segmentation image and submitting it to a professional physician for review, and modifying the lung tumor region segmentation image again according to the professional physician's opinion to obtain a lung tumor region segmentation annotated image.

6. The method for predicting the efficacy of lung cancer targeted drugs based on convolutional neural networks according to claim 1 or 2, characterized in that: The feature parameters in step S21 include the width of the histogram bin, the Gaussian filter standard deviation sigma, the interpolator type, the pixel spacing and the voxel array offset; the image types in step S22 include the original image, the image after Laplace-Gaussian filtering and the image after wavelet transformation; the tumor imaging feature categories in step S23 include first-order features, shape features and high-order texture features, and the high-order texture features are extracted through the gray-level co-occurrence matrix GLCM, the gray-level dependency matrix GLDM and the gray-level run length matrix GLRLM; the preprocessing in step S25 includes normalization and denoising; the feature selection and optimization in step S26 include feature dimensionality reduction and selection of the most informative tumor imaging features.

7. The method for predicting the efficacy of lung cancer targeted drugs based on convolutional neural networks according to claim 1 or 5, characterized in that: In the process of layer-by-layer segmentation and annotation in step S13, the connection between each layer of the lung CT image needs to be considered so that a smooth transition is maintained between each layer of the obtained lung tumor region segmentation image.

8. The method for predicting the efficacy of targeted drugs for lung cancer based on convolutional neural networks according to claim 1, characterized in that: A flattening layer is provided after the attention module in step S34.

9. A drug effect prediction device, used for predicting the drug effect of lung cancer targeted drugs according to the method for predicting the drug effect of lung cancer targeted drugs based on convolutional neural network according to any one of claims 1 to 8, characterized in that: include: An image acquisition module, used for acquiring a number of lung CT images including lung tumor areas; A segmentation and annotation module, used for drawing a segmentation and annotation image of the lung tumor region on the lung CT image based on medical image processing software; A feature extraction module, used to extract features of the corresponding lung CT image based on the segmentation and annotation image of the lung tumor region, and obtain several tumor imaging features of the lung CT image; The model prediction module is used to input the tumor imaging feature value based on the lung tumor targeted drug effect prediction model, adjust the model training parameters, and obtain the prediction result of the lung tumor targeted drug effect.

10. A computing device, characterized in that: include: a memory configured to store computer-executable instructions; A processor, which is configured to perform the method according to any one of claims 1 to 8 when the computer executable instructions are executed by the processor.

Images