Deep learning and radiomics fused CT image stone identification method and device
Through the integration of deep learning and radiomics, combined with three-dimensional convolutional neural network and radiomic feature extraction, the problem of insufficient subjectivity and accuracy in preoperative stone species recognition is solved, and efficient and accurate stone species recognition is achieved .
Patent Information
- Application Number
- CN202411647852.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-18
- Publication Date
- 2025-02-07
AI Technical Summary
The prior art has problems of insufficient subjectivity and accuracy in the recognition of preoperative stone types, especially under the limited data volume of stone CT images, it is difficult for deep learning models to achieve higher accuracy.
Using the method of fusion of deep learning and radiomics, deep learning features are extracted by constructing a three-dimensional convolutional neural network, and morphological, statistical and texture features are extracted using radiomics methods. After the fusion is regularized and feature selection is performed, and finally classified by LightGBM The instrument realizes the identification of stone types.
It realizes automated, objective and efficient identification of stone species, improves the accuracy and stability of stone species recognition, and reduces the recognition pressure of doctors.
Smart Images
Figure CN119399602A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a CT image stone recognition method and device integrating deep learning and radiomics, belonging to the technical field of medical image analysis, and in particular to stone CT image recognition in the field of medical image analysis, which can be applied to the diagnosis of stone patients by medical institutions and the processing of stone CT images. Background Art
[0002] Stone disease is caused by the abnormal accumulation of crystal substances (such as calcium, oxalic acid, uric acid, etc.) in the kidneys. It is a common and frequently-occurring disease of the urinary system. 40% to 75% of stone patients have different degrees of low back pain problems, and there is still a high risk of recurrence after 6 to 7 years of treatment, which has a great impact on the patients' lives. The incidence of stone disease has been increasing year by year, which has a negative impact on the patients' lives and medical resources. Therefore, studying the symptoms of stones and finding better detection and identification methods is one of the main issues in today's medical research. The treatment strategy for stone disease mainly depends on the type, size, number, and location of stones in the patient's body. Among them, the type of stone is the most important. Knowing the type of stone in the patient's body in advance can assist doctors in formulating preoperative treatment strategies, and intervene in patients from two aspects of treatment methods and lifestyle, and prevent the recurrence of stone disease in patients after surgery. For preoperative stone type identification, doctors with rich clinical experience usually use CT images and clinical pathology information to identify the type of stone in patients. Although doctors can use CT images and clinical pathological information such as the patient's age and gender to identify the type of stone before surgery, this method is subjective and the results vary from person to person. Different doctors may give different results. Therefore, studying the automatic identification algorithm of preoperative stone types can reduce the pressure on doctors, help doctors choose treatments and patients recover, and has certain scientific research value and social significance.
[0003] For the identification of preoperative stone types, doctors usually make stone type identification based on the patient's CT images and their own clinical experience. However, due to differences in clinical experience among different doctors, different stone types may be judged for the same CT image. Therefore, it is particularly important to design an objective and accurate stone type identification method. Radiomics can extract features that reflect the pathological state through in-depth analysis of CT images, thereby providing an accurate basis for the diagnosis and treatment of the disease. However, if only radiomics is used for feature extraction, the infectious and non-infectious pathological features of stones cannot be fully extracted, resulting in the inability to effectively distinguish the types of stones in subsequent experiments. On the other hand, the neural network model based on deep learning methods can automatically extract advanced pathological features from CT by mining information from medical data.
[0004] However, deep learning usually requires thousands or even tens of thousands of data to achieve better results, while the amount of data in CT images of stones is only a few hundred due to the influence of data specifications, personal privacy, differences in acquisition systems, etc., which is very limited. Therefore, it is difficult to achieve a high accuracy rate for stone type identification using a neural network model directly. The present invention makes full use of the advantages of radiomics and deep learning methods in feature extraction of CT images, and constructs a stone type identification framework combining deep learning with radiomics, so as to achieve efficient stone type identification before surgery. Summary of the invention
[0005] In order to solve the above problems, the present invention provides a CT image stone recognition method and device that integrates deep learning and radiomics, which fully utilizes the advantages of radiomics and deep learning methods in feature extraction of CT images to achieve efficient preoperative stone type recognition.
[0006] The technical solution of the present invention is: in the first aspect, a CT image stone recognition method integrating deep learning and radiomics, the method comprising:
[0007] Step 1: Build a classification neural network composed of three-dimensional convolution and use the network to extract deep learning features from calculus CT images;
[0008] Step 2: Use radiomics to extract the morphological features, first-order statistical features, and texture features of the shallow output of the neural network, and finally obtain the radiomics features;
[0009] Step 3: Fuse the radiomics features with the deep learning features in the neural network, perform regularization and LASSO feature selection on the fused features, and send the screened features to the LightGBM classifier to achieve preoperative stone type identification.
[0010] Furthermore, the Step 1 includes:
[0011] (1) The classification neural network composed of three-dimensional convolution has a total of 15 layers, referred to as a 3D convolutional neural network; the input dimension of the network is 136×118×110×1, where 136×118×110 represents the input size of each sample, and 1 represents the number of channels; after the sample input, it undergoes 1 3D convolution, 1 batch normalization, and 1 3D maximum pooling to extract shallow features in the CT image;
[0012] (2) In the process of deep learning feature extraction of images, in order to prevent the problems of gradient disappearance and gradient explosion during training, three groups of 3D residual modules were constructed; at the same time, in order to reduce the feature output dimension and improve training efficiency, a pooling layer was followed by each residual module; then the two-dimensional features were flattened into one-dimensional vector features by Flatten, which were used to match the dimension with the fully connected layer to achieve classification probability prediction; due to the excessive number of features after flattening, two fully connected layers were used for feature dimensionality reduction, outputting 64-dimensional and 512-dimensional network features respectively; in order to prevent the network from overfitting during training, after these two fully connected layers, Dropout was used to randomly drop the weights of the network to improve the generalization ability of the model; finally, after a fully connected layer and Softmax activation function, a 2×1 probability tensor was obtained to obtain the classification result; the model was trained using the binary cross entropy loss function, and the loss function is shown in formula (1):
[0013]
[0014] In the formula, n represents the number of samples, y i represents the network prediction probability, represents the true label.
[0015] The above process of obtaining deep learning features is essentially a supervised learning process, which is supervised by infectious or non-infectious label information to train a kidney stone classification network. At this point, the network has the ability to classify kidney stones. However, the network at this time only uses deep learning features to classify kidney stones, and the performance is not ideal. Therefore, the outputs of the fully connected layer 1 and the fully connected layer 2 of the 3D convolutional neural network are used as deep learning features; the deep learning feature F dl It is expressed as shown in formula (2):
[0016] F dl =Concat(F dl-1 ,F dl-2 ) (2)
[0017] in:
[0018]
[0019] Among them, Concat means concatenation, X means input sample information, 3DConNetP means the output of the average pooling layer of the 3D convolutional neural network, Fla means flattening, and FC means full connection.
[0020] Furthermore, the Step 2 includes:
[0021] The shallow feature map in the 3D convolutional neural network, that is, the output of the first pooling layer, is averaged in the channel dimension and used as the input of the radiomics method; the dimension of the shallow feature map is 57×53×66×64, 64 represents the number of channels, and after the average operation in the channel dimension, the image obtained as the input of the radiomics method has a dimension of 57×53×66;
[0022] When using the radiomics method to extract features, the input image is first subjected to wavelet transform and Laplace transform to obtain the corresponding transformed image; then, the first-order statistical features, morphological features, and texture features in the original input image, wavelet transform image, and Laplace transform image are extracted respectively; finally, the various features extracted by radiomics are spliced to obtain the radiomics feature F ra , as shown in formula (3):
[0023] F ra =Concat(F o ,F w ,F l ) (3)
[0024] Among them, F o represents the features extracted from the original image, F w represents the features extracted by wavelet transform, F l Represents the features extracted by Laplace transform, and Concat represents concatenation.
[0025] Furthermore, the Step 3 includes:
[0026] The multi-level deep learning features extracted by deep learning are fused with the radiomics features to integrate the advantages of the two feature expressions. The feature fusion method is shown in formula (4):
[0027] F co =Con(F dl ,F ra ) (4)
[0028] Among them, F co is a multidimensional feature vector containing rich information about stone types. Concat means concatenation. In addition, due to the problem of uneven numerical distribution of the fused features, regularization is performed on them, as shown in formula (5):
[0029]
[0030] in,
[0031]
[0032] in, It represents the kth feature in the fused features, n represents the total number of samples, and then LASSO is used to select the features.
[0033] In a second aspect, the present invention provides a CT image stone recognition device that integrates deep learning and radiomics, including a module for executing the method described in the first aspect.
[0034] The beneficial effects of the present invention are:
[0035] 1. The present invention constructs a complete recognition framework based on stone CT images, which can automatically and efficiently identify the types of stones.
[0036] 2. The present invention extracts the pathological features of stone CT images through radiomics and deep learning methods respectively, and fuses the radiomics features with the deep learning features to increase the richness of the features and improve the utilization rate of the features through screening; after the radiomics features and the deep learning features are fused, the stone type is identified through a classification model. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 It is a network framework diagram in the present invention;
[0038] Figure 2 Extracting features of the pooling layer for the radiomics method of the present invention;
[0039] Figure 3 ROC curve and confusion matrix for each classifier. DETAILED DESCRIPTION
[0040] The embodiments of the present invention are described below in conjunction with the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention.
[0041] It should be noted that the execution subject of the embodiments of the present invention can be various types of terminals, such as computers, smart phones, tablet computers, wearable devices, personal digital assistants (English: Personal Digital Assistant, abbreviated: PDA), mobile Internet devices (English: Mobile Internet Device, abbreviated: MID) and other terminals that can perform text processing, and the present invention does not impose any restrictions on this.
[0042] Example 1: Figure 1-Figure 2 As shown, a CT image stone recognition method integrating deep learning and radiomics, the method comprising:
[0043] Step 1, construct a classification neural network composed of three-dimensional convolution, and use the network to extract deep learning features in stone CT images; stones have different sizes and morphological structures in three-dimensional space, and each slice in a set of stone CT images is related. Using 3D convolution can extract the correlation information between slices, retain the integrity of stone information, and is conducive to extracting infectious and non-infectious features. In view of the characteristics of stone CT images, the present invention constructs a 3D convolutional neural network to extract multi-level deep learning features in CT images;
[0044] Furthermore, the Step 1 includes:
[0045] (1) The specific parameters of the 3D convolutional neural network constructed by the present invention are shown in Table 1. The classification neural network composed of three-dimensional convolution has a total of 15 layers, referred to as a 3D convolutional neural network; the input dimension of the network is 136×118×110×1, where 136×118×110 represents the input size of each sample, and 1 represents the number of channels; after the sample input, it undergoes 1 3D convolution, 1 batch normalization, and 1 3D maximum pooling to extract shallow features in CT images;
[0046] (2) In the process of deep learning feature extraction of images, in order to prevent the problems of gradient disappearance and gradient explosion during training, three groups of 3D residual modules were constructed; at the same time, in order to reduce the feature output dimension and improve training efficiency, a pooling layer was followed by each residual module; then the two-dimensional features were flattened into one-dimensional vector features by Flatten, which were used to match the dimension with the fully connected layer to achieve classification probability prediction; due to the excessive number of features after flattening, two fully connected layers were used for feature dimensionality reduction, outputting 64-dimensional and 512-dimensional network features respectively; in order to prevent the network from overfitting during training, after these two fully connected layers, Dropout was used to randomly drop the weights of the network to improve the generalization ability of the model; finally, after a fully connected layer and Softmax activation function, a 2×1 probability tensor was obtained to obtain the classification result; the model was trained using the binary cross entropy loss function, and the loss function is shown in formula (1):
[0047]
[0048] In the formula, n represents the number of samples, y i represents the network prediction probability, represents the true label.
[0049] Table 1: 3D Convolutional Neural Network Parameters
[0050]
[0051] Note: Conv3D means 3D convolution, BatchNorm means batch normalization, MaxPooling3D means 3D maximum pooling, AveragePooling3D means 3D average pooling, Flatten means flattening, and Dense means fully connected layer. The part marked with red font indicates the network layer for feature extraction.
[0052] Furthermore, as the number of network layers increases, the 3D convolutional neural network can extract high-level semantic features from CT images of stones. These features contain characteristic information for stone type identification, which can be sent to the classifier to obtain classification results. If these high-level semantic features are combined with radiomics features, the classification accuracy will be further improved. Figure 1 As shown in Figure 1, after the 3D convolutional neural network training is completed, the network has the ability to identify the types of stones. The deep semantic features in the network, that is, the outputs of the fully connected layer 1 and the fully connected layer 2 shown in Table 1, are used as deep learning features, which are later fused with the radiomics features obtained through the shallow output of the network, and the fused features are used for classification. Among them, the deep learning feature F dl It is expressed as shown in formula (2):
[0053] F dl =Concat(F dl-1 ,F dl-2 ) (2)
[0054] in:
[0055]
[0056] Among them, Concat means concatenation, X means input sample information, 3DConNetP means the output of the average pooling layer of the 3D convolutional neural network, Fla means flattening, and FC means full connection.
[0057] Step 2: Use radiomics to extract the morphological features, first-order statistical features, and texture features of the shallow output of the neural network, and finally obtain the radiomics features;
[0058] Furthermore, the Step 2 includes:
[0059] Radiomics uses automated, high-throughput feature extraction methods to transform images into feature data that can be mined. Features containing pathological information extracted by radiomics methods can be used to identify the type of stones. Wavelet transform and Laplace transform are commonly used feature extraction methods in radiomics. Wavelet transform has basically no information loss and redundancy in the process of image decomposition and reorganization, and has a relatively complete reconstruction capability. Laplace transform calculates the second-order derivative in the image space. The Laplace operator transform emphasizes areas with strong grayscale changes in the image. The greater the grayscale change, the greater the value of the image after transformation. In this text, the shallow feature map in the 3D convolutional neural network, that is, the output of the first pooling layer, is averaged in the channel dimension and used as the input of the radiomics method; among them, the dimension of the shallow feature map is 57×53×66×64, 64 represents the number of channels, and after the averaging operation in the channel dimension, the image obtained as the input of radiomics has a dimension of 57×53×66;
[0060] When using the radiomics method to extract features, the input image is first subjected to wavelet transform and Laplace transform to obtain the corresponding transformed image; then, the first-order statistical features, morphological features, and texture features in the original input image, wavelet transform image, and Laplace transform image are extracted respectively; finally, the various features extracted by radiomics are spliced to obtain the radiomics feature F ra , as shown in formula (3):
[0061] F ra =Concat(F o ,F w ,F l ) (3)
[0062] Among them, F o represents the features extracted from the original image, F w represents the features extracted by wavelet transform, F l Represents the features extracted by Laplace transform, and Concat represents concatenation.
[0063] Step 3, merge the radiomics features with the deep learning features in the neural network, perform regularization and LASSO feature selection on the fused features, and send the screened features to the LightGBM classifier to realize the identification of preoperative stone types. Fusion of deep learning features with radiomics features is a widely used method. The deep learning features and radiomics features can be combined, and then input into the feature dimension reduction algorithm for subsequent analysis. Fusion of radiomics features with deep learning features can obtain richer feature information in the image, increase the credibility of the features, and remove invalid features. After the features are selected, a model needs to be established to classify and identify the CT images of stones. The present invention uses the LightGBM classifier in the ensemble learning method to construct a stone type recognition model. The LightGBM classifier is an implementation of the Gradient Boosting Decision Tree (GBDT) algorithm, which has the advantages of fast training speed, low memory consumption, and high accuracy.
[0064] Furthermore, the Step 3 includes:
[0065] The multi-level deep learning features extracted by deep learning are fused with the radiomics features to integrate the advantages of the two feature expressions. The feature fusion method is shown in formula (4):
[0066] F co =Con(F dl ,F ra ) (4)
[0067] Among them, F co is a multidimensional feature vector containing rich information about stone types. Concat means concatenation. In addition, due to the problem of uneven numerical distribution of the fused features, regularization is performed on them, as shown in formula (5):
[0068]
[0069] in,
[0070]
[0071] in, It represents the kth feature in the fused features, n represents the total number of samples, and then LASSO is used to select the features.
[0072] The following is an embodiment of the device of the present invention. The embodiment of the device of the present invention is used to execute the method implemented by the first embodiment of the method of the present invention. For ease of explanation, only the relevant parts of the embodiment of the present invention are shown. For the specific parts not disclosed, please refer to the first embodiment of the present invention.
[0073] The embodiment of the present invention provides a CT image stone recognition device integrating deep learning and radiomics, including a module for executing the method described in the first aspect, specifically including:
[0074] Deep learning feature acquisition module: used to construct a classification neural network composed of three-dimensional convolution, and use this network to extract deep learning features from stone CT images;
[0075] Radiomics feature acquisition module: uses radiomics methods to extract the morphological features, first-order statistical features, and texture features of the shallow output of the neural network, and finally obtains the radiomics features;
[0076] Feature fusion and identification module: used to fuse radiomics features with deep learning features in neural networks, regularize and select LASSO features on the fused features, and feed the screened features into the LightGBM classifier to identify the types of stones before surgery.
[0077] In order to illustrate the effect of the present invention, the present invention conducted the following experiments for verification and obtained corresponding results; experimental process and results:
[0078] (1) Dataset used in the experiment
[0079] The data set used in the experiment of the present invention comes from the urinary tract stone disease database of the cooperative hospital. It is a proprietary data set and has not been publicly released. The data set includes 165 cases of infectious stones and 349 cases of non-infectious stones.
[0080] (2) Evaluation indicators
[0081] The present invention adopts the overall classification accuracy (OA) as the performance evaluation standard of the model, as shown in formula (6):
[0082]
[0083] In the formula, TP (True Positive) represents the total number of samples correctly classified as infectious stones by the model, FP (False Positive) represents the total number of samples incorrectly classified as infectious stones, TN (True Negative) represents the total number of samples correctly classified as non-infectious stones by the model, and FN (False Negative) represents the total number of samples incorrectly classified as non-infectious stones by the model.
[0084] In addition, the present invention also uses AUC (Area Under Curve), precision, recall and F1 score as performance indicators. Among them, AUC is related to the receiver operating characteristic curve (ROC). The ROC curve uses the true positive rate (TPR) and the false positive rate (FPR) as the horizontal and vertical coordinates to depict the performance of the classifier at different thresholds. The true positive rate and the false positive rate are calculated as shown in formula (7):
[0085]
[0086] AUC is equal to the area between the ROC curve and the coordinate axis. In addition, the calculation method of precision and recall is shown in formula (8), and the calculation method of F1 score is shown in formula (9):
[0087]
[0088]
[0089] (3) Comparative experiment of classification methods
[0090] In order to prove the effectiveness of the method proposed in the present invention, the method of the present invention is compared with other medical image classification methods on the data set of the present invention, and the results are shown in Table 2. As can be seen from the table, the OA value of the method of the present invention is 9%-16% higher than that of other classification methods. In addition, the AUC score also shows that the framework proposed in the present invention has better stability in identifying the types of kidney stones.
[0091] Table 2: Comparative experiment of classification methods
[0092]
[0093] (4) Feature selection comparison experiment
[0094] In order to verify the superiority of the feature selection method used in the present invention, the LASSO method used in the present invention is compared with different feature selection methods, and the experimental results are shown in Table 3. It can be seen that the evaluation indexes of the LASSO feature selection method are the highest. Relative to other methods, the total classification accuracy is improved by 6% to 8%. It can be seen that the LASSO method can effectively screen out the features that distinguish whether the kidney stones are infectious or not, and then improve the kidney stone type recognition accuracy.
[0095] Table 3: Feature selection comparison experiment
[0096]
[0097] (5) Classifier comparison experiment
[0098] In order to demonstrate the superiority of LightGBM in the framework proposed in this invention, this invention uses different ensemble learning classifiers for comparative experiments, and the results are shown in Table 4. It can be seen that the OA and F1 scores of the LightGBM classifier are the highest. Compared with other classifiers, the overall classification accuracy is improved by 3%-5%. This shows that for the extracted fusion features, LightGBM has better kidney stone type recognition capabilities and can effectively improve classification accuracy. The AUC score also proves that LightGBM has improved robustness and generalization capabilities compared to other classifications. As can be seen from Table 4, LightGBM has the highest F1 score among the four classifiers, indicating that when weighing the precision and recall rates, the framework proposed in this chapter uses the LightGBM classifier for the best results.
[0099] Table 4: Classifier comparison experiment
[0100]
[0101] In addition, the ROC curves and confusion matrices of each classifier are shown in Figure 3 As shown in the confusion matrix, it can be seen that for the case of correctly identifying non-infectious kidney stones, LightGBM has the highest index of 88%. Among the various classifiers, LightGBM has the lowest error rate of identifying non-infectious kidney stones as infected kidney stones, which is 12%.
[0102] (6) Ablation experiment
[0103] The present invention conducted an ablation experiment on the proposed kidney stone type recognition framework to verify the effectiveness of the deep learning combined with radiomics method. The present invention conducted a total of three groups of ablation experiments, which were respectively identified using a 3D convolutional neural network, identified using only radiomics features, and identified using only deep learning features extracted from the neural network. The experimental results are shown in Table 5. As can be seen from the table, the total accuracy of kidney stone type recognition using a 3D convolutional neural network is only 79.6%. The total accuracy is only 83.5% when only radiomics features are used for recognition or only deep learning features are used for feature recognition. After fusing the two features, the total accuracy is increased by 1% compared to using only a single feature, and the total accuracy is increased by 15% compared to using a 3D convolutional neural network for recognition. This shows that the method of deep learning combined with radiomics effectively improves the accuracy of kidney stone type recognition.
[0104] Table 5: Ablation experiment
[0105]
[0106]
[0107] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.
Claims
1. A CT image stone recognition method based on deep learning and radiomics, characterized by: The method comprises: Step 1: Build a classification neural network composed of three-dimensional convolution and use the network to extract deep learning features from calculus CT images; Step 2: Use radiomics to extract the morphological features, first-order statistical features, and texture features of the shallow output of the neural network, and finally obtain the radiomics features; Step 3: Fuse the radiomics features with the deep learning features in the neural network, perform regularization and LASSO feature selection on the fused features, and send the screened features to the LightGBM classifier to achieve preoperative stone type identification.
2. The method for identifying stones in CT images by integrating deep learning and radiomics according to claim 1, characterized in that: The Step 1 includes: (1) The classification neural network composed of three-dimensional convolution has a total of 15 layers, referred to as a 3D convolutional neural network; the input dimension of the network is 136×118×110×1, where 136×118×110 represents the input size of each sample, and 1 represents the number of channels; after the sample input, it undergoes 1 3D convolution, 1 batch normalization, and 1 3D maximum pooling to extract shallow features in the CT image; (2) In the process of image deep learning feature extraction, in order to prevent the problems of gradient disappearance and gradient explosion during training, three groups of 3D residual modules were constructed; at the same time, in order to reduce the feature output dimension and improve training efficiency, a pooling layer was followed by each residual module; then the two-dimensional features were flattened into one-dimensional vector features by Flatten, which were used to match the dimension with the fully connected layer to achieve classification probability prediction; due to the excessive number of features after flattening, two fully connected layers were used for feature dimensionality reduction, outputting 64-dimensional and 512-dimensional network features respectively; in order to prevent the network from overfitting during training, after these two fully connected layers, Dropout was used to randomly drop the network weights to improve the generalization ability of the model; finally, after a fully connected layer and Softmax activation function, a 2×1 probability tensor was obtained to obtain the classification result; the model was trained using the binary cross entropy loss function, and the loss function is shown in formula (1): In the formula, n represents the number of samples, y i represents the network prediction probability, represents the true label; The outputs of the fully connected layer 1 and the fully connected layer 2 of the 3D convolutional neural network are used as deep learning features; the deep learning feature F dl It is expressed as shown in formula (2): F dl =Concat(F dl-1 ,F dl-2 ) (2) in: Among them, Concat means concatenation, X means input sample information, 3DConNetP means the output of the average pooling layer of the 3D convolutional neural network, Fla means flattening, and FC means full connection.
3. The method for identifying stones in CT images by integrating deep learning and radiomics according to claim 1, characterized in that: The Step 2 includes: The shallow feature map in the 3D convolutional neural network, that is, the output of the first pooling layer, is averaged in the channel dimension and used as the input of the radiomics method; the dimension of the shallow feature map is 57×53×66×64, 64 represents the number of channels, and after the average operation in the channel dimension, the image obtained as the input of the radiomics method has a dimension of 57×53×66; When using the radiomics method to extract features, the input image is first subjected to wavelet transform and Laplace transform to obtain the corresponding transformed image; then, the first-order statistical features, morphological features, and texture features in the original input image, wavelet transform image, and Laplace transform image are extracted respectively; finally, the various features extracted by radiomics are spliced to obtain the radiomics feature F ra , as shown in formula (3): F ra =Concat(F o ,F w ,F l ) (3) Among them, F o represents the features extracted from the original image, F w represents the features extracted by wavelet transform, F l Represents the features extracted by Laplace transform, and Concat represents concatenation.
4. The method for identifying stones in CT images by integrating deep learning and radiomics according to claim 1, characterized in that: The Step 3 includes: The multi-level deep learning features extracted by deep learning are fused with the radiomics features to integrate the advantages of the two feature expressions. The feature fusion method is shown in formula (4): F co =Con(F dl ,F ra ) (4) Among them, F co is a multidimensional feature vector containing rich information about stone types. Concat means concatenation. In addition, due to the problem of uneven numerical distribution of the fused features, regularization is performed on them, as shown in formula (5): in, in, It represents the kth feature in the fused features, n represents the total number of samples, and then LASSO is used to select the features.
5. A CT image stone recognition device integrating deep learning and radiomics, characterized by: The method comprises a module for executing the method according to any one of claims 1 to 4.