Lung nodule benign and malignant classification method, device and equipment and medium
By fusing the characteristics of label data, numerical data and image data, using deep learning models to classify benign and malignant lung nodules, solving the problem of ignoring clinical data and imaging means in the prior art, and improving the accuracy of classification results.
Patent Information
- Application Number
- CN202411737125.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-03-21
AI Technical Summary
Existing lung cancer diagnosis methods rely on expensive imaging methods, ignoring the information in clinical symptoms, risk factors and routine test data, resulting in poor accuracy of classification results.
By obtaining the patient's label data, numerical data and image data, preprocessing and feature extraction, integrating different types of data features, and using a multi-layer fully connected network and a multi-head attention mechanism for classification, outputting benign and malignant classification results of lung nodules.
It improves the accuracy of the classification results of benign and malignant lung nodules, comprehensively considers a variety of clinical data, and enhances the reliability of diagnosis.
Smart Images
Figure CN119673472A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence processing technology, and specifically to a method, device, equipment and medium for classifying benign and malignant lung nodules. Background Art
[0002] Lung cancer is one of the most common malignant tumors in the world. Due to the atypical and lack of specificity of the early clinical symptoms of lung cancer, most patients are already in the middle or late stages when diagnosed, with multiple metastases. Existing treatments are ineffective, the tumor progresses rapidly, and the mortality rate is high. The 5-year survival rate of advanced lung cancer is only 4%-17%. Therefore, early diagnosis and treatment are crucial to improving patient survival rates.
[0003] Traditional lung cancer diagnosis methods rely on clinical symptoms, tumor markers, imaging and pathological examinations, but there are problems such as atypical early symptoms, high misdiagnosis rate, and poor compliance with invasive examinations. With the rapid development of artificial intelligence technology, deep learning algorithms are used in the diagnosis of lung cancer, but these algorithms are based on relatively expensive imaging methods, ignoring the patient's clinical symptoms, risk factors, and a large amount of information hidden in routine test data. There is no effective fusion of multi-source data, and there is a technical problem of poor accuracy of classification results. Summary of the invention
[0004] In view of the above analysis, the embodiments of the present invention aim to provide a method, device, equipment and medium for classifying benign and malignant pulmonary nodules, so as to solve one or more of the above problems existing in the prior art.
[0005] The object of the present invention is achieved in that:
[0006] A first aspect of the present invention provides a method for classifying benign and malignant pulmonary nodules, comprising:
[0007] Acquire input data of a patient, wherein the input data includes label data, numerical data, and image data;
[0008] Preprocessing and feature extraction are performed on the input data to obtain feature vectors corresponding to various types of input data;
[0009] The feature vectors corresponding to various types of input data are fused to obtain fused features;
[0010] The fusion features are input into a classification model, and a classification result of pulmonary nodules is output.
[0011] Furthermore, the preprocessing of the input data includes: binarizing the label data, normalizing the numerical data, and performing image recognition and image segmentation on the image data.
[0012] Furthermore, the feature extraction of the input data to obtain feature vectors corresponding to each type of input data includes: weighted splicing of the label data after binarization to obtain a label feature vector; splicing of the normalized numerical data to obtain a numerical feature vector; extracting shape features of the lung nodule area in the image data after image segmentation to obtain an image feature vector.
[0013] The label data after binarization is weightedly concatenated, and the feature vectors corresponding to various types of input data are fused to obtain fused features, including: using the attention mechanism to weightedly fuse the label feature vector, the numerical feature vector and the image feature vector to generate the fused feature, which is expressed as: X' r =X' b +X' s +X' t ; where X' b =Σ(α bs ·X s +α bt ·X t ), X' s =Σ(α sb ·X b +α st ·X t ), X' t =∑(α tb ·X b +α ts ·X s ), X b represents the label feature vector, X s represents the numerical feature vector, X t represents the image feature vector, α bs represents the correlation weight between the label feature vector and the numerical feature vector, α sb represents the correlation weight between the numerical feature vector and the label feature vector, α tb represents the correlation weight between the image feature vector and the label feature vector, X' b is the weighted representation of the label feature vector, X' s is the weighted representation of the numerical feature vector, X' t is the weighted representation of the image feature vector, W q represents the query weight matrix, W k Represents the key weight matrix.
[0014] Furthermore, the classification model includes: a multi-layer fully connected network, used to perform layer-by-layer nonlinear mapping on the fused features to obtain high-dimensional features; a multi-head attention mechanism, used to perform weighted calculation on each feature vector in the fused features to obtain weighted features; and a global average pooling layer, used to obtain the global features of the fused features.
[0015] Furthermore, the classification model concatenates the high-dimensional features, weighted features and global features into a joint feature vector, calculates the probability distribution of benign and malignant through a Softmax classification layer, and outputs the classification results and confidence levels of the pulmonary nodules.
[0016] Furthermore, it also includes: searching in the case database according to the fusion feature, recalling the case with the highest similarity to the fusion feature as a reference case; and adjusting the parameters of the classification model according to the probability of the classification result in the reference case.
[0017] A second aspect of the present invention provides a device for classifying benign and malignant pulmonary nodules, comprising:
[0018] An acquisition module, used for acquiring input data of a patient, wherein the input data includes label data, numerical data and image data;
[0019] A feature extraction module is used to preprocess and extract features from the input data to obtain feature vectors corresponding to various types of input data;
[0020] The feature fusion module is used to fuse the feature vectors corresponding to various types of input data to obtain fused features;
[0021] The classification module is used to input the fusion features into the classification model and output the classification results of the pulmonary nodules.
[0022] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method for classifying benign and malignant pulmonary nodules described in any embodiment is implemented.
[0023] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for classifying benign and malignant pulmonary nodules described in any one of the embodiments is implemented.
[0024] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:
[0025] The method for classifying benign and malignant lung nodules provided by the present invention integrates the features of the patient's label data, numerical data and image data, and comprehensively classifies the benign and malignant lung nodules of the patient by combining different types of clinical data, thereby improving the accuracy of the classification results. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0027] Figure 1 A flow chart of a method for classifying benign and malignant pulmonary nodules provided in Example 1 of the present invention;
[0028] Figure 2 A schematic diagram of a device for classifying benign and malignant pulmonary nodules provided in Example 2 of the present invention;
[0029] Figure 3 This is a schematic diagram of the electronic device architecture provided in Example 3 of the present invention. DETAILED DESCRIPTION
[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. It should be noted that, in the absence of conflict, the embodiments and features in the embodiments disclosed in this disclosure can be combined, separated, interchanged and / or rearranged with each other. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0031] Example 1
[0032] A specific embodiment of the present invention, as Figure 1 As shown, a method for classifying benign and malignant pulmonary nodules is disclosed, comprising the following steps:
[0033] S1. Obtain input data of a patient, wherein the input data includes label data, numerical data and image data.
[0034] In this embodiment, the label data is obtained through clinical questioning, including clinical characteristics and risk factors, and the data content is yes or no, as shown in Table 1 and Table 2:
[0035] Table 1 Clinical characteristics
[0036] Serial number Chest pain cough Coughing up blood Chest tightness hoarse 1 no yes no no no
[0037] Table 2 Risk factors
[0038] Serial number Smoking alcoholism Family history of cancer Occupational carcinogens History of chronic lung disease 1 yes yes no no no
[0039] The numerical data are data obtained from clinical tests, such as blood routine tests, tumor marker tests, CRP tests, biochemical tests, etc., as shown in Table 3:
[0040] Table 3 Tumor marker tests
[0041] Serial number NSE CEA CYFRA211 SCC PROGRP 1 9.96 1.37 1.72 1.12 42.87
[0042] The image data refers to medical imaging data, such as chest CT, MRI, etc.
[0043] S2. Preprocess and extract features of the input data to obtain feature vectors corresponding to various types of input data.
[0044] In this embodiment, step S2 specifically includes the following steps:
[0045] S201, performing binarization processing on the label data, normalization processing on the numerical data, and image recognition and image segmentation processing on the image data;
[0046] Specifically, the value of the label data is converted into a binary representation, where 0 represents yes and 1 represents no, and interpolation is used to fill in the missing data; the value of the numerical data is scaled to between [0,1], and interpolation is used to fill in the missing data; image recognition algorithms such as YOLO are used to identify lung nodules in image data, and image segmentation algorithms are used to segment the lung nodules.
[0047] S202, performing weighted concatenation on the binarized label data to obtain a label feature vector;
[0048] Specifically, different factors in the label data contribute differently to the judgment of the benign and malignant nature of nodules. The weight of each factor can be set through expert scoring, and then the label feature vector can be obtained through vectorization, providing a more accurate basis for subsequent judgment of benign and malignant nature.
[0049] S203, concatenating the normalized numerical data to obtain a numerical feature vector;
[0050] S204, extracting shape features of the lung nodule region in the image data after image segmentation processing to obtain an image feature vector.
[0051] For example, a pre-trained deep learning model can be used to extract shape features such as area, perimeter, edge irregularity, aspect ratio, texture features such as contrast, homogeneity, entropy, and density distribution of nodules in the image after image segmentation, and then spliced into an image feature vector.
[0052] S3. Fuse the feature vectors corresponding to various types of input data to obtain fused features.
[0053] In this embodiment, step S3 specifically includes:
[0054] The label feature vector, numerical feature vector and image feature vector are weightedly fused using the attention mechanism to generate the fused feature, which is expressed as:
[0055] X' r =X' b +X' s +X' t ;
[0056] Among them, X' b =Σ(α bs ·X s +α bt ·X t ), X' s =Σ(α sb ·X b +α st ·X t ), X' t =Σ(α tb ·X b +α ts ·X s ), X b represents the label feature vector, X s represents the numerical feature vector, X t represents the image feature vector, α bs represents the correlation weight between the label feature vector and the numerical feature vector, α sb represents the correlation weight between the numerical feature vector and the label feature vector, α tb represents the correlation weight between the image feature vector and the label feature vector, X' b is the weighted representation of the label feature vector, X' s is the weighted representation of the numerical feature vector, X' t is the weighted representation of the image feature vector, W q represents the query weight matrix, W k Represents the key weight matrix.
[0057] In some embodiments, the method further includes constructing a hemodynamic model based on the image data to extract biological features including blood flow impedance and blood oxygen supply rate, and adding the biological features to the image feature vector.
[0058] S4. Input the fusion features into a classification model, and output the classification results of lung nodules.
[0059] In this embodiment, the classification model includes:
[0060] A multi-layer fully connected network is used to perform layer-by-layer nonlinear mapping on the fused features to obtain high-dimensional features;
[0061] A multi-head attention mechanism is used to perform weighted calculation on each feature vector in the fusion feature to obtain a weighted feature;
[0062] The global average pooling layer is used to obtain the global features of the fused features.
[0063] The specific use process of the classification model is as follows: the classification model concatenates the high-dimensional features, weighted features and global features into a joint feature vector, calculates the probability distribution of benign and malignant through the Softmax classification layer, and outputs the classification results and confidence of the lung nodules.
[0064] Specifically, the classification model needs to be trained, and the training steps include: collecting a data set including fusion features and annotating it; initializing model parameters; inputting the data set into the model and obtaining the predicted probability through forward propagation; using the cross entropy loss function to calculate the error between the predicted probability and the true label; back-propagating the gradient of the cross entropy loss function to the model parameters; updating the model parameters using the gradient descent method; and using the test set to verify the performance of the model.
[0065] It should be noted that the classification model provided in this embodiment can be implemented using a variety of models such as Transformer, XGBoost, etc., and this embodiment does not limit this.
[0066] Preferably, a classification model is constructed based on a Multi-level Deep Evidential Fusion Algorithm.
[0067] In some embodiments, further comprising:
[0068] Searching the case database according to the fusion feature, and recalling the case with the highest similarity to the fusion feature as a reference case;
[0069] The parameters of the classification model are adjusted according to the probability of the classification results in the reference cases.
[0070] Specifically, the case database is composed of annotated patient data, including label data, numerical data, image data and their corresponding classification results. The features of each case are stored in vector form, and a retrieval index based on the feature vector is constructed; the input fusion features are standardized to have the same distribution and scale as the feature vectors in the case database, and the similarity between the fusion features and the feature vectors in the case database is calculated using cosine similarity, Euclidean distance or Mahalanobis distance, and the case with the highest similarity is retrieved, and a fixed recall number k (such as 3 or 5) is set, and the first k reference cases and their classification results are returned, and the probability distribution of benign and malignant classification results in the reference cases is summarized. For example, if 5 cases are recalled, 4 of which are benign and 1 is malignant, then the reference classification probability distribution is P (benign) = 0.8, P (malignant) = 0.2. If the error between the probability of the classification result output by the classification model and the probability distribution is greater than 20%, the parameters of the classification layer in the classification model are adjusted according to the probability distribution of the pathology database to reduce the error.
[0071] Compared with the prior art, the method for classifying benign and malignant lung nodules provided in this embodiment integrates the features of the patient's label data, numerical data, and image data, and comprehensively classifies the patient's lung nodules into benign and malignant types of clinical data, thereby improving the accuracy of the classification results.
[0072] In some embodiments, the process further includes generating treatment recommendations based on the classification results, specifically including:
[0073] Use a large language model to identify keywords in the classification results, and search the database for the pathology with the highest similarity based on the keywords. Search the drug library for drugs that treat related symptoms based on the symptoms of the pathology, extract the drug name, usage method and precautions, and output them.
[0074] Example 2
[0075] This embodiment provides a device for classifying benign and malignant pulmonary nodules. Figure 2 As shown, including:
[0076] An acquisition module, used for acquiring input data of a patient, wherein the input data includes label data, numerical data and image data;
[0077] A feature extraction module is used to preprocess and extract features from the input data to obtain feature vectors corresponding to various types of input data;
[0078] The feature fusion module is used to fuse the feature vectors corresponding to various types of input data to obtain fused features;
[0079] The classification module is used to input the fusion features into the classification model and output the classification results of the pulmonary nodules.
[0080] Example 3
[0081] This embodiment provides an electronic device, such as Figure 3 As shown, it includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the method for classifying benign and malignant pulmonary nodules as described in any of the above embodiments is implemented.
[0082] Example 4
[0083] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for classifying benign or malignant pulmonary nodules as described in any of the above embodiments is implemented.
[0084] Computer readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0085] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0086] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0087] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for classifying benign and malignant pulmonary nodules, characterized in that: include: Acquire input data of a patient, wherein the input data includes label data, numerical data, and image data; Preprocessing and feature extraction are performed on the input data to obtain feature vectors corresponding to various types of input data; The feature vectors corresponding to various types of input data are fused to obtain fused features; The fusion features are input into a classification model, and a classification result of pulmonary nodules is output.
2. The method for classifying benign and malignant pulmonary nodules according to claim 1, characterized in that: The preprocessing of the input data comprises: The label data is binarized, the numerical data is normalized, and the image data is image recognized and segmented.
3. The method for classifying benign and malignant pulmonary nodules according to claim 2, characterized in that: The feature extraction of the input data to obtain feature vectors corresponding to various types of input data includes: Perform weighted concatenation on the binary label data to obtain the label feature vector; The normalized numerical data are concatenated to obtain the numerical feature vector; The shape features of the lung nodule area in the image data after image segmentation processing are extracted to obtain the image feature vector.
4. The method for classifying benign and malignant pulmonary nodules according to claim 3, characterized in that: The feature vectors corresponding to various types of input data are fused to obtain fused features, including: The label feature vector, numerical feature vector and image feature vector are weightedly fused using the attention mechanism to generate the fused feature, which is expressed as: X' r =X' b +X' s +X' t ; Among them, X' b =∑(α bs ·X s +α bt ·X t ), X' s =Σ(α sb ·X b +α st ·X t ), X' t =∑(α tb ·X b +α ts ·X s ), X b represents the label feature vector, X s represents the numerical feature vector, X t represents the image feature vector, α bs represents the correlation weight between the label feature vector and the numerical feature vector, α sb represents the correlation weight between the numerical feature vector and the label feature vector, α tb represents the correlation weight between the image feature vector and the label feature vector, X' b is the weighted representation of the label feature vector, X' s is the weighted representation of the numerical feature vector, X' t is the weighted representation of the image feature vector, W q represents the query weight matrix, W k Represents the key weight matrix.
5. The method for classifying benign and malignant pulmonary nodules according to claim 1, characterized in that: The classification model includes: A multi-layer fully connected network is used to perform layer-by-layer nonlinear mapping on the fused features to obtain high-dimensional features; A multi-head attention mechanism is used to perform weighted calculation on each feature vector in the fusion feature to obtain a weighted feature; The global average pooling layer is used to obtain the global features of the fused features.
6. The method for classifying benign and malignant pulmonary nodules according to claim 5, characterized in that: The classification model concatenates the high-dimensional features, weighted features and global features into a joint feature vector, calculates the probability distribution of benign and malignant through a Softmax classification layer, and outputs the classification results and confidence levels of the pulmonary nodules.
7. The method for classifying benign and malignant pulmonary nodules according to any one of claims 1 to 6, characterized in that: Also includes: Searching the case database according to the fusion feature, and recalling the case with the highest similarity to the fusion feature as a reference case; The parameters of the classification model are adjusted according to the probability of the classification results in the reference cases.
8. A device for classifying benign and malignant pulmonary nodules, characterized in that: The device comprises: An acquisition module, used for acquiring input data of a patient, wherein the input data includes label data, numerical data and image data; A feature extraction module is used to preprocess and extract features from the input data to obtain feature vectors corresponding to various types of input data; The feature fusion module is used to fuse the feature vectors corresponding to various types of input data to obtain fused features; The classification module is used to input the fusion features into the classification model and output the classification results of the pulmonary nodules.
9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method for classifying benign and malignant pulmonary nodules according to any one of claims 1 to 7 is implemented.
10. A storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method for classifying benign and malignant pulmonary nodules according to any one of claims 1 to 7 is implemented.