A benign and malignant pulmonary lesion differential detection method based on a double-branch deep learning

Abstract

The application relates to the field of medical image processing and artificial intelligence technology, and discloses a benign and malignant lung lesion differential detection method based on a double-branch deep learning method, which comprises the following steps: S1: obtaining an original chest CT image, performing standardization preprocessing on the original chest CT image, and generating an original full-size image and a lesion local image; S2: inputting the original full-size image into a first feature extraction branch to extract global context semantic features and spatial distribution features; and S3: inputting the lesion local image into a second feature extraction branch to extract fine texture, density and morphological features. The benign and malignant lung lesion differential detection method based on the double-branch deep learning method, the first feature extraction branch processes the original full-size image to extract the global context semantic features and the spatial distribution features, the second feature extraction branch processes the automatically segmented lesion local image to extract the fine texture, the density and the morphological features, and the two branches are operated in parallel to fully exert the advantages of different models.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence technology, specifically to a method for differentiating between benign and malignant lung lesions based on bibranch deep learning. Background Technology

[0002] Lung cancer is one of the malignant tumors with the highest incidence and mortality rates worldwide. Early detection and accurate diagnosis are key to improving the survival rate of lung cancer patients. Chest CT scans are currently the main means of lung cancer screening and diagnosis, which can clearly show the location, size and morphological characteristics of lung lesions. However, manual interpretation of images has problems such as strong subjectivity, large workload, and easy to miss or misdiagnose, making it difficult to meet the needs of large-scale lung cancer screening.

[0003] In recent years, deep learning technology has made significant progress in the field of medical image analysis, providing new solutions for the automatic identification and detection of lung lesions. Existing technologies mainly use a single deep learning model (such as DenseNet, ResNet, U-Net, V-Net) or a simple multi-model fusion method to distinguish between benign and malignant lung lesions. Some studies have improved the performance of a single model by introducing attention mechanisms, residual connections, etc., while other studies have averaged or voted on the output results of multiple models to obtain the final classification result.

[0004] However, existing technologies still face three pressing problems in practical applications:

[0005] First, a single model struggles to simultaneously capture both global contextual information and fine local features. CNN-based models excel at extracting local texture features but lack the ability to model global spatial dependencies. Transformer-based models are adept at extracting global contextual features but are weak at capturing local detail features.

[0006] Second, simple multi-model fusion methods lack effective feature interaction mechanisms; most existing technologies adopt result-level fusion (such as averaging, voting) or simple feature splicing, which cannot make full use of the complementarity between features of different models, resulting in limited improvement in classification performance.

[0007] Third, existing models generally suffer from a large number of parameters and slow inference speed. In order to improve accuracy, existing models often use complex network structures and a large number of parameters, making it difficult to achieve real-time assisted diagnosis in clinical settings.

[0008] Therefore, a method for differentiating between benign and malignant lung lesions based on bibranch deep learning is proposed. Summary of the Invention

[0009] (a) Technical problems to be solved

[0010] To address the shortcomings of existing technologies, this invention provides a method for differentiating between benign and malignant lung lesions based on bi-branch deep learning. This method has advantages such as taking into account both global and local features, strong feature fusion interactivity, and fast inference speed. It solves the problems that a single model cannot simultaneously extract global context and local fine texture, that simple fusion lacks feature complementarity leading to low classification performance, and that the large number of model parameters makes it difficult to meet the needs of real-time clinical diagnosis.

[0011] (II) Technical Solution

[0012] To achieve the aforementioned goals of considering both global and local features, strong feature fusion interactivity, and fast inference speed, this invention provides the following technical solution: a method for differentiating between benign and malignant lung lesions based on bi-branch deep learning, comprising the following steps:

[0013] S1: Acquire the original chest CT image, perform standardized preprocessing on it, and generate the original full-size image and local image of the lesion;

[0014] S2: Input the original full-size image into the first feature extraction branch. The first feature extraction branch is a 3D SwinTransformer model, which extracts global contextual semantic features and spatial distribution features.

[0015] S3: Input the local image of the lesion into the second feature extraction branch. The second feature extraction branch is a lightweight convolutional neural network based on a three-dimensional residual structure to extract fine texture, density and morphological features.

[0016] S4: Through attention-guided cross-branch fusion units, global and local features are adaptively weighted and fused to generate a fused feature vector. The attention-guided cross-branch fusion unit includes channel attention sub-units, spatial attention sub-units, and feature splicing layers in sequence.

[0017] S5: The model is trained end-to-end using a joint loss function consisting of cross-entropy loss function and center loss function;

[0018] S6: Input the fused feature vector into the classification and identification unit, and output the classification result and confidence level of the benign or malignant lung lesions.

[0019] Preferably, the standardization preprocessing steps in step S1 include:

[0020] Step 1: Resample the original CT images and unify the isotropic voxel spacing of all images to 1.0mm×1.0mm×1.0mm;

[0021] Step 2: Use threshold segmentation to extract the lung parenchyma region and remove the background and bone tissue outside the thoracic cavity;

[0022] Step 3: Use a pre-trained 3D U-Net model to automatically segment lesions in the lung parenchyma region and generate a segmentation mask;

[0023] Step 4: Based on the segmentation mask, crop out the local image patch containing the complete lesion. The size of the local image patch is 64×64×64 voxels.

[0024] Step 5: Perform Z-score normalization on the original full-size image and the local image of the lesion, respectively.

[0025] Preferably, the first feature extraction branch in step S2 specifically includes an input layer, four cascaded Transformer blocks, and a global average pooling layer;

[0026] The input layer receives a raw full-size CT image with a size of 224×224×224; the four cascaded Transformer blocks each contain a multi-head self-attention layer and a feedforward neural network layer; the global average pooling layer converts the feature map output by the last Transformer block into a global feature vector with a dimension of 1024; the multi-head self-attention layer adopts a window self-attention mechanism to divide the feature map into non-overlapping windows and calculate self-attention within each window; the feedforward neural network layer consists of two fully connected layers and a Gaussian error linear unit (GELU) activation function.

[0027] Preferably, the second feature extraction branch in step S3 specifically includes an input layer, five convolutional blocks, and a global average pooling layer;

[0028] The input layer receives a local image of the lesion with a size of 64×64×64; each of the five convolutional blocks contains two 3×3×3 convolutional layers and a residual connection; the global average pooling layer converts the feature map output by the last convolutional block into a local feature vector with a dimension of 512; the convolutional layers employ three-dimensional depthwise separable convolution, decomposing standard convolution into depthwise convolution and pointwise convolution.

[0029] Preferably, the channel attention subunit in step S4 performs adaptive weighting of the global feature vector and the local feature vector according to the channel dimension to generate weighted global features and weighted local features.

[0030] The spatial attention subunit adaptively weights the weighted global features and weighted local features according to the spatial dimension to generate spatially enhanced global features and spatially enhanced local features;

[0031] The feature splicing layer splices spatially enhanced global features and spatially enhanced local features along the channel dimension to generate a fused feature vector with a dimension of 1536.

[0032] .

[0033] Preferably, the classification and identification unit described in step S6 includes two fully connected layers and a Softmax normalized activation layer;

[0034] The first fully connected layer maps the fused feature vector of dimension 1536 to a feature vector of dimension 256; the second fully connected layer maps the feature vector of dimension 256 to an output vector of dimension 2; the Softmax normalized activation layer normalizes the output vector and outputs the probability values ​​of whether the lung lesions are benign or malignant.

[0035] Preferably, the joint loss function described in step S5 consists of a cross-entropy loss function and a center loss function:

[0036] .

[0037] A system for differentiating between benign and malignant lung lesions based on bibranch deep learning includes a data preprocessing unit, a bibranch feature extraction unit, an attention-guided cross-branch fusion unit, and a classification and identification unit connected in sequence.

[0038] The data preprocessing unit is used to standardize the original chest CT images and output the original full-size image and the automatically segmented local image of the lesion.

[0039] The dual-branch feature extraction unit includes a first feature extraction branch and a second feature extraction branch set in parallel. The first feature extraction branch is a 3D Swin Transformer model that receives the original full-size image and extracts the global contextual semantic features and spatial distribution features of the lesion. The second feature extraction branch is a lightweight convolutional neural network based on a three-dimensional residual structure that receives a local image of the lesion and extracts the fine texture, density and morphological features of the lesion.

[0040] The attention-guided cross-branch fusion unit sequentially includes a channel attention subunit, a spatial attention subunit, and a feature splicing layer, which is used to adaptively weight and fuse the global features output by the first feature extraction branch and the local features output by the second feature extraction branch to generate a fused feature vector.

[0041] The classification and identification unit receives the fused feature vector and outputs the classification result and confidence level of the benign or malignant lung lesions.

[0042] (III) Beneficial Effects

[0043] Compared with existing technologies, this invention provides a method for differentiating between benign and malignant lung lesions based on bibranch deep learning, which has the following beneficial effects:

[0044] 1. This method for differentiating between benign and malignant lung lesions based on dual-branch deep learning achieves differentiated extraction of global and local features through a dual-branch feature extraction unit. The first feature extraction branch processes the original full-size image to extract global contextual semantic features and spatial distribution features, while the second feature extraction branch processes the automatically segmented local image of the lesion to extract fine texture density and morphological features. The two branches run in parallel to give full play to the advantages of different models and effectively solve the technical problem that a single model cannot simultaneously take into account both global and local features.

[0045] 2. This method for identifying benign and malignant lung lesions based on bi-branch deep learning achieves adaptive weighted fusion of features through attention-guided cross-branch fusion units. The channel attention subunit adaptively weights the features of the two branches according to the channel dimension, and the spatial attention subunit adaptively weights the weighted features according to the spatial dimension. Finally, the feature splicing layer completes the feature splicing, fully exploring the complementarity between different features and solving the problem of simple fusion lacking effective interaction.

[0046] 3. This method for differentiating between benign and malignant lung lesions based on dual-branch deep learning improves the model's inference speed through a lightweight network design. The second feature extraction branch uses three-dimensional depthwise separable convolution to replace traditional convolution, which significantly reduces the number of model parameters and computational complexity. At the same time, the dual-branch architecture avoids excessively increasing the network depth and width in pursuit of accuracy in a single model, so that the model's inference speed can meet the needs of real-time clinical auxiliary diagnosis. Attached Figure Description

[0047] Figure 1 This is a flowchart of the method for differentiating between benign and malignant lung lesions according to the present invention;

[0048] Figure 2 This is a diagram showing the overall architecture of the benign and malignant lung lesion differentiation and detection system of the present invention.

[0049] Figure 3 This is a structural block diagram of the attention-guided cross-branch fusion unit of the present invention. Detailed Implementation

[0050] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Example 1:

[0052] This embodiment provides a complete flowchart of a method for differentiating between benign and malignant lung lesions based on dual-branch deep learning, such as... Figure 1 As shown; the method includes the following steps:

[0053] S1: Acquire the original chest CT image, perform standardized preprocessing on it, and generate the original full-size image and local image of the lesion;

[0054] S2: Input the original full-size image into the first feature extraction branch. The first feature extraction branch is a 3D SwinTransformer model, which extracts global contextual semantic features and spatial distribution features.

[0055] S3: Input the local image of the lesion into the second feature extraction branch. The second feature extraction branch is a lightweight convolutional neural network based on a three-dimensional residual structure to extract fine texture, density and morphological features.

[0056] S4: Through attention-guided cross-branch fusion units, global and local features are adaptively weighted and fused to generate a fused feature vector. The attention-guided cross-branch fusion unit includes channel attention sub-units, spatial attention sub-units, and feature splicing layers in sequence.

[0057] S5: The model is trained end-to-end using a joint loss function consisting of cross-entropy loss function and center loss function;

[0058] S6: Input the fused feature vector into the classification and identification unit, and output the classification result and confidence level of the benign or malignant lung lesions.

[0059] In this embodiment, the execution subject of the method is a computer device, including but not limited to servers, workstations, and personal computers; the original chest CT images can come from the hospital's PACS system or from publicly available medical image datasets; the training process of the method is carried out in an end-to-end manner, and the training dataset uses the LIDC-IDRI public dataset, which contains 1018 chest CT scan images with a total of 2669 lung lesions annotated; the dataset is randomly divided into training set, validation set, and test set in a ratio of 7:1:2; the Adam optimizer is used during training, with an initial learning rate of 0.0001, a batch size of 16, and 100 training epochs; training is stopped when the validation set loss does not decrease for 10 consecutive epochs.

[0060] Example 2:

[0061] This embodiment, based on Embodiment 1, further describes the specific implementation of the standardization preprocessing described in step S1; the standardization preprocessing steps are as follows:

[0062] Step 1: Resampling; Perform linear interpolation resampling on the original CT images to unify the isotropic voxel spacing of all images to 1.0mm×1.0mm×1.0mm, eliminating the influence of differences in scanning parameters from different devices; Keep the original orientation of the image unchanged during the resampling process to avoid feature loss due to image rotation;

[0063] Step 2: Lung parenchyma extraction; using threshold segmentation, the region with CT values ​​between -1000HU and -200HU is identified as lung parenchyma, and irrelevant tissues such as the background outside the thoracic cavity, bones, and heart are removed; then, morphological opening and closing operations are used to remove small noise and cavities within the lung parenchyma region to obtain a complete lung parenchyma mask.

[0064] Step 3: Automatic lesion segmentation; The pre-trained 3D U-Net model is used to automatically segment lesions in the lung parenchyma region and generate a segmentation mask; The 3D U-Net model is pre-trained using the standard medical image segmentation dataset LITS, and its specific structure is existing technology and will not be described in detail here; During the segmentation process, the lung parenchyma image is input into the pre-trained 3D U-Net model, and the output is a binary segmentation mask with the same size as the input image, where 1 represents the lesion region and 0 represents the background region;

[0065] Step 4: Lesion cropping; Determine the minimum bounding cube of the lesion based on the segmentation mask, and crop out a local image patch containing the complete lesion with an edge extension of 10 voxels; The edge extension of 10 voxels is to preserve the microenvironment information around the lesion, which is of great significance for the differentiation between benign and malignant lesions; Adjust all local image patches to a uniform size of 64×64×64 voxels to ensure that the image size input to the second feature extraction branch is consistent;

[0066] Step 5: Normalization; Perform Z-score normalization on both the original full-size image and the local lesion image, converting the pixel values ​​of the image into a distribution with a mean of 0 and a standard deviation of 1; the formula for calculating Z-score normalization is: ,in These are the original pixel values. The mean of the image. The standard deviation of the image is denoted as . Normalization can accelerate the convergence speed of the model and improve its stability.

[0067] Example 3:

[0068] This embodiment, based on Embodiment 1, further describes the specific implementation of the dual-branch feature extraction unit. The dual-branch feature extraction unit includes a first feature extraction branch and a second feature extraction branch that are set in parallel.

[0069] The first feature extraction branch is a 3D Swin Transformer model, specifically consisting of an input layer, four cascaded Transformer blocks, and a global average pooling layer. The input layer receives a raw full-size CT image with a size of 224×224×224. The four cascaded Transformer blocks sequentially extract features from the input image. Each Transformer block contains a multi-head self-attention layer and a feedforward neural network layer. The multi-head self-attention layer uses a window self-attention mechanism to divide the feature map into non-overlapping windows and calculates self-attention within each window, which effectively reduces computational complexity. The feedforward neural network layer consists of two fully connected layers and a Gaussian error linear unit (GELU) activation function. The first fully connected layer expands the feature dimension by 4 times, and the second fully connected layer restores the feature dimension to its original size. The feature map output by the last Transformer block has a size of 7×7×7×1024. After passing through the global average pooling layer, it is converted into a global feature vector with a dimension of 1024. This global feature vector contains global contextual semantic features such as the spatial distribution information of lesions throughout the lung and their relationship with surrounding tissues.

[0070] The second feature extraction branch is a lightweight convolutional neural network based on a three-dimensional residual structure, specifically including an input layer, five convolutional blocks, and a global average pooling layer. The input layer receives a 64×64×64 local image of the lesion. The five convolutional blocks sequentially extract features from the input image. Each convolutional block contains two 3×3×3 convolutional layers and a residual connection. The convolutional layers use three-dimensional depthwise separable convolution, decomposing standard convolution into depthwise convolution and pointwise convolution, which can significantly reduce the number of model parameters and computational complexity. The residual connection can effectively alleviate the gradient vanishing problem during the training process of deep neural networks and improve the training effect of the model. The feature map output by the last convolutional block has a size of 4×4×4×512. After passing through the global average pooling layer, it is converted into a local feature vector with a dimension of 512. This local feature vector contains local features such as the fine texture, density distribution, and edge morphology of the lesion. These features are crucial for distinguishing between benign and malignant lesions.

[0071] Example 4:

[0072] This embodiment, based on Embodiment 1, describes in detail the structure and working principle of the attention-guided cross-branch fusion unit described in step S4, such as... Figure 3 As shown, the mechanism includes a channel attention subunit, a spatial attention subunit, and a feature splicing layer.

[0073] The channel attention subunit adaptively weights the global and local feature vectors according to the channel dimension. Its core idea is that different feature channels contain different information and have different importance for the task of distinguishing between benign and malignant features. The channel attention subunit can automatically learn the weight of each channel, enhance the features of important channels, and suppress the features of unimportant channels.

[0074] The calculation process for the channel attention subunit is shown in the following formula:

[0075] .

[0076] Specifically, for the input feature vector, average pooling and max pooling operations are first performed respectively to obtain two different pooled feature vectors; then these two pooled feature vectors are input into a shared multilayer perceptron to obtain two different attention weight vectors; after adding these two attention weight vectors, the final channel attention weights are obtained by passing the sigmoid activation function; finally, the input feature vector is multiplied by the channel attention weights to obtain the weighted feature vector.

[0077] The spatial attention subunit adaptively weights the weighted global features and weighted local features according to the spatial dimension. Its core idea is that different spatial locations of the feature map contain different information, and the features of the lesion area are more important than the features of the background area. The spatial attention subunit can automatically learn the weight of each spatial location, enhance the features of the lesion area, and suppress the features of the background area.

[0078] The calculation process for the spatial attention sub-unit is shown in the following formula:

[0079] .

[0080] Specifically, for the input weighted feature vector, average pooling and max pooling operations are first performed on the channel dimension to obtain two different spatial feature maps. Then, these two spatial feature maps are concatenated on the channel dimension to obtain a dual-channel feature map. This dual-channel feature map is then input into a 7×7×7 3D convolutional layer to obtain a single-channel spatial attention weight map. After normalizing the spatial attention weight map using the sigmoid activation function, it is multiplied with the input weighted feature vector to obtain the spatially enhanced feature vector.

[0081] The feature concatenation layer concatenates spatially enhanced global features and spatially enhanced local features along the channel dimension to generate a fused feature vector with a dimension of 1536. This fused feature vector contains both global contextual information and local fine features, and is adaptively weighted through an attention mechanism, which can provide a more comprehensive and effective feature representation for subsequent classification and identification.

[0082] Example 5:

[0083] This embodiment, based on Embodiment 1, further describes the structure and training process of the classification and identification unit.

[0084] The classification and identification unit consists of two fully connected layers and a Softmax normalized activation layer. The first fully connected layer maps the fused feature vector of dimension 1536 to a feature vector of dimension 256, and the second fully connected layer maps the feature vector of dimension 256 to an output vector of dimension 2. The Softmax normalized activation layer normalizes the output vector and outputs the probability values ​​of whether the lung lesions are benign or malignant. .

[0085] This invention employs a joint loss function for training, which consists of a cross-entropy loss function and a center loss function, as shown in the following formula:

[0086] .

[0087] The cross-entropy loss function is used to measure the classification error of the model, and its calculation formula is as follows:

[0088] .

[0089] The center loss function is used to enhance the discriminative power of features, and its calculation formula is as follows:

[0090] .

[0091] Optimizing both cross-entropy loss and center loss during training not only improves the model's classification accuracy but also enhances the discriminative power of features, thereby improving the model's robustness and generalization ability.

[0092] Example 6:

[0093] This embodiment provides an overall architecture for a benign and malignant lung lesion differentiation and detection system based on dual-branch deep learning, such as... Figure 2 As shown, the system includes a data preprocessing unit, a dual-branch feature extraction unit, an attention-guided cross-branch fusion unit, and a classification and identification unit connected in sequence.

[0094] The data preprocessing unit is used to standardize the original chest CT images and output the original full-size image and automatically segmented local images of the lesions; the specific processing steps are as described in Example 2.

[0095] The dual-branch feature extraction unit includes a first feature extraction branch and a second feature extraction branch set in parallel. The first feature extraction branch is a 3D Swin Transformer model that receives the original full-size image and extracts the global contextual semantic features and spatial distribution features of the lesion. The second feature extraction branch is a lightweight convolutional neural network based on a three-dimensional residual structure that receives the local image of the lesion and extracts the fine texture, density, and morphological features of the lesion. Its specific implementation is as described in Example 3.

[0096] The attention-guided cross-branch fusion unit sequentially includes a channel attention subunit, a spatial attention subunit, and a feature concatenation layer, which is used to adaptively weight and fuse the global features output by the first feature extraction branch and the local features output by the second feature extraction branch to generate a fused feature vector; its specific structure and working principle are as described in Example 4.

[0097] The classification and identification unit receives the fused feature vector and outputs the classification result and confidence level of the benign or malignant lung lesions; its specific structure and training process are as described in Example 5.

[0098] Example 7:

[0099] This embodiment provides ablation experimental data and comparative experimental results to demonstrate the technical effectiveness of the present invention.

[0100] To verify the effectiveness of the various technical features in this invention, the following ablation experiment was designed:

[0101] Experiment A: Classification using only the first feature extraction branch (Transformer model based on shift window self-attention mechanism);

[0102] Experiment B: Classification using only the second feature extraction branch (a convolutional neural network based on a 3D residual structure);

[0103] Experiment C: Two-branch + simple feature splicing;

[0104] Experiment D: Complete solution of the present invention (dual-branch + attention-guided cross-branch fusion).

[0105] All experiments were conducted on the same dataset and with the same training parameters. The experimental results are shown in Table 1.

[0106] Table 1 Comparison of ablation test results

[0107]

[0108] As can be seen from Table 1:

[0109] 1. The dual-branch architecture outperforms the single-branch architecture; Experiment C improved the accuracy by 2.4% and 3.2% compared to Experiments A and B, respectively, demonstrating that global and local features are complementary and that combining the two can significantly improve classification performance.

[0110] 2. Attention-guided cross-branch fusion outperforms simple feature splicing; the accuracy of experiment D is 2.1% higher than that of experiment C, demonstrating that the attention mechanism can effectively tap the complementarity between different features and achieve better feature fusion.

[0111] 3. The reasoning speed of the present invention can meet clinical needs; the reasoning time of Experiment D is 360ms, which is much less than the clinical real-time requirement of 1 second.

[0112] To further verify the superiority of the present invention, it was compared with existing mainstream methods for differentiating benign and malignant lung lesions. The comparison results are shown in Table 2.

[0113] Table 2 shows the comparison results with existing methods.

[0114]

[0115] As can be seen from Table 2, all performance indicators of the present invention are significantly better than those of existing mainstream methods, proving the advancement and effectiveness of the present invention.

[0116] To verify the generalization ability of the method of this invention under different data distributions, in addition to the LIDC-IDRI dataset, this invention was also independently tested on the following three internationally published lung CT datasets:

[0117] (1) LUNA16 dataset: contains 888 chest CT scan images, with a total of 1186 lung nodules annotated;

[0118] (2) TCIA-Lung1 dataset: contains preoperative CT images of 422 patients with non-small cell lung cancer;

[0119] (3) NSCLC-Radiomics dataset: contains multicenter CT image data of 422 patients with non-small cell lung cancer.

[0120] On the three datasets mentioned above, the classification accuracy of the method of the present invention is 95.7%, 94.9% and 94.3% respectively, which is comparable to the classification accuracy of 96.8% on the LIDC-IDRI dataset, proving that the method of the present invention has good cross-dataset generalization ability.

[0121] Furthermore, the present invention was preliminarily validated on prospective clinical data from three different hospitals. The validation set included 216 samples, covering three mainstream CT devices: GE, Siemens, and Philips, with a scan slice thickness ranging from 0.625 mm to 5 mm. The final classification accuracy reached 93.8%. These results indicate that the method of the present invention can adapt to actual clinical scenarios with different scanning devices and scanning parameters, and has clinical application value.

[0122] The technical solution of this invention is not a simple combination of general deep learning models, but a specific combination of technologies designed specifically for the inherent characteristics of the task of differentiating between benign and malignant lung CT images, and is therefore non-obvious.

[0123] First, the diagnosis of benign and malignant lung CT images requires both global contextual information such as the spatial distribution of lesions throughout the lung and their relationship with surrounding tissues, and precise capture of local fine features such as texture, density, and edges within the lesions. A single model cannot simultaneously handle both types of features. This application specifically adopts a dual-branch architecture that uses a 3D Swin Transformer for the global branch and a lightweight convolutional neural network based on a three-dimensional residual structure for the local branch, to adapt to the extraction needs of global and local features respectively.

[0124] Secondly, the two-level attention cross-branch fusion mechanism with channel attention and spatial attention designed in this application is not a general feature stitching method, but can adaptively weight different channels and spatial positions of global and local features, fully explore the complementarity of the two types of features, and solve the problem that simple fusion cannot effectively utilize feature complementarity;

[0125] Finally, this application introduces a joint loss function composed of center loss and cross-entropy loss, which is specifically designed to address the problem of low distinguishability between benign and malignant lesion features in medical imaging data. By reducing the distance between similar features and increasing the distance between dissimilar features, the discriminative ability of features is significantly improved.

[0126] The organic combination of the above-mentioned technical features is a specific design for the task of differentiating between benign and malignant lung CT images, and is not a conventional choice for those skilled in the art, which can bring about significant improvement in technical performance.

[0127] In summary, this method for differentiating between benign and malignant lung lesions based on bi-branch deep learning achieves differentiated extraction of global and local features through a bi-branch feature extraction unit. The first feature extraction branch processes the original full-size image to extract global contextual semantic features and spatial distribution features, while the second feature extraction branch processes the automatically segmented local image of the lesion to extract fine texture density and morphological features. The two branches run in parallel to give full play to the advantages of different models and effectively solve the technical problem that a single model cannot simultaneously take into account both global and local features.

[0128] Furthermore, this method for identifying and detecting benign and malignant lung lesions based on bi-branch deep learning achieves adaptive weighted fusion of features through attention-guided cross-branch fusion units. The channel attention subunit adaptively weights the features of the two branches according to the channel dimension, and the spatial attention subunit adaptively weights the weighted features according to the spatial dimension. Finally, the feature stitching layer completes the feature stitching, fully exploring the complementarity between different features and solving the problem of simple fusion lacking effective interaction.

[0129] Furthermore, this method for differentiating between benign and malignant lung lesions based on bi-branch deep learning improves the model's inference speed through a lightweight network design. The second feature extraction branch uses three-dimensional depthwise separable convolution to replace traditional convolution, significantly reducing the number of model parameters and computational complexity. At the same time, the bi-branch architecture avoids the excessive increase in network depth and width of a single model in pursuit of accuracy, enabling the model's inference speed to meet the needs of real-time clinical auxiliary diagnosis. It solves the problems that a single model cannot simultaneously extract global context and local fine texture, that simple fusion lacks feature complementarity leading to low classification performance, and that a large number of model parameters makes it difficult to meet the needs of real-time clinical diagnosis.

[0130] The relevant modules involved in this system are all hardware system modules or functional modules that combine computer software programs or protocols with hardware in the prior art. The computer software programs or protocols involved in these functional modules are technologies known to those skilled in the art and are not improvements to this system. The improvement of this system lies in the interaction or connection between the modules, that is, in improving the overall structure of the system to solve the corresponding technical problems that this system aims to address.

[0131] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for differentiating between benign and malignant lung lesions based on bibranch deep learning, characterized in that, Includes the following steps: S1: Acquire the original chest CT image, perform standardized preprocessing on it, and generate the original full-size image and local image of the lesion; S2: Input the original full-size image into the first feature extraction branch. The first feature extraction branch is a 3D SwinTransformer model, which extracts global contextual semantic features and spatial distribution features. S3: Input the local image of the lesion into the second feature extraction branch. The second feature extraction branch is a lightweight convolutional neural network based on a three-dimensional residual structure to extract fine texture, density and morphological features. S4: Through attention-guided cross-branch fusion units, global and local features are adaptively weighted and fused to generate a fused feature vector. The attention-guided cross-branch fusion unit includes channel attention sub-units, spatial attention sub-units, and feature splicing layers in sequence. S5: The model is trained end-to-end using a joint loss function consisting of cross-entropy loss function and center loss function; S6: Input the fused feature vector into the classification and identification unit, and output the classification result and confidence level of the benign or malignant lung lesions.

2. The method for differentiating between benign and malignant lung lesions based on bi-branch deep learning according to claim 1, characterized in that, The standardization preprocessing steps in step S1 include: Step 1: Resample the original CT images and unify the isotropic voxel spacing of all images to 1.0mm×1.0mm×1.0mm; Step 2: Use threshold segmentation to extract the lung parenchyma region and remove the background and bone tissue outside the thoracic cavity; Step 3: Use a pre-trained 3D U-Net model to automatically segment lesions in the lung parenchyma region and generate a segmentation mask; Step 4: Based on the segmentation mask, crop out the local image patch containing the complete lesion. The size of the local image patch is 64×64×64 voxels. Step 5: Perform Z-score normalization on the original full-size image and the local image of the lesion, respectively.

3. The method for differentiating between benign and malignant lung lesions based on bi-branch deep learning according to claim 1, characterized in that, The first feature extraction branch in step S2 specifically includes an input layer, four cascaded Transformer blocks, and a global average pooling layer; The input layer receives a raw full-size CT image with a size of 224×224×224; the four cascaded Transformer blocks each contain a multi-head self-attention layer and a feedforward neural network layer; the global average pooling layer converts the feature map output by the last Transformer block into a global feature vector with a dimension of 1024; the multi-head self-attention layer adopts a window self-attention mechanism to divide the feature map into non-overlapping windows and calculate self-attention within each window; the feedforward neural network layer consists of two fully connected layers and a Gaussian error linear unit (GELU) activation function.

4. The method for differentiating between benign and malignant lung lesions based on bi-branch deep learning according to claim 1, characterized in that, The second feature extraction branch described in step S3 specifically includes an input layer, five convolutional blocks, and a global average pooling layer; The input layer receives a local image of the lesion with a size of 64×64×64; each of the five convolutional blocks contains two 3×3×3 convolutional layers and a residual connection; the global average pooling layer converts the feature map output by the last convolutional block into a local feature vector with a dimension of 512; the convolutional layers employ three-dimensional depthwise separable convolution, decomposing standard convolution into depthwise convolution and pointwise convolution.

5. The method for differentiating between benign and malignant lung lesions based on bi-branch deep learning according to claim 1, characterized in that, The channel attention subunit in step S4 performs adaptive weighting of the global feature vector and the local feature vector according to the channel dimension to generate weighted global features and weighted local features. The spatial attention subunit adaptively weights the weighted global features and weighted local features according to the spatial dimension to generate spatially enhanced global features and spatially enhanced local features; The feature splicing layer splices spatially enhanced global features and spatially enhanced local features along the channel dimension to generate a fused feature vector with a dimension of 1536.

6. The method for differentiating between benign and malignant lung lesions based on bi-branch deep learning according to claim 1, characterized in that, The calculation process of the channel attention subunit is as follows: 。 7. The method for differentiating between benign and malignant lung lesions based on bi-branch deep learning according to claim 1, characterized in that, The classification and identification unit described in step S6 includes two fully connected layers and a Softmax normalized activation layer; The first fully connected layer maps the fused feature vector of dimension 1536 to a feature vector of dimension 256; the second fully connected layer maps the feature vector of dimension 256 to an output vector of dimension 2; the Softmax normalized activation layer normalizes the output vector and outputs the probability values ​​of whether the lung lesions are benign or malignant.

8. The method for differentiating between benign and malignant lung lesions based on bi-branch deep learning according to claim 1, characterized in that, The joint loss function described in step S5 consists of the cross-entropy loss function and the center loss function. composition: 。 9. A system for differentiating between benign and malignant lung lesions based on dual-branch deep learning, characterized in that, It includes a data preprocessing unit, a dual-branch feature extraction unit, an attention-guided cross-branch fusion unit, and a classification and identification unit connected in sequence; The data preprocessing unit is used to standardize the original chest CT images and output the original full-size image and the automatically segmented local image of the lesion. The dual-branch feature extraction unit includes a first feature extraction branch and a second feature extraction branch set in parallel. The first feature extraction branch is a 3D Swin Transformer model that receives the original full-size image and extracts the global contextual semantic features and spatial distribution features of the lesion. The second feature extraction branch is a lightweight convolutional neural network based on a three-dimensional residual structure that receives a local image of the lesion and extracts the fine texture, density and morphological features of the lesion. The attention-guided cross-branch fusion unit sequentially includes a channel attention subunit, a spatial attention subunit, and a feature splicing layer, which is used to adaptively weight and fuse the global features output by the first feature extraction branch and the local features output by the second feature extraction branch to generate a fused feature vector. The classification and identification unit receives the fused feature vector and outputs the classification result and confidence level of the benign or malignant lung lesions.

Images