Pneumonic medical data analysis and processing method, system and equipment and storage medium

Abstract

The invention provides a pneumonia medical data analysis and processing method, system and device and a storage medium, and belongs to the field of medical image processing. Multi-scale features of the lung CT image are extracted layer by layer through convolution, global pooling operation down-sampling is carried out based on the multi-scale features, and saliency features of lung lobe and lesion areas are captured from local details to global semantics; performing convolution and multi-scale pooling on the saliency features to generate a feature map fusing global and local information; and recovering the feature map through layer-by-layer up-sampling to obtain a segmentation result of the lung lobe and the focus. Quantitative indicators are calculated based on regions and pixels of the segmentation results of the lung lobes and lesions. And performing model processing based on the quantitative index, the segmentation result of the lung lobe and the focus and clinical data of the lung lobe and the patient to obtain pneumonia focus characteristic data. And a doctor can conveniently make an objective and reasonable treatment scheme according to a result of lesion segmentation and feature data calculation based on the pneumonia lesion feature data.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of medical image processing, and particularly relates to a pneumonia medical data analysis processing method, system, device and storage medium. BACKGROUND

[0002] Lobe segmentation aims to accurately segment each lobe from the lung CT image, helping doctors to identify and locate abnormal areas, and providing support for early detection of diseases. Lesion segmentation focuses on extracting the lesion area in the lung image to help doctors assess the severity of the disease, the range and location of the lesion, and other key information. Therefore, the lobe and lesion segmentation of pneumonia is a key link in medical image processing, which is crucial for diagnosis, treatment and disease monitoring, and has important clinical significance.

[0003] Automatic segmentation methods based on deep learning have made significant progress in speed and accuracy compared to manual segmentation. However, in the segmentation task of CT images of pneumonia patients, existing segmentation models still face many challenges and deficiencies. First, in the process of lobe segmentation, pneumonia patients face more technical challenges than normal people. The interference of lesion areas such as ground glass shadows and consolidation often leads to blurred boundaries of lobes, and is accompanied by collapse, displacement and even structure loss of lung tissue, making the shape structure and density distribution of lobes abnormal, thus increasing the difficulty of segmentation and being easily ignored or missegmented by the model. Traditional methods are difficult to cope with these complex changes and are prone to segmentation bias. This difference in lesion size and distribution makes it difficult for algorithms to capture multi-scale features, resulting in suboptimal segmentation results. Second, most cases only consider single image data, which is not conducive to subsequent pneumonia medical data analysis and treatment planning. SUMMARY

[0004] In order to solve the problem that the interference of pneumonia lesions affects the segmentation effect of the model, resulting in deficiencies in pneumonia medical data analysis, the present application provides a pneumonia medical data analysis processing method, system, device and storage medium.

[0005] In order to achieve the above-mentioned purpose, the present application provides the following technical solutions: A pneumonia image analysis processing method, comprising the following steps: Obtaining CT images and clinical data of patients with pneumonia; The multi-scale features of the lung CT image are extracted, global information at the channel level is extracted based on the multi-scale feature extraction channel, and feature enhancement is adaptively performed from local details to global semantics to capture significant features of the lung lobes and lesion regions; multi-scale pooling calculation is performed on the significant features to generate a feature map that fuses global and local information; feature splicing is performed through step-by-step upsampling to extract and enhance the spliced features, and an enhanced feature map is obtained, which is mapped to the classification probability of each pixel to output the segmentation result of the lung lobes and lesions; and a plurality of quantitative index data is calculated based on the regions and pixels of the segmentation result of the lung lobes and lesions. Different levels of features of the pneumonia CT image are extracted, and a plurality of convolution kernels are used to extract multi-scale features to obtain a multi-scale feature map; adaptive average pooling is performed on the multi-scale feature map to obtain a global average feature map, and feature integration is performed on the global average feature map through weights to obtain CT image feature data; clinical feature data is obtained by extracting features from the clinical data and the plurality of quantitative index data; and pneumonia lesion feature data is obtained by weighted fusion of the CT image feature data and the clinical feature data.

[0006] Preferably, the lung CT image is processed by an improved Unet CT pneumonia lobe lesion segmentation network to obtain a segmentation result of lung lobes and lesions, and the improved Unet CT pneumonia lobe lesion segmentation network specifically includes an encoder, a bridging module and a decoder; a compression excitation attention mechanism is introduced in the encoder and the decoder; and a spatial pyramid pooling module is introduced in the bridging module.

[0007] Preferably, the encoder extracts features layer by layer through four encoding convolution blocks, each encoding convolution block contains a convolution block, and each convolution block is composed of two convolution layers, batch normalization and a ReLU activation function, and a compression excitation attention mechanism is introduced; The bridging module first performs convolution operation through a center convolution block, then performs pooling operation at multiple scales using the spatial pyramid pooling module, and the pooled features are fused through 1x1 convolution to maintain the same number of channels, thereby generating a feature map that fuses global and local information; In the decoder, the encoder features are spliced with the corresponding upsampled features by using a jump connection, and the spliced features are extracted and enhanced by a convolution block with an SE module, and the enhanced feature map is mapped to the classification probability of each pixel.

[0008] Preferably, an improved large kernel convolutional neural network is used to process the pneumonia CT image to obtain CT image feature data, and the improved large kernel convolutional neural network includes an attention convolution module, a large kernel selective convolution module and a high-efficiency multi-scale attention module.

[0009] Preferably, the improved large nuclear convolutional neural network further comprises a convolutional layer, the convolutional layer comprising a standard convolutional operation and a convolutional operation for generating an attention map, for extracting different levels of features of the CT image of pneumonia.

[0010] Preferably, the clinical data and the plurality of quantitative index data are processed by a clinical data fusion network to obtain clinical feature data, the clinical data fusion network comprising a convolutional neural network, a Transformer and a long short-term memory module; the clinical data and the plurality of quantitative index data are processed by the convolutional neural network to obtain preliminary feature data, the preliminary feature data is input into the Transformer module to capture global dependency relationships to obtain global features; meanwhile, the preliminary feature data is input into the long short-term memory module to obtain time series feature data; the global features and the time series feature data are weighted and fused to obtain the clinical feature data.

[0011] Preferably, the plurality of quantitative index data specifically comprises a total volume of each lung lobe, a lesion volume, a lesion ratio, an average density and an average density of total lesions.

[0012] The application further provides a pneumonia medical data analysis processing system, specifically comprising: A data module is configured to acquire CT images of pneumonia and clinical data of a patient.

[0013] A model segmentation module is configured to extract multi-scale features of the CT images of the lung, capture significant features of lung lobes and lesion regions from local details to global semantics based on the multi-scale features, extract and enhance the significant features, and output segmentation results of the lung lobes and the lesions.

[0014] A feature extraction module is configured to extract different levels of features in the CT images of pneumonia, extract multi-scale features through a plurality of parallel convolution kernels, obtain a multi-scale feature map, perform adaptive average pooling on the multi-scale feature map to obtain a global average feature map, integrate features of the global average feature map through weights to obtain CT image feature data, extract clinical feature data from the clinical data and the plurality of quantitative index data, and perform weighted fusion on the CT image feature data and the clinical feature data to obtain pneumonia lesion feature data.

[0015] The application further provides a computer device comprising a memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to implement the steps in the pneumonia image analysis processing method.

[0016] The application further provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program can execute the steps in the pneumonia image analysis processing method when loaded by a processor.

[0017] The pneumonia image analysis processing method provided by the application has the following beneficial effects: The application extracts multi-scale features of the lung CT image, effectively pays attention to features such as multi-scale features, boundary blur and shape change, generates a feature map that fuses global and local information by convolution and multi-scale pooling, and enhances the extraction ability of different scale lesion features. Based on the segmentation results of the lung lobes and lesions, a plurality of quantitative index data can be calculated to make up for the deficiency of subjective judgment in traditional image diagnosis, provide an objective basis for the diagnosis and prognosis evaluation of pneumonia patients, and be beneficial to the follow-up diagnosis of diseases.

[0018] The application extracts multi-scale features of the lung CT image by using parallel multi-scale convolution kernels, performs adaptive average pooling (AGAP) on the feature map, improves the feature acquisition ability at different scales, and obtains image feature data. The clinical data and a plurality of quantitative index data are extracted to extract the features of the data from different angles and capture local and global long-term dependencies. The processed data are fused to obtain pneumonia lesion feature data. The two complement each other to improve the reliability of the data. The obtained pneumonia lesion feature data are helpful for doctors to identify the subsequent disease condition and course, and improve the efficiency and service capability of the overall medical system. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the application and the design scheme thereof, the drawings required by the embodiments will be briefly introduced as follows. The drawings in the following description are only part of the embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the drawings.

[0020] Figure 1 The application improves the CT pneumonia lobe lesion segmentation network framework of Unet.

[0021] Figure 2 The application provides a structure diagram of the compression excitation attention mechanism in the embodiment.

[0022] Figure 3 The application provides a structure diagram of the spatial pyramid pooling module in the embodiment.

[0023] Figure 4 Figure 6 is a comparison chart of the original Unet model in the embodiment of the present application and other existing medical image segmentation models for the results of the pneumonia lobe segmentation.

[0024] Figure 5 Figure 7 is a comparison chart of the original Unet model and the model with different modules in the embodiment of the present application for the results of the A pneumonia and immune pneumonia data set.

[0025] Figure 6 Figure 8 is a comparison chart of the method of the present application and the existing method in the embodiment of the present application for the results of the A pneumonia and immune pneumonia data set.

[0026] Figure 7 Figure 9 is a comparison chart of the basic model and the model with different modules in the embodiment of the present application for the results of the A pneumonia and immune pneumonia data set.

[0027] Figure 8 Figure 10 is a flowchart of the pneumonia sign quantitative calculation in the embodiment of the present application.

[0028] Figure 9 Figure 11 is a framework diagram based on the improved large kernel convolutional neural network in the embodiment of the present application.

[0029] Figure 10 Figure 12 is a high-efficiency multi-scale attention module in the embodiment of the present application.

[0030] Figure 11 Figure 13 is a large kernel selective convolution module in the embodiment of the present application.

[0031] Figure 12 Figure 14 is a comprehensive network framework diagram of fusing CT images and clinical data in the embodiment of the present application.

[0032] Figure 13 Figure 15 is a flowchart of a pneumonia medical data analysis processing method in the embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make those skilled in the art better understand the technical solutions of the present application and can be implemented, the present application will be described in detail below in conjunction with the drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0034] EMBODIMENT The present application provides a pneumonia image analysis processing method, as shown in Figure 1, which specifically comprises the following steps: Figure 13 Step one: obtaining the CT image of the patient with pneumonia and the clinical data.

[0035] ​Step two: use the improved Unet CT pneumonia lobe lesion segmentation network to process the lung CT image, and get the lobe and lesion segmentation results. On the basis of Unet model, introduce spatial pyramid pooling module (SPP) and compression excitation attention mechanism (SE), get the improved Unet CT pneumonia lobe lesion segmentation network, as shown in Figure 1

[0036] Specifically, the segmentation network model adopts the classic architecture of encoder-bridge module-decoder, fully utilizes multi-scale feature extraction and context information fusion, ensures that the complex characteristics of lung structure and lesion can be captured, and the detail information is retained for segmentation.

[0037] (1) In the encoder part, the multi-scale features of CT image are extracted through four EncoderConvBlock layer-by-layer feature extraction modules. Each EncoderConvBlock contains a convolution block, and each convolution block is composed of two convolution layers, batch normalization and ReLU activation function. The compression excitation attention mechanism (SE) is introduced after the convolution block. The implementation of SE module is shown in Figure 2 Global pooling operation is used to extract global information at the channel level, and important features are enhanced adaptively by learning the weights of the channels, and redundant information is suppressed. It helps to improve the feature expression ability of lung structure area and pneumonia lesion significant area, especially in the refinement of lobe and lesion edge. In the whole feature extraction process, the front EncoderConvBlock captures the basic edge and texture features of the lung, and through continuous down sampling, the rear EncoderConvBlock extracts information from local details to global semantics. The introduction of SE module in each EncoderConvBlock in the encoder enhances the attention to the lobe lesion features of pneumonia patients, and improves the model's ability to capture significant features in the lobe and lesion area.

[0038] (2) The design of the bridge module is used to make up for the semantic gap between the encoder and the decoder, and at the same time, multi-scale context information is introduced. First, the center convolution block (CenterConvBlock) performs convolution operation. The role of this layer is to further integrate and process the input features, enhance the feature expression ability, and ensure the modeling of high semantic information in the lobe and lesion area. Then, the spatial pyramid pooling module (SPP) is introduced after the center convolution block. The structure of SPP module is as shown in Figure 3 ​As shown, different scale context information is captured by the pooling operation of multiple scales (pooling window size is 1x1, 2x2, 4x4), especially in lesion segmentation, this multi-scale design can effectively deal with the size and distribution changes of the lesion. The pooled features are fused by 1x1 convolution to keep the number of channels consistent, thereby generating a feature map that integrates global and local information by modeling the multi-scale context of lung lobes and lesion regions, providing multi-scale context support for the decoder and improving the model's adaptability to large-scale lung structures and small lesion features.

[0039] (3) The decoder part gradually recovers the spatial resolution of the feature map by layer-by-layer upsampling, while using skip connections to splice the features of each stage of the encoder with the corresponding upsampled features, combining high-level semantic information with low-level detail information. After each splicing operation in the skip connection, the convolution block with the SE module is used to further extract and enhance the spliced features, ensuring that the resolution is restored while effectively preserving the details of the lung lobe boundaries and lesions. For example, in lung lobe segmentation, the skip connection combined with the SE module can further focus on the salient features of the lung lobe boundaries; in lesion segmentation, it can enhance the feature expression of the lesion region and filter background interference. In the process of gradually restoring the resolution, the introduction of the SE module enhances the decoder's ability to focus on important channels, especially when the lesion area has varying shapes and sizes. Finally, the output layer of the decoder uses 1x1 convolution to map the feature map to the classification probability of each pixel, obtaining the segmentation results of the lung lobes and lesions, and the output resolution is consistent with the input CT image.

[0040] The entire network architecture highlights the salient features of lung lobes and lesions through the SE module in the encoder, captures multi-scale context information through the SPP in the bridge module, and gradually refines the features through the skip connection and SE module in the decoder to restore the resolution. In the pneumonia CT segmentation task, this design has significant advantages in recognizing complex lung lobe structures, detailed segmentation of lesion regions, and balancing global and local features.

[0041] Step three: Calculate multiple quantitative index data according to the lung lobe and lesion segmentation results to facilitate auxiliary diagnosis of the disease later. The quantitative calculation flowchart is as follows Figure 8As shown, by calculating the segmentation results of the lung lobes and lesions, the segmented region labels are obtained to distinguish the left lung, right lung, and lesion areas. According to the label map of the segmentation results, the number of pixels in each lung lobe area and the number of pixels in the corresponding lesion area are counted, reflecting the distribution of the lesion in the lung lobe. On this basis, the original CT image is adjusted to the size consistent with the segmentation results, and combined with the pixel spacing information (voxel volume) of the CT image, the total volume of each lung lobe and the volume of the corresponding lesion are calculated. At the same time, by the gray value of the original CT image, combined with the segmentation results, the gray value of each lung lobe lesion area is counted, the average density is calculated, and the average density of the total lesion area is further calculated. Finally, the above results are summarized, and multiple quantitative index data specifically include the total volume of each lung lobe, the lesion volume, the lesion ratio, the average density, and the average density of the total lesion, forming a quantitative analysis report.

[0042] Through the calculation of the above quantitative indicators, the evaluation of the lesion range and severity can be realized. The comparison of the lung lobe volume and the lesion volume can directly reflect the proportion of the lesion in the lung lobe, helping to judge the spread range and severity of the lesion. Quantitative analysis is used for prognosis evaluation and risk prediction, and the total lesion volume and density are closely related to the prognosis of patients. Higher lesion density may indicate tissue fibrosis or other adverse outcomes. These indicators can also support the development of individualized treatment plans, helping doctors to select local treatment or systemic treatment by understanding the spatial distribution and density characteristics of the lesion, providing objective and quantitative data support for the diagnosis and prognosis evaluation of pneumonia patients. These indicators can make up for the shortcomings of subjective evaluation in traditional image diagnosis, improve the accuracy of diagnosis and the pertinence of treatment plan, and also provide a scientific basis for prognosis prediction.

[0043] Step four: Different pneumonia conditions may have stronger pathogenicity, more complex immune response mechanisms, and greater uncertainty in disease progression, often showing multiple lesions in both lungs, extensive ground glass shadows, and consolidation, etc. Some patients may rapidly deteriorate into severe cases or even endanger their lives. Based on the segmentation of lung lobes and lesions in lung CT images, multi-modal analysis is performed by combining CT images and clinical data, significantly improving the accuracy and robustness of classification. CT images provide intuitive structural information such as lesion range, density, and distribution, while clinical data reflect the patient's underlying disease, laboratory indicators, etc. Both complement each other and improve the ability to distinguish complex conditions.

[0044] For CT image data, the attention convolution module (AttnConv), large kernel selective convolution module (LSKBlock) and efficient multi-scale attention module (EMA) are introduced on the basis of convolutional neural network to obtain an improved large kernel convolutional neural network, so that the network model can efficiently process multi-scale features in the CT image of pneumonia and accurately extract lesion features through the attention mechanism. For clinical data, including quantitative data obtained in step three, patient age, underlying disease, laboratory test indicators, etc., long short-term memory module (LSTM) and transformer are integrated on the basis of traditional convolutional neural network to obtain a clinical data fusion network, and the outputs of the improved large kernel convolutional neural network and the clinical data fusion network are weighted and integrated. The network framework is shown in Figure 9 .

[0045] Step five: use the improved large kernel convolutional neural network to process the CT image of pneumonia to obtain image feature data; use the clinical data fusion network to process the clinical data and multiple quantitative index data to obtain clinical feature data; fuse the image feature data and the clinical feature data to obtain pneumonia lesion feature data, and the overall framework is shown in Figure 12 . The pneumonia lesion feature data is used to assist doctors in subsequent medical diagnosis. Specifically, the following steps are included:

[0046] S51: Based on the convolutional neural network model, an attention convolution module (AttnConv) is added after the standard convolution of the original convolutional neural network model, a large kernel selective convolution module (LSKBlock) is introduced after each convolution layer, and an efficient multi-scale attention module (EMA) is connected after the convolution is completed to obtain an improved large kernel convolutional neural network. The improved large kernel convolutional neural network is used to process pneumonia CT images. Specifically: the input image passes through the first convolutional layer, which includes a standard convolution operation and a convolution operation for generating an attention map. Specifically, two outputs are generated: one is a feature map calculated by standard convolution, and the other is a feature map obtained by attention convolution. By applying a Sigmoid activation function, an attention map is obtained, which assigns a weight to each convolution output. Then, the output of the standard convolution is multiplied by the attention map to highlight important features. This weighted output is passed to the batch normalization layer for normalization, making the data distribution more stable and reducing fluctuations during training. Next, the pooling layer is used for downsampling, reducing the image size by half, thereby reducing the computational load and helping to extract key features from the image. The subsequent layer structure (attnconv2 and attnconv3) is similar to the first convolutional layer, with each layer having a corresponding attention convolution operation, batch normalization, and pooling. Each convolution operation is used to extract features at different levels, and as the network deepens, the extracted features gradually change from low-level (such as edges, textures) to high-level (such as shapes, parts, etc.). After each convolution operation, a large kernel selective convolution module (Lskblock) is introduced, as shown in the structure of Figure 11 . This module extracts multi-scale features through multiple parallel convolution kernels (3x3, 5x5, 7x7) and selects the most important features through a gating mechanism. Specifically, Lskblock applies different size convolution kernels to the input feature map and generates multiple convolution outputs. These outputs are passed through a gating mechanism, i.e., a 1x1 convolution layer generates gating weights to determine which convolution output is more important in the current task. In this way, the network can adapt to different spatial patterns of different scales, improving its ability to perceive different details. In addition, the network also introduces an efficient multi-scale attention module (EMA), as shown in the structure of Figure 10 . The EMA module performs adaptive average pooling (AGAP) on each feature map to generate a global average feature map, then reduces and increases the dimension through a fully connected layer, and finally generates a weight coefficient through a Sigmoid activation function to adjust the response of each feature channel. This can reduce the impact of noise during training and strengthen the focus on important features, thereby improving the accuracy and robustness of the model.

[0047] After processing through all the convolutional layers and modules, the feature map is flattened and fed into two fully connected layers for recognition, obtaining image feature data. The first fully connected layer fc1 connects the flattened features with a 512-dimensional hidden layer, which enhances the non-linear representation ability through the ReLU activation function. The second fully connected layer fc2 outputs the final classification result. The output of this layer is converted into feature data through the softmax or Sigmoid activation function. In addition, the network introduces residual connections after each convolutional layer, adding the input directly to the output of the convolutional layer. The introduction of residual connections helps to solve the gradient vanishing problem commonly encountered in deep networks, enabling the network to more effectively pass information during training, and enhancing the stability and training efficiency of the network.

[0048] S52: processing the pneumonia patient clinical data and multiple quantitative index data based on a clinical data fusion network. The clinical data fusion network combines convolutional neural networks (CNN), Transformers, and long short-term memory modules (LSTM) to handle complex tasks with local features and global dependencies. Through the combination of different modules, it can efficiently extract multi-dimensional features of the input data and model them, learning the clinical data features of the patients. The network structure is as shown in Figure 13

[0049] First, the input data is processed by the CNN module for preliminary feature extraction. The CNN module consists of four convolutional layers, each followed by batch normalization and ReLU activation functions. The convolutional layers use a 3x3 convolutional kernel to gradually extract local features from the input data. The output of the convolution is down-sampled through average pooling to reduce the feature dimension while preserving important feature information.

[0050] Next, the features extracted by the CNN module are fed into the Transformer encoder module. The core of the Transformer module is the multi-head self-attention mechanism, which can capture the global dependencies between positions in the sequence. The Transformer obtains the position information of the elements in the input sequence through position encoding, and the order of the input sequence is crucial for capturing dependencies between positions. In the Transformer encoder, in addition to the multi-head self-attention mechanism, there is also a feedforward neural network that further processes and transforms the features to enhance the model's representation ability. Each layer of the Transformer encoder uses residual connections and layer normalization to enhance the stability and training effect of the model.

[0051] ​The clinical data fusion network also contains an LSTM module. The features extracted by the CNN module are input into the LSTM module. The local features of the input data are extracted through convolution and pooling operations, and the long-term time dependence in the data is captured through LSTM. Compared with traditional RNN, LSTM can better remember important sequence information and ignore unimportant parts. In this network, the output of LSTM is taken as another feature representation, which is fused with the output of the Transformer part.

[0052] The output of the model is generated through the decoder part. The decoder consists of fully connected layers, each followed by a ReLU activation function and a batch normalization operation. The decoder maps the outputs of the Transformer and LSTM networks to the final prediction space. In the decoder, the features are mapped step by step through fully connected layers, and non-linear transformations and overfitting prevention are introduced through activation functions and batch normalization layers.

[0053] In order to combine the advantages of CNN, Transformer and LSTM, the network adopts a weighted fusion strategy. CNN, Transformer and LSTM produce independent outputs, and the network will fuse the outputs of the three parts according to the preset weights (the weight of CNN is 0.3, and the weight of LSTM is 0.7).

[0054] In summary, this network structure combines the advantages of CNN, Transformer and LSTM to handle complex tasks with local features and global dependencies. The CNN part extracts local features of the input data through convolution and pooling operations. The Transformer part captures long-term dependencies and contextual information using multi-head self-attention mechanisms. The LSTM part handles long-term dependencies in time series, and finally generates clinical feature data through the decoder. The outputs of different modules are weighted and fused. The combination of CNN and Transformer in the network can effectively extract features of data from different angles and capture local and global patterns. This multi-modal feature extraction method helps to fully exploit the rich information in the data and improve the model's expressive power. The outputs of the CNN+Transformer model and the CNN+LSTM model are weighted and fused. This model integration strategy can well leverage the strengths of different models and complement each other's weaknesses. The CNN+LSTM model is good at capturing temporal dependencies, while the CNN+Transformer model focuses more on modeling global information. Through weighted fusion, knowledge transfer and complementarity between models can be achieved, improving the generalization ability and robustness of the overall model. The output weight of the CNN+Transformer model is set to 0.3, while the output weight of the CNN+LSTM model is set to 0.7. This design is a trade-off between the importance of local and global information. In medical data sets, temporal dependencies and local patterns are more critical, so it is reasonable to give a higher weight to the CNN+LSTM model. At the same time, global information cannot be ignored, and appropriate weight given to the CNN+Transformer model can provide auxiliary information and enhance the model's judgment ability.

[0055] The results of using the improved large kernel convolutional neural network and the clinical data fusion network to process pneumonia CT images and clinical data, respectively, are fused to obtain pneumonia lesion feature data. Weighted average is chosen as the fusion method. Pneumonia lesion feature data is helpful for subsequent identification of high-risk patients, early implementation of intensive treatment and intensive care intervention measures, effectively reducing the severity rate and mortality rate, while also avoiding over-treatment of mild patients, improving the efficiency and service capability of the overall medical system.

[0056] Taking the actual data set as an example, the above processing process of the model is further described: (I) In the task of processing lung CT images using an improved Unet CT pneumonia lobe lesion segmentation network to obtain lung lobe and lesion segmentation results, the publicly available A pneumonia dataset was used, while the segmentation task for immune pneumonia used a dataset collected from Hospital A. In both experiments, 60% of the data was allocated to the training set for model training; 20% of the data was allocated to the validation set for adjusting the model's hyperparameters and avoiding overfitting; and the remaining 20% ​​of the data was used as the test set.

[0057] (1) Comparison of lung lobe segmentation models: Cross-entropy loss was used as the loss function to guide the updating of network parameters. In addition, the Dice coefficients calculated from the validation set were used as checkpoints to obtain the optimal model during training.

[0058] Based on the aforementioned dataset, the improved UNet CT pneumonia lobe lesion segmentation network was compared with advanced medical image segmentation models. TransUNet combines the Transformer and U-Net architectures, introducing a globally modeling Transformer module and skip connections from U-Net, achieving both global feature capture and fine-grained feature representation capabilities. ResUNet adds residual connections to U-Net to alleviate the vanishing gradient problem and enhance deep feature extraction capabilities. AttentionU-Net dynamically adjusts the importance of feature channels through an attention mechanism, improving its ability to focus on target regions, making it suitable for segmentation tasks with complex backgrounds. U²-Net employs a nested U-shaped structure, multi-level feature extraction and fusion, achieving refined optimization from global to local levels, and performs particularly well in edge detection and small object segmentation. TA-UNet introduces a dual attention mechanism (channel attention and spatial attention) and integrates the Transformer structure to enhance the ability to model long-range dependencies, thereby improving the ability to perceive complex lesion areas; while CAD-UNet combines capsule networks with the U-Net structure, obtains rich spatial and pose information through parallel capsule paths, and integrates it with the trunk encoder features, effectively improving the model's ability to segment fine-grained regions of pneumonia A infection.

[0059] The A pneumonia lobe segmentation task comparison results are shown in Table 1. The performance of TransUNet is relatively weak, especially in the IoU and mIoU indicators, which mainly relies on the advantages of global feature capture, but may be insufficient in handling complex lung lobe morphology and boundary details, resulting in poor segmentation performance. The performance of ResUNet has improved significantly, indicating that the introduction of residual connection has enhanced the expression ability of deep features, showing more accurate capture ability of lung lobe structure. AttentionU-Net further improves the attention ability to the target area through attention mechanism, but is slightly lower than ResUNet in IoU and mIoU. U²-Net performs better in IoU and mIoU, which benefits from the advantages of its nested U-shaped structure in multi-scale feature fusion. TA-UNet achieves better results in multiple indicators. The introduction of the capsule structure in CAD-UNet enhances the spatial relationship modeling capability, making the model perform better in capturing complex lung lobe boundaries and structural consistency, and the segmentation performance is comprehensively improved. The proposed method achieves the best performance in all indicators, fully demonstrating its comprehensive advantages in capturing multi-scale features, optimizing global and local information, and improving lung lobe segmentation accuracy.

[0060] Table 1 Comparison results of A pneumonia lobe segmentation The immune pneumonia lobe segmentation task comparison results are shown in Table 2. From the results in the table, it can be seen that CAD-UNet and TA-UNet have obvious performance improvement compared with other models, and perform better in Dice, IoU and mIoU indicators. In contrast, the performance of TransUNet is weak, especially in the IoU and mIoU indicators, which is significantly lower than other methods, which may be related to its strong modeling ability for global features but insufficient capture of edge details. The performance of ResUNet and AttentionU-Net is similar. U2-Net is better than ResUNet and AttentionU-Net in mIoU, but its performance in Dice is slightly inferior. TA-UNet further optimizes U2-Net and improves in all indicators. The performance of CAD-UNet is better than TA-UNet. Finally, the Ours model achieves the highest value in all indicators, indicating that it has obvious improvement in global feature modeling and achieves higher segmentation accuracy, proving the effectiveness and superiority of the proposed method.

[0061] Table 2 Comparison results of immune pneumonia lobe segmentation As Figure 4The visualization shows the comparative segmentation results of the method on the data sets with the above methods. The first row is the original slice in the test set, the second row is the true labeled segmentation contour, and the following rows are the segmentation contours of all comparison methods. (a) column and (b) column are A pneumonia segmentation result figures, (c) column and (d) column are immune pneumonia segmentation result figures. As can be seen from the figures, the method performs significantly better in the lung lobe segmentation task, with the highest degree of fit to the true labeled segmentation contour, especially in the segmentation of small areas and complex boundaries of lung lobes, showing stronger refinement and accuracy. In contrast, TransUNet and Res-UNet have deficiencies in detail processing and boundary segmentation of lung lobes, and are prone to missing and misclassification; AttentionU-Net and U2-Net have improved feature capture in important areas, but there are still deviations in lung lobe boundary details. TA-UNet and CAD-UNet perform better than the previous methods. The method achieves more accurate segmentation through more efficient global and local feature fusion, showing stronger robustness and generalization ability, which is superior to all comparison methods.

[0062] (2) Analysis of comparative results of lesion segmentation model: Based on the A pneumonia and immune pneumonia data sets, the improved Unet CT pneumonia lobe lesion segmentation network is compared with advanced medical image segmentation models. The comparison methods are the same as the lung lobe segmentation task.

[0063] The comparison results of A pneumonia lesion segmentation task are shown in Table 3. TransUNet performs relatively weak, although its global feature capture ability is strong, but there may be some deficiencies in dealing with complex boundaries and small lesions. ResUNet enhances feature expression ability by introducing residual connection, significantly better than TransUNet, indicating that residual structure has advantages in improving deep feature learning and dealing with fuzzy boundaries. Although AttentionU-Net enhances the attention to lesion areas through attention mechanism, it is slightly inferior to ResUNet. U2-Net performs outstandingly in IoU and mIoU, which benefits from its nested U-shaped structure that can effectively capture complex and small lesion information in multi-scale feature fusion. TA-UNet and CAD-UNet perform better for pneumonia lesion segmentation. The proposed method achieves the best performance in all indicators, significantly outperforming other methods. Its excellent performance is mainly due to the optimization of multi-scale feature capture, boundary detail processing and global and local feature fusion, which enables it to more accurately segment the lesion area, especially in dealing with complex boundaries, small lesion capture and fuzzy area division, showing outstanding robustness and generalization ability.

[0064] Table 3 Comparison results of A pneumonia lesion segmentation Table 6 shows the comparison results of the immune-mediated pneumonia lesion segmentation task. In this table, TransUNet performs relatively weakly, with its segmentation accuracy and overall performance lagging behind other models, especially in the mIoU metric. ResUNet significantly outperforms TransUNet. AttentionU-Net and ResUNet perform similarly. U2-Net outperforms AttentionU-Net and ResUNet overall, particularly in IoU and mIoU metrics. TA-UNet and CAD-UNet show well-balanced performance across all metrics. The method with the best performance outperforms the others, demonstrating its superior performance in the segmentation task.

[0065] Table 4. Results of comparative experiments on the segmentation of lesions in immune-mediated pneumonia. like Figure 6 As shown, the visualization presents the segmentation results of our method compared with the aforementioned methods on the A pneumonia and immune pneumonia datasets. The first row shows the original slices from the test set, the second row shows the ground truth labeled segmentation contours, and the following rows show the segmentation contours of all the comparison methods. Columns (a), (b), and (c) are the segmentation results for A pneumonia, and columns (d) and (e) are the segmentation results for immune pneumonia.

[0066] The results show that while TransUNet demonstrates some ability in capturing global features, it performs poorly in segmenting boundary details and small lesion regions, exhibiting both missed and incorrect segmentations. ResUNet improves deep feature representation by introducing residual connections, enhancing the continuity of lesion boundaries and regional integrity, but still falls short in complex boundary regions. AttentionU-Net enhances attention to key regions through its attention mechanism, but still exhibits biases in the detailed segmentation of small lesions. U2-Net, with its nested U-shaped structure, excels in multi-scale feature fusion and provides good segmentation of lesion regions, but still has limitations in boundary refinement. TA-UNet and CAD-UNet provide more refined segmentation of pneumonia lesions than the previously compared methods. In contrast, the proposed method demonstrates a significant advantage in segmentation results, accurately capturing large-scale lesions while maintaining higher refinement and accuracy in small lesions and complex boundary regions, achieving the highest degree of fit with ground truth annotations.

[0067] (II) In the task of processing pneumonia CT images and clinical data using an improved large-kernel convolutional neural network and a clinical data fusion network to obtain pneumonia lesion feature data, CT image data of 40 patients with pneumonia A, totaling more than 4800 slides, were collected from Hospital A. CT image data of 50 patients with immune-mediated pneumonia, totaling more than 6000 slides, were also collected. Analysis of these data allows for in-depth exploration of the lesion features in CT images of pneumonia A and immune-mediated pneumonia at different disease stages, providing more support and evidence for early diagnosis and prognostic assessment. This, in turn, provides clinicians with effective auxiliary decision-making tools to optimize treatment plans.

[0068] Using a dataset of CT images from 40 patients with pneumonia A and 50 patients with immune-mediated pneumonia collected at Hospital A, 60% of the data was allocated to the training set for model training; 20% was allocated to the validation set for adjusting model hyperparameters and avoiding overfitting; and the remaining 20% ​​was used as the test set. The Adam optimizer was used to iteratively update the network weights, with mean squared error loss used as the loss function to guide the update of the network parameters.

[0069] (1) Analysis of the lesion feature identification results of A pneumonia and immune pneumonia Based on the aforementioned dataset, binary classification experiments were first conducted on pneumonia A and immune-mediated pneumonia. The method was compared with state-of-the-art medical image classification models. Specifically, ResNet50 addresses the vanishing gradient problem in deep networks through residual connections, enhancing training stability. GoogLeNet employs multi-scale convolutional parallel computation to improve feature extraction capabilities. DenseNet121 uses dense connections to enhance feature reuse and improve efficiency, reducing the vanishing gradient problem. SwinTransformerV2 aims to improve model capacity and resolution adaptability. CovC-ReDRNet combines convolutional neural networks and deep stochastic vector function linking networks (dRVFL).

[0070] The comparison results are shown in Table 5. The proposed method achieved the best classification performance in the binary classification task of distinguishing between pneumonia A and immune-mediated pneumonia, demonstrating the network's powerful expressive ability in handling complex lung imaging features. CovC-ReDRNet also performed very well. SwinTransformerV2, relying on its window-based multi-scale attention mechanism, achieved a balanced development in local and global feature extraction, and also achieved high accuracy and F1 score. DenseNet121 also performed well; its dense connection structure enhanced feature reuse and gradient propagation capabilities, which helped identify different types of pneumonia with similar imaging manifestations. GoogLeNet had certain advantages in multi-scale feature extraction, and its performance was slightly better than ResNet50. ResNet50 struggled to fully capture the subtle imaging differences between pneumonia A and immune-mediated pneumonia, and its overall performance was the weakest. In summary, the method of this invention is more targeted in its structural design and feature modeling strategy, and can more effectively distinguish similar pneumonia types, providing strong support for accurate diagnosis.

[0071] Table 5. Comparison of Classifications of Pneumonia and Immune Pneumonia (A) (2) Analysis of the results of identifying the characteristics of pneumonia lesions Based on the above dataset, a binary classification experiment was conducted to classify mild and severe cases of pneumonia A. The method was compared with advanced medical image classification models using the same comparison method.

[0072] The comparison results are shown in Table 6. The method proposed in this invention performs excellently on all metrics. In comparison, DenseNet121's performance is also very close to that of the method in this invention. It enhances feature reuse through dense connections, effectively capturing subtle lesion changes and improving the accuracy and recall of identification. GoogLeNet's performance is relatively weak. ResNet performs the worst. Although ResNet50 alleviates the gradient vanishing problem in deep networks through residual connections, its shallow network structure may prevent it from fully capturing subtle lesion differences in CT images of patients with pneumonia A. SwinTransformerV2 and CovC-ReDRNet perform better than the previous networks. In summary, the results in the table show that it can better overcome the difficulties of data complexity and subtle differences when identifying lesion features in CT images of patients with pneumonia A.

[0073] Table 6 Comparison of CT image lesion feature identification results for pneumonia A. (3) Analysis of the results of identifying the characteristics of immune-mediated pneumonia lesions Based on the above data, the method of this invention was compared with advanced medical image classification models. The comparison results are shown in Table 7. The method of this invention achieved the best performance in the task of identifying lesion features in immune-mediated pneumonia, demonstrating excellent feature extraction and classification capabilities. In comparison, CovC-ReDRNet and SwinTransformerV2 also performed very well. DenseNet121 enhances feature propagation and reuse capabilities through dense connections, maintaining good classification performance even in complex feature scenarios. While GoogLeNet has advantages in multi-scale feature extraction, it still has some shortcomings in identifying subtle differences between features of different diseases; and ResNet50 has relatively limited feature expression capabilities, making it difficult to fully capture the complex changes in CT images of immune-mediated pneumonia.

[0074] Table 7. Results of CT imaging lesion feature identification for immune-mediated pneumonia In processing CT images and clinical data, clinical and chest CT data of 40 patients with pneumonia A were collected from Hospital A, including 20 deaths and 20 improvements, totaling over 4800 slides. A pre-trained segmentation model was used for lung lobe segmentation and intrapulmonary lesion segmentation of pneumonia A. Based on the segmentation results, quantitative indicators of chest CT were calculated, including the volume of the whole lung and individual lobes, lesion volume and its proportion in the lung lobe, and average lesion density. Statistical analysis was performed on the differences in chest CT quantitative indicators and clinical data between the two groups of patients. Indicators with statistically significant differences were included in multivariate logistic regression analysis to identify independent risk factors for severe pneumonia A. The correlation between various indicators and the severity of pneumonia A was explored.

[0075] Specifically, clinical data includes the patient's age, sex, presence of diabetes, hypertension, and cancer; white blood cell count; neutrophil count; lymphocyte count; neutrophil / lymphocyte ratio; neutrophil ratio; lymphocyte ratio; high-sensitivity C-reactive protein; and albumin / globulin ratio (liver function test). The segmentation results are used to calculate quantitative indicators for chest CT, including the volume of the entire lung, left lung, and right lung; the volume of the upper and lower lobes of the left lung; the volume of the upper, middle, and lower lobes of the right lung; the volume of inflammatory lesions in each lung lobe; the percentage of lesion volume to the total lung lobe volume; and the lesion volume in each lung lobe within each density range.

[0076] SPSS 22.0 software was used for statistical analysis. The Kolmogorov-Smirnov test was used to test the normality of continuous data. Continuous data conforming to a normal distribution in the death and improvement groups of pneumonia A were expressed as mean ± standard deviation (Mean ± SD) and analyzed using an independent samples t-test. Continuous data not conforming to a normal distribution were expressed as mean ± standard deviation (M, P25, P75) and analyzed using the Mann-Whitney U test. Categorical data were expressed as percentages and analyzed using a chi-square test. Statistically significant indicators were included in multivariate logistic regression analysis to identify independent risk factors for severe pneumonia A.

[0077] The following are the results of univariate analysis: Significant differences were found in lesion volume and its percentage of total lung volume, left lower lobe volume, and right lower lobe volume, respectively, between the left and right lower lobes. Conversely, there were no significant differences in lesion volume and its percentage between the left upper lobe and the right upper and middle lobes. Differences in severity between mild and severe cases were generally more pronounced in the left and right lower lobes. These two sites may be areas more susceptible to infection and progression of pneumonia A. In admission laboratory tests, white blood cell count, neutrophils, neutrophil / lymphocyte ratio, neutrophil ratio, and high-sensitivity C-reactive protein were significantly higher in the mortality group than in the improvement group (P<0.01). Lymphocyte count, lymphocyte ratio, and albumin / globulin ratio (liver function albumin / globulin ratio) were significantly lower in the mortality group than in the improvement group (P<0.01). Significant differences were found between the two groups in terms of gender, with males having a higher mortality rate than females. No significant differences were found in the presence of hypertension, diabetes, or cancer, as shown in Tables 8 to 11. However, the P-value for diabetes was small, close to the threshold for statistical significance, and we learned from doctors that diabetes may have a certain impact on the development of pneumonia A. The lack of significant difference in diabetes may be due to the small amount of data.

[0078] Table 8 Results of Univariate Analysis of Pneumonia A Table 9. Results of Univariate Analysis of Pneumonia A (2) Table 10 Results of Univariate Analysis of Pneumonia A Table 11 Results of Univariate Analysis of Pneumonia A 4 Indicators showing statistically significant differences in univariate analysis were included in multivariate logistic regression analysis. The results of the multivariate analysis are as follows: Statistical analysis revealed that age, gender, volume of the left lower lobe and right middle lobe, volume and proportion of right pneumonia lesions, volume and proportion of right upper and lower lobes inflammatory lesions, mean density of right pneumonia lesions, mean density of right lower lobe inflammatory lesions, neutrophils, neutrophil ratio, and high-sensitivity C-reactive protein are important influencing factors on whether a patient with pneumonia A improves or progresses to severe illness. Among the quantitative indicators of lung CT, independent risk factors for severe pneumonia A are mostly concentrated in the right lung region. Therefore, in the early screening process, attention should be paid to the volume and proportion of lesions and the density of lesions in the right lung region to facilitate early and effective treatment and minimize the risk of disease deterioration.

[0079] (1) Significance analysis of clinical data of immune pneumonia Clinical and chest CT data of 50 patients with immune-mediated pneumonia were collected from Hospital A, including 25 deaths and 25 improvements, totaling over 6000 slides. A pre-trained segmentation model was used for lung lobe and lesion segmentation. Based on the segmentation results, quantitative indicators of chest CT were calculated, including the volume of the entire lung and individual lobes, lesion volume and its proportion in the lobe, and average lesion density. Statistical analysis was performed on the differences in chest CT quantitative indicators and clinical data between the two groups. Indicators with statistically significant differences were included in multivariate logistic regression analysis to identify independent risk factors for severe immune-mediated pneumonia. The correlation between various indicators and the severity of immune-mediated pneumonia was explored.

[0080] Specifically, clinical data includes the patient's age, sex, TICA, underlying disease status, blood oxygen saturation, BMI, ECOG, smoking status, alcohol consumption, whether immunotherapy is combined with chemotherapy, whether it is combined with bevacizumab, whether it is concurrent radiotherapy, whether it is immunotherapy combined with radiotherapy, and the number of immunotherapy cycles. The segmentation results are used to calculate quantitative indicators for chest CT, including the volume of the entire lung, left lung, and right lung; the volume of the upper and lower lobes of the left lung; the volume of the upper, middle, and lower lobes of the right lung; the volume of inflammatory lesions in each lung lobe; the percentage of lesion volume to the lobe volume; and the lesion volume in each lung lobe within each density range.

[0081] The analysis method was the same as above. The following are the results of the univariate analysis: Significant differences were found in the volume of the left middle and lower lobes, the volume of the right middle and lower lobes, the percentage of inflammatory lesions in the left and right lower lobes, the mean density of pneumonia lesions in the right lung, the mean density of lesions in the left middle and lower lobes, and the mean density of inflammatory lesions in the right middle and lower lobes (P<0.01). It can be seen that the differences between severe and mild cases are generally more pronounced in the left and right lower lobes, which may be areas where pneumonia A is more easily infected and progresses. Among the laboratory tests at admission, significant differences were found in blood oxygen saturation, BMI, number of immune cycles, TICA, underlying diseases, alcohol consumption, whether concurrent radiotherapy was administered, and ECOG (P<0.01), as shown in Tables 12 and 13.

[0082] Table 12 Results of Univariate Analysis of Immune Pneumonia 1 Table 13 Results of Univariate Analysis of Immune Pneumonia 2 Indicators showing statistically significant differences in univariate analysis were included in multivariate logistic regression analysis. The results of the multivariate analysis are as follows: Statistical analysis revealed that the volume of the left lower lobe, the volume of the right lower lobe, the proportion of inflammatory lesions in the right lower lobe, the average density of inflammatory lesions in the right lung, the average density of inflammatory lesions in the right middle lobe, the average density of inflammatory lesions in the right lower lobe, TICA, blood oxygen saturation, ECOG, and whether concurrent radiotherapy was administered were important influencing factors on whether patients with immune-mediated pneumonia improved or progressed to severe illness. Among the quantitative indicators of lung CT, independent risk factors for severe A pneumonia were concentrated in the left and right lower lobes. Therefore, in the early screening process, attention should be paid to the volume and proportion of lesions in these areas, as well as the lesion density, to facilitate early and effective treatment and minimize the deterioration of patients with immune-mediated pneumonia.

[0083] CT imaging data of 50 patients with immune-mediated pneumonia were collected at Hospital A. 25 patients gradually improved and eventually recovered, while the condition of another 25 patients worsened, totaling over 6000 slides. Corresponding quantitative indicators and clinical data, screened through significance analysis, were also collected, including left lower lobe volume, right lower lobe volume, percentage of inflammatory lesions in the right lower lobe, mean density of inflammatory lesions in the right lung, mean density of inflammatory lesions in the right middle lobe, mean density of inflammatory lesions in the right lower lobe, TICA, blood oxygen saturation, ECOG, and whether concurrent radiotherapy was administered.

[0084] (1) Analysis of the lesion feature identification results of pneumonia A based on clinical data The method of this invention was compared with other classification models, including: KNN (K-Nearest Neighbors), suitable for data with simple sample distribution; SVM (Support Vector Machine), suitable for tasks with high dimensionality and clear boundaries, especially performing well in small sample cases; and RandomForest, suitable for handling high-dimensional, non-linear problems with a lot of feature noise.

[0085] The comparison results are shown in Table 14. In the clinical data-based task of identifying lesion features in pneumonia A, the method of this invention achieved the best performance on all evaluation metrics. KNN performed relatively poorly; although its accuracy was high (82.61%), its recall was low (63.33%), indicating that it had significant missed cases when identifying critically ill patients, affecting the overall F1 score. SVM achieved classification by constructing a maximum margin hyperplane, achieving a certain balance between accuracy and recall, but it was still slightly inferior to the model of this invention. RandomForest performed well in terms of recall (83.33%). In comparison, the method of this invention outperformed other methods in accuracy, precision, recall, and F1 score, indicating that it can provide more reliable decision support for the assessment and grading of pneumonia A patients.

[0086] Table 14. Results of Clinical Data on Pneumonia Lesion Feature Identification (2) Analysis of the results of identifying the lesion features of immune-mediated pneumonia based on clinical data The comparison results are shown in Table 15. The method of this invention achieved the best performance across all evaluation metrics. KNN had high precision (81.89%) but low recall (62.75%), indicating a significant false negative rate when identifying critically ill patients, with an overall F1 score of 71.10%. SVM achieved a good balance between precision and recall, but its overall performance was still slightly inferior to the model of this invention. RandomForest performed well in recall (83.01%), demonstrating its ability to identify critically ill patients, but due to its relatively low precision (76.12%), it still had some false positives, with an F1 score of 79.12%. In comparison, the method of this invention outperformed other models, demonstrating significant advantages in fully utilizing clinical feature information, improving model generalization ability, and balancing classification performance, providing more accurate and stable auxiliary support for lesion feature identification and assessment.

[0087] Table 15 Clinical data on lesion characteristics identification results of immune-mediated pneumonia (3) Analysis of the comparative results of lesion feature identification of pneumonia A by integrating CT images and clinical data The comparison results are shown in Table 16. The method of fusing CT images and clinical data (CT+Clin) outperforms the model using only CT images in the task of identifying lesion features in pneumonia A, especially in terms of recall, indicating that the model is more sensitive and comprehensive in identifying critically ill patients. This fully demonstrates that the complementarity of clinical data and imaging information plays a key role in classification tasks. The fusion strategy can more comprehensively reflect the patient's condition, thereby improving the model's ability to discriminate complex conditions and providing more reliable support for clinical decision-making.

[0088] Table 16. Results of Lesion Feature Identification in Pneumonia CT Data and Fusion Data (4) Analysis of the lesion feature identification results of immune pneumonia fused with CT images and clinical data The comparison results are shown in Table 17. In the identification of lesion features in immune-mediated pneumonia, the model integrating CT images and clinical data (CT+Clin) outperformed the model using only CT images in overall performance. Specifically, accuracy improved from 98.14% to 98.86%, indicating a significant enhancement in the model's overall classification ability; precision remained at 98.32%, demonstrating the model's continued accuracy in predicting positive cases; recall increased from 97.45% to 97.79%, reflecting a more comprehensive model in identifying critically ill patients; and the F1 score also improved from 98.21% to 98.67%, indicating a better balance between accuracy and recall. The overall trend shows that integrating clinical data effectively compensates for details that may be missed by relying solely on images, improves the comprehensive ability to judge the severity of the disease, and provides more valuable auxiliary decision-making information for clinicians.

[0089] Table 17. Lesion feature identification results from CT and fusion data of immune-mediated pneumonia. The present invention also provides a pneumonia image analysis and processing system, comprising: The data module is used to acquire CT images of patients with pneumonia and clinical data.

[0090] The model segmentation module is used to extract multi-scale features from lung CT images. Based on the multi-scale features, it extracts channel-level global information and adaptively enhances features, capturing salient features of lung lobes and lesion regions from local details to global semantics. Multi-scale pooling is performed on the salient features to generate feature maps that fuse global and local information. Feature stitching is performed through progressive upsampling, and the stitched features are extracted and enhanced to obtain enhanced feature maps. The enhanced feature maps are mapped to the classification probability of each pixel, and the segmentation results of lung lobes and lesions are output. Based on the regions and pixels of the segmentation results of lung lobes and lesions, multiple quantitative index data are calculated.

[0091] The feature extraction module is used to extract features at different levels of pneumonia CT images, and then extract multi-scale features through multiple parallel convolutional kernels to obtain multi-scale feature maps. Adaptive average pooling is performed on the multi-scale feature maps to obtain global average feature maps. The global average feature maps are then integrated with weights to obtain CT image feature data. Features are extracted from clinical data and multiple quantitative index data to obtain clinical feature data. The CT image feature data and clinical feature data are then weighted and fused to obtain pneumonia lesion feature data.

[0092] The modules in the aforementioned pneumonia image analysis and processing system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0093] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in an embodiment of a pneumonia image analysis and processing method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0094] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the aforementioned method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in an embodiment of a pneumonia image analysis and processing method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0095] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0097] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0098] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0099] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for analyzing and processing medical data related to pneumonia, characterized in that, Includes the following steps: Obtain CT images and clinical data of the patient with pneumonia; Multi-scale features of the lung CT images are extracted, channel-level global information is extracted based on the multi-scale features, and feature enhancement is adaptively performed to capture salient features of lung lobes and lesion areas from local details to global semantics. Multi-scale pooling is performed on the salient features to generate a feature map that integrates global and local information; features are concatenated by progressive upsampling, and the concatenated features are extracted and enhanced to obtain an enhanced feature map. The enhanced feature map is mapped to the classification probability of each pixel, and the segmentation results of the lung lobe and lesion are output; multiple quantitative index data are calculated based on the regions and pixels of the segmentation results of the lung lobe and lesion. Features at different levels of pneumonia CT images are extracted, and then multi-scale features are extracted using multiple convolutional kernels in parallel to obtain multi-scale feature maps. Adaptive average pooling is performed on the multi-scale feature maps to obtain global average feature maps. The global average feature maps are then integrated with weights to obtain CT image feature data. Features are extracted from the clinical data and multiple quantitative index data to obtain clinical feature data. The CT image feature data and clinical feature data are then weighted and fused to obtain pneumonia lesion feature data.

2. The method for analyzing and processing pneumonia images according to claim 1, characterized in that, The lung CT images are processed by an improved Unet CT pneumonia lobar lesion segmentation network to obtain segmentation results of the lung lobes and lesions. The improved Unet CT pneumonia lobar lesion segmentation network specifically includes an encoder, a bridging module, and a decoder. A compressed excitation attention mechanism is introduced in the encoder and decoder. A spatial pyramid pooling module is introduced in the bridging module.

3. The method for analyzing and processing pneumonia images according to claim 1, characterized in that, The encoder extracts features layer by layer through four coding convolutional blocks. Each coding convolutional block contains a convolutional block, and each convolutional block consists of two convolutional layers, batch normalization, and a ReLU activation function. At the same time, a compressed excitation attention mechanism is introduced. The bridging module first performs convolution operation through the central convolution block, and then uses the spatial pyramid pooling module to perform pooling operation at multiple scales. The pooled features are fused through 1×1 convolution to maintain the same number of channels and generate a feature map that fuses global and local information. The decoder uses skip connections to concatenate encoder features with corresponding upsampled features. After concatenation, the concatenated features are extracted and enhanced through convolutional blocks with SE modules, and the enhanced feature map is mapped to the classification probability of each pixel.

4. The method for analyzing and processing pneumonia images according to claim 1, characterized in that, An improved large-kernel convolutional neural network was used to process pneumonia CT images to obtain CT image feature data. The improved large-kernel convolutional neural network includes an attention convolution module, a large-kernel selective convolution module, and an efficient multi-scale attention module.

5. The method for analyzing and processing pneumonia images according to claim 1, characterized in that, The improved large-kernel convolutional neural network also includes a convolutional layer, which includes a standard convolution operation and a convolutional operation for generating an attention map to extract features at different levels of pneumonia CT images.

6. The method for analyzing and processing pneumonia images according to claim 1, characterized in that, The clinical data and multiple quantitative indicator data are processed using a clinical data fusion network to obtain clinical feature data. The clinical data fusion network specifically includes a convolutional neural network, a Transformer, and a long short-term memory module. The convolutional neural network is used to process the clinical data and multiple quantitative indicator data to obtain preliminary feature data. This preliminary feature data is then input into the Transformer module to capture global dependencies, resulting in global features. Simultaneously, the preliminary feature data is input into the long short-term memory module to obtain temporal feature data. Finally, the global features and temporal feature data are weighted and fused to obtain the clinical feature data.

7. The method for analyzing and processing pneumonia medical data according to claim 1, characterized in that, The quantitative indicators specifically include the total volume of each lung lobe, the volume of lesions, the proportion of lesions, the average density, and the average density of the overall lesions.

8. A pneumonia medical data analysis and processing system, characterized in that, include: The data module is used to acquire CT images and clinical data of patients with pneumonia. The model segmentation module is used to extract multi-scale features from the lung CT images, extract channel-level global information based on multi-scale features, and adaptively perform feature enhancement to capture salient features of lung lobes and lesion areas from local details to global semantics. Multi-scale pooling is performed on the salient features to generate a feature map that integrates global and local information; features are concatenated by progressive upsampling, and the concatenated features are extracted and enhanced to obtain an enhanced feature map. The enhanced feature map is mapped to the classification probability of each pixel, and the segmentation results of the lung lobe and lesion are output; multiple quantitative index data are calculated based on the regions and pixels of the segmentation results of the lung lobe and lesion. The feature extraction module is used to extract features at different levels of pneumonia CT images, and then extract multi-scale features through multiple parallel convolutional kernels to obtain multi-scale feature maps; adaptive average pooling is performed on the multi-scale feature maps to obtain global average feature maps, and the global average feature maps are integrated with weights to obtain CT image feature data; features are extracted from the clinical data and multiple quantitative index data to obtain clinical feature data; the CT image feature data and clinical feature data are weighted and fused to obtain pneumonia lesion feature data.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 7.

Images