Pneumonia CT (Computed Tomography) image diagnosis model training method, diagnosis method and equipment

Abstract

The invention provides a pneumonia CT image diagnosis model training method, diagnosis method and equipment, and the training method comprises the steps: inputting a 3D chest CT image into a multi-task deep learning model, enabling a shared encoder in the model to extract multi-scale feature data, and enabling a connection module and a decoder to obtain pneumonia focus region prediction result data according to the multi-scale feature data, the classification head obtains pneumonia diagnosis type prediction result data according to the multi-scale feature data; determining the joint loss of the model in the current iteration round and updating model parameters; and if the current multi-task deep learning model satisfies a training termination condition, outputting the current model as a pneumonia CT image diagnosis model. According to the method, the problems of low model feature utilization rate and low pneumonia diagnosis process efficiency caused by incapability of simultaneously completing focus segmentation and type classification due to task simplification of an existing pneumonia diagnosis model can be solved.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to training methods, diagnostic methods and equipment for pneumonia CT image diagnostic models. Background Technology

[0002] Pneumonia, an infectious disease of the lungs caused by bacteria, viruses, or other pathogens, is a significant public health problem with high morbidity and mortality rates worldwide. Chest computed tomography (CT) scans, due to their high resolution, have become a crucial non-invasive imaging tool for diagnosing and differentiating various types of pneumonia. In clinical practice, based on the anatomical distribution of lung involvement, pneumonia on CT imaging can primarily manifest as lobar pneumonia, lobular pneumonia (also known as bronchopneumonia), and interstitial pneumonia. These three patterns present distinctly different imaging characteristics: lobar pneumonia typically presents as well-defined lobar consolidation; lobular pneumonia is characterized by patchy peribronchial nodules and consolidation; while interstitial pneumonia mainly presents as ground-glass opacities, reticular shadows, and scattered nodules.

[0003] Traditional analysis of pneumonia CT images relies primarily on manual interpretation by radiologists. However, each pneumonia case's CT scan typically contains hundreds of slices, making manual analysis not only time-consuming and labor-intensive, but also susceptible to diagnostic accuracy issues influenced by factors such as physician experience, subjective judgment, and visual fatigue. More importantly, significant anatomical variability and feature overlap exist among the three pneumonia patterns mentioned above. For instance, the patchy consolidation in lobular pneumonia and the ground-glass opacity in interstitial pneumonia are highly similar in appearance, making accurate differentiation challenging even for experienced radiologists. Therefore, automated prediction or identification of these pneumonia CT images is of crucial clinical significance for determining etiology, guiding targeted treatment, and assessing prognosis.

[0004] Currently, recent studies have attempted to develop automated diagnostic systems for pneumonia using deep learning technology. However, these existing deep learning-based solutions have several significant limitations. Most models are single-task models, focusing only on either segmentation of pneumonia lesions or classification of pneumonia types, neglecting the inherent correlation and synergistic effect between segmentation and classification tasks. This fails to fully utilize the complementary information between the two tasks, resulting in low model feature utilization and low efficiency in the pneumonia diagnosis process. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method for training a pneumonia CT image diagnostic model, a diagnostic method, and a device to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of this application provides a method for training a pneumonia CT image diagnostic model, comprising: In the current iteration, a 3D chest CT image is input into a multi-task deep learning model. The shared encoder in this model extracts multi-scale feature data corresponding to the 3D chest CT image. The connection module and decoder in the model then use this multi-scale feature data to obtain predicted pneumonia lesion regions corresponding to the 3D chest CT image. Additionally, the classification head in the model uses this multi-scale feature data to obtain predicted pneumonia diagnosis types corresponding to the 3D chest CT image. The shared encoder, connection module, and decoder constitute the segmentation sub-network of the multi-task deep learning model. The shared encoder and classification head also constitute the classification sub-network of the multi-task deep learning model. Based on the preset segmentation labels, preset classification labels, pneumonia lesion region prediction results, and pneumonia diagnosis type prediction results corresponding to the 3D chest CT images, the joint loss of the segmentation sub-network and the classification sub-network in the current iteration is determined, and the parameters of the multi-task deep learning model are updated based on the joint loss. If the current multi-task deep learning model meets the preset training termination condition, then the multi-task deep learning model with updated parameters will be used as a pneumonia CT image diagnostic model for output.

[0007] In some embodiments of this application, the shared encoder includes: an encoding module, a voxel-based spatial and channel squeezing and excitation module, and a downsampling module; the encoding module includes a plurality of encoding stage units arranged sequentially; the downsampling module includes a first max pooling layer corresponding to each of the encoding stage units except the first encoding stage unit; The first encoding stage unit is used to extract features from the 3D chest CT image to obtain encoded feature data, and input the encoded feature data into the voxel-based spatial and channel compression and excitation module. The voxel-based spatial and channel compression and excitation module is used to recalibrate the dimensions of the channel and spatial directions of the encoded feature data to obtain multi-scale feature data, and transmit the multi-scale feature data to the connection module; and, if it is determined that the encoded feature data currently being processed is transmitted by the last encoded stage unit, then the multi-scale feature data is transmitted to the classification head in the classification sub-network; if it is determined that the encoded feature data currently being processed is transmitted by a non-last encoded stage unit, then the multi-scale feature data is transmitted to the first max pooling layer corresponding to the next encoded stage unit; The first max pooling layer is used to downsample the multi-scale feature data to obtain a first downsampled feature map, and inputs the first downsampled feature map into the corresponding encoding stage unit. The non-first encoding stage unit is used to extract features from the downsampled feature map to obtain encoded feature data, and input the encoded feature data into the voxel-based spatial and channel squeezing and excitation module.

[0008] In some embodiments of this application, the connection module includes: a cross-fusion Transformer module; The cross-fusion Transformer module includes: A normalization layer is used to normalize each of the multi-scale feature data currently received from the shared encoder along the channel dimension, so as to obtain the normalized feature map corresponding to each of the multi-scale feature data. Two consecutive 3D convolutional layers are used to extract features from each of the normalized feature maps to obtain the convolutional feature maps corresponding to each of the multi-scale feature data. The first ReLU activation layer is used to perform non-linear activation processing on each of the convolutional feature maps based on the ReLU activation function, so as to obtain the second activated feature map corresponding to each of the multi-scale feature data. The second max pooling layer is used to downsample each of the second activated feature maps to obtain the second downsampled feature map corresponding to each of the multi-scale feature data. A voxel embedding layer is used to flatten each of the second downsampled feature maps into a sequence to obtain the serialized feature data corresponding to each of the multi-scale feature data. The convolutional projection layer is used to generate corresponding query tags, key tags, and value tags for each of the serialized feature data; A multi-head attention layer is used to concatenate the key and value tags of each of the serialized feature data to obtain a key-value library, and to perform multi-head attention calculation on the query tag of each of the serialized feature data and the key-value library to obtain the corresponding attention-weighted feature data. A residual connection layer is used to perform residual connections between the attention-weighted feature data and each of the multi-scale feature data to obtain enhanced feature data, and the enhanced feature data is input into the decoder.

[0009] In some embodiments of this application, the decoder includes: a decoding module, a voxel-based spatial and channel compression and excitation module, an upsampling module, and an output layer; the decoding module includes a plurality of decoding stage units arranged sequentially; the upsampling module includes transposed convolutional layers corresponding to each of the decoding stage units except the first decoding stage unit; The first decoding stage unit is used to extract features from the enhanced feature data corresponding to the last decoding stage unit to obtain decoded feature data, and input the decoded feature data into the voxel-based spatial and channel squeezing and excitation module. The voxel-based spatial and channel compression and excitation module is used to recalibrate the dimensions of the channel and spatial directions of the decoded feature data to obtain optimized decoded feature data, and transmit the optimized decoded feature data to the output layer; and, if it is determined that the decoded feature data currently being processed is transmitted by a unit other than the last decoded stage unit, then the optimized decoded feature data is transmitted to the transposed convolutional layer corresponding to the next decoded stage unit. The transposed convolutional layer is used to upsample the optimized decoding feature data to obtain an upsampled feature map, and input the upsampled feature map into the corresponding decoding stage unit. The non-first decoding stage unit is used to perform feature concatenation on the received enhanced feature data and the upsampled feature map to obtain concatenated feature data, and to extract features from the concatenated feature data to obtain decoded feature data, and input the decoded feature data into the voxel-based spatial and channel squeezing and excitation module; The output layer is used to map the optimized decoding feature data corresponding to each of the decoding stage units into a pixel-level multi-class probability distribution to generate the pneumonia lesion area prediction result data corresponding to the 3D chest CT image.

[0010] In some embodiments of this application, the voxel-based spatial and channel extrusion and excitation module includes: a channel-direction extrusion and excitation unit, a spatial-direction extrusion and excitation unit, and a feature fusion layer; The channel-direction squeezing and excitation unit is used to recalibrate the channel-direction dimension of the first feature data to obtain a channel recalibrated feature map; wherein, the first feature data includes encoded feature data or decoded feature data; The spatial compression and excitation unit is used to recalibrate the spatial dimension of the first feature data to obtain a spatially recalibrated feature map. The feature fusion layer is used to add the channel recalibration feature map and the spatial recalibration feature map to obtain target feature data; the target feature data includes: multi-scale feature data or optimized decoding feature data; The compression and excitation unit in the channel direction includes: A global average pooling layer is used to compress the first feature data along the spatial dimension to obtain a channel statistics vector for capturing global context information of each channel of the first feature data. The first convolutional layer is used to reduce the channel dimension of the channel statistical vector to obtain the dimension-reduced feature map. The second ReLU activation layer is used to perform nonlinear activation processing on the dimensionality-reduced feature map based on the ReLU activation function to obtain the first activated feature map. The second convolutional layer is used to perform original channel dimension restoration processing on the first activated feature map to obtain the original dimension feature map; wherein, the original channel dimension is the channel dimension of the first feature data; The first Sigmoid activation layer is used to normalize the channel weights of the original dimensional feature map to the [0,1] interval based on the Sigmoid activation function, so as to obtain the channel weight vector. The channel rescaling layer is used to multiply the channel weight vector with the first feature data channel by channel to obtain a channel recalibrated feature map. The spatial compression and excitation unit includes: The third convolutional layer is used to compress the first feature data along the channel dimension to obtain a spatial projection tensor for fusing the spatial information of all channels of the first feature data. The second Sigmoid activation layer is used to normalize the spatial weights corresponding to the spatial projection tensor to the [0,1] interval based on the Sigmoid activation function, so as to obtain the spatial weight vector. A spatial rescaling layer is used to multiply the spatial weight vector with the first feature data position by position to obtain a spatially recalibrated feature map.

[0011] In some embodiments of this application, the classification header includes: The feature extraction layer is used to encode and decode the multi-scale feature data transmitted by the voxel-based spatial and channel compression and excitation module in sequence to obtain a deep semantic feature map. The cross-fusion Transformer module is used to sequentially perform layer normalization, feature extraction, nonlinear activation based on the ReLU activation function, downsampling, voxel embedding, convolutional projection, multi-head attention calculation, and residual connection on the deep semantic feature map to obtain the global context information feature map corresponding to the deep semantic feature map. A fully connected layer is used to perform a linear transformation on the global context information feature map to learn the high-level semantic relationships between different pneumonia patterns and output a 4-dimensional classification logical value vector. The Softmax activation layer is used to convert the 4-dimensional classification logic value vector into a probability distribution along the vector dimension based on the Softmax function, so as to obtain the pneumonia diagnosis type prediction result data corresponding to the 3D chest CT image, wherein the pneumonia diagnosis type includes: lobar, lobular, interstitial and normal.

[0012] In some embodiments of this application, the step of determining the joint loss of the segmentation sub-network and the classification sub-network in the current iteration based on the preset segmentation label, preset classification label, pneumonia lesion region prediction result data, and pneumonia diagnosis type prediction result data corresponding to the 3D chest CT image, and updating the parameters of the multi-task deep learning model based on the joint loss, includes: Based on the probability value of each voxel belonging to a specific category in the pneumonia lesion region prediction result data, the preset segmentation label, and the preset total number of voxels participating in the calculation, the Dice loss of the segmentation sub-network corresponding to the current iteration round is determined; and based on the prediction probability of each voxel being correctly classified in the pneumonia lesion region prediction result data, the preset focusing parameter, and the category balance weight, the Focal loss of the segmentation sub-network corresponding to the current iteration round is determined; the segmentation loss of the segmentation sub-network corresponding to the current iteration round is determined by the sum of the Dice loss and the Focal loss. Based on the pneumonia diagnosis type prediction results of each of the 3D chest CT images input to the multi-task deep learning model in the current iteration, the preset classification labels, the total number of 3D chest CT images input to the multi-task deep learning model in the current iteration, and the preset total number of categories, the classification loss of the segmentation sub-network in the current iteration is determined. The segmentation loss and the classification loss are linearly combined according to preset weights to obtain the joint loss for the current iteration. Based on the joint loss, the gradients of each parameter of the multi-task deep learning model are calculated using the stochastic gradient descent algorithm, and all parameters of the multi-task deep learning model are updated according to the polynomial learning rate descent strategy; wherein, the polynomial learning rate descent strategy is used to dynamically adjust the learning rate according to the ratio of the current iteration round to the total number of iteration rounds.

[0013] In some embodiments of this application, before inputting the 3D chest CT image into the multi-task deep learning model, the following steps are also included: Acquire the original 3D chest CT image and its corresponding preset segmentation and classification labels; A lung parenchyma mask is generated based on the Henle unit value of the original 3D chest CT image, and a bounding box is determined based on the lung parenchyma mask. The original 3D chest CT image is then cropped based on the bounding box to obtain the cropped image corresponding to the original 3D chest CT image. An adaptive resampling algorithm is used to process the cropped image to obtain the corresponding resampled image; The resampled image is truncated and normalized based on the statistical information of the foreground voxels in the pre-acquired training set to obtain a 3D chest CT image for training a multi-task deep learning model.

[0014] The second aspect of this application provides a method for CT imaging diagnosis of pneumonia, comprising: Acquire target 3D chest CT images; The target 3D chest CT image is input into the pneumonia CT image diagnosis model, so that the pneumonia CT image diagnosis model outputs the prediction result data of the pneumonia lesion area and the prediction result data of the pneumonia diagnosis type corresponding to the target 3D chest CT image. The pneumonia CT image diagnosis model is pre-trained based on the pneumonia CT image diagnosis model training method provided in the first aspect above.

[0015] A third aspect of this application provides a training device for a pneumonia CT image diagnostic model, comprising: An iterative training module is used to input 3D chest CT images into a multi-task deep learning model in the current iteration. This allows the shared encoder in the multi-task deep learning model to extract multi-scale feature data corresponding to the 3D chest CT images. The connection module and decoder in the multi-task deep learning model then obtain predicted pneumonia lesion regions corresponding to the 3D chest CT images based on the multi-scale feature data. Additionally, the classification head in the multi-task deep learning model obtains predicted pneumonia diagnosis types corresponding to the 3D chest CT images based on the multi-scale feature data. The shared encoder, connection module, and decoder constitute the segmentation sub-network of the multi-task deep learning model. The shared encoder and classification head also constitute the classification sub-network of the multi-task deep learning model. The joint optimization module is used to determine the joint loss of the segmentation sub-network and the classification sub-network in the current iteration based on the preset segmentation label, preset classification label, the pneumonia lesion area prediction result data and the pneumonia diagnosis type prediction result data corresponding to the 3D chest CT image, and update the parameters of the multi-task deep learning model based on the joint loss. The model output module is used to output the multi-task deep learning model with updated parameters as a pneumonia CT image diagnostic model if the current multi-task deep learning model meets the preset training termination condition.

[0016] A fourth aspect of this application provides a pneumonia CT imaging diagnostic device, comprising: Image acquisition module, used to acquire target 3D chest CT images; The model prediction module is used to input the target 3D chest CT image into the pneumonia CT image diagnosis model, so that the pneumonia CT image diagnosis model outputs the prediction result data of the pneumonia lesion area and the prediction result data of the pneumonia diagnosis type corresponding to the target 3D chest CT image. The pneumonia CT image diagnosis model is pre-trained based on the pneumonia CT image diagnosis model training method provided in the first aspect above.

[0017] A fifth aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the pneumonia CT image diagnostic model training method, and / or to implement the pneumonia CT image diagnostic method.

[0018] The sixth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the pneumonia CT image diagnostic model training method and / or the pneumonia CT image diagnostic method.

[0019] The seventh aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, implements the pneumonia CT image diagnostic model training method and / or the pneumonia CT image diagnostic method.

[0020] The pneumonia CT image diagnostic model training method provided in this application, in the current iteration, inputs a 3D chest CT image into a multi-task deep learning model, so that the shared encoder in the multi-task deep learning model extracts multi-scale feature data corresponding to the 3D chest CT image, and the connection module and decoder in the multi-task deep learning model obtain the predicted pneumonia lesion region data corresponding to the 3D chest CT image based on the multi-scale feature data, and the classification head in the multi-task deep learning model obtains the predicted pneumonia diagnosis type data corresponding to the 3D chest CT image based on the multi-scale feature data; wherein, the shared encoder, the connection module and the decoder constitute the segmentation sub-network of the multi-task deep learning model; the shared encoder also... The classification subnetwork of the multi-task deep learning model is formed by the classification head. Based on the preset segmentation labels, preset classification labels, predicted pneumonia lesion regions, and predicted pneumonia diagnosis types corresponding to the 3D chest CT images, the joint loss of the segmentation subnetwork and the classification subnetwork in the current iteration is determined, and the parameters of the multi-task deep learning model are updated based on this joint loss. If the current multi-task deep learning model meets the preset training termination condition, the multi-task deep learning model with updated parameters is output as the pneumonia CT image diagnosis model. The multi-task learning framework achieves synergistic effects between segmentation and classification. The segmentation and classification tasks share features extracted by the encoder, effectively promoting each other and improving the overall model performance. The joint loss function also ensures balanced optimization of the segmentation and classification tasks during training, avoiding overfitting of a single task. This improves the effectiveness and reliability of the results of pneumonia CT image diagnosis using this model, thereby effectively improving the model's feature utilization rate, increasing the efficiency and automation of pneumonia diagnosis, and achieving higher accuracy, precision, sensitivity, and specificity, making it a valuable tool to assist clinicians in pneumonia diagnosis.

[0021] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0022] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings: Figure 1(a) is a schematic diagram illustrating an example of a CT imaging pattern of a lobar pneumonia slice.

[0024] Figure 1(b) is a schematic diagram illustrating another example of a CT imaging pattern for a lobar pneumonia slice.

[0025] Figure 1(c) is a schematic diagram illustrating an example of a CT imaging pattern of a lobular pneumonia slice.

[0026] Figure 1(d) is a schematic diagram illustrating another example of a CT imaging pattern for a slice of lobular pneumonia.

[0027] Figure 1(e) is a schematic diagram illustrating an example of a CT imaging pattern of a section of interstitial pneumonia.

[0028] Figure 1(f) is a schematic diagram illustrating another example of a CT imaging pattern for a section of interstitial pneumonia.

[0029] Figure 2 This is a schematic diagram of the first process of training a pneumonia CT image diagnostic model according to an embodiment of this application.

[0030] Figure 3 This is a schematic diagram of the architecture of the shared encoder in a multi-task deep learning model according to an embodiment of this application.

[0031] Figure 4 This is a schematic diagram of the architecture of the cross-fusion Transformer module in one embodiment of this application.

[0032] Figure 5 This is a schematic diagram of the decoder architecture in one embodiment of this application.

[0033] Figure 6 This is a schematic diagram of the architecture of a voxel-based space and channel compression and excitation module in one embodiment of this application.

[0034] Figure 7 This is a schematic diagram of the classification head architecture in one embodiment of this application.

[0035] Figure 8 This is a schematic diagram of the second process of training a pneumonia CT image diagnostic model in one embodiment of this application.

[0036] Figure 9(a) is a comparison of lobar pneumonia images before and after preprocessing.

[0037] Figure 9(b) is a comparison of images of lobular pneumonia before and after pretreatment.

[0038] Figure 9(c) is a comparison of images of interstitial pneumonia before and after pretreatment.

[0039] Figure 9(d) is a comparison of the 2D images before and after cropping.

[0040] Figure 10 This document presents an architecture diagram of an automatic diagnostic system based on a 3D multi-task deep learning model, which serves as an application example for this application.

[0041] Figure 11 This is a flowchart illustrating the process of using an automated diagnostic system based on a 3D multi-task deep learning model to perform a pneumonia CT image diagnostic method, which serves as an application example of this application.

[0042] Figure 12 A schematic diagram of the architecture of the pneumonia CT image diagnostic model provided for the application example of this application.

[0043] Figure 13 A comparative schematic diagram of two-dimensional visualization images of lobar pneumonia segmentation results provided as an application example of this application.

[0044] Figure 14 A comparative schematic diagram of two-dimensional visualization images of lobular pneumonia segmentation results provided for application examples of this application.

[0045] Figure 15 A comparative schematic diagram of two-dimensional visualization images of interstitial pneumonia segmentation results provided as an application example of this application.

[0046] Figure 16 The micro-average ROC curves of 3D U-Net (green), nnU-Net (yellow), nnFormer (blue) and the model designed in this application (red) are provided for application examples of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0048] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the scheme according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0049] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0050] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0051] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0052] First, in one example, referring to Figures 1(a) through 1(f), CT imaging patterns of lobar pneumonia, lobular pneumonia, and interstitial pneumonia are shown, respectively. To illustrate these variations, each type of pneumonia is represented by two slices from different cases. In Figure 1(a), lobar pneumonia is characterized by well-defined, homogeneous consolidation of the lung tissue, with visible airways. The consolidation areas in both Figures 1(a) and 1(b) are highlighted in red in the right lobe. It is noteworthy that even within the same type of pneumonia, the pattern of consolidation can vary depending on location, shape, and area. In Figures 1(c) and 1(d), lobular pneumonia, also known as bronchopneumonia, presents as patchy peribronchial or centrilobular nodules that progress to dense consolidation. Patchy peribronchial and centrilobular nodules are highlighted in green in Figures 1(c) and 1(d), respectively. Diagnosing lobular pneumonia using CT images is more challenging than diagnosing lobar pneumonia due to its complexity and variability. In Figures 1(e) and 1(f), interstitial pneumonia presents with ground-glass opacities, linear-reticular patterns, scattered nodules, and patchy consolidation. Ground-glass opacities in both lungs are highlighted in blue in Figures 1(e) and 1(f). Lobular and interstitial pneumonia can have overlapping radiographic features, making them difficult to distinguish based on radiological findings alone. As shown in Figures 1(d) and 1(e), the two types share similarities in patchy opacities and consolidation. Lobular pneumonia presents as peribronchial patchy consolidation, while interstitial pneumonia shows a similar patchy ground-glass opacity pattern.

[0053] To achieve accurate and efficient pneumonia diagnosis, extensive research has been conducted on automated diagnostic systems using advanced machine learning and deep learning (DL) technologies. Some studies have attempted to classify pneumonia based on three anatomical categories. However, these early methods were primarily based on handcrafted features or traditional machine learning methods. These methods have limited ability to capture the complex and heterogeneous imaging patterns of pneumonia and often struggle to generalize across different patient populations. For example, researchers have developed a diagnostic framework integrating multiple 2D convolutional neural network (CNN) models for classifying coronavirus pneumonia, community-acquired pneumonia (CAP), and non-pneumonia cases. Studies have used squeeze and excitation (SE) blocks to enhance ResNet50, constructing a novel CT image analysis model. Researchers have employed the Inception-ResNet architecture for pneumonia classification. Similarly, researchers have combined U-Net and ResNet50 to segment and analyze CT images, introducing a weighted mask global average pooling strategy to improve lesion sensitivity and reduce background noise interference. Other studies have proposed a 3D-CNN model to distinguish between active tuberculosis (APTB) and CAP, while some researchers have used a 3D ResNet50-based framework to differentiate between coronavirus pneumonia and further tested the generalization ability of various 3D deep learning models. While these efforts have demonstrated the potential of deep learning in pneumonia diagnosis, they neglect the anatomical patterns of pneumonia, namely lobar, lobular, and interstitial pneumonia. These three patterns exhibit distinct imaging characteristics and have significant clinical implications for treatment planning and patient management. Ignoring these anatomical differences reduces the clinical value and applicability of existing models.

[0054] In detail, examples of relevant existing technologies are as follows: 1. DL model for pneumonia segmentation DL models have achieved considerable success in pneumonia segmentation. Researchers have developed a segmentation network for point-level labeled data, using transform consistency constraints to enhance the model's robustness through weakly supervised training. To address the lack of segmentation labels, researchers have introduced a pixel-level anomaly auto-modeling network within an encoder-decoder framework, achieving top-tier performance among unsupervised methods. Various 3D U-Net variants have been created to improve accuracy. Attention mechanisms can identify highly discriminative features within the network. Researchers have proposed an attention-based U-Net to capture fine structures in medical images, making it suitable for COVID-19 applications to segment lesions and lung nodules. Researchers have introduced Inf-Net, a semi-supervised segmentation framework designed to reduce reliance on labeled data. Researchers have implemented noise-resistant Dice loss in the network to better handle noisy training labels for COVID-19. UNet++ has also been applied to lesion segmentation for COVID-19. Researchers have used UNet++ as the backbone segmentation network to develop a system for detecting COVID-19 infection regions, although it is difficult to train due to its high computational cost. One study introduced a lightweight 3D CNN for coronavirus pneumonia segmentation by replacing traditional 3D convolutional layers with attention-based convolutional blocks. Another study developed a feature variation block that adaptively adjusts global features within a VNet, enhancing feature representation capabilities for different cases. A third study designed a multi-encoder segmentation framework based on 3DU-Net, using a hybrid encoder learning strategy to explore ways to enhance coronavirus pneumonia lesion segmentation performance through external prior information. Dedicated encoders and adaptive encoders were used to extract coronavirus pneumonia lesion features. In a previous study, researchers developed a 3D-SE-UNet model that fully leverages pixel-level spatial knowledge to segment lesions and classify lobar, lobular, and interstitial pneumonia types; however, this model has limitations in learning global and long-range semantic information interactions.

[0055] 2. DL model for pneumonia classification Pneumonia can be classified using various schemes. The earliest classifications stemmed from the anatomical distribution of the infection process observed in medical imaging. Advances in microbiology further led to etiologically based classifications (bacterial, viral, fungal). With the widespread use of antibiotics and increasing drug resistance, a classification scheme considering the environment in which pneumonia is acquired was introduced to guide empirical treatment. Within this framework, pneumonia is classified into community-acquired pneumonia (CAP), healthcare-associated pneumonia (HCAP), ventilator-associated pneumonia (VAP), and hospital-acquired pneumonia (HAP). Based on these classification schemes, various deep learning (DL) models have been developed for pneumonia diagnosis. Some researchers claim that CNN-based intelligent systems have improved radiologists' ability to differentiate between coronavirus pneumonia and CAP on chest CT scans. Researchers evaluated 10 state-of-the-art CNNs and found that ResNet-101 was the most effective in distinguishing between coronavirus pneumonia and non-coronavirus pneumonia cases, with an AUC of 0.994. Other researchers used a double-sampled attention network to diagnose coronavirus pneumonia and CAP, achieving a sensitivity of 86.9%. Additionally, a study proposed a 3D DL model that combines coronavirus pneumonia classification and lesion segmentation for real-time and interpretable chest CT diagnosis of coronavirus pneumonia. Some researchers have introduced a multi-scale attention network based on various advanced deep learning technologies for the automatic classification of coronavirus pneumonia and other types of pneumonia. Other studies have applied several CNN models to develop an intelligent system capable of distinguishing between coronavirus pneumonia, common pneumonia, and normal lung tissue.

[0056] 3. Multi-task DL model for pneumonia diagnosis Compared to single-task models, multi-task models focus more on the correlation features between segmentation and classification tasks. One study introduced a multi-task, multi-slice deep neural network to distinguish between COVID-19 and healthy individuals, H1N1, and CAP cases. Abdel-Basset designed a two-stage deep learning method using CT scans to differentiate between CAP and COVID-19. Another researcher developed a deep neural network utilizing CT scans and clinical metadata to distinguish COVID-19 from other viral pneumonias. A study introduced a network based on a three-level attention mechanism for disease classification, lesion segmentation, and disease severity prediction; this involves pre-training a sub-network for lesion segmentation and integrating the obtained weights into the other two tasks. Similarly, a study proposed an automated COVID-19 diagnosis system. They initially trained an independent classification network with a ResNet backbone and then fused the weights of the classification network with the segmentation network to obtain lesion segmentation results. Another study proposed a linear optimization process that uses the results of the segmentation network to guide classification. Still another researcher proposed a joint deep learning model for 3D lesion segmentation and classification to diagnose COVID-19. This network comprises three subnetworks, extracting shared features through a cross-task feature subnetwork and proposing a task-aware loss to enhance the task interaction between the classification and segmentation subnetworks. Previous research introduced MultiR-Net to combine COVID-19 classification and lesion segmentation. The proposed network includes a multi-scale feature fusion UNet-like subnetwork for lesion segmentation and a classification subnetwork for disease diagnosis. This model achieves classification of COVID-19, CAP, and healthy individuals.

[0057] 4. Transformer-based medical image models With the increasing prominence of Transformer models, several medical imaging tasks have integrated Transformer architectures. TransUNet addresses the limitations of UNet models in long-range dependency modeling by introducing Transformer into the encoder as middleware for long-range dependency modeling. Transformer-UNet introduces a Transformer encoder to directly process the raw image, achieving better performance than TransUNet. Research has proposed Medical Transformer, a gated axial attention model that introduces additional control mechanisms in its self-attention module. The UNETR model integrates Transformer into the U-Net architecture for 3D medical image segmentation. This approach leverages the Transformer's ability to model long-range dependencies in volumetric data. Swin UNETR combines Swin Transformers with a U-Net-like architecture for 3D medical image segmentation. The hierarchical structure of the Swin Transformer helps capture multi-scale features and improves segmentation performance by utilizing shifted windows for efficient self-attention computation.

[0058] In other words, existing deep learning models for pneumonia diagnosis have limitations: CNN-based solutions are limited in learning global and long-range semantic information interactions due to the inherent locality of convolutional operations; existing deep learning techniques are single-task models, ignoring the correlation between segmentation and classification tasks. If classification models can fully utilize global and long-range semantic information from the segmentation task, diagnostic accuracy can be further improved.

[0059] To address the shortcomings of existing automated pneumonia diagnosis methods, such as neglecting the inherent correlation and synergistic effect between segmentation and classification tasks, failing to fully utilize the complementary information between the two tasks, resulting in low model feature utilization and low efficiency in the pneumonia diagnosis process, this application provides a pneumonia CT image diagnostic model training method, a pneumonia CT image diagnostic method, a pneumonia CT image diagnostic model training device for executing the pneumonia CT image diagnostic model training method, a pneumonia CT image diagnostic device for executing the pneumonia CT image diagnostic method, an electronic device, a computer-readable storage medium, and a computer program product. It proposes a multi-task learning model that integrates the segmentation and classification of various pneumonia types into a unified framework. The multi-task architecture effectively enhances the learning of the correlation between segmentation and classification tasks, enabling the multi-task learning model of this application to achieve superior performance compared to various state-of-the-art models.

[0060] The following examples will provide a detailed description.

[0061] Based on this, embodiments of this application provide a method for training a pneumonia CT image diagnostic model that can be implemented by a pneumonia CT image diagnostic model training device, see [link to relevant documentation]. Figure 2 The training method for the pneumonia CT image diagnostic model specifically includes the following: Step 100: In the current iteration, a 3D chest CT image is input into a multi-task deep learning model. The shared encoder in this model extracts multi-scale feature data corresponding to the 3D chest CT image. The connection module and decoder in the model then obtain predicted pneumonia lesion regions corresponding to the 3D chest CT image based on the multi-scale feature data. The classification head in the model also obtains predicted pneumonia diagnosis types corresponding to the 3D chest CT image based on the multi-scale feature data. The shared encoder, connection module, and decoder constitute the segmentation sub-network of the multi-task deep learning model. The shared encoder and classification head also constitute the classification sub-network of the multi-task deep learning model.

[0062] It is understood that the 3D chest CT image refers to three-dimensional computed tomography scan data containing the lung region. Its original size is usually 512×512×N (N is the number of slices), which can be adjusted to the standard size of 256×256×96 after preprocessing.

[0063] In one or more embodiments of this application, the segmentation sub-network is based on the nnU-Net architecture, which is built on top of 3D U-Net. The shared encoder refers to an encoder that serves as the feature extraction backbone network of the multi-task deep learning model and is shared by the segmentation sub-network and the classification sub-network. The shared encoder can be configured based on an improved 3DU-Net architecture, containing multiple (e.g., 5) encoding stages, responsible for extracting multi-level feature representations, i.e., multi-scale feature data, from the 3D chest CT images input to the multi-task deep learning model. The multi-scale feature data corresponding to each encoding stage can include: (1) High-resolution features (early stage): capturing detailed information such as texture and edges; (2) Medium-resolution features (intermediate stage): Extracting local structural and pattern features; (3) Low-resolution features (deep stage): contain global semantics and contextual information.

[0064] In the segmentation sub-network, the connection module can adopt the skip connections in 3D U-Net. However, in order to further explore the multi-scale global context and learn long-range semantic information interaction, the cross-fusion Transformer block proposed for the first time in this application can be used to replace the skip connections of traditional U-Net. The cross-fusion Transformer block is used to receive multi-scale feature data from each stage of the shared encoder, establish long-range dependencies through a multi-head attention mechanism, and bridge the semantic gap between the encoder and the decoder. This will be described in detail in subsequent embodiments.

[0065] In the segmentation sub-network, the decoder can be configured based on an improved 3D U-Net architecture, comprising multiple (e.g., 5) decoding stages. Spatial resolution is gradually restored by processing the enhanced feature data rich in global context information output by the connection module, and feature recalibration is performed at each decoding stage. Finally, the segmentation prediction result is output, i.e., the predicted pneumonia lesion region data. The predicted pneumonia lesion region data can refer to a probability tensor of size 256×256×96×4, where each voxel contains 4 values, representing the probability of belonging to the background, lobar pneumonia region, lobular pneumonia region, and interstitial pneumonia region, respectively.

[0066] In one or more embodiments of this application, in the classification sub-network, the classification head refers to a dedicated classification branch attached to the shared encoder. It primarily utilizes multi-scale feature data from the deep stages of the encoder to further extract features, capture global semantic information, map the features into a 4-dimensional vector, and apply the Softmax function to generate the final classification probability, thereby obtaining pneumonia diagnosis type prediction data. The pneumonia diagnosis type prediction data can refer to a 4-dimensional probability vector, with the format: [P(lobular), P(small lobular), P(interstitial), P(normal)], satisfying the probability normalization condition.

[0067] Step 200: Based on the preset segmentation label, preset classification label, pneumonia lesion region prediction result data, and pneumonia diagnosis type prediction result data corresponding to the 3D chest CT image, determine the joint loss of the segmentation sub-network and the classification sub-network in the current iteration round, and update the parameters of the multi-task deep learning model based on the joint loss.

[0068] In step 200, the preset segmentation label can adopt the pixel-level segmentation gold standard annotated by radiology experts, and the size is consistent with the prediction result; the preset classification label can adopt a one-hot encoded vector based on clinical diagnosis results.

[0069] Step 300: If the current multi-task deep learning model meets the preset training termination condition, then the multi-task deep learning model with updated parameters is output as a pneumonia CT image diagnosis model.

[0070] In step 300, the training termination condition may include any of the following: reaching the maximum number of iterations, the validation set performance no longer improving, or the joint loss value converging. For example: the maximum number of iterations (250 rounds) has been reached, the validation set performance has not improved for 10 consecutive rounds, or the joint loss value has converged (rate of change < 1e-5).

[0071] It is understood that the pneumonia CT image diagnostic model refers to a multi-task deep learning model that has been trained and has fixed parameters. It has the ability to simultaneously segment pneumonia lesions and diagnose their types. It can also be validated and tested before output. For example, if it achieves a Dice coefficient of 71.2% and a classification accuracy of 93.7% on the test set, it can be used online.

[0072] As described above, the pneumonia CT image diagnostic model training method provided in this application achieves synergistic effects between segmentation and classification through a multi-task learning framework. The segmentation and classification tasks share features extracted by the encoder, effectively promoting each other and improving the overall model performance. The use of a joint loss function ensures balanced optimization of the segmentation and classification tasks during training, avoiding overfitting of a single task. This improves the effectiveness and reliability of the results obtained from diagnosing pneumonia CT images using this model. Furthermore, it effectively increases the utilization rate of model features, enhances the efficiency and automation of pneumonia diagnosis, and achieves higher accuracy, precision, sensitivity, and specificity, making it a valuable tool to assist clinicians in diagnosing pneumonia.

[0073] To further address the problems in existing image diagnostic methods, such as the encoder's inability to adaptively calibrate the importance of feature maps, the unclear feature sharing and distribution paths in multi-task learning, and the impact of the order of downsampling and feature enhancement on network performance, this application provides a pneumonia CT image diagnostic model training method, see [link to relevant documentation]. Figure 3 The shared encoder in the pneumonia CT image diagnostic model training method specifically includes the following: The system includes an encoding module, a voxel-based spatial and channel squeeze & excitation (vscSE) module, and a downsampling module.

[0074] The encoding module includes a plurality of encoding stage units arranged sequentially; the downsampling module includes a first max pooling layer corresponding to each of the encoding stage units except the first encoding stage unit.

[0075] It should be noted that the shared encoder is an improved architecture based on the 3D U-Net encoding path, specifically designed to simultaneously serve the two subtasks of segmentation and classification. It modularizes the feature extraction, feature enhancement, and downsampling processes, and achieves intelligent feature distribution through conditional judgment. In one example, the encoding module in the shared encoder can consist of five sequentially connected encoding stage units, forming a hierarchical structure for feature extraction: a first encoding stage unit, a second encoding stage unit, a third encoding stage unit, a fourth encoding stage unit, and a fifth encoding stage unit executed sequentially. Each encoding stage unit can contain two 3×3×3 convolutional layers, each followed by instance normalization and a Leaky ReLU activation function.

[0076] The first encoding stage unit is used to extract features from the 3D chest CT image to obtain encoded feature data, and input the encoded feature data into the voxel-based spatial and channel compression and excitation module.

[0077] In other words, the first encoding stage unit directly receives the preprocessed 3D chest CT image (size: 256×256×96×1), and outputs 32-channel encoded feature data after convolution. This encoded feature data refers to the intermediate feature representation extracted by the convolutional neural network, which contains local pattern, texture, and edge information learned from the 3D chest CT image.

[0078] The voxel-based spatial and channel squeezing and excitation (vscSE) module is used to recalibrate the dimensions of the channel and spatial directions of the encoded feature data to obtain multi-scale feature data, and transmit the multi-scale feature data to the connection module; and, if it is determined that the encoded feature data currently being processed is transmitted by the last encoded stage unit, then the multi-scale feature data is transmitted to the classification head in the classification sub-network; if it is determined that the encoded feature data currently being processed is transmitted by a non-last encoded stage unit, then the multi-scale feature data is transmitted to the first max pooling layer corresponding to the next encoded stage unit.

[0079] It should be noted that the vscSE module is the core innovative component of the shared encoder, responsible for adaptive recalibration of features: channel orientation recalibration and spatial orientation recalibration, so that the multi-scale feature data after recalibration by the vscSE module retains important channel and spatial information.

[0080] Additionally, when the vscSE module processes the input of the last encoding stage unit, it transmits the multi-scale feature data to the classification head in the classification sub-network; when the vscSE module processes the input of a non-last encoding stage unit, it transmits the multi-scale feature data to the first max pooling layer of the next stage.

[0081] The first max pooling layer is used to downsample the multi-scale feature data to obtain a first downsampled feature map, and inputs the first downsampled feature map into the corresponding encoding stage unit.

[0082] Specifically, the downsampling module consists of four first max-pooling layers, corresponding to the second to fifth encoding stage units. Each first max-pooling layer uses a 2×2×2 kernel size with a stride of 2. It receives multi-scale feature data from the previous stage's vscSE module, halves the spatial size, doubles the number of channels through pooling, and then outputs the first downsampled feature map to its corresponding encoding stage unit. The first downsampled feature map refers to the feature representation after spatial downsampling, possessing a larger receptive field and a higher level of feature abstraction. The increased number of channels also enhances the expressive power of the features.

[0083] The non-first encoding stage unit is used to extract features from the downsampled feature map to obtain encoded feature data, and input the encoded feature data into the voxel-based spatial and channel squeezing and excitation module.

[0084] Specifically, starting from the second encoding stage unit, the processing procedure of each encoding stage unit is as follows: receive the first downsampled feature map from the previous stage (i.e., its corresponding first max pooling layer), perform deeper feature extraction through two 3×3×3 convolutional layers, and send the output encoded feature data into the vscSE module for feature enhancement.

[0085] As can be seen from the above description, the pneumonia CT image diagnostic model training method provided in this application can simultaneously enhance important features and suppress secondary features from both channel and spatial dimensions; it can dedicate deep semantic features to classification tasks and use full-scale features for segmentation tasks; and it can also ensure the optimal execution order of feature extraction, enhancement and downsampling.

[0086] To further address the semantic gap issues inherent in traditional skip connections in existing image diagnostic methods, the limitations of CNN models in capturing long-range semantic dependencies, and the poor performance of multi-scale feature fusion, this application provides a pneumonia CT image diagnostic model training method. The connection module in this method specifically includes the following components: The cross-fusion Transformer module is one of the core innovations of this application, used to replace the simple skip connections in traditional U-Net. This module is specifically designed to capture multi-scale global contextual information and bridge the semantic gap between the encoder and decoder.

[0087] See Figure 4 The cross-fusion Transformer (CTB) module specifically includes the following components: (1) Layer Normalization layer, used to normalize each of the multi-scale feature data currently received from the shared encoder along the channel dimension, so as to obtain the normalized feature map corresponding to each of the multi-scale feature data.

[0088] Specifically, the layer normalization layer normalizes the multi-scale feature data of the five stages of the input self-shared encoder. That is, the layer normalization normalizes each multi-scale feature data along the channel dimension to obtain a normalized feature map. The size of the normalized feature map is consistent with the input scale feature data, but it has a more stable numerical distribution.

[0089] (2) Two consecutive 3D convolutional layers are used to extract features from each of the normalized feature maps to obtain the convolutional feature maps corresponding to each of the multi-scale feature data.

[0090] Specifically, two consecutive 3D convolutional layers constitute a feature extraction network. Each 3D convolutional layer can use a 3×3×3 convolutional kernel with a stride of 1 and padding of 1. This is used to extract deep features from each normalized feature map separately, and output a convolutional feature map with unchanged channel number and spatial size.

[0091] (3) The first ReLU activation layer is used to perform non-linear activation processing on each of the convolutional feature maps based on the ReLU activation function, so as to obtain the second activated feature map corresponding to each of the multi-scale feature data.

[0092] Specifically, the first ReLU activation layer introduces non-linear expressive power to the convolutional feature map based on the ReLU activation function, enhances the model's ability to fit complex functions, and outputs the second activated feature map.

[0093] (4) The second max pooling layer is used to downsample each of the second activated feature maps to obtain the second downsampled feature map corresponding to each of the multi-scale feature data.

[0094] Specifically, the pooling parameters of the second max-pooling layer can be set to: 2×2×2 kernel size, stride = 2. This reduces subsequent computation while expanding the receptive field, resulting in a second downsampled feature map with half the spatial size.

[0095] (5) Voxel Embedding layer, used to flatten each of the second downsampled feature maps into a sequence, so as to obtain the serialized feature data corresponding to each of the multi-scale feature data.

[0096] Specifically, the voxel embedding layer is used to realize the 3D to 2D transformation by flattening each second downsampled feature map into a vector sequence, thereby outputting serialized feature data, which is convenient for subsequent Transformer processing.

[0097] (6) Convolutional Projection layer, used to generate corresponding query token Q, key token K and value token V for each of the serialized feature data.

[0098] Specifically, the convolutional projection layer uses 1×1×1 convolution instead of nonlinear projection to preserve spatial structure information and generate attention markers. The query marker Q represents the content to be focused on; the key marker K represents the content being compared; and the value marker V represents the actual information being transmitted. This effectively solves the problem of spatial information loss in traditional Transformers.

[0099] (7) Multi-head Attention layer, used to concatenate the key and value tags of each of the serialized feature data to obtain a key-value library, and to perform multi-head attention calculation on the query tag of each of the serialized feature data and the key-value library to obtain the corresponding attention-weighted feature data.

[0100] Specifically, the multi-head attention layer can be set to 2 heads to balance performance and computational efficiency. By concatenating the key and value tags of each stage to form a unified key-value library, multi-head attention calculation is then performed to obtain attention-weighted feature data containing rich cross-scale dependencies.

[0101] (8) Residual Connection layer, used to perform residual connection between the attention-weighted feature data and each of the multi-scale feature data to obtain enhanced feature data, and input the enhanced feature data into the decoder.

[0102] Specifically, the residual connection layer adds the attention-weighted feature data to each of the multi-scale feature data element by element to promote gradient flow, prevent network degradation, and preserve the original feature information.

[0103] As can be seen from the above description, the pneumonia CT image diagnostic model training method provided in this application has the following features: the CTB module captures global contextual information through a multi-head attention mechanism, effectively bridging the semantic gap between the encoder and decoder; convolutional projection can preserve spatial information, overcoming the problem of spatial information loss in traditional Transformers; and through multi-scale feature fusion capabilities, it can significantly improve the model's recognition accuracy for complex pneumonia patterns.

[0104] To further address the problems of detail loss during decoder feature reconstruction, insufficient fusion of upsampled and encoder features, and blurred segmentation boundaries and insufficient accuracy in existing image diagnostic methods, this application provides a pneumonia CT image diagnostic model training method, see [link to relevant documentation]. Figure 5 The decoder in the pneumonia CT image diagnostic model training method specifically includes the following components: The module consists of a decoding module, a voxel-based spatial and channel compression and excitation module, an upsampling module, and an output layer.

[0105] Specifically, the decoder is a multi-scale reconstruction network based on an improved 3D U-Net decoding path. It is responsible for progressively restoring the spatial resolution from the deep semantic features extracted by the encoder, generating an accurate segmentation map of pneumonia lesions. It integrates multi-scale feature fusion, dual feature optimization, and deep supervision mechanisms.

[0106] The decoding module comprises multiple decoding stage units arranged sequentially, such as the first, second, third, fourth, and fifth decoding stage units executed in sequence, forming a progressive reconstruction path from deep features to fine segmentation. Each decoding stage unit contains two 3×3×3 convolutional layers equipped with instance normalization and Leaky ReLU activation functions to progressively fuse deep semantic information with shallow detailed features, restoring lesion boundaries and spatial structure.

[0107] The upsampling module includes transposed convolutional layers corresponding to each of the decoding stage units except for the first decoding stage unit.

[0108] Specifically, the upsampling module can consist of four transposed convolutional layers to achieve gradual restoration of spatial resolution.

[0109] The first decoding stage unit is used to extract features from the enhanced feature data corresponding to the last decoding stage unit to obtain decoded feature data, and input the decoded feature data into the voxel-based spatial and channel squeezing and excitation module in the decoder.

[0110] Specifically, the first decoding stage unit, as the starting point of the decoding path, directly receives the enhanced feature data (corresponding to the deepest stage of the encoder) from the connection module, performs preliminary feature transformation through convolution operations, outputs decoded feature data, and then transmits the decoded feature data to the vscSE module for optimization.

[0111] The voxel-based spatial and channel compression and excitation module is used to recalibrate the dimensions of the channel and spatial directions of the decoded feature data to obtain optimized decoded feature data, and transmit the optimized decoded feature data to the output layer; and if it is determined that the decoded feature data currently being processed is transmitted by a unit other than the last decoded stage unit, then the optimized decoded feature data is transmitted to the transposed convolutional layer corresponding to the next decoded stage unit.

[0112] Specifically, the vscSE module in the decoder plays a crucial role in feature refinement. It optimizes the channel orientation by adaptively adjusting the contribution weights of each feature channel and optimizes the spatial orientation by highlighting the response intensity of important spatial regions, outputting optimized decoded feature data containing enhanced discriminative features. If the input being processed is not from the last decoding stage unit, the optimized decoded feature data is transferred to the transposed convolutional layer and the output layer of the next stage; if the input being processed is from the last decoding stage unit, only the optimized decoded feature data is transferred to the output layer (achieving multi-scale depth supervision).

[0113] The transposed convolutional layer is used to upsample the optimized decoding feature data to obtain an upsampled feature map, and input the upsampled feature map into the corresponding decoding stage unit.

[0114] Specifically, the transposed convolutional layer can use a 2×2×2 kernel size and a stride of 2. By transposing the convolution, the spatial size is doubled while the number of channels is halved, resulting in an upsampled feature map. Transposed convolution, through learning a parameterized upsampling process, can recover detailed information better than simple interpolation methods.

[0115] The non-first decoding stage unit is used to perform feature concatenation on the received enhanced feature data and the upsampled feature map to obtain concatenated feature data, and to extract features from the concatenated feature data to obtain decoded feature data, and input the decoded feature data into the voxel-based spatial and channel squeezing and excitation module in the decoder.

[0116] Specifically, starting from the second decoding stage unit, a feature concatenation and fusion strategy is adopted for processing. That is, the enhanced feature data and the upsampled feature map are concatenated along the channel dimension to ensure accurate spatial alignment (center cropping is performed if necessary). Then, the concatenated features are deeply fused through two 3×3×3 convolutional layers to obtain the decoded feature data.

[0117] The output layer is used to map the optimized decoding feature data corresponding to each of the decoding stage units into a pixel-level multi-class probability distribution to generate the pneumonia lesion area prediction result data corresponding to the 3D chest CT image.

[0118] Specifically, the output layer adopts a multi-scale deep supervision architecture, integrating optimized features from all decoding stages. That is, it receives optimized decoding feature data from all 5 decoding stage units, upsamples feature maps of different scales to a uniform size (256×256×96), concatenates features of all scales along the channel dimension, and then maps the number of channels to the number of classes K=4 through 1×1×1 convolution. Finally, it applies the Softmax function to generate a probability distribution along the channel dimension and outputs the predicted data of the pneumonia lesion area with a size of 256×256×96×4.

[0119] As can be seen from the above description, the pneumonia CT image diagnostic model training method provided in this application can effectively restore spatial resolution while preserving detailed information by using progressive upsampling combined with feature stitching; by integrating the vscSE module in the decoder, secondary optimization can be performed during feature reconstruction; and by designing a multi-stage decoding architecture, the segmentation results can be gradually refined to improve boundary accuracy.

[0120] To further address the problems of imbalanced importance of channels and spatial locations in feature maps, computationally complex existing attention mechanisms making them difficult to apply effectively in 3D medical imaging, and insufficient integration of local features with global context in existing image diagnostic methods, this application provides a pneumonia CT image diagnostic model training method, see [link to relevant documentation]. Figure 6 The voxel-based spatial and channel compression and excitation module in the pneumonia CT image diagnostic model training method specifically includes the following: The compression and excitation units in the channel direction, the compression and excitation units in the spatial direction, and the feature fusion layer.

[0121] The channel-direction squeezing and excitation unit is used to recalibrate the channel-direction dimension of the first feature data to obtain a channel recalibrated feature map; wherein, the first feature data includes encoded feature data or decoded feature data.

[0122] The compression and excitation unit in the channel direction specifically includes the following components: (1) A global average pooling layer is used to compress the first feature data along the spatial dimension to obtain a channel statistics vector for capturing the global context information of each channel of the first feature data.

[0123] Specifically, the first feature data It can be viewed as a set of channels ;in, C represents the number of channels; to achieve space compression, this application applies a global average pooling layer to obtain a voxel-based vector, namely the channel statistics vector. : (Formula 1) In Formula (I), H represents the height of the first feature data, i.e., the size of the spatial dimension in the vertical direction; W represents the input width of the first feature data, i.e., the size of the spatial dimension in the horizontal direction; D represents the depth of the first feature data, i.e., the size of the spatial dimension in the depth direction; h represents the summation index of the height dimension, from 1 to H, used to traverse all height positions; w represents the summation index of the width dimension, from 1 to W, used to traverse all width positions; and d represents the summation index of the depth dimension, from 1 to D, used to traverse all depth positions. This represents the value of the first feature data at channel c and spatial location (h,w,d).

[0124] This operation embeds global spatial information into the channel statistical vector z.

[0125] (2) The first convolutional layer is used to reduce the channel dimension of the channel statistical vector to obtain the dimension-reduced feature map.

[0126] (3) The second ReLU activation layer is used to perform nonlinear activation processing on the dimensionality-reduced feature map based on the ReLU activation function to obtain the first activated feature map.

[0127] (4) The second convolutional layer is used to perform original channel dimension recovery processing on the first activated feature map to obtain the original dimension feature map; wherein the original channel dimension is the channel dimension of the first feature data.

[0128] (5) The first Sigmoid activation layer is used to normalize the channel weights of the original dimensional feature map to the [0,1] interval based on the Sigmoid activation function to obtain the channel weight vector.

[0129] The first convolutional layer and the second ReLU activation layer constitute a fully connected layer, and the second convolutional layer and the first Sigmoid activation layer constitute another fully connected layer.

[0130] In other words, after passing through the first convolutional layer and the first sigmoid activation layer, the channel statistics vector z is transformed into... ,in and These are the weights of two fully connected layers. It's the ReLU operator. This transformation encodes the dependencies between channels by passing it through a sigmoid layer. ,Will The dynamic range is adjusted to the interval [0, 1]. The resulting channel statistics vector is then used for recalibration or excitation X.

[0131] (6) Channel rescaling layer, used to multiply the channel weight vector with the first feature data channel by channel to obtain a channel recalibrated feature map.

[0132] Specifically, channel recalibration feature map The following formula (2) is used for calculation: (Formula 2) In Formula 2, Indicates the channel compression and excitation function; , , ..., These refer to the first feature data respectively. X 3D feature maps of each channel; , , ..., These values ​​reflect the weight coefficients of each channel and are then rescaled. As the network trains, these activation values ​​are adaptively adjusted to reduce the influence of less important channels while enhancing the influence of important channels.

[0133] The spatial compression and excitation unit is used to recalibrate the spatial dimension of the first feature data to obtain a spatially recalibrated feature map.

[0134] The spatial compression and excitation unit specifically includes the following components: (1) The third convolutional layer is used to compress the first feature data along the channel dimension to obtain a spatial projection tensor for fusing the spatial information of all channels of the first feature data.

[0135] (2) The second Sigmoid activation layer is used to normalize the spatial weights corresponding to the spatial projection tensor to the [0,1] interval based on the Sigmoid activation function, so as to obtain the spatial weight vector.

[0136] Specifically, the third convolutional layer (channel compression) compresses the first feature data into a single-channel spatial weight map, i.e., a spatial projection tensor. The second sigmoid activation layer is used to normalize the spatial weights to the [0,1] interval.

[0137] In other words, for the first feature data X= Another representation of, in Corresponding to spatial locations (h, w, d), where , , Spatial compression operations use convolution. Execution, weight This yields a projection tensor. Each in the projection This represents a linear combination of all channels C at spatial location (h, w, d). This projection is then passed through a sigmoid layer. This allows the activation values ​​to be rescaled to [0, 1], thereby facilitating spatial recalibration or excitation of X.

[0138] (3) Spatial rescaling layer, used to multiply the spatial weight vector with the first feature data position by position to obtain a spatial recalibrated feature map.

[0139] Specifically, spatial recalibration feature maps The following formula (3) is used for calculation: (Formula 3) In Formula 3, Represents space compression and excitation function; These represent the C-dimensional feature vectors corresponding to the first feature data at each spatial location. , ,…, Each C-dimensional feature vector has its own spatial location weight coefficient, representing the relative importance of spatial locations (h, w, d) in the first feature data. This recalibration enhances the importance of relevant spatial locations while ignoring irrelevant locations.

[0140] The feature fusion layer is used to add the channel recalibration feature map and the spatial recalibration feature map to obtain target feature data; the target feature data includes: multi-scale feature data or optimized decoding feature data.

[0141] Specifically, the feature fusion layer is the final processing stage of the VSCSE module. It is responsible for integrating and fusing the outputs of the two recalibrated paths in the channel and spatial directions to generate the final target feature data. .

[0142] Right now: .

[0143] As can be seen from the above description, the pneumonia CT image diagnostic model training method provided in this application can optimize the importance weights of the channel and spatial dimensions respectively through the dual-path recalibration design; it can control the computational complexity while ensuring performance through the lightweight module design; and it can improve the feature discrimination ability by combining local feature enhancement with global context awareness.

[0144] To further address the shortcomings of existing imaging diagnostic methods, such as insufficient utilization of global contextual information in classification tasks, inadequate extraction of pneumonia type discrimination features, and poor interpretability of classification results, this application provides a pneumonia CT image diagnostic model training method, see [link to relevant documentation]. Figure 7 The classification head in the pneumonia CT image diagnostic model training method specifically includes the following: (1) Feature extraction layer, used to encode and decode the multi-scale feature data transmitted by the voxel-based spatial and channel compression and excitation module in sequence to obtain a deep semantic feature map.

[0145] Specifically, the feature extraction layer receives multi-scale feature data from the last vscSE module of the encoder. It further refines and transforms the features through two encoding convolutional layers (3×3×3 convolutions for extracting deeper features) and two decoding convolutional layers (3×3×3 convolutions for adjusting feature representations to adapt to classification tasks), thereby enhancing the ability to discriminate pneumonia patterns and obtaining a deep semantic feature map.

[0146] (2) Cross-fusion Transformer module, used to perform layer normalization, feature extraction, nonlinear activation based on ReLU activation function, downsampling, voxel embedding, convolutional projection, multi-head attention calculation and residual connection on the deep semantic feature map in sequence to obtain the global context information feature map corresponding to the deep semantic feature map.

[0147] It is understood that the cross-fusion Transformer module in the decoder and the cross-fusion Transformer module in the encoder can adopt the exact same architectural settings.

[0148] (3) Fully connected layer, used to perform linear transformation on the global context information feature map to learn the high-level semantic relationship between different pneumonia patterns and output a 4-dimensional classification logical value vector.

[0149] Specifically, the fully connected layer achieves high-level semantic mapping by performing a linear transformation on the global context information feature map. The meanings of each vector in the 4-dimensional classification logical value vector z=[z1,z2,z3,z4] are as follows: 1) z1: Raw score for lobar pneumonia; 2) z2: Raw score for lobular pneumonia; 3) z3: Raw score for interstitial pneumonia; 4) z4: Normal raw score.

[0150] (4) Softmax activation layer, used to convert the 4-dimensional classification logic value vector into a probability distribution along the vector dimension based on the Softmax function, so as to obtain the pneumonia diagnosis type prediction result data corresponding to the 3D chest CT image, wherein the pneumonia diagnosis type includes: lobar, lobular, interstitial and normal.

[0151] Specifically, the Softmax activation layer transforms logistic values ​​into a probability distribution, outputting pneumonia diagnosis type prediction data P=[p1,p2,p3,p4], where: 1) p 1: The probability of lobar pneumonia (which can be simply referred to as lobar pneumonia) is ∈ [0,1]; 2) p2: The probability of lobular pneumonia (which can be simply referred to as lobular) ∈ [0,1]; 3) p3: The probability of interstitial pneumonia (which can be simply referred to as interstitial) is ∈ [0,1]; 4) p4: The probability of normality is ∈ [0,1].

[0152] It satisfies: p1+p2+p3+p4=1.

[0153] As can be seen from the above description, the pneumonia CT image diagnostic model training method provided in this application can provide rich global semantic information for classification tasks by designing a dedicated CTB module; it can effectively learn the high-level semantic features of pneumonia types by performing deep feature extraction and transformation; and it can effectively enhance the credibility and interpretability of classification results by outputting a clear probability distribution.

[0154] To further address the problems of class imbalance in medical image segmentation, uneven contribution of easy and difficult samples to model training, and difficulty in allocating multi-task loss weights in existing image diagnostic methods, this application provides a pneumonia CT image diagnostic model training method, see [link to relevant documentation]. Figure 8 Step 200 in the pneumonia CT image diagnostic model training method specifically includes the following: Step 210: Based on the probability value of each voxel in the pneumonia lesion region prediction result data belonging to a specific category, the preset segmentation label, and the preset total number of voxels participating in the calculation, determine the Dice loss of the segmentation sub-network in the current iteration round; and based on the prediction probability of each voxel in the pneumonia lesion region prediction result data being correctly classified, the preset focusing parameter, and the category balance weight, determine the Focal loss of the segmentation sub-network in the current iteration round; determine the segmentation loss of the segmentation sub-network in the current iteration round based on the sum of the Dice loss and the Focal loss.

[0155] Specifically, the Dice loss is calculated based on the overlap between the predicted segmentation map and the ground truth label: (Formula 4) In Formula 4, Indicates Dice loss; The preset segmentation label of the i-th voxel in the predicted pneumonia lesion area data is the actual segmentation label value (0 or 1). The probability value (between 0 and 1) represents the probability value of the i-th voxel in the predicted data of the pneumonia lesion area belonging to a specific category; N represents the total number of voxels involved in the calculation, N = H × W × D × batch size.

[0156] Focal loss addresses the imbalance between easy and difficult samples by using a modulation factor: (Formula 5) In Formula 5, This represents the probability that the model predicts the correct category; Indicates Focal loss; This represents the class balancing weights, used to address the imbalance between positive and negative samples, in pneumonia segmentation: =0.25 (foreground weight), background weight is 0.75; The focal parameter is a constant, typically set to 2. Focal loss is achieved by introducing a modulation factor. This reduces the contribution of easily classified samples and addresses the imbalance between easy and difficult samples. This allows the model to focus more on the challenging samples.

[0157] in, ; y is the preset segmentation label; p is the probability of a correct prediction; that is: if y=1, then =p; otherwise =1-p.

[0158] Segmentation loss This is the sum or weighted sum of the Dice loss and the Focal loss, such as: (Formula 6) And, step 220: based on the pneumonia diagnosis type prediction results of each of the 3D chest CT images input to the multi-task deep learning model in the current iteration round, the preset classification labels, the total number of the 3D chest CT images input to the multi-task deep learning model in the current iteration round, and the preset total number of categories, determine the classification loss corresponding to the segmentation sub-network in the current iteration round.

[0159] Specifically, the classification loss is calculated using the classification cross-entropy loss function: (Formula 7) In Formula 7, It is classification loss; B M is the batch size, i.e., the number of samples (CT scans) in the current batch; M is the number of pneumonia diagnosis types, M = 4 (lobar, lobular, interstitial, normal). j The sample index is 1 to n; m is the category index, from 1 to 4. Indicates the preset category labels; This represents the predicted results of pneumonia diagnosis types.

[0160] Step 230: Combine the segmentation loss and the classification loss linearly according to preset weights to obtain the joint loss for the current iteration round.

[0161] Specifically, to balance the losses of segmentation and classification tasks on the same scale and enhance the effectiveness of multi-task learning, the classification loss and segmentation loss are linearly combined using the hyperparameter λ to form a multi-task loss. Joint Loss Defined as: (Formula 8) In Formula 8, These are the weights of the classification loss.

[0162] Step 240: Based on the joint loss, calculate the gradient of each parameter of the multi-task deep learning model using the stochastic gradient descent algorithm, and update all parameters of the multi-task deep learning model according to the polynomial learning rate descent strategy; wherein, the polynomial learning rate descent strategy is used to dynamically adjust the learning rate according to the ratio of the current iteration round to the total number of iteration rounds.

[0163] Specifically, all models are trained from scratch and evaluated using five-fold cross-validation on the training set.

[0164] During training, the SGD optimizer is used with an initial learning rate of 1e-2. The learning rate is adjusted using a Poly learning rate strategy, defined as: (Formula 9) In Formula Nine, It's the new learning rate. is the initial learning rate, t is the current number of cycles, T is the maximum number of cycles, and exp is a parameter that controls the shape of the learning rate curve, set to 0.9.

[0165] As can be seen from the above description, the pneumonia CT image diagnostic model training method provided in this application can effectively solve the problems of foreground-background imbalance and easy-sampling imbalance by combining Dice loss and Focal loss; by setting adjustable loss weight parameters, it can achieve balanced optimization of segmentation and classification tasks; and by adopting a dynamic learning rate strategy, it can ensure stable convergence of training and improve the final performance of the model.

[0166] To further address the problems of inconsistency in data acquired by different CT devices, the impact of background interference on model learning, and insufficient model generalization ability caused by differences in grayscale distribution in existing imaging diagnostic methods, this application provides a pneumonia CT image diagnostic model training method, see [link to relevant documentation]. Figure 8 The training method for the pneumonia CT image diagnostic model specifically includes the following content before step 100: Step 010: Obtain the original 3D chest CT image and the corresponding preset segmentation labels and preset classification labels.

[0167] Step 020: Generate a lung parenchyma mask based on the Henle unit value of the original 3D chest CT image, determine a bounding box based on the lung parenchyma mask, and crop the original 3D chest CT image based on the bounding box to obtain the cropped image corresponding to the original 3D chest CT image.

[0168] Step 030: The cropped image is processed using an adaptive resampling algorithm to obtain the corresponding resampled image.

[0169] Step 040: The resampled image is truncated, and the resampled image is normalized based on the statistical information of the foreground voxels in the pre-acquired training set to obtain a 3D chest CT image for training a multi-task deep learning model.

[0170] Specifically, the preprocessing process enhances spatial semantic learning, including pruning, resampling, and normalization.

[0171] In the cropping step, all CT images are cropped to the lung parenchyma region. CT images are characterized by varying grayscale values, reflecting the degree to which organs and tissues absorb X-rays. The absorption coefficient of X-rays indicates density and is measured in Henle units (HU). The HU value of lung tissue typically hovers around -500, sometimes exceeding this value, such as -320 or -400. By separating areas in the lung CT scan with HU values ​​exceeding -500, the contour of the lung tissue can be delineated. A fill function is then applied to this contour, creating a lung mask. Furthermore, by performing a bitwise XOR operation on the lung mask and the lung contour, a mask representing only the lung cavity region can be generated. The slice with the largest area is selected from the axial and coronal views of the mask. Boundary values ​​are then established using the height h, width w, and depth d coordinates of the selected slice, defining a bounding box for cropping the lung parenchyma region. Implementing this bounding box on the original image helps extract pixels corresponding to the lung region. This extraction process results in the separation of the lung parenchyma, which is the primary region of interest. The cropping step helps eliminate interference from irrelevant information, ultimately contributing to the learning of spatial semantics.

[0172] Excessive spacing can hinder the network's ability to effectively learn spatial context information. Therefore, before inputting CT images into the network for training, this application applies resampling and normalization steps to further improve the region to be segmented. This application uses Algorithm 1 as shown in Table 1 for resampling and Algorithm 2 as shown in Table 2 for normalization.

[0173] Table 1 Table 2 Nearest neighbor interpolation is used for axes with large spacing, while cubic spline interpolation is used for other axes to ensure that each voxel in different image data has a consistent physical spatial representation. Normalization ensures that the gray values ​​of each image in the training set have the same distribution. Z-score normalization is applied by subtracting the mean and dividing by the standard deviation. The processed data follows a standard normal distribution with a mean of 0 and a standard deviation of 1. The mean and standard deviation are calculated using statistics from the entire foreground of the training set. Utilizing HU values ​​(which reflect the physical characteristics of different tissues in CT images), the statistics from the entire foreground effectively capture the additional information encoded in the HU values.

[0174] Figures 9(a) through 9(c) show the 3D images before and after preprocessing to illustrate how cropping, resampling, and normalization ensure data consistency, reduce computational requirements, and enhance the reliability of segmentation results. Figure 9(a) , 9(b)In Figure 9(c), the left image represents the original 3D chest CT image before preprocessing, and the right image represents the preprocessed 3D chest CT image. Irrelevant background information in the original 3D chest CT image is highlighted with white circles. Figure 9(d) illustrates the effect of cropping in 2D. By reducing the input image size, cropping reduces the computational resources required for training and inference. Simultaneously, cropping eliminates irrelevant background information, allowing the network to focus on important anatomical structures, which improves segmentation accuracy.

[0175] During training, data augmentation techniques such as random rotation, random scaling, random elastic transformation, gamma correction, and mirroring are employed to increase the amount of training data and enhance the model's generalization ability.

[0176] As can be seen from the above description, the pneumonia CT image diagnostic model training method provided in this application can ensure the consistency and comparability of input data by designing a standardized preprocessing process; it can eliminate irrelevant background interference by accurately extracting the lung parenchyma region; and it can enhance the model's generalization ability to data from different sources by designing adaptive data normalization.

[0177] Based on the above-described embodiments of the pneumonia CT image diagnostic model training method, this application also provides embodiments of a pneumonia CT image diagnostic method, which specifically includes the following: Step 400: Acquire the target 3D chest CT image.

[0178] Step 500: Input the target 3D chest CT image into the pneumonia CT image diagnosis model, so that the pneumonia CT image diagnosis model outputs the prediction result data of the pneumonia lesion area and the prediction result data of the pneumonia diagnosis type corresponding to the target 3D chest CT image, wherein the pneumonia CT image diagnosis model is pre-trained based on the pneumonia CT image diagnosis model training method.

[0179] The pneumonia CT image diagnostic model used in step 500 of this application can be specifically referred to the architecture of the pneumonia CT image diagnostic model trained by the pneumonia CT image diagnostic model training method in the above embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the above pneumonia CT image diagnostic model training method embodiments.

[0180] As can be seen from the above description, the pneumonia CT imaging diagnostic method provided in this application embodiment can significantly improve clinical work efficiency, reduce the influence of human factors, and reduce the risk of missed diagnosis and misdiagnosis.

[0181] From a software perspective, this application also provides a pneumonia CT image diagnostic model training device for executing all or part of the pneumonia CT image diagnostic model training method, wherein the pneumonia CT image diagnostic model training device specifically includes the following: An iterative training module is used to input 3D chest CT images into a multi-task deep learning model in the current iteration. This allows the shared encoder in the multi-task deep learning model to extract multi-scale feature data corresponding to the 3D chest CT images. The connection module and decoder in the multi-task deep learning model then obtain predicted pneumonia lesion regions corresponding to the 3D chest CT images based on the multi-scale feature data. Additionally, the classification head in the multi-task deep learning model obtains predicted pneumonia diagnosis types corresponding to the 3D chest CT images based on the multi-scale feature data. The shared encoder, connection module, and decoder constitute the segmentation sub-network of the multi-task deep learning model. The shared encoder and classification head also constitute the classification sub-network of the multi-task deep learning model. The joint optimization module is used to determine the joint loss of the segmentation sub-network and the classification sub-network in the current iteration based on the preset segmentation label, preset classification label, the pneumonia lesion area prediction result data and the pneumonia diagnosis type prediction result data corresponding to the 3D chest CT image, and update the parameters of the multi-task deep learning model based on the joint loss. The model output module is used to output the multi-task deep learning model with updated parameters as a pneumonia CT image diagnostic model if the current multi-task deep learning model meets the preset training termination condition.

[0182] The embodiments of the pneumonia CT image diagnostic model training device provided in this application can be used to execute the processing flow of the pneumonia CT image diagnostic model training method in the above embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the above embodiments of the pneumonia CT image diagnostic model training method.

[0183] The training portion of the pneumonia CT image diagnostic model in the aforementioned pneumonia CT image diagnostic model training device can be completed on either a server or a client device. The choice can be made based on the processing capabilities of the client device and the limitations of the user's usage scenario. This application does not impose any limitations in this regard. If all operations are completed on the client device, the client device may further include a processor for the specific processing of the pneumonia CT image diagnostic model training.

[0184] Furthermore, from a software perspective, this application also provides a pneumonia CT image diagnostic device for performing all or part of the pneumonia CT image diagnostic method, wherein the pneumonia CT image diagnostic device specifically includes the following: The image acquisition module is used to acquire 3D chest CT images of the target.

[0185] The model prediction module is used to input the target 3D chest CT image into the pneumonia CT image diagnosis model, so that the pneumonia CT image diagnosis model outputs the prediction result data of the pneumonia lesion area and the prediction result data of the pneumonia diagnosis type corresponding to the target 3D chest CT image. The pneumonia CT image diagnosis model is pre-trained based on the pneumonia CT image diagnosis model training method provided in the aforementioned embodiment.

[0186] The pneumonia CT image diagnosis component of the aforementioned pneumonia CT image diagnostic device can be performed on either a server or a client device. The choice can be made based on the processing capabilities of the client device and the limitations of the user's usage scenario. This application does not impose any limitations in this regard. If all operations are performed on the client device, the client device may further include a processor for the specific processing of the pneumonia CT image diagnosis.

[0187] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0188] The server and the client device can communicate using any suitable network protocol, including those not yet developed as of the date of this application. Such network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Furthermore, such network protocols may also include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer Protocol) protocols used on top of the aforementioned protocols.

[0189] To further illustrate the above embodiments, this application also provides an automatic diagnostic system based on a 3D multi-task deep learning model, which is described in detail below: Anatomical CT patterns of pneumonia, including lobar, lobular, and interstitial types, are crucial for accurate diagnosis and effective treatment. However, automated diagnostic systems based on these patterns are rare due to feature overlap between different types of pneumonia. Existing deep learning (DL) models have limitations in capturing global and long-range semantic information crucial for distinguishing complex anatomical patterns. Furthermore, the gap between encoder and decoder features in these models makes it difficult to fully utilize detailed feature maps in segmentation and classification tasks. To address these challenges, this application proposes an automated diagnostic system based on a 3D multi-task deep learning model. This model integrates segmentation and classification sub-networks. For the segmentation task, this application develops a standardized preprocessing pipeline specifically for CT slices containing lobar, lobular, and interstitial pneumonia. To better capture global and long-range semantic features, this application introduces a cross-fusion Transformer module (CTB), which fuses multi-scale channel and spatial features from different stages of the encoder. In addition, this application incorporates a voxel-based spatial and channel compression and excitation (vscSE) module to enhance local feature learning by recalibrating feature maps. For classification, a dedicated branch attached to the segmentation subnetwork uses enhanced features to classify pneumonia patterns. The model was tested on a dataset containing 180 pneumonia patients, and the results show significant improvements over state-of-the-art models, achieving higher accuracy, precision, sensitivity, and specificity, making it a valuable tool to assist clinicians in diagnosing pneumonia.

[0190] While U-Net and its variants capture global context by aggregating multi-scale features using skip connections, they typically lack a comprehensive analysis of how to effectively bridge the semantic gap between the encoder and decoder by fusing multi-scale channel and spatial information. To address the limitations of existing research, this application proposes an automated diagnostic system based on a 3D multi-task deep learning model. The overall process of the proposed system is as follows: Figure 10 and Figure 11 As shown, this 3D multi-task model comprises a segmentation subnetwork and a classification subnetwork. For the segmentation task, this application example integrates the Transformer architecture into the nnU-Net framework. nnU-Net is a state-of-the-art framework for medical image segmentation, renowned for its adaptability and standardized preprocessing steps. These characteristics make nnU-Net suitable for a wide range of medical image segmentation tasks, including pneumonia lesion segmentation. Non-architectural aspects of nnU-Net, such as the standardized preprocessing steps, have a significant impact on medical image segmentation tasks, although they are often underestimated.

[0191] To enhance the extraction of contextual information and fine-grained features from pneumonia lesions, this application example first introduces a standardized preprocessing workflow specifically designed for lobar, lobular, and interstitial pneumonia slides. This workflow includes cropping, resampling, and normalization. To bridge semantic gaps and achieve global and long-range semantic information interaction, this application example proposes a Cross-Fusion Transformer Module (CTB) to merge multi-scale channel and spatial features from different stages of the encoder. The CTB consists of a voxel embedding module and a convolutional attention projection module. It bridges the encoder and decoder by utilizing multi-scale global context and replaces the original skip connections to effectively capture non-local semantic dependencies. Due to the limitations of the Transformer in local feature learning, this application example adds a voxel-based spatial and channel squeezing and activation (vscSE) module at each stage of the encoder. The vscSE module recalibrates the feature maps along the channel and spatial dimensions, respectively, thereby amplifying meaningful features and suppressing weaker features. For classification tasks, this application example introduces a classification branch based on CTB. This branch, attached to the segmentation sub-network, classifies the feature maps extracted by the encoder, effectively utilizing global and long-range semantic information to classify the three anatomical patterns. Beyond methodological advancements, the system in this application instance is designed with clinical applicability in mind: it reduces CT interpretation time by automatically identifying and classifying pneumonia patterns, improves diagnostic consistency by minimizing inter-observer variability, and provides anatomical insights directly supporting treatment planning and patient management. This application instance tested the proposed 3D multi-task model on a dataset of CT scans from 180 patients diagnosed with lobar, lobular, and interstitial pneumonia, totaling 80,972 CT slices. Extensive experiments were conducted, comparing the model to various state-of-the-art DL models. The final results demonstrate that the model in this application instance exhibits superior performance in both segmentation and classification tasks compared to state-of-the-art models. In summary, the contributions of this application instance are as follows: (1) The application example in this application improves upon previous work by integrating the Transformer architecture and the vscSE module into nnU-Net. CTB facilitates long-range semantic information interaction. The vscSE module enhances local feature learning in CTB.

[0192] (2) In the application example of this application, a multi-task learning model is proposed, which integrates the segmentation and classification of various pneumonia types into a unified framework. The multi-task architecture enhances the learning of the correlation between segmentation and classification tasks, enabling the multi-task model of the application example of this application to achieve superior performance compared with various state-of-the-art models.

[0193] (3) Unlike existing multi-task models that study etiology or clinical classification, the application example in this application focuses on the anatomical patterns of lobar, lobular, and interstitial pneumonia. The classification subnetwork of the application example in this application is attached to the segmentation subnetwork to classify the feature maps extracted by the encoder, effectively utilizing global and long-range semantic information to classify the three anatomical patterns.

[0194] (4) Based on the research of the application examples of this application, the model of the application examples of this application is the first model to integrate the Transformer architecture into pneumonia diagnosis. The application examples of this application introduce a CTB to merge multi-scale channel and spatial features from different stages of the encoder, thereby enabling the capture of non-local semantic dependencies.

[0195] Specifically, this application proposes a 3D multi-task model, namely the aforementioned pneumonia CT image diagnostic model (hereinafter referred to as the model), see [link to application example]. Figure 12 The model comprises a segmentation subnetwork and a classification subnetwork. The segmentation and classification subnetworks share the same encoder, which can be termed a shared encoder. This shared encoder (shrinking path) consists of five stages. Each stage contains two 3×3×3 convolutional layers (stride = 1, padding = 1). Each convolutional layer contains an instance normalization (IN) layer and a Leaky ReLU (Leaky Rectified Linear Unit). In the first stage of the encoder, an instance is applied to obtain a 3D chest CT image after the preprocessing stage. The 3D chest CT image is then sent to the convolutional layer, which outputs a feature map. The feature map is then used to generate target feature data through a voxel-based spatial and channel compression and excitation (vscSE) module. Inspired by the introduced compression and excitation (SE) block, the vscSE module recalibrates the feature map along both the channel and spatial dimensions, and then combines the outputs. Immediately following the vscSE module, a 2×2×2 max pooling layer with a stride of 2 is applied at the end of each stage. Each stage of the encoder doubles the number of feature channels while halving the spatial dimension. In this application example, the original skip connections are replaced with a Cross-Fusion Transformer Module (CTB). The CTB fuses feature maps from each stage, which allows for exploration of multi-scale global context and learning of long-range semantic information interactions.

[0196] CTB first applies layer normalization to the input feature map, followed by two convolutional layers and ReLU activation. These steps encode image information better than directly creating a voxel-by-voxel projection of the image first. The output of the max-pooling layer is then processed by a voxel embedding module, where the convolutional projection maps Q, K, and V labels. Multi-head attention is then computed. Considering computational cost, model complexity, and the specific requirements of the application instance task, the application instance sets the number of attention heads to 2. This choice strikes a balance between capturing sufficient feature diversity and maintaining efficiency, ensuring good model performance without unnecessary computational overhead or overfitting risk. Convolutional projection mapping helps preserve spatial information in the feature map as an alternative to positional encoding. This effectively addresses the spatial information loss problem caused by point mapping in traditional Transformer structures, thereby improving the model's classification accuracy. During forward propagation, convolutional layers are used instead of linear layers to preserve the spatial context in the image. Residual connections are introduced between each module layer to enhance feature propagation. The decoder (extended path) structure is similar to the encoder, except that the max-pooling layer is replaced with a 2×2×2 transposed convolutional layer. The connection between the encoder and decoder can involve two 3×3×3 convolutional layers (Conv3D+IN+Leaky Relu) and a vscSE module.

[0197] The classification and segmentation subnetworks are interconnected and share a common encoder. The classification subnetwork may consist of four convolutional layers (two for the encoder and two for the decoder) and a CTB module. The input and output sizes of the CTB are 320×8×8×6 and 8×8×8×6 = 3072, respectively. A fully connected (FC) layer then receives these features and combines them to transform the feature space to better represent the relationships between different pneumonia patterns. The input and output sizes of the FC are 3072 and 4, respectively. A softmax function is applied to transform the output of the FC into a probability distribution over four categories (lobular, microlobular, mesenchymal, and normal).

[0198] In other words, this application proposes a novel automatic diagnostic system based on a 3D multi-task deep learning model for anatomical CT patterns of pneumonia; it develops a standardized preprocessing workflow specifically for CT slices of lobar, lobular, and interstitial pneumonia, optimizing segmentation accuracy; it introduces a cross-fusion Transformer module (CTB), which captures global and long-range semantic information by fusing multi-scale features, significantly improving model performance; and it adds a voxel-based spatial and channel compression and excitation (vscSE) module, which recalibrates feature maps to enhance the learning of meaningful features while suppressing weaker features.

[0199] Furthermore, this application provides an application example that compares the proposed model with various state-of-the-art DL models to offer a comprehensive overview of its performance. The dataset, evaluation metrics, experimental results, and discussion are shown below: (1) Dataset This application thoroughly investigated publicly available datasets for pneumonia segmentation and classification tasks. The findings of this application indicate that no publicly available datasets focus on segmentation and classification based on lobar, lobular, and interstitial patterns. Most existing datasets either focus on specific types of pneumonia, such as coronavirus pneumonia [54, 55], or on various nodular patterns. To evaluate the proposed model, this application constructed a dataset containing 80,972 CT images from 180 patients diagnosed with lobar, lobular, and interstitial pneumonia. The CT scans were obtained from a tertiary hospital in Beijing, with 60 patients each of lobar, lobular, and interstitial pneumonia. All patients presented to the fever clinic and underwent chest CT scans between June 2019 and May 2022. CT image reconstruction used a 512×512 matrix, with slice thicknesses ranging from 1 to 1.5 mm. To ensure diversity and generalization, the application example in this application randomly divided 180 patients into three groups: a training set (n = 108), a validation set (n = 27), and a test set (n = 45). This partitioning strategy allows for robust training, validation, and evaluation on different subsets of the dataset.

[0200] (2) Evaluation indicators The application examples in this application use the five evaluation metrics recommended by the ISIC-2017 Challenge to evaluate the performance of the proposed method, including Dice coefficient (DI), accuracy (AC), precision (PC), sensitivity (SE), and specificity (SC).

[0201] For segmentation tasks, image overlap detection (DI) is used to evaluate the segmentation performance of a model. DI is a method that measures the overlap between any two images and is widely used to evaluate the performance of medical image segmentation. DI is defined as follows: For classification tasks, AC, PC, SE, and SC are used to evaluate classification performance.

[0202] AC is the proportion of correctly predicted instances (true positives and true negatives) out of the total number of instances, defined as follows: PC, also known as positive predictive value, is the proportion of correctly predicted positive instances out of all instances predicted as positive, and is defined as follows: SE, also known as recall or true positive rate, is the proportion of correctly predicted positive instances out of all actual positive instances, defined as follows: SC, also known as the true negative rate, is the proportion of correctly predicted negative instances out of all actual negative instances, and is defined as follows: (3) Evaluation results Extensive experiments were conducted in the application examples of this application, comparing the proposed model with 3D U-Net, nn-UNet, and nnFormer. The reasons for choosing these DL models are as follows: 3D U-Net is an extension of the original U-Net architecture, specifically designed for volumetric data segmentation (such as 3D medical images). The segmentation sub-network of the model in the application examples of this application is based on 3D U-Net, making it the baseline model for this application example. nnU-Net is another state-of-the-art framework for medical image segmentation. Its adaptability, automated configuration, and normalized preprocessing make it a versatile and efficient choice for various medical image segmentation tasks. Comparing 3D U-Net and nnU-Net helps illustrate the effectiveness of normalized preprocessing in the pneumonia segmentation task. nnFormer is also a state-of-the-art DL model for medical image segmentation, combining the principles of Transformer architecture and CNN. Unlike nnU-Net, nnFormer integrates CNN and Transformer to leverage local and global feature extraction capabilities and capture long-range dependencies. A comparison of nnU-Net and nnFormer demonstrates whether integrating the Transformer architecture can effectively improve segmentation performance by leveraging multi-scale global context and replacing original skip connections to facilitate long-range semantic information. Finally, an application example comparing the proposed model with nnFormer illustrates whether the multi-task model with the vscSE module outperforms existing single-task models in pneumonia segmentation and classification tasks.

[0203] The application example in this application uses PyTorch to implement all models and executes them on an NVIDIA A6000 GPU with 48 GB of memory. Before training the model for 250 epochs, the application example performs a 50-epoch warm-up training.

[0204] (4) Task splitting The segmentation results of all models are shown in Table 3. The results show that the models of nnU-Net, nnFormer, and the application example of this application continuously improve upon existing technologies. 3D U-Net achieved an average Dice index (DI) of 60.3 in segmenting lobar, lobular, and interstitial lesions, but performed worst in segmenting the lobular pattern, with a DI of only 55.9. In contrast, nnU-Net achieved an average DI of 68.3, a 13.3% improvement over 3D U-Net. Besides the improvement in average DI, nnU-Net also demonstrated stronger adaptability in segmenting the three types of pneumonia lesions. The comparison between 3D U-Net and nnU-Net validates the effectiveness of standardized preprocessing in pneumonia diagnosis, highlighting the significant non-architectural impact of segmentation methods. Compared to nnU-Net, nnFormer showed a slight improvement in average DI, with the greatest improvement in the lobar pattern, but performed poorly in the lobular and interstitial patterns. The model in the application example of this application, which integrates the vscSE module with 3D U-Net, showed significant improvements in all three pneumonia patterns. This result demonstrates the effectiveness of using vscSE to amplify meaningful features and suppress weaker features in pneumonia segmentation.

[0205] Table 3 Application examples of this application are still available. Figure 13 , Figure 14 and Figure 15 The segmentation results of comparable models are visualized. Red, green, and blue represent the lesion areas of lobar, lobular, and interstitial pneumonia, respectively.

[0206] For lobar pneumonia, the segmentation results of nnFormer and the model in the application example of this application are closer to the ground reality than those of 3D U-Net and nnU-Net. For example, the segmentation results of 3D U-Net and nnU-Net contain more non-real regions (highlighted with yellow arrows) than those of nnFormer and the model in the application example of this application. In the nnFormer image, some small regions outside the real region are incorrectly labeled as lobar lesions, while the model image in the application example of this application does not show this effect.

[0207] Segmentation of lobular pneumonia is particularly challenging because the size and shape of the patchy consolidation vary greatly. Compared to ground reality, the results from 3D U-Net and nnU-Net appear more rounded with fewer patchy features, while the models from nnFormer and the application examples in this application are visually closer to ground reality.

[0208] Interstitial pneumonia is also difficult to classify because it shares features with lobular pneumonia, such as patchy opacities and consolidation. For Figure 15 In the interstitial pneumonia cases, all models incorrectly segmented a region at the right lobe edge (marked with a yellow arrow). However, the model in this application example and nnU-Net showed a relatively small incorrect segmentation region.

[0209] (5) Classification tasks The classification results are shown in Table 4, demonstrating that the nnFormer and the application examples in this application show continuous improvements over previous methods. Since 3D U-Net, nnU-Net, and nnFormer are single-task segmentation models, the application examples in this application use NumPy to generate classification outputs by statistically analyzing pixel-wise information within the segmentation results and label values.

[0210] Table 4 On average, 3D U-Net performed the worst among the four models, with an AC score of 85.2, a PC score of 78.8, a SE score of 77.8, and a SC score of 85.3. In contrast, nnU-Net outperformed 3D U-Net, with an AC score of 87.0, a PC score of 81.0, a SE score of 80.4, and a SP score of 90.3. Since nnU-Net uses the same model as 3D U-Net, it also shares the same classification patterns. Detailed results show that 3D U-Net and nnU-Net performed best in classifying lobar pneumonia, but worst in classifying lobular pneumonia.

[0211] nnFormer showed slight improvements over nnU-Net in average AC, PC, SE, and SP, performing best in classifying interstitial pneumonia but worst in classifying lobular pneumonia.

[0212] Unlike nnU-Net and nnFormer, the multi-task model proposed in this application instance includes a classification subnetwork connected to the segmentation subnetwork. This classification subnetwork utilizes global and long-range semantic information from the feature maps extracted by the encoder to classify three pneumonia patterns. The model in this application instance achieves significant improvements across all evaluation metrics, with an average improvement of 8.5% in AC, 12.4% in PC, 13.5% in SE, and 6.1% in SP. Specifically, the model in this application instance performs exceptionally well in classifying lobar pneumonia, achieving a 9.9% improvement in AC compared to nnU-Net. Although the model in this application instance performs the worst in classifying interstitial pneumonia, it still outperforms the other two models.

[0213] For all comparable models, the application examples in this application also use the micro-average ROC curve method to comprehensively demonstrate their classification performance. The four micro-average ROC curves are as follows: Figure 16 As shown.

[0214] Based on the quantitative results introduced in the segmentation and classification tasks, the analysis is as follows: 1. All three comparable models, as well as the model used in this application example, achieved the worst performance in segmenting lobular lesions. This finding aligns with the challenges faced by clinicians in real-world diagnosis, primarily due to the more complex shapes and variable sizes exhibited by patchy consolidation in lobular lesions.

[0215] 2. The model in this application example may outperform nnU-Net and nnFormer in distinguishing between lobular pneumonia and interstitial pneumonia. According to Table 4, nnU-Net and nnFormer both exhibit the worst performance in classifying lobular pneumonia. Although the model in this application example performs the worst in classifying interstitial pneumonia, it still outperforms the other two models overall. The significant improvement in lobular pneumonia classification without compromising interstitial pneumonia classification performance indicates that the model in this application example performs better in distinguishing overlapping features between these two types of pneumonia.

[0216] 3. Although nnFormer achieves minor quantitative improvements over nnU-Net in segmentation and classification tasks, the visualization results clearly demonstrate the effectiveness of integrating the Transformer architecture. This integration leverages multi-scale global context and facilitates long-range semantic information interaction. The visualized segmentation results of nnFormer and the application instance model in this application are closer to ground reality than those of 3DU-Net and nnU-Net.

[0217] 4. Not all skip connections with simple replication are useful for pneumonia segmentation. Before determining the architecture for capturing nonlocal semantic dependencies, this application investigated three categories of solutions focused on reconstructing skip connections. The first category involves directly replacing ordinary skip connections with nested, dense skip connection paths. The most representative model in this category is UNet++, which achieves better segmentation performance by introducing a series of dense convolutional connections to bridge the semantic gap between the encoder and decoder subnetworks. This is an improvement over the restrictive skip connections in U-Net, which only requires fusing feature maps of the same scale. The second category aims to strengthen skip connections by introducing additional nonlinear transformations. MultiResUNe is the most representative model in this category, balancing the potential semantic gap. The third category focuses on bridging the semantic gap between the encoder and decoder through multi-scale channel information fusion, effectively capturing nonlocal semantic dependencies. The most representative model here is UCTransNet, which includes a channel cross-fusion Transformer that fuses multi-scale context from a channel perspective through cross-attention.

[0218] This application example tested the performance of various models from these categories on the dataset used in this application example. The results show that the optimal design varies depending on the dataset. Tests indicate that all models from the aforementioned categories perform similarly to or worse than 3D U-Net, while U-Net integrating the Transformer architecture achieves a significant improvement in visualization results. Based on this finding, this application example further integrates a voxel-based spatial and channel compression and activation model at each stage of the encoder. This model effectively amplifies meaningful features in the feature maps, thereby further improving quantitative results. This application example also tested models without the Transformer architecture but containing vscSE, whose performance was close to or even worse than the traditional 3D U-Net. Only when 3D U-Net combines both vscSE and the Transformer architecture does it achieve optimal pneumonia segmentation performance.

[0219] To provide a comprehensive understanding of the computational performance of the model proposed in this application example, its complexity, parameter scale, and inference time under both high-performance and resource-constrained hardware settings are systematically evaluated. Using the THOP library in PyTorch, the number of parameters and FLOPs are calculated via a profile function given the shape of the input tensor, and the results are converted to a human-readable form using the cleverformat function to ensure standardized reporting. The proposed 3D multi-task model contains approximately 76.18 million trainable parameters, totaling 620.83 GFLOPs per forward propagation. On an NVIDIA A6000 GPU with 48 GB of memory, the model achieves an average inference time of 2.94 seconds per scan, enabling efficient processing of volumetric CT data and thus reducing the interpretation burden on radiologists in clinical practice.

[0220] This application example further analyzes the impact of the voxel-based Spatial and Channel Compression and Excitation (vscSE) module on computational efficiency. Including vscSE increases the number of parameters by less than 2.2% (from 74.52 million to 76.18 million) and FLOPs by less than 1% (from 615.09 G to 620.83 G), while maintaining the same inference time of approximately 2.94 seconds. Despite this negligible overhead, the module enhances feature recalibration and delivers performance improvements in lesion localization and pneumonia subtype differentiation, demonstrating a good balance between representational power and computational efficiency.

[0221] It is also noteworthy that the parameter size of the application example model in this application is comparable to other state-of-the-art 3DTransformer-based networks (such as nnFormer), which typically have between 70 and 90 million parameters. This indicates that the method in the application example of this application operates within a computationally similar range to existing Transformer-based architectures, while achieving superior segmentation and classification performance due to the integration of the Cross-Fusion Transformer Module (CTB) and the vscSE module. Therefore, the model in the application example of this application achieves a strong trade-off between parameter size and diagnostic accuracy.

[0222] To assess real-world feasibility, the application example in this application was further tested on a resource-constrained GPU (NVIDIA RTX 3060, 12 GB of memory). In this environment, the forward propagation time per scan was 7.15 seconds, longer than the 2.94 seconds observed on the A6000, but still within acceptable limits in radiology workflows that typically allow for several seconds of time for automated CT analysis. Considering the RTX 3060 is a consumer-grade GPU with a significantly reduced number of CUDA cores, lower memory bandwidth, and much smaller video memory than workstation GPUs, these results highlight the adaptability of the method in the application example in this application to different deployment environments. Importantly, this finding suggests that the proposed system is not only suitable for high-end clinical infrastructures but also has the potential for wider adoption in hospitals or institutions with limited computing resources.

[0223] In other words, this application presents a novel DL-based CT pattern diagnostic system for pneumonia anatomy. The system includes a standardized preprocessing pipeline for segmentation of lobar, lobular, and interstitial pneumonia. The proposed pipeline has been tested and proven effective in enhancing contextual information and fine-grained feature extraction. In the segmentation task, this application proposes integrating a vscSE module at each stage of the encoder and using a cross-fusion Transformer block (CTB) to combine multi-scale channel and spatial features. This approach aims to capture more meaningful long-range semantic information and enhance local feature learning. This application also proposes a multi-task learning model that integrates segmentation and classification of various pneumonia types into a unified framework. The multi-task architecture enhances the learning of correlations between segmentation and classification tasks, enabling the multi-task model of this application to achieve superior performance compared to various state-of-the-art models.

[0224] This application example introduces a deep learning-based diagnostic system designed to diagnose lobar, lobular, and interstitial pneumonia. The proposed system is based on a 3D multi-task model that integrates a segmentation sub-network and a classification sub-network, utilizing a standardized preprocessing pipeline tailored to the unique anatomical CT imaging patterns of pneumonia. A voxel-based spatial and channel compression and excitation (vscSE) module is incorporated at each stage of the encoder, along with a cross-fusion Transformer module (CTB) to fuse multi-scale features, significantly enhancing the model's ability to capture meaningful long-range semantic information and local features, thereby improving pneumonia segmentation. Furthermore, the classification branch connected to the segmentation sub-network effectively utilizes global and long-range semantic information to accurately classify the three pneumonia patterns. This application example evaluates the model on a comprehensive dataset from a top-tier hospital in Beijing and conducts extensive comparisons with leading state-of-the-art deep learning models. The results demonstrate that the proposed model exhibits superior performance in both segmentation and classification tasks, achieving substantial improvements in key metrics such as the Dice index, accuracy, precision, sensitivity, and specificity. These promising results underscore the potential of the system in the application examples of this application to significantly assist physicians and radiologists in accurately and efficiently diagnosing pneumonia.

[0225] This application also provides an electronic device, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the pneumonia CT image diagnostic model training method and / or pneumonia CT image diagnostic method mentioned in the above embodiments. The processor and memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and memory via wired or wireless means.

[0226] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0227] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the pneumonia CT image diagnostic model training method and / or pneumonia CT image diagnostic method in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the pneumonia CT image diagnostic model training method and / or pneumonia CT image diagnostic method in the above method embodiments.

[0228] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0229] The one or more modules are stored in the memory, and when executed by the processor, they execute the pneumonia CT image diagnostic model training method and / or the pneumonia CT image diagnostic method in the embodiment.

[0230] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0231] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.

[0232] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.

[0233] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned pneumonia CT image diagnostic model training method and / or pneumonia CT image diagnostic method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0234] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned pneumonia CT image diagnostic model training method and / or pneumonia CT image diagnostic method.

[0235] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0236] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0237] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0238] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to the embodiments of this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a pneumonia CT image diagnostic model, characterized in that, include: In the current iteration, a 3D chest CT image is input into a multi-task deep learning model. The shared encoder in this model extracts multi-scale feature data corresponding to the 3D chest CT image. The connection module and decoder in the model then use this multi-scale feature data to obtain predicted pneumonia lesion regions corresponding to the 3D chest CT image. Additionally, the classification head in the model uses this multi-scale feature data to obtain predicted pneumonia diagnosis types corresponding to the 3D chest CT image. The shared encoder, connection module, and decoder constitute the segmentation sub-network of the multi-task deep learning model. The shared encoder and classification head also constitute the classification sub-network of the multi-task deep learning model. Based on the preset segmentation labels, preset classification labels, pneumonia lesion region prediction results, and pneumonia diagnosis type prediction results corresponding to the 3D chest CT images, the joint loss of the segmentation sub-network and the classification sub-network in the current iteration is determined, and the parameters of the multi-task deep learning model are updated based on the joint loss. If the current multi-task deep learning model meets the preset training termination condition, then the multi-task deep learning model with updated parameters will be used as a pneumonia CT image diagnostic model for output.

2. The method for training a pneumonia CT image diagnostic model according to claim 1, characterized in that, The shared encoder includes: an encoding module, a voxel-based spatial and channel squeezing and excitation module, and a downsampling module; the encoding module includes a plurality of encoding stage units arranged sequentially; the downsampling module includes a first max pooling layer corresponding to each of the encoding stage units except the first encoding stage unit; The first encoding stage unit is used to extract features from the 3D chest CT image to obtain encoded feature data, and input the encoded feature data into the voxel-based spatial and channel compression and excitation module. The voxel-based spatial and channel compression and excitation module is used to recalibrate the dimensions of the channel and spatial directions of the encoded feature data to obtain multi-scale feature data, and transmit the multi-scale feature data to the connection module; and, if it is determined that the encoded feature data currently being processed is transmitted by the last encoded stage unit, then the multi-scale feature data is transmitted to the classification head in the classification sub-network; if it is determined that the encoded feature data currently being processed is transmitted by a non-last encoded stage unit, then the multi-scale feature data is transmitted to the first max pooling layer corresponding to the next encoded stage unit; The first max pooling layer is used to downsample the multi-scale feature data to obtain a first downsampled feature map, and inputs the first downsampled feature map into the corresponding encoding stage unit. The non-first encoding stage unit is used to extract features from the downsampled feature map to obtain encoded feature data, and input the encoded feature data into the voxel-based spatial and channel squeezing and excitation module.

3. The method for training a pneumonia CT image diagnostic model according to claim 1, characterized in that, The connection module includes: a cross-fusion Transformer module; The cross-fusion Transformer module includes: A normalization layer is used to normalize each of the multi-scale feature data currently received from the shared encoder along the channel dimension, so as to obtain the normalized feature map corresponding to each of the multi-scale feature data. Two consecutive 3D convolutional layers are used to extract features from each of the normalized feature maps to obtain the convolutional feature maps corresponding to each of the multi-scale feature data. The first ReLU activation layer is used to perform non-linear activation processing on each of the convolutional feature maps based on the ReLU activation function, so as to obtain the second activated feature map corresponding to each of the multi-scale feature data. The second max pooling layer is used to downsample each of the second activated feature maps to obtain the second downsampled feature map corresponding to each of the multi-scale feature data. A voxel embedding layer is used to flatten each of the second downsampled feature maps into a sequence to obtain the serialized feature data corresponding to each of the multi-scale feature data. The convolutional projection layer is used to generate corresponding query tags, key tags, and value tags for each of the serialized feature data; A multi-head attention layer is used to concatenate the key and value tags of each of the serialized feature data to obtain a key-value library, and to perform multi-head attention calculation on the query tag of each of the serialized feature data and the key-value library to obtain the corresponding attention-weighted feature data. A residual connection layer is used to perform residual connections between the attention-weighted feature data and each of the multi-scale feature data to obtain enhanced feature data, and the enhanced feature data is input into the decoder.

4. The method for training a pneumonia CT image diagnostic model according to claim 3, characterized in that, The decoder includes: a decoding module, a voxel-based spatial and channel compression and excitation module, an upsampling module, and an output layer; the decoding module includes a plurality of decoding stage units arranged sequentially; the upsampling module includes transposed convolutional layers corresponding to each of the decoding stage units except the first decoding stage unit; The first decoding stage unit is used to extract features from the enhanced feature data corresponding to the last decoding stage unit to obtain decoded feature data, and input the decoded feature data into the voxel-based spatial and channel squeezing and excitation module. The voxel-based spatial and channel compression and excitation module is used to recalibrate the dimensions of the channel and spatial directions of the decoded feature data to obtain optimized decoded feature data, and transmit the optimized decoded feature data to the output layer; and, if it is determined that the decoded feature data currently being processed is transmitted by a unit other than the last decoded stage unit, then the optimized decoded feature data is transmitted to the transposed convolutional layer corresponding to the next decoded stage unit. The transposed convolutional layer is used to upsample the optimized decoding feature data to obtain an upsampled feature map, and input the upsampled feature map into the corresponding decoding stage unit. The non-first decoding stage unit is used to perform feature concatenation on the received enhanced feature data and the upsampled feature map to obtain concatenated feature data, and to extract features from the concatenated feature data to obtain decoded feature data, and input the decoded feature data into the voxel-based spatial and channel squeezing and excitation module; The output layer is used to map the optimized decoding feature data corresponding to each of the decoding stage units into a pixel-level multi-class probability distribution to generate the pneumonia lesion area prediction result data corresponding to the 3D chest CT image.

5. The method for training a pneumonia CT image diagnostic model according to claim 2 or 4, characterized in that, The voxel-based spatial and channel extrusion and excitation module includes: a channel-direction extrusion and excitation unit, a spatial-direction extrusion and excitation unit, and a feature fusion layer; The channel-direction squeezing and excitation unit is used to recalibrate the channel-direction dimension of the first feature data to obtain a channel recalibrated feature map; wherein, the first feature data includes encoded feature data or decoded feature data; The spatial compression and excitation unit is used to recalibrate the spatial dimension of the first feature data to obtain a spatially recalibrated feature map. The feature fusion layer is used to add the channel recalibration feature map and the spatial recalibration feature map to obtain target feature data; the target feature data includes: multi-scale feature data or optimized decoding feature data; The compression and excitation unit in the channel direction includes: A global average pooling layer is used to compress the first feature data along the spatial dimension to obtain a channel statistics vector for capturing global context information of each channel of the first feature data. The first convolutional layer is used to reduce the channel dimension of the channel statistical vector to obtain the dimension-reduced feature map. The second ReLU activation layer is used to perform nonlinear activation processing on the dimensionality-reduced feature map based on the ReLU activation function to obtain the first activated feature map. The second convolutional layer is used to perform original channel dimension restoration processing on the first activated feature map to obtain the original dimension feature map; wherein, the original channel dimension is the channel dimension of the first feature data; The first Sigmoid activation layer is used to normalize the channel weights of the original dimensional feature map to the [0,1] interval based on the Sigmoid activation function, so as to obtain the channel weight vector. A channel rescaling layer is used to multiply the channel weight vector with the first feature data channel by channel to obtain a channel recalibrated feature map; The spatial compression and excitation unit includes: The third convolutional layer is used to compress the first feature data along the channel dimension to obtain a spatial projection tensor for fusing the spatial information of all channels of the first feature data. The second Sigmoid activation layer is used to normalize the spatial weights corresponding to the spatial projection tensor to the [0,1] interval based on the Sigmoid activation function, so as to obtain the spatial weight vector. A spatial rescaling layer is used to multiply the spatial weight vector with the first feature data position by position to obtain a spatially recalibrated feature map.

6. The method for training a pneumonia CT image diagnostic model according to claim 2, characterized in that, The classification header includes: The feature extraction layer is used to encode and decode the multi-scale feature data transmitted by the voxel-based spatial and channel compression and excitation module in sequence to obtain a deep semantic feature map. The cross-fusion Transformer module is used to sequentially perform layer normalization, feature extraction, nonlinear activation based on the ReLU activation function, downsampling, voxel embedding, convolutional projection, multi-head attention calculation, and residual connection on the deep semantic feature map to obtain the global context information feature map corresponding to the deep semantic feature map. A fully connected layer is used to perform a linear transformation on the global context information feature map to learn the high-level semantic relationships between different pneumonia patterns and output a 4-dimensional classification logical value vector. The Softmax activation layer is used to convert the 4-dimensional classification logic value vector into a probability distribution along the vector dimension based on the Softmax function, so as to obtain the pneumonia diagnosis type prediction result data corresponding to the 3D chest CT image, wherein the pneumonia diagnosis type includes: lobar, lobular, interstitial and normal.

7. The method for training a pneumonia CT image diagnostic model according to claim 1, characterized in that, The step of determining the joint loss of the segmentation sub-network and the classification sub-network in the current iteration based on the preset segmentation labels, preset classification labels, pneumonia lesion region prediction results, and pneumonia diagnosis type prediction results corresponding to the 3D chest CT image, and updating the parameters of the multi-task deep learning model based on the joint loss, includes: Based on the probability value of each voxel belonging to a specific category in the pneumonia lesion region prediction result data, the preset segmentation label, and the preset total number of voxels participating in the calculation, the Dice loss of the segmentation sub-network corresponding to the current iteration round is determined; and based on the prediction probability of each voxel being correctly classified in the pneumonia lesion region prediction result data, the preset focusing parameter, and the category balance weight, the Focal loss of the segmentation sub-network corresponding to the current iteration round is determined; the segmentation loss of the segmentation sub-network corresponding to the current iteration round is determined by the sum of the Dice loss and the Focal loss. Based on the pneumonia diagnosis type prediction results of each of the 3D chest CT images input to the multi-task deep learning model in the current iteration, the preset classification labels, the total number of 3D chest CT images input to the multi-task deep learning model in the current iteration, and the preset total number of categories, the classification loss of the segmentation sub-network in the current iteration is determined. The segmentation loss and the classification loss are linearly combined according to preset weights to obtain the joint loss for the current iteration. Based on the joint loss, the gradients of each parameter of the multi-task deep learning model are calculated using the stochastic gradient descent algorithm, and all parameters of the multi-task deep learning model are updated according to the polynomial learning rate descent strategy; wherein, the polynomial learning rate descent strategy is used to dynamically adjust the learning rate according to the ratio of the current iteration round to the total number of iteration rounds.

8. The method for training a pneumonia CT image diagnostic model according to claim 1, characterized in that, Before inputting the 3D chest CT images into the multi-task deep learning model, the following steps are also included: Acquire the original 3D chest CT image and its corresponding preset segmentation and classification labels; A lung parenchyma mask is generated based on the Henle unit value of the original 3D chest CT image, and a bounding box is determined based on the lung parenchyma mask. The original 3D chest CT image is then cropped based on the bounding box to obtain the cropped image corresponding to the original 3D chest CT image. An adaptive resampling algorithm is used to process the cropped image to obtain the corresponding resampled image; The resampled image is truncated and normalized based on the statistical information of the foreground voxels in the pre-acquired training set to obtain a 3D chest CT image for training a multi-task deep learning model.

9. A method for CT imaging diagnosis of pneumonia, characterized in that, include: Acquire target 3D chest CT images; The target 3D chest CT image is input into the pneumonia CT image diagnosis model, so that the pneumonia CT image diagnosis model outputs the prediction result data of the pneumonia lesion area and the prediction result data of the pneumonia diagnosis type corresponding to the target 3D chest CT image. The pneumonia CT image diagnosis model is pre-trained based on the pneumonia CT image diagnosis model training method according to any one of claims 1 to 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the pneumonia CT image diagnostic model training method as described in any one of claims 1 to 8, and / or implements the pneumonia CT image diagnostic method as described in claim 9.

Images