A medical image processing method and apparatus based on chest CT

Abstract

The application relates to a medical image processing method and device based on chest CT. Compared with the single prediction task model or the mode of combining multiple independent models in the prior art, firstly, the method adopts a shared feature encoder, feature extraction is performed on a lung parenchyma region of interest through the shared feature encoder, so that multiple tasks can share and learn general features of lung images, the sharing mechanism promotes knowledge transfer and collaborative learning between tasks, and the generalization ability and prediction accuracy of the model are improved. Furthermore, a multi-task joint loss function based on homoscedastic uncertainty dynamic weighting is introduced, the weights of the loss of each task are dynamically adjusted, and through the adaptive weight adjustment mechanism, it is ensured that all tasks can be balanced and fully trained, and the efficiency of task analysis using medical images and the accuracy of multi-task are significantly improved.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a medical image processing method and apparatus based on chest CT. Background Technology

[0002] In recent years, deep learning has been applied to medical image analysis. Convolutional neural networks such as ResNet and MobileNet, as well as radiomics methods, can extract quantitative features from CT images and perform risk prediction based on these quantitative features.

[0003] However, the shortcomings of existing technologies are: CT image analysis models fail to identify the correlation between features across multiple tasks. Independent training of a single task leads to low data utilization efficiency and the inability of tasks to enhance each other through shared representations. Furthermore, simple fixed-weight multi-task models are prone to gradient competition due to differences in the magnitude of loss and convergence speed among tasks, resulting in one task dominating training while other tasks underfit, making it impossible to achieve accurate output for prediction of multiple risk tasks simultaneously.

[0004] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0005] To achieve the above objectives, a first aspect of this application proposes a medical image processing method based on chest CT scans, the method comprising: Acquire target chest CT images; The target chest CT image is input into a pre-trained lung parenchyma segmentation network to obtain a lung parenchyma segmentation probability map. A lung parenchyma mask is generated based on post-processing, and the region of interest in the lung parenchyma is extracted based on the lung parenchyma mask. The region of interest (ROI) of the lung parenchyma is input into a pre-trained multi-task deep learning network. The multi-task deep learning network includes a shared feature encoder and multiple task branches. The shared feature encoder extracts features from the ROI of the lung parenchyma to obtain shared image features associated with multiple prediction tasks. The multiple task branches output prediction results for the corresponding prediction tasks based on the shared image features. The multi-task deep learning network is trained using a multi-task joint loss function based on dynamic weighting of homoscedastic uncertainty.

[0006] The method provided in this application embodiment has at least the following beneficial effects: Compared to existing technologies that mostly employ single-prediction-task models or combinations of multiple independent models, this method firstly uses a shared feature encoder to extract features from the regions of interest in the lung parenchyma. This allows multiple tasks to share and learn common features of lung images, promoting knowledge transfer and collaborative learning between tasks and improving the model's generalization ability and prediction accuracy. Secondly, it introduces a multi-task joint loss function based on homoscedastic uncertainty dynamic weighting. By dynamically adjusting the weights of each task's loss, an adaptive weight adjustment mechanism ensures that all tasks receive balanced and sufficient training, significantly improving the efficiency and accuracy of task analysis using medical images.

[0007] To achieve the above objectives, a second aspect of this application provides a medical image processing apparatus based on chest CT scans, the apparatus comprising: The data acquisition module is used to acquire the target chest CT image; The region of interest extraction module is used to input the target chest CT image into a pre-trained lung parenchyma segmentation network to obtain a lung parenchyma segmentation probability map, generate a lung parenchyma mask based on post-processing, and extract the region of interest of the lung parenchyma based on the lung parenchyma mask. A risk prediction module is used to input the region of interest (ROI) of the lung parenchyma into a pre-trained multi-task deep learning network. The multi-task deep learning network includes a shared feature encoder and multiple task branches. The shared feature encoder extracts features from the ROI to obtain shared image features associated with multiple prediction tasks. The multiple task branches output prediction results for the corresponding prediction tasks based on the shared image features. The multi-task deep learning network is trained using a multi-task joint loss function based on dynamic weighting of homoscedastic uncertainty.

[0008] To achieve the above objectives, a third aspect of this application provides an electronic device including at least one controller and a memory for communicatively connecting to the controller; the memory stores instructions executable by the at least one controller, which, when executed by the at least one controller, causes the at least one controller to perform: a medical image processing method based on chest CT as described in the first aspect embodiment.

[0009] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform: a medical image processing method based on chest CT as described in the first aspect embodiment.

[0010] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of a medical image processing method based on chest CT provided in an embodiment of this application; Figure 2 This is a schematic diagram of a medical image processing method based on chest CT provided in another embodiment of this application; Figure 3 This is a schematic diagram of a medical image processing device based on chest CT provided in an embodiment of this application; Figure 4 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0014] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0016] Some embodiments of this application are described with reference to... Figure 1 and Figure 2 This paper provides a medical image processing method based on chest CT, the method comprising: Step S110: Acquire the target chest CT image; Step S120: Input the target chest CT image into the pre-trained lung parenchyma segmentation network to obtain the lung parenchyma segmentation probability map, generate a lung parenchyma mask based on post-processing, and extract the lung parenchyma region of interest based on the lung parenchyma mask. Step S130: Input the region of interest (ROI) of the lung parenchyma into a pre-trained multi-task deep learning network; wherein, the multi-task deep learning network includes a shared feature encoder and multiple task branches, the shared feature encoder extracts features from the ROI of the lung parenchyma to obtain shared image features associated with multiple prediction tasks, and the multiple task branches output prediction results for the corresponding multiple prediction tasks based on the shared image features; wherein, the multi-task deep learning network is trained by a multi-task joint loss function based on homoscedastic uncertainty dynamic weighting.

[0017] It should be noted that this application only applies to image processing of existing CT images, that is, it uses computer vision technology to mine shared features between multiple tasks and then outputs prediction results for multiple tasks, and does not involve disease diagnosis.

[0018] For ease of understanding, the following explains some key terms in this embodiment: Chest CT images: refer to three-dimensional digital image data of the internal structure of the human chest obtained through computed tomography (CT) technology.

[0019] Lung parenchyma segmentation network: A deep learning-based image processing model whose main function is to automatically identify and accurately separate the lung parenchyma region from input chest CT images. Lung parenchyma segmentation probability map: An image representation output by the lung parenchyma segmentation network after processing the chest CT image. Lung parenchyma mask: A binary image generated after post-processing based on the lung parenchyma segmentation probability map. This mask clearly delineates the precise boundaries and extent of the lung parenchyma.

[0020] Region of Interest (ROI) in lung parenchyma: This refers to the image region containing only the lung parenchyma extracted from the original chest CT image based on a generated lung parenchyma mask. This region excludes non-lung parenchymal structures such as the chest wall, bones, and mediastinum, allowing subsequent analysis to focus more on the imaging features of the lungs themselves.

[0021] Multi-task deep learning networks refer to deep learning models capable of handling multiple related prediction tasks simultaneously. By sharing some underlying feature learning mechanisms, the network enables different tasks to mutually reinforce each other, thereby improving overall prediction performance and efficiency.

[0022] Shared feature encoder: This refers to a core component of a multi-task deep learning network. Its function is to extract features from the input lung parenchyma region of interest, generating a global feature vector containing rich semantic information.

[0023] Multiple task branches: In multi-task deep learning networks, after the shared feature encoder, each branch is dedicated to a specific prediction task. Each branch typically contains independent network layers that transform the shared feature vectors into predictions for the corresponding task.

[0024] This embodiment provides a medical image processing method based on chest CT.

[0025] In step S110, the method first acquires the target chest CT image. In one implementation, the target chest CT image can be obtained by manually selecting and loading the patient's CT image file. For example, healthcare workers can manually retrieve and import the patient's CT image data from a Picture Archiving and Communication System (PACS). However, this method may be inefficient or prone to introducing human error.

[0026] In step S120, as one implementation, the lung parenchyma segmentation network can be a model based on traditional image processing algorithms, such as attempting to identify lung regions by setting a fixed grayscale threshold or using a region growing algorithm. Alternatively, it can be a convolutional neural network that learns to distinguish between lung parenchyma and non-lung parenchyma through a limited stack of convolutional layers. These methods may face problems such as insufficient segmentation accuracy or sensitivity to noise when processing complex lung images or images with abnormalities.

[0027] One approach is to directly binarize the segmentation probability map with a fixed threshold and then perform connected component analysis to remove some small, isolated regions.

[0028] Based on a lung parenchyma mask, the region of interest (ROI) of the lung parenchyma is extracted. Specifically, the corresponding rectangular region in the original CT image can be directly cropped based on the bounding box of the lung parenchyma mask.

[0029] In step S130, the region of interest (ROI) of the lung parenchyma is input into a pre-trained multi-task deep learning network, which includes a shared feature encoder and multiple task branches. In one implementation, a system consisting of multiple independent single-task deep learning networks can be constructed, with each network responsible for predicting one task. The predictions from these independent networks are then combined. However, this independent training approach fails to fully utilize the shared information that may exist between different tasks, resulting in low data utilization efficiency and preventing the tasks from being collaboratively enhanced through shared representations.

[0030] A shared feature encoder extracts features from the region of interest (ROI) in the lung parenchyma, and multiple task branches output multi-task prediction results based on the global feature vector. As one implementation, the shared feature encoder can be a network structure composed of several stacked general convolutional layers, which performs multi-layer convolution and pooling operations on the input ROI of the lung parenchyma to extract features at different scales. These features are then fed into multiple task branches, each independently performing classification or regression prediction.

[0031] In one implementation, multiple prediction tasks may include: Task 1: Detect connected regions within the region of interest of the lung parenchyma whose pixel intensity is below a preset threshold, and classify them into multiple levels according to the proportion of the volume of the connected region to the volume of the lung parenchyma. Task 2: Measure the cross-sectional diameter of the tubular structure of the pulmonary artery in the hilar region and determine whether the measured value exceeds the preset threshold range; Task 3: Calculate the transverse diameter parameters and cardiothoracic ratio of the mediastinal cardiac silhouette and determine whether the calculated values ​​exceed the preset reference range.

[0032] The three tasks described above exhibit a spatially progressive correlation in lung CT images: the low-density connected regions processed in Task 1 are distributed in the peripheral lung fields of both lungs; the tubular structures of the pulmonary arteries processed in Task 2 are located in the central area of ​​the hilum; the volume proportion of the low-density regions in the peripheral lung fields and the cross-sectional diameter of the pulmonary artery in the hilum are statistically correlated in the same chest CT image; the cardiac silhouette processed in Task 3 is anatomically adjacent to the pulmonary artery in the hilum, and changes in the cross-sectional diameter of the pulmonary artery and changes in the transverse diameter of the cardiac silhouette are geometrically correlated. The shared feature encoder simultaneously encodes three types of correlated features—peripheral lung field density distribution, hilar pulmonary artery geometric parameters, and mediastinal cardiac silhouette morphological parameters—through a unified feature space. This allows a single feature extraction to serve multiple tasks, avoiding redundant computation caused by building a separate feature extraction network for each task, and effectively improving image processing efficiency.

[0033] Alternatively, another embodiment is provided, and the above technical solution will be explained in more detail below through a more specific example: First, after user A undergoes a chest CT scan at a medical institution, the raw chest CT image data of user A is obtained and stored in the medical institution's database. This image data may contain various formats and scan parameters.

[0034] Subsequently, the acquired target chest CT images are input into a pre-trained lung parenchyma segmentation network. The network performs pixel-level analysis of the images, outputting a lung parenchyma segmentation probability map, where each voxel carries a probability value indicating that it belongs to the lung parenchyma. For example, the probability value for the central lung region may be close to 1, while the probability value for the chest wall or mediastinal region may be close to 0. Following this, a series of post-processing operations are performed based on the probability map, such as preserving the largest connected components, performing 3D hole filling, and morphological closure operations, to generate an accurate lung parenchyma mask. The mask clearly delineates the precise boundaries of patient A's lungs and removes non-lung parenchyma regions such as the chest wall, bones, and mediastinum. Based on the high-quality lung parenchyma mask, the region of interest in patient A's lung parenchyma can be accurately extracted, thus focusing the analysis on the lungs themselves.

[0035] Next, the extracted regions of interest (ROIs) in the lung parenchyma are input into a pre-trained multi-task deep learning network. The core of the network is a shared feature encoder, which performs multi-level feature extraction on the input ROIs, generating a global feature vector containing rich semantic information about patient A's lung images. For example, the feature vector might encode key information such as the distribution of emphysema, the degree of airway wall thickening, and the morphology of pulmonary vessels. Following the shared feature encoder, the network contains multiple parallel task branches. These branches receive the global feature vector and, based on it, output the severity classification of user A's COPD (Chronic Obstructive Pulmonary Disease) (e.g., COPD classification GOLD 2), the predicted risk of pulmonary hypertension (e.g., high risk), and the predicted risk of pulmonary heart disease (e.g., moderate risk). It is worth noting that the above tasks are for illustrative purposes only and cannot be directly applied to disease diagnosis; a specific diagnosis requires a physician to determine it based on medical indicators and consultation results.

[0036] During training, multi-task deep learning networks are optimized using a dynamically weighted joint loss function based on homoscedastic uncertainty. This means that during training, the network can automatically adjust the weights of each prediction task in the total loss function according to their inherent uncertainties, such as severity rating, structural parameter discrimination, and morphological parameter discrimination. For example, if a task has high prediction uncertainty, the network may assign it a higher weight in the early stages of training to promote faster convergence; conversely, it may reduce the weight. This dynamic adjustment mechanism ensures that all tasks receive balanced and sufficient training, avoiding the gradient competition problem that may occur with traditional fixed-weight methods, thus enabling the network to accurately complete all prediction tasks simultaneously.

[0037] As can be seen from the above examples, the method provided in this embodiment has made significant technical contributions in solving existing technical problems.

[0038] Compared to existing technologies that mostly employ single-prediction-task models or combinations of multiple independent models, this method firstly uses a shared feature encoder to extract features from the regions of interest in the lung parenchyma. This allows multiple tasks to share and learn common features of lung images, promoting knowledge transfer and collaborative learning between tasks and improving the model's generalization ability and prediction accuracy. Secondly, it introduces a multi-task joint loss function based on homoscedastic uncertainty dynamic weighting. By dynamically adjusting the weights of each task's loss, an adaptive weight adjustment mechanism ensures that all tasks receive balanced and sufficient training, significantly improving the efficiency and accuracy of task analysis using medical images.

[0039] In some embodiments of this application, the lung parenchyma segmentation network includes: 3D U-Net; The 3D U-Net encoder-decoder consists of four downsampling levels with an initial channel count of 32, followed by 64, 128, 256, and 512 channels. Downsampling uses 2×2×2 max pooling or 3D convolution with a stride of 2. Upsampling uses transposed convolution or trilinear interpolation followed by 3×3×3 convolution, and is concatenated with the encoded features of the same level through skip connections. Each encoded block in the encoder-decoder consists of two 3×3×3 convolutional layers, followed by instance normalization and a Leaky ReLU activation function with a negative slope of 0.01. The output layer of 3D U-Net uses 1×1×1 convolution to generate a lung parenchyma segmentation probability map.

[0040] Among them, 3D U-Net is a deep learning network architecture specifically designed for 3D image segmentation. It can effectively learn and extract features from 3D medical images to achieve pixel-level classification, thereby accurately identifying and segmenting the lung parenchyma region. In addition to 3D U-Net, other 3D segmentation networks such as V-Net, nnU-Net, or Transformer-based segmentation models can also be used for lung parenchyma segmentation.

[0041] The encoder-decoder structure is the core of U-Net. The encoder extracts abstract features of the image by downsampling layer by layer, while the decoder restores the spatial resolution by upsampling layer by layer and fuses the high-level features of the encoder to generate fine segmentation results.

[0042] Four downsampling levels provide sufficient depth to capture features at different scales, from fine textures to macroscopic structures. The progressively increasing number of channels allows the network to learn more complex feature representations at deeper layers while maintaining information capacity. The downsampling operation aims to reduce the spatial size of the feature map while increasing the receptive field, thereby extracting more abstract and semantic features. Max pooling preserves the most salient features by selecting the maximum value in a local region, while a 2-stride 3D convolution performs feature transformation during downsampling; both effectively reduce computation and enhance feature robustness. In addition to max pooling and a 2-stride 3D convolution, downsampling can also be performed using average pooling or an asymmetric convolution kernel with a 2-stride.

[0043] Upsampling aims to restore the spatial resolution of the feature maps in order to generate a segmentation mask that matches the size of the original input image. Transposed convolution and trilinear interpolation are commonly used upsampling methods; the former performs upsampling through learning, while the latter uses an interpolation algorithm. Following upsampling with a 3×3×3 convolution can further fuse the upsampled features.

[0044] Skip connections are a key innovation of U-Net. They directly pass the fine-grained features captured in the encoder to the corresponding upsampling layer in the decoder, which helps to preserve spatial detail information and avoid losing important information such as edges and textures in deep networks, thereby generating more accurate segmentation boundaries.

[0045] Besides transposed convolution and trilinear interpolation, upsampling can also employ nearest neighbor interpolation, bilinear interpolation, and other methods. The encoding block is the basic building block of the network, and two 3×3×3 convolutional layers are used to extract local features. Instance normalization helps stabilize the training process, reduce internal covariate shifts, and improve the model's generalization ability.

[0046] The LeakyReLU activation function introduces a small negative slope, solving the problem of zero gradient in the negative region of standard ReLU, avoiding neuron death, and helping the network learn and converge better. Besides instance normalization, batch normalization or layer normalization can also be used. The output layer is the last layer of the network; the 1×1×1 convolution maps the deep feature maps to the required number of output channels and generates the probability value of each voxel belonging to the lung parenchyma.

[0047] This embodiment concretizes the lung parenchyma segmentation network as 3D U-Net, which can effectively process 3D CT data. 3D U-Net, through its unique encoder-decoder structure, can effectively process 3D CT data. The encoder part extracts abstract features of the lung parenchyma layer by layer through four downsampling levels, gradually expanding from an initial 32 channels to 512 channels, thereby capturing information of different scales and complexities. Each encoding block contains two 3×3×3 convolutional layers, combined with instance normalization and the LeakyReLU activation function, ensuring the stability of feature extraction and non-linear expressive power. The downsampling operation is implemented through max pooling or 3D convolutions with a stride of 2, reducing spatial dimensionality while expanding the network's receptive field. The decoder part performs upsampling through transposed convolutions or trilinear interpolation combined with 3×3×3 convolutions, gradually restoring the spatial resolution of the feature maps. The key is that the decoder concatenates the corresponding fine-grained features from the encoder at each upsampling level through skip connections. This mechanism effectively compensates for the details that deep networks may lose during downsampling, enabling the network to fuse high-level semantic information and low-level spatial details when restoring spatial resolution, thereby generating a high-precision lung parenchyma segmentation probability map. Finally, the output layer maps these fused features to the probability that each voxel belongs to the lung parenchyma through 1×1×1 convolutions, providing a reliable basis for subsequent lung parenchyma mask generation. This refined 3D U-Net structure allows the lung parenchyma segmentation network to more accurately identify and delineate the boundaries of the lung parenchyma, laying the foundation for subsequent multi-task deep learning networks to extract high-quality lung parenchyma region of interest features, thereby improving the accuracy and efficiency of medical image processing based on chest CT.

[0048] In some embodiments of this application, the target chest CT image is obtained in the following ways: The DICOM sequences corresponding to the initial chest CT images were sorted according to the scan location; The window width and window level were adjusted using lung window parameters, with a window level of -600HU and a window width of 1500HU. The HU values ​​were cropped to the range of -1024HU to 400HU and normalized to the range of 0 to 1. The images were resampled to 1.0mm×1.0mm×1.0mm isotropic voxels using trilinear interpolation to obtain the target chest CT image.

[0049] The DICOM sequences corresponding to the initial chest CT images are sorted according to their scan locations to ensure the spatial continuity and anatomical accuracy of the 3D image data. Images in a DICOM sequence may not be stored strictly in scan order; sorting allows for the reconstruction of the correct anatomical 3D structure, which is crucial for subsequent image processing and analysis. Sorting can be performed by reading label information such as Patient or SliceLocation from the DICOM file header, or by utilizing sequence number information such as Instance Number.

[0050] Lung window parameters were used to adjust the window width and level, with a window level of -600 HU and a window width of 1500 HU, to optimize the display of lung parenchyma in CT images. Window width and level adjustment is a fundamental technique in CT image processing. By setting specific window widths and levels, tissue structures within a specific density range can be highlighted. Lung window parameters enable a clear contrast between the lung parenchyma and surrounding tissues such as the chest wall and mediastinum, thus facilitating subsequent lung parenchyma segmentation and feature extraction. Adjustment can be achieved by linearly transforming the original HU values ​​to the display grayscale range; or by using a pre-defined lookup table (LUT).

[0051] Cropping HU values ​​to the range of -1024 HU to 400 HU and normalizing them to the range of 0 to 1 aims to remove extreme HU values ​​in images that are irrelevant to lung parenchyma analysis and to unify the intensity range of the images. CT images may contain extremely low HU values ​​(e.g., external air) or extremely high HU values ​​(e.g., metal artifacts, dense bone), which can interfere with subsequent deep learning models. Cropping effectively focuses on the density range relevant to lung tissue, reducing noise. Normalization maps the cropped HU values ​​to a uniform range of 0 to 1, which is a common preprocessing step for input to deep learning networks, helping the network converge faster and improving performance. HU value cropping can be achieved by setting a threshold to directly truncate HU values ​​that exceed the range; or by using a non-linear mapping such as the sigmoid function for soft cropping. Normalization typically uses linear scaling, i.e., (HU - min_HU) / (max_HU - min_HU).

[0052] Resampling images to 1.0mm × 1.0mm × 1.0mm isotropic voxels using trilinear interpolation aims to eliminate voxel size differences caused by different CT scanning equipment and protocols, ensuring consistency in spatial resolution across all images. Original CT images typically have anisotropic voxels, and voxel sizes may vary between scans. Resampling images to isotropic voxels provides standardized input for 3D deep learning networks, avoiding feature bias caused by inconsistent voxel sizes, thereby improving the model's generalization ability and robustness. Trilinear interpolation is a commonly used interpolation method that effectively preserves image details. Resampling can be achieved using resampling functions provided by image processing libraries (such as ITK, SimpleITK, SciPy, etc.); or by using a custom interpolation algorithm to calculate the gray value of the new voxel through a weighted average of the gray values ​​of adjacent voxels.

[0053] This embodiment transforms raw, heterogeneous chest CT images into standardized, high-quality target chest CT images through a series of refined preprocessing steps, providing optimized input for subsequent lung parenchyma segmentation and multi-task deep learning prediction. First, the DICOM sequences are sorted to ensure the anatomical accuracy of the 3D images, laying the foundation for subsequent 3D processing. Next, lung window parameter adjustments significantly enhance the contrast of the lung parenchyma, making the lung structures clearer, which greatly facilitates the accurate identification and delineation of lung regions by the lung parenchyma segmentation network. Subsequently, HU value clipping and normalization effectively filter out irrelevant extreme density information and unify the image intensity to a range easily handled by the deep learning model, thereby reducing the difficulty and instability of model training. Finally, the images are resampled to isotropic voxels, eliminating spatial resolution differences caused by different scanning devices and protocols, ensuring the consistency of all input images in the spatial dimension, enabling the deep learning network to learn more generalizable features. These preprocessing steps work synergistically to address the challenges posed by the heterogeneity of raw CT images to subsequent analysis, significantly improving the accuracy and efficiency of the entire chest CT-based medical image processing method.

[0054] In some embodiments of this application, the post-processing procedure includes: The two largest connected regions of the left and right lungs are preserved, and three-dimensional hole filling and morphological closure operations are performed to remove the chest wall, bones, mediastinum, bed board and extra-lung air background areas. Based on the central connecting structure of the upper lung field and the location of the airway opening, the trachea and main bronchus regions are removed or weakened. Set the area outside the lung parenchyma mask to a uniform background value.

[0055] This embodiment employs a series of refined post-processing operations on the lung parenchyma segmentation probability map output by the lung parenchyma segmentation network to ensure high accuracy and purity of the final extracted lung parenchyma region of interest. Specifically, firstly, by retaining the two largest connected regions of the left and right lungs, small, discrete noise points or mis-segmented regions that may be generated by the segmentation network can be effectively filtered out, thereby focusing on the main lung structures. Based on this, three-dimensional hole filling and morphological closure operations are performed to repair cavities within the lung parenchyma, smooth the mask boundaries, and connect any possible breaks, making the lung parenchyma mask more complete and continuous in morphology, and more consistent with the actual anatomical structure. Subsequently, by removing the chest wall, bones, mediastinum, bed lamina, and extrapulmonary air background areas, the lung parenchyma is further precisely separated from surrounding non-lung tissues and external interference, ensuring that the area for subsequent analysis is limited to the lung parenchyma itself. To avoid interference from large airways in the extraction of lung parenchyma features, the trachea and main bronchus regions are removed or weakened based on the central connectivity structure of the upper lung field and the location of airway openings, making the lung parenchyma mask purer. Finally, the region outside the lung parenchyma mask is set to a uniform background value, standardizing the non-lung parenchyma region and providing clear, interference-free input for subsequent feature extraction by multi-task deep learning networks. Through the above synergistic effect, the scheme in this application can significantly improve the extraction quality of regions of interest in lung parenchyma, providing more reliable and accurate input data for subsequent multi-task prediction.

[0056] In some embodiments of this application, the shared feature encoder includes any one of 3D ResNet-18, MobileNetV2, ShuffleNet, EfficientNet, or ConvNet.

[0057] When regions of interest (ROIs) in the lung parenchyma are fed into a pre-trained multi-task deep learning network, these carefully designed shared feature encoders come into play. For example, if 3D ResNet-18 is chosen as the encoder, it will efficiently extract discriminative deep features from the 3D ROIs of the lung parenchyma using its unique residual connections and 3D convolutional layers. Choosing lightweight networks such as MobileNetV2 or ShuffleNet significantly reduces computational resource consumption while maintaining feature extraction capabilities, making the entire prediction process more efficient. EfficientNet, through its composite scaling strategy, optimizes the efficiency and accuracy of feature extraction under different resource constraints. These encoders pass the extracted high-quality global feature vectors to multiple task branches, enabling these branches to output COPD severity grading, pulmonary hypertension risk prediction, and pulmonary heart disease risk prediction results in parallel based on these rich and representative features. By selecting these optimized shared feature encoders, this application can ensure the extraction of sufficiently rich and representative features from complex chest CT images, thereby overcoming the problems of low feature extraction efficiency or insufficient accuracy in traditional methods, and providing a solid foundation for subsequent chest CT-based medical image processing.

[0058] In some embodiments of this application, prior to acquiring the target chest CT image, the method further includes: A multi-task joint loss function is used to train the multi-task deep learning network; the multi-task joint loss function is: ; in, Indicates the task sequence number. For the first The loss of each task For the first Learnable parameters for each task The variance represents the uncertainty of the task loss.

[0059] A multi-task joint loss function is a function used in multi-task learning to simultaneously optimize the losses of multiple related tasks. Its core idea is to combine the loss terms of multiple tasks into a single optimization objective, enabling the model to learn and improve the performance of all tasks simultaneously. This function is designed to address potential conflicts or imbalances between different tasks, guiding the model to learn shared representations that are beneficial to all tasks through appropriate weighting or combination strategies. It is a specific form of loss function, a dynamically weighted loss function based on homoscedastic uncertainty. Among them, This represents the total joint loss, which is the objective that the model needs to minimize during training. This represents the summation of losses across all tasks. Task number This is used to differentiate between different prediction tasks in multi-task deep learning networks, such as COPD severity grading, pulmonary hypertension risk prediction, and pulmonary heart disease risk prediction. Each task has its own independent loss. . Representing the The loss function value for the nth task is used to measure the model's performance on the nth task. The difference between the predicted result and the true label on a task. For example, for a classification task, It could be cross-entropy loss; for regression tasks, It could be the mean squared error loss. It is a related to the first The learnable parameters associated with each task, which are expressed as variances of the uncertainty of the task loss. The logarithm is used to obtain this parameter. This parameter is automatically adjusted during training as the network weights are updated, and its function is to dynamically adjust the contribution weight of each task in the total loss. Indicates the first The uncertainty variance of the loss function reflects the inherent noise level or learning difficulty of the task. A larger variance indicates higher uncertainty or noise in the task, potentially leading to lower confidence in the model's predictions. By incorporating this uncertainty variance into the loss function, the model can assign lower weights to tasks with higher uncertainty and higher weights to tasks with lower uncertainty during training, thus achieving an adaptive balance among tasks.

[0060] This embodiment, through this adaptive weight adjustment, enables the network to avoid the loss of a single task being too large or too small, thus preventing it from dominating or being ignored. This facilitates the shared feature encoder in learning a global feature vector that is robust and generalizable across all tasks. After receiving these optimized global feature vectors, multiple task branches can more effectively output COPD severity classification results, pulmonary hypertension risk prediction results, and pulmonary heart disease risk prediction results, thereby improving overall prediction performance.

[0061] In some embodiments of this application, the first 5 epochs of training the multi-task deep learning network use fixed weights. Learnable parameters are enabled starting from the 6th epoch. The weights are automatically adjusted by a multi-task deep learning network based on the uncertainty of the loss of each task.

[0062] This phased training strategy cleverly combines the stability of early training with the adaptability of later training, enabling multi-task deep learning networks to converge more effectively and ultimately achieve better performance on all prediction tasks.

[0063] like Figure 3 One embodiment of this application provides a medical image processing device based on chest CT, the device comprising: Data acquisition module 1001 is used to acquire target chest CT images; The region of interest extraction module 1002 is used to input the target chest CT image into the pre-trained lung parenchyma segmentation network to obtain the lung parenchyma segmentation probability map, generate a lung parenchyma mask based on post-processing, and extract the region of interest of the lung parenchyma based on the lung parenchyma mask. The risk prediction module 1003 is used to input the region of interest in the lung parenchyma into a pre-trained multi-task deep learning network. The multi-task deep learning network includes a shared feature encoder and multiple task branches. The shared feature encoder extracts features from the region of interest in the lung parenchyma, and the multiple task branches output COPD severity classification results, pulmonary hypertension risk prediction results, and pulmonary heart disease risk prediction results based on global feature vectors. The multi-task deep learning network is trained using a multi-task joint loss function based on homoscedastic uncertainty dynamic weighting.

[0064] It should be noted that the medical image processing device based on chest CT in this application embodiment and the aforementioned medical image processing method based on chest CT are based on the same inventive concept. Therefore, the medical image processing device based on chest CT in this application embodiment corresponds to the aforementioned medical image processing method based on chest CT. For the specific implementation process, please refer to the aforementioned medical image processing method based on chest CT, which will not be repeated here.

[0065] like Figure 4 As shown, one embodiment of this application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned medical image processing method based on chest CT. The electronic device includes: At least one battery; At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the above-described medical image processing method based on chest CT.

[0066] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0067] The electronic devices according to embodiments of this application will now be described in detail.

[0068] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to perform a medical image processing method based on chest CT according to an embodiment of this application.

[0069] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0070] This application embodiment also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described medical image processing method based on chest CT.

[0071] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0072] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0073] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0074] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0075] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0076] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0077] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0078] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0079] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0081] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A medical image processing method based on chest CT, characterized in that, The method includes: Acquire target chest CT images; The target chest CT image is input into a pre-trained lung parenchyma segmentation network to obtain a lung parenchyma segmentation probability map. A lung parenchyma mask is generated based on post-processing, and the region of interest in the lung parenchyma is extracted based on the lung parenchyma mask. The region of interest (ROI) of the lung parenchyma is input into a pre-trained multi-task deep learning network. The multi-task deep learning network includes a shared feature encoder and multiple task branches. The shared feature encoder extracts features from the ROI of the lung parenchyma to obtain shared image features associated with multiple prediction tasks. The multiple task branches output prediction results for the corresponding prediction tasks based on the shared image features. The multi-task deep learning network is trained using a multi-task joint loss function based on dynamic weighting of homoscedastic uncertainty.

2. The medical image processing method based on chest CT according to claim 1, characterized in that, The lung parenchyma segmentation network includes: 3D U-Net; The encoder-decoder of the 3D U-Net contains four downsampling levels with an initial channel count of 32, followed by 64, 128, 256, and 512 channels. Downsampling uses 2×2×2 max pooling or 3D convolution with a stride of 2. Upsampling uses transposed convolution or trilinear interpolation followed by 3×3×3 convolution, and is concatenated with the encoded features of the same level through skip connections. Each coding block in the encoder-decoder contains two 3×3×3 convolutional layers, followed by instance normalization and a LeakyReLU activation function with a negative slope of 0.

01. The output layer of the 3D U-Net uses 1×1×1 convolution to generate a lung parenchyma segmentation probability map.

3. The medical image processing method based on chest CT according to claim 1, characterized in that, The target chest CT image was obtained through the following methods: The DICOM sequences corresponding to the initial chest CT images were sorted according to the scan location; The window width and window level were adjusted using lung window parameters, with a window level of -600HU and a window width of 1500HU. The HU values ​​were cropped to the range of -1024HU to 400HU and normalized to the range of 0 to 1. The image was resampled to a 1.0mm×1.0mm×1.0mm isotropic voxel using trilinear interpolation to obtain the target chest CT image.

4. The medical image processing method based on chest CT according to claim 1, characterized in that, The post-processing process includes: The two largest connected regions of the left and right lungs are preserved, and three-dimensional hole filling and morphological closure operations are performed to remove the chest wall, bones, mediastinum, bed board and extra-lung air background areas. Based on the central connecting structure of the upper lung field and the location of the airway opening, the trachea and main bronchus regions are removed or weakened. Set the area outside the lung parenchyma mask to a uniform background value.

5. The medical image processing method based on chest CT according to claim 1, characterized in that, The shared feature encoder includes any one of 3D ResNet-18, MobileNetV2, ShuffleNet, EfficientNet, or ConvNet.

6. The medical image processing method based on chest CT according to claim 1, characterized in that, Prior to acquiring the target chest CT image, the procedure also includes: The multi-task deep learning network is trained using a multi-task joint loss function; wherein the multi-task joint loss function is: ； in, Indicates the task sequence number. For the first The loss of each task For the first Learnable parameters for each task The variance represents the uncertainty of the task loss.

7. A medical image processing method based on chest CT according to claim 6, characterized in that, The multi-task deep learning network uses fixed weights for the first 5 epochs of training; from the 6th epoch onwards, the learnable parameters are enabled. The weights are automatically adjusted by the multi-task deep learning network based on the uncertainty of the loss of each task.

8. A medical image processing device based on chest CT, characterized in that, The device includes: The data acquisition module is used to acquire the target chest CT image; The region of interest extraction module is used to input the target chest CT image into a pre-trained lung parenchyma segmentation network to obtain a lung parenchyma segmentation probability map, generate a lung parenchyma mask based on post-processing, and extract the region of interest of the lung parenchyma based on the lung parenchyma mask. A risk prediction module is used to input the region of interest (ROI) of the lung parenchyma into a pre-trained multi-task deep learning network. The multi-task deep learning network includes a shared feature encoder and multiple task branches. The shared feature encoder extracts features from the ROI to obtain shared image features associated with multiple prediction tasks. The multiple task branches output prediction results for the corresponding prediction tasks based on the shared image features. The multi-task deep learning network is trained using a multi-task joint loss function based on dynamic weighting of homoscedastic uncertainty.

9. An electronic device, characterized in that, include: At least one processor and a memory for communicatively connecting to said at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a medical image processing method based on chest CT as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a medical image processing method based on chest CT as described in any one of claims 1 to 7.

Images