A method and device for constructing a dual-task lung collapse assessment model

Abstract

The application provides a method and device for constructing a double-task lung collapse evaluation model, which comprises the following steps: obtaining lung images with different lung collapse degrees, and marking the collapsed lung region and the chest cavity background region; extracting the collapsed lung region image and marking the collapsed region and the non-collapsed region label; training a lung volume task segmentation module and a lung color task segmentation module based on the marked image; constructing a fusion evaluation module for obtaining the lung collapse evaluation result based on the segmentation result of the lung volume task segmentation module and the lung color task segmentation module; and obtaining the double-task lung collapse evaluation model based on the fusion evaluation module and the trained lung volume task segmentation module and lung color task segmentation module. The application solves the problem in the prior art that the lung collapse evaluation mainly depends on the subjective experience of doctors during surgery, lacks artificial intelligence assistance, and results in large differences in scoring results, which easily leads to misjudgment.

Description

Technical Field

[0001] This invention belongs to the field of clinical medical technology, and in particular relates to a method and device for constructing a dual-task lung collapse assessment model. Background Technology

[0002] In current thoracic surgery anesthesia and thoracoscopic surgery, the assessment of the degree of lung collapse mainly relies on the subjective experience of the anesthesiologist or surgeon during the operation. This method is highly subjective and has poor repeatability. Different doctors have different experience and subjective standards, resulting in large differences in the scoring results.

[0003] With the development of artificial intelligence algorithms, the probability of misjudging lung collapse can be greatly reduced by using pre-trained models for auxiliary assessment. Therefore, there is an urgent need for a reliable, standardized, objective, and traceable lung collapse assessment model for real-time intraoperative evaluation and postoperative data analysis, so as to provide a scientific basis for optimizing clinical anesthesia strategies and postoperative rehabilitation. Summary of the Invention

[0004] Based on the above analysis, the present invention aims to provide a method and device for constructing a dual-task lung collapse assessment model, which solves the problem that the lung collapse assessment in the prior art mainly relies on the subjective experience judgment of the surgeon during operation, lacks artificial intelligence assistance, and leads to large differences in scoring results, which is prone to misjudgment.

[0005] The objective of this invention is mainly achieved through the following technical solutions:

[0006] On the one hand, the present invention provides a method for constructing a dual-task lung collapse assessment model, comprising: Multiple lung images with different degrees of lung collapse are acquired, and the collapsed lung region and the pleural background region are labeled to obtain the first training sample set; the lung volume task segmentation module is trained based on the first training sample set. Based on the lung images, images of collapsed lung regions are extracted, and the collapsed lung regions are labeled with collapsed and non-collapsed regions to obtain a second training sample set; a lung color task segmentation module is trained based on the second training sample set. A fusion assessment module is constructed, which is used to obtain the corresponding lung volume task assessment result and lung color task assessment result based on the segmentation results of the lung volume task segmentation module and the lung color task segmentation module; and to fuse the two task assessment results to obtain the final lung collapse assessment result. Based on the fusion evaluation module and the trained lung volume task segmentation module and lung color task segmentation module, a dual-task lung collapse evaluation model is obtained.

[0007] Furthermore, both the lung volume task segmentation module and the lung color task segmentation module are improved Unet models; The lung volume task segmentation module is constructed by sequentially setting a first multi-scale feature fusion layer and a lung volume attention layer between the encoder and decoder of the Unet model. The lung color task segmentation module is constructed by sequentially setting a second multi-scale feature fusion layer and a lung color attention layer between the encoder and decoder of the Unet model.

[0008] Furthermore, the first multi-scale feature fusion layer and the second multi-scale feature fusion layer have the same structure, including: Standard convolution module: It uses a 3×3 receptive field to perform convolution operations on the received feature map in order to extract detailed features and maintain the consistency of local features; Medium dilation rate convolution module: The dilation rate is set to 2, and a 5×5 receptive field is used to capture the structural relationships and contextual dependencies of the region; High dilation rate convolution module: The dilation rate is set to 4, and a 9×9 receptive field is used to model global semantic constraints and long-distance spatial relationships; Feature fusion module: This module is used to weight the feature maps obtained from each convolution module and then concatenate them along the channel dimension; and then perform dimensionality reduction through 1×1 convolutional channels to obtain a multi-scale fused feature map. When performing multi-view feature extraction through each convolutional module, padding is used to pre-fill the feature map with different amounts of padding to ensure that the extracted feature map is the same size as the original feature map.

[0009] Furthermore, the lung volume attention layer is used to extract features from the received multi-scale fusion feature map through a lung volume attention mechanism that includes a channel attention mechanism and a spatial attention mechanism, to obtain a channel attention feature map and a spatial attention feature map; and to fuse and output the channel attention feature map and the spatial attention feature map to enhance the features of important channels and important spaces.

[0010] Furthermore, the lung color attention layer includes a color statistics attention module, a local color distribution attention module, and a fusion module; The color statistics attention module is used to extract macroscopic color statistics features of the image using the following method: Macroscopic color statistical features are extracted from the multi-scale fusion feature maps output by the corresponding multi-scale feature fusion layers through both global average pooling and max pooling paths. The two macroscopic color statistical feature maps are concatenated along the channel dimension, and the concatenated feature maps are then used to generate channel attention weights for each channel via the sigmoid activation function. The multi-scale fused feature map is multiplied channel by channel with the corresponding channel attention weight to obtain the output of the color statistical attention module; The local color distribution attention module is used to extract local spatial patterns between different color channel groups using the following method: The received multi-scale fused feature maps are grouped along the channel dimension, and convolution operations are performed on each group of features. The convolution results are then concatenated within the group to obtain the feature map corresponding to each group. Each set of corresponding feature maps is converted to a single-channel attention map by 1x1 convolution to reduce the number of channels to 1. Each group of the multi-scale fused feature map is multiplied by the corresponding single-channel attention map to obtain the output of the local color distribution attention module.

[0011] Furthermore, both the lung volume segmentation module and the lung color segmentation module are trained using the following loss function: ; ; Where N is the number of pixels, Let be the predicted probability of the i-th pixel for its true class. As a focal regulation factor, Focus-adjusting weights are used to identify and prioritize difficult-to-classify boundary regions. Boundary enhancement factor; For the first The gradient of each pixel; Gradient-aware weights; Based on cross-entropy loss; To predict the probability in Gradient of direction; To predict the probability in Gradient of direction.

[0012] Furthermore, the segmentation results of the lung volume task segmentation module include the segmentation results of the collapsed lung region and the pleural background region; Based on the segmentation results of the lung volume task segmentation module, the corresponding lung volume task evaluation results are obtained, including: Based on the segmentation results of the collapsed lung region and the pleural background region, lung masks and pleural background region masks are obtained; Based on the lung mask and the pleural cavity background region mask, the number of pixels occupied by the collapsed lung region and the pleural cavity background region are obtained respectively, and the proportion of the collapsed lung region to the entire pleural cavity is obtained based on the number of pixels. The first lung collapse level is obtained based on the ratio and used as the result of the lung volume task assessment.

[0013] Furthermore, the segmentation results of the lung color task segmentation module include segmentation results of collapsed areas and non-collapsed areas; The lung color task evaluation result is obtained based on the segmentation result of the lung color task segmentation module, including: The segmentation results of the collapsed and non-collapsed areas are converted from RGB to grayscale images, and the average grayscale value is extracted for each. The average gray values ​​are normalized to obtain the color weights corresponding to the two regions. The area ratio of each region is obtained based on the number of pixels occupied by the collapsed and non-collapsed regions, and the area ratio is used as the area weight of each region. Based on the color weight and area weight, the lung color index is obtained using the following formula: ; in, As an indicator of lung color, Color weights for the collapsed areas. Color weights for the non-collapsed areas. The area weight of the collapsed area, The area weight of the non-collapsed area; The second lung collapse level is obtained based on the lung color index and used as the assessment result of the lung color task.

[0014] Furthermore, the proportion of the collapsed lung area to the total thoracic cavity area is divided into n levels in descending order to obtain the first lung collapse degree level; The second degree of lung collapse was obtained by the following method: Lung color indices were obtained from multiple lung images with different degrees of lung collapse. The maximum and minimum values ​​of these lung color indices were then represented as follows: and ; by The result of +0.1 is used as the upper bound for evaluation. ,like If +0.1>1, then take 1; The result of -0.1 is used as the lower bound for evaluation. ,like If -0.1 < 0, then the value is 0; based on - The value is used to construct n equally divided intervals, and then arranged in descending order to obtain n levels, which serve as the second lung collapse degree level.

[0015] On the other hand, an electronic device is also provided, including at least one processor and at least one memory communicatively connected to said processor; The memory stores instructions that can be executed by the processor to implement the aforementioned method for constructing a dual-task lung collapse assessment model.

[0016] The beneficial effects of this technical solution are: 1. In order to achieve a comprehensive assessment of lung collapse, this invention designs two assessment tasks for lung volume and lung color. Through two corresponding segmentation modules, it realizes the accurate calculation of the lung area as a percentage of the thoracic cavity area after lung collapse and the accurate analysis of the lung surface color. Finally, it completes a comprehensive assessment of the patient's lung collapse and finally integrates the assessment results of the two tasks of lung volume and lung color to obtain the final lung collapse assessment result. This avoids the misjudgment problem caused by a single assessment indicator and improves the accuracy of lung collapse assessment. 2. During model training, this invention combines an improved gradient-aware focus loss function with a three-pronged approach, demonstrating significant advantages in medical image segmentation: First, the basic cross-entropy loss ensures overall segmentation accuracy; second, the focus adjustment mechanism automatically identifies and prioritizes difficult-to-classify samples, which are often located in complex boundary regions; finally, the gradient-aware weights, based on the spatial variation characteristics of predicted probabilities, can accurately identify segmentation boundaries and assign higher optimization weights to these key regions, improving the precision and accuracy of image segmentation. 3. This invention considers that the segmentation of collapsed lung regions and pleural cavity background regions relies on their spatial features in lung images, and introduces a lung volume attention mechanism to accurately locate key segmentation structures. This mechanism is based on a deep convolutional neural network and integrates two core technology modules: channel attention and spatial attention. The channel attention mechanism adjusts the contribution of each channel to enhance useful feature channels and suppress feature channels with smaller contributions, allowing the network to focus on channels with greater information content. The spatial attention mechanism learns the importance of spatial locations, making the algorithm focus more on important spatial locations in the image, thereby enhancing the features of important spatial locations. 4. This invention proposes an enhanced color attention mechanism for lung color scoring tasks, specifically optimizing the detection capability of light-colored regions caused by poor lung collapse. It extracts macroscopic color statistical features of lung images through color statistical attention; and analyzes fine-grained color texture changes through local color distribution attention, enabling the network to learn local spatial patterns between different color channel groups. This helps capture local texture changes on different color components, improving the segmentation accuracy of collapsed and non-collapsed lung regions. Attached Figure Description

[0017] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Figure 1 This is a flowchart of the construction method of the dual-task lung collapse assessment model according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the dual-task lung collapse assessment model according to an embodiment of the present invention. Detailed Implementation

[0018] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0019] One embodiment of the present invention provides a method for constructing a dual-task lung collapse assessment model, such as... Figure 1 As shown, it includes: Step S1: Acquire multiple lung images with different degrees of lung collapse, and label the collapsed lung region and the pleural background region to obtain the first training sample set; train the lung volume task segmentation module based on the first training sample set; Specifically, this application uses a clinically standard thoracoscopic probe to acquire lung images. When acquiring lung images, it can be adapted to various surgical approaches, such as multi-port thoracoscopic probes or single-port thoracoscopic probes.

[0020] To obtain the optimal field of view for intelligent assessment, the thoracoscopic probe is preferably oriented towards the patient's head for image acquisition. This orientation helps to acquire a panoramic image of the lung surface, reducing anatomical obstruction and thus improving the accuracy of subsequent image analysis and assessment of the degree of lung collapse. After image acquisition, preprocessing is performed, including image stabilization, white balance, and noise reduction. To meet the input dimension requirements of general image data models for convolutional neural networks (such as ResNet), the lung images are also standardized, such as being uniformly scaled to 224×224 pixels. This standardization process has dual advantages: firstly, compared to the high resolution of the original image, the standardized image saves training memory and improves training efficiency; secondly, the standardized image size matches the original input size of the model, facilitating the use of pre-trained model parameters and fully leveraging the capabilities of transfer learning. After standardizing the lung images, various data augmentation techniques can be employed, including random rotation, horizontal and vertical flipping, random translation, and photometric enhancement. These techniques allow the model to access diverse image transformations, resist common interferences from image orientation and lighting conditions, enhance its generalization ability to unseen data, and reduce the risk of overfitting. In this embodiment, the random rotation range is set to (-15°, 15°), the probability of horizontal and vertical flipping is set to 0.5, the maximum displacement of random translation is set to 0.2 (20% of the original image's horizontal size), and photometric enhancement includes adjusting the image's brightness, contrast, and saturation, all within a range of (0.8, 1.2), i.e., randomly adjusted from 80% to 120% of the original values.

[0021] After processing the lung images as described above, labels are added to the collapsed lung region and the pleural background region to construct the first training sample set, which is used for training the lung volume segmentation module.

[0022] Preferably, the lung volume task segmentation module is an improved Unet model; the lung volume task segmentation module is constructed by sequentially setting a first multi-scale feature fusion layer and a lung volume attention layer between the encoder and decoder of the Unet model. More specifically, the lung volume task segmentation module includes a first encoder, a first multi-scale feature fusion layer, a lung volume attention layer, and a first decoder; The first encoder adopts the encoder structure of the Unet model and is used to perform multiple feature extractions and feature compressions based on the lung image to obtain feature maps at multiple scales. The first multi-scale feature fusion layer extracts and fuses features from the feature map output by the last layer of the first encoder through convolutional kernels of different scales set in parallel, thereby obtaining the first multi-scale fused feature map. The lung volume attention layer is used to extract features from the first multi-scale fusion feature map through a lung volume attention mechanism that includes channel attention and spatial attention mechanisms, to obtain channel attention feature maps and spatial attention feature maps; and to fuse and output the channel attention feature maps and spatial attention feature maps to enhance the features of important channels and important spaces; The first decoder adopts the decoder structure of the Unet model, which is used to perform multiple upsampling on the feature map output by the lung volume attention layer, and to perform skip connections with the feature map of the corresponding scale output by the first encoder to obtain the segmentation results of the collapsed lung region and the pleural background region.

[0023] Specifically, the first multi-scale feature fusion layer includes: Standard convolution module: It uses a 3×3 receptive field to perform convolution operations on the received feature map in order to extract detailed features and maintain the consistency of local features; Medium dilation rate convolution module: The dilation rate is set to 2, and a 5×5 receptive field is used to capture the structural relationships and contextual dependencies of the region; High dilation rate convolution module: The dilation rate is set to 4, and a 9×9 receptive field is used to model global semantic constraints and long-distance spatial relationships; Feature fusion module: This module is used to weight the feature maps obtained from each convolution module and then concatenate them along the channel dimension; and then perform dimensionality reduction through 1×1 convolutional channels to obtain the first multi-scale fused feature map. When performing multi-view feature extraction through each convolutional module, padding is used to pre-fill the feature map with different amounts of padding to ensure that the extracted feature map is the same size as the original feature map.

[0024] Specifically, this embodiment proposes a multi-scale feature fusion module. By applying convolutional kernels with different dilation rates in parallel, it extracts high-level semantic feature maps from the output of the last layer of the U-Net encoding layer from multiple perspectives. The standard convolutional kernel uses a 3×3 receptive field to focus on local feature consistency and detail preservation; the medium dilation rate convolutional kernel has a dilation rate of 2 and uses a 5×5 receptive field to capture regional structural relationships and contextual dependencies; and the large dilation rate convolutional kernel has a dilation rate of 4 and uses a 9×9 receptive field to model global semantic constraints and long-range spatial relationships. When using multi-view feature extraction, padding is used to pre-fill the feature map with different amounts of padding to ensure that the extracted feature map is the same size as the original feature map. The three different scale feature representations are fused through weighted summation, where the weight parameters are learned. The learned weight parameters are multiplied by the corresponding multi-scale features, concatenated along the channel direction, and then subjected to 1×1 convolutional channel dimensionality reduction to finally obtain a multi-scale fused feature map with the same size as the last layer of the original U-Net encoding layer.

[0025] Furthermore, feature extraction in the lung volume task segmentation module using the channel attention mechanism includes: Global average pooling is performed on the input first multi-scale fusion feature map to compress all spatial information of each channel into a global feature; The global features are learned through two fully connected layers set sequentially to learn the non-linear relationship between channels; the first fully connected layer reduces the number of channels, and the second fully connected layer restores the original number of channels. The output of the second fully connected layer is restricted to between 0 and 1 by using the Sigmoid activation function, which serves as the weight for the corresponding channel. After multiplying each channel of the first multi-scale fusion feature map by its corresponding weight, the two channels are concatenated along the channel dimension to obtain the channel attention feature map.

[0026] Feature extraction via the spatial attention mechanism includes: The first multi-scale fusion feature map input is subjected to average pooling and max pooling along the channel dimension to obtain two spatial feature maps; The two spatial feature maps are spliced ​​together to obtain a feature map with a size of H×W×2; The concatenated feature map is convolved to learn the importance of spatial location, and a weight map between 0 and 1 is generated by the Sigmoid function. The weight map is multiplied position-by-position with the input first multi-scale fusion feature map to obtain the spatial attention feature map.

[0027] This embodiment considers that the segmentation of collapsed lung regions and pleural cavity background regions is based on their spatial features in lung images, and introduces a lung volume attention mechanism to accurately locate key segmentation structures. This mechanism is based on a deep convolutional neural network and integrates two core technology modules: channel attention and spatial attention. The channel attention mechanism adjusts the contribution of each channel to enhance useful feature channels and suppress those with smaller contributions, allowing the network to focus on channels with greater information content. The spatial attention mechanism learns the importance of spatial locations, making the algorithm focus more on important spatial locations in the image, thereby enhancing the features of important spatial locations.

[0028] After extracting features from the multi-scale fused feature map through channel attention and spatial attention branches respectively, the output weights of the two branches are multiplied by the original input feature map to obtain two weighted feature maps. The two weighted feature maps are then summed to obtain the output of the lung volume attention layer.

[0029] Preferably, in order to improve the image segmentation accuracy and precision of the lung volume task segmentation module, the lung volume task segmentation module is trained using the following loss function: ; ; Where N is the number of pixels, Let be the predicted probability of the i-th pixel for its true class. As a focal regulation factor, Focus-adjusting weights are used to identify and prioritize difficult-to-classify boundary regions. Boundary enhancement factor; For the first The gradient of each pixel; Gradient-aware weights are used to accurately identify segmentation boundaries; To predict the probability in Gradient of direction; To predict the probability in Gradient of direction.

[0030] This embodiment demonstrates significant advantages in medical image segmentation through an improved loss function and a three-pronged approach: First, the basic cross-entropy loss ensures overall segmentation accuracy; second, the focus adjustment mechanism automatically identifies and prioritizes difficult-to-classify samples, which are often located in complex boundary regions; and finally, the gradient-aware weights, based on the spatial variation characteristics of predicted probabilities, accurately identify segmentation boundaries and assign higher optimization weights to these key regions, thereby improving the precision and accuracy of image segmentation.

[0031] Step S2: Extract images of collapsed lung regions based on the lung images, and label the collapsed lung regions with collapsed and non-collapsed regions to obtain a second training sample set; train a lung color task segmentation module based on the second training sample set; the lung color task segmentation module and the lung volume task segmentation module are trained using the same loss function.

[0032] The lung color task segmentation module is also an improved Unet model; the lung color task segmentation module is constructed by sequentially setting a second multi-scale feature fusion layer and a lung color attention layer between the encoder and decoder of the Unet model.

[0033] More specifically, the lung color task segmentation module includes a second encoder, a second multi-scale feature fusion layer, a lung color attention layer, and a second decoder arranged sequentially. The second encoder, the second multi-scale feature fusion layer, and the second decoder have the same structure as the corresponding first encoder, the first multi-scale feature fusion layer, and the first decoder. The lung color attention mechanism includes a color statistical attention mechanism and a local color distribution attention mechanism; the lung color attention layer includes a color statistical attention module, a local color distribution attention module, and a fusion module. The color statistics attention module is used to extract the macroscopic color statistics features of the image; The local color distribution attention module is used to extract local spatial patterns between different color channel groups; The fusion module is used to perform weighted fusion of the outputs of the color statistics attention module and the local color distribution attention module to obtain the lung color feature extraction result; Based on the lung color feature extraction results, the second decoder performs multiple upsampling operations and performs skip connections with the feature maps of the corresponding scale output by the second encoder to obtain the segmentation results of the collapsed and non-collapsed lung regions.

[0034] The color statistics attention module extracts features using the following method: Macroscopic color statistical features are extracted from the second multi-scale fusion feature map output by the second multi-scale feature fusion layer through both global average pooling and max pooling paths. The two macroscopic color statistical feature maps are concatenated along the channel dimension, and the concatenated feature maps are then used to generate channel attention weights for each channel via the sigmoid activation function. The second multi-scale fusion feature map is multiplied channel by channel with the corresponding channel attention weight to obtain the output of the color statistical attention module; The local color distribution attention module extracts features using the following method: The second multi-scale fusion feature map is grouped along the channel dimension, and a convolution operation is performed on each group of features. The convolution results are then connected within the group to obtain the feature map corresponding to each group. Each set of corresponding feature maps is converted to a single-channel attention map by 1x1 convolution to reduce the number of channels to 1. Each group of the second multi-scale fused feature map is multiplied by the corresponding single-channel attention map to obtain the output of the local color distribution attention module.

[0035] Specifically, this embodiment proposes an enhanced color attention mechanism for the lung color scoring task, specifically optimizing the detection capability of light-colored regions caused by poor lung collapse. Macroscopic color statistical features of the lung image are extracted through color statistical attention; fine-grained color texture changes are analyzed based on grouped convolutions (e.g., group=4) through local color distribution attention. Each group is convolved separately, with the output channel number of each group equal to the input channel number divided by the group number. The outputs of all groups are then concatenated to obtain C / 4 channels. Finally, a 1x1 convolution reduces the number of channels in each group to 1, generating a single-channel attention map. This approach allows the network to learn local spatial patterns between different color channel groups, rather than mixing all color channels together. This helps capture local texture changes in different color components, improving the segmentation accuracy of collapsed and non-collapsed lung regions.

[0036] In summary, to achieve a comprehensive assessment of lung collapse and improve its accuracy, this embodiment designs two assessment tasks: lung volume and lung color. Two corresponding segmentation models (assessment modules) are used to accurately calculate the ratio of lung area to pleural cavity area after lung collapse and to accurately analyze the lung surface color, ultimately completing a comprehensive assessment of the patient's lung collapse condition. The two segmentation models are based on the U-Net framework, combining multi-scale feature fusion and attention mechanisms designed for different tasks to integrate the lung volume and lung color assessment tasks, avoiding misjudgments caused by a single assessment indicator.

[0037] Step S3: Construct a fusion assessment module, which is used to obtain the corresponding lung volume task assessment result and lung color task assessment result based on the segmentation results of the lung volume task segmentation module and the lung color task segmentation module; and fuse the two task assessment results to obtain the final lung collapse assessment result. Specifically, the segmentation results of the lung volume task segmentation module include the segmentation results of the collapsed lung region and the pleural background region; Based on the segmentation results of the lung volume task segmentation module, the corresponding lung volume task evaluation results are obtained, including: Based on the segmentation results of the collapsed lung region and the pleural background region, lung masks and pleural background region masks are obtained; Based on the lung mask and the pleural cavity background region mask, the number of pixels occupied by the collapsed lung region and the pleural cavity background region are obtained respectively, and the proportion of the collapsed lung region to the entire pleural cavity is obtained based on the number of pixels. The first lung collapse level is obtained based on the ratio and used as the result of the lung volume task assessment.

[0038] Specifically, for the lung volume task, this embodiment adopts the U-Net main framework, adding a multi-scale feature fusion module and a lung volume attention mechanism after its last encoding layer. The output is used as the input to the first decoding layer, and the final model output is a mask for the collapsed lung region and a mask for the pleural cavity background region (excluding the lung). By calculating the number of pixels occupied by each mask, the final ratio of the collapsed lung region to the pleural cavity area is: lung pixels / (lung pixels + pure pleural cavity pixels).

[0039] Furthermore, the segmentation results of the lung color task segmentation module include segmentation results of collapsed areas and non-collapsed areas; The lung color task evaluation result is obtained based on the segmentation result of the lung color task segmentation module, including: The segmentation results of the collapsed and non-collapsed areas are converted from RGB to grayscale images, and the average grayscale value is extracted for each. The average gray values ​​are normalized to obtain the color weights corresponding to the two regions. The area ratio of each region is obtained based on the number of pixels occupied by the collapsed and non-collapsed regions, and the area ratio is used as the area weight of each region. Based on the color weight and area weight, the lung color index is obtained using the following formula: ; in, As an indicator of lung color, Color weights for the collapsed areas. Color weights for the non-collapsed areas. The area weight of the collapsed area, The area weight of the non-collapsed area; The second lung collapse level is obtained based on the lung color index and used as the assessment result of the lung color task.

[0040] Furthermore, the proportion of the collapsed lung area to the total thoracic cavity area is divided into n levels in descending order to obtain the first lung collapse degree level; The second degree of lung collapse was obtained by the following method: Lung color indices were obtained from multiple lung images with different degrees of lung collapse. The maximum and minimum values ​​of these lung color indices were then represented as follows: and ; by The result of +0.1 is used as the upper bound for evaluation. ,like If +0.1>1, then take 1; The result of -0.1 is used as the lower bound for evaluation. ,like If -0.1 < 0, then the value is 0; based on - The value is used to construct n equally divided intervals, and then arranged in descending order to obtain n levels, which serve as the second lung collapse degree level.

[0041] It should be noted that during the lung collapse procedure, some areas of the lung may collapse well, while others may not collapse as well as expected. For example, the surface features of lung tissue with good lung collapse include: fine or reticular folds on the surface of the visceral pleura; dark red or purplish-red color: due to slowed blood flow and accumulation of venous blood, it is in a state of relative ischemia; The surface characteristics of lung tissue with poor lung collapse include: a smooth, full visceral pleura without obvious wrinkles; and a pink or bright red color, indicating that there is still considerable arterial blood perfusion and oxygen exchange.

[0042] This embodiment uses a lung color segmentation module to extract features and segment regions from images of collapsed lung areas. It outputs masks for the darker, less collapsed areas (collapsed regions) and masks for the lighter, less collapsed areas (non-collapsed regions). Based on these masks, images of the collapsed and non-collapsed lung regions are obtained. The OpenCV `cvtColor` function is used to convert the two segmented images from RGB to grayscale, and the average grayscale value is extracted for each. The average grayscale value is then normalized by dividing by 255 to obtain the color weights. , Calculate the area percentage of each entity to obtain their respective area weight. , For lung collapse, a darker color indicates better collapse, while poor collapse will appear too white. For grayscale images, a darker color corresponds to 0, while a lighter color corresponds to 255. To maintain this correspondence and facilitate comparison, color weight and area weight are used to obtain the lung color index using the following formula. A larger lung color index y indicates better lung collapse.

[0043] To facilitate the assessment of lung collapse, a unified assessment standard needs to be established. For example, five levels—A, B, C, D, and E—can be used to correspond to five levels of lung collapse: excellent, good, average, poor, and very poor, respectively. For lung volume assessment tasks, based on physician experience and previous research, the lung area assessment results are divided into five levels (AE) based on the proportion of the collapsed lung area to the thoracic cavity area. The proportion corresponding to each level can be adaptively set based on actual application. For lung color assessment tasks after collapse, samples of excellent and very poor lung collapse selected by the physician are used as references. Lung color indicators are obtained using the aforementioned methods, and five equal intervals are constructed based on the maximum and minimum values ​​of the lung color indicators. These five intervals, from largest to smallest, correspond one-to-one with the five levels (AE), yielding the second lung collapse degree level, i.e., the lung color task assessment result.

[0044] By fusing the lung volume task assessment results and the lung color task assessment results through a dual-task fusion unit, in this embodiment, the lung collapse condition of the patient is considered qualified only if both the lung volume task assessment results and the lung color task assessment results reach grade B or above.

[0045] Step S4: Based on the fusion evaluation module and the trained lung volume task segmentation module and lung color task segmentation module, a dual-task lung collapse evaluation model is obtained.

[0046] Specifically, such as Figure 2 As shown, the trained lung volume segmentation module and lung color segmentation module are set sequentially, with the collapsed lung region image output by the lung volume segmentation module used as the input to the lung color segmentation module. The fusion evaluation module receives the segmentation results from the lung volume segmentation module and the lung color segmentation module, and the fusion evaluation yields the final lung collapse evaluation result.

[0047] The experiments in this embodiment were conducted on an NVIDIA A100 GPU equipped with 24GB of memory, with Python as the primary programming language and PyTorch library used for data processing and deep learning model building.

[0048] All lung image data used for training were extracted from clear images captured by thoracoscopy or surgical camera systems. Annotators used polygon tools to precisely delineate the outline of each target and assigned corresponding label names to different categories, ultimately generating a single-channel mask image of the same size as the original image. In this mask image, the value of each pixel represents its category ID (e.g., 0 for background, 1 for foreground). The labeled data was converted to a model-readable digital format and subjected to identical data augmentation processing simultaneously with the original images, including random rotation (±15°), random flipping (both horizontal and vertical flip probabilities are 0.5), and random translation (maximum displacement 20%). This ensured that the geometric and spatial correspondence between the images and labels remained strictly consistent, providing the model with high-quality training samples.

[0049] To evaluate the model's robustness and generalization ability, a rigorous five-fold cross-validation strategy was employed. In each fold, the dataset was randomly divided into training and test sets in an 8:2 ratio. This process was repeated five times to generate five distinct training-test splits. Each split was used independently to train and evaluate the network, and the final performance metric was obtained by taking the median of the results from all folds. The final model parameters were selected based on the intermediate model. This approach mitigates the impact of a single split and ensures reliable model performance on diverse and unseen datasets.

[0050] During training, the batch size was set to 32, the model ran for 150 epochs, and the training dataset was randomly sampled. The Adam optimizer was used with an initial learning rate of 0.001, fine-tuning was performed on the pre-trained model, and the OneCycleLR learning rate scheduler was used to adjust the learning rate during training.

[0051] Another embodiment of the present invention also discloses an electronic device, including at least one processor and at least one memory communicatively connected to said processor; The memory stores instructions that can be executed by the processor to implement the aforementioned method for constructing a dual-task lung collapse assessment model.

[0052] In summary, the dual-task lung collapse assessment model construction method of the present invention, in order to achieve an accurate and comprehensive assessment of the lung collapse state, designs two assessment tasks for lung volume and lung color. Through two corresponding segmentation modules, it realizes the accurate calculation of the lung area as a percentage of the thoracic cavity area after lung collapse and the accurate analysis of the lung surface color, respectively. Finally, it completes a comprehensive assessment of the patient's lung collapse and finally integrates the dual-task assessment results of lung volume and lung color to obtain the final lung collapse assessment result. This avoids the misjudgment problem caused by a single assessment indicator and improves the accuracy of lung collapse assessment.

[0053] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0054] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a dual-task lung collapse assessment model, characterized in that, include: Multiple lung images with different degrees of lung collapse were acquired, and the collapsed lung areas and the pleural background areas were labeled to obtain the first training sample set; The lung volume task segmentation module is trained based on the first training sample set; Based on the lung images, images of collapsed lung regions are extracted, and the collapsed lung regions are labeled with collapsed and non-collapsed regions to obtain a second training sample set; a lung color task segmentation module is trained based on the second training sample set. A fusion assessment module is constructed, which is used to obtain the corresponding lung volume task assessment result and lung color task assessment result based on the segmentation results of the lung volume task segmentation module and the lung color task segmentation module. The results of the two tasks are then merged to obtain the final lung collapse assessment result; Based on the fusion evaluation module and the trained lung volume task segmentation module and lung color task segmentation module, a dual-task lung collapse evaluation model is obtained.

2. The method for constructing the dual-task lung collapse assessment model according to claim 1, characterized in that, Both the lung volume segmentation module and the lung color segmentation module are improved Unet models; The lung volume task segmentation module is constructed by sequentially setting a first multi-scale feature fusion layer and a lung volume attention layer between the encoder and decoder of the Unet model. The lung color task segmentation module is constructed by sequentially setting a second multi-scale feature fusion layer and a lung color attention layer between the encoder and decoder of the Unet model.

3. The method for constructing the dual-task lung collapse assessment model according to claim 2, characterized in that, The first multi-scale feature fusion layer and the second multi-scale feature fusion layer have the same structure, including: Standard convolution module: It uses a 3×3 receptive field to perform convolution operations on the received feature map in order to extract detailed features and maintain the consistency of local features; Medium dilation rate convolution module: The dilation rate is set to 2, and a 5×5 receptive field is used to capture the structural relationships and contextual dependencies of the region; High dilation rate convolution module: The dilation rate is set to 4, and a 9×9 receptive field is used to model global semantic constraints and long-distance spatial relationships; Feature fusion module: This module is used to weight the feature maps obtained from each convolution module and then concatenate them along the channel dimension; and then perform dimensionality reduction through 1×1 convolutional channels to obtain a multi-scale fused feature map. When performing multi-view feature extraction through each convolutional module, padding is used to pre-fill the feature map with different amounts of padding to ensure that the extracted feature map is the same size as the original feature map.

4. The method for constructing the dual-task lung collapse assessment model according to claim 2, characterized in that, The lung volume attention layer is used to extract features from the received multi-scale fusion feature map through a lung volume attention mechanism that includes channel attention mechanism and spatial attention mechanism, so as to obtain channel attention feature map and spatial attention feature map; The channel attention feature map and spatial attention feature map are fused and output to enhance the features of important channels and important spaces.

5. The method for constructing the dual-task lung collapse assessment model according to claim 2, characterized in that, The lung color attention layer includes a color statistics attention module, a local color distribution attention module, and a fusion module; The color statistics attention module is used to extract macroscopic color statistics features of the image using the following method: Macroscopic color statistical features are extracted from the multi-scale fusion feature maps output by the corresponding multi-scale feature fusion layers through both global average pooling and max pooling paths. The two macroscopic color statistical feature maps are concatenated along the channel dimension, and the concatenated feature maps are then used to generate channel attention weights for each channel via the sigmoid activation function. The multi-scale fused feature map is multiplied channel by channel with the corresponding channel attention weight to obtain the output of the color statistical attention module; The local color distribution attention module is used to extract local spatial patterns between different color channel groups using the following method: The received multi-scale fused feature maps are grouped along the channel dimension, and convolution operations are performed on each group of features. The convolution results are then concatenated within the group to obtain the feature map corresponding to each group. Each set of corresponding feature maps is converted to a single-channel attention map by 1x1 convolution to reduce the number of channels to 1. Each group of the multi-scale fused feature map is multiplied by the corresponding single-channel attention map to obtain the output of the local color distribution attention module.

6. The method for constructing the dual-task lung collapse assessment model according to claim 1, characterized in that, Both the lung volume segmentation module and the lung color segmentation module are trained using the following loss function: ； ； Where N is the number of pixels, Let be the predicted probability of the i-th pixel for its true class. As a focal regulation factor, Focus-adjusting weights are used to identify and prioritize difficult-to-classify boundary regions. Boundary enhancement factor; For the first The gradient of each pixel; Gradient-aware weights; To predict the probability in Gradient of direction; To predict the probability in Gradient of direction.

7. The method for constructing the dual-task lung collapse assessment model according to claim 1, characterized in that, The segmentation results of the lung volume task segmentation module include the segmentation results of the collapsed lung region and the pleural background region; Based on the segmentation results of the lung volume task segmentation module, the corresponding lung volume task evaluation results are obtained, including: Based on the segmentation results of the collapsed lung region and the pleural background region, lung masks and pleural background region masks are obtained; Based on the lung mask and the pleural cavity background region mask, the number of pixels occupied by the collapsed lung region and the pleural cavity background region are obtained respectively, and the proportion of the collapsed lung region to the entire pleural cavity is obtained based on the number of pixels. The first lung collapse level is obtained based on the ratio and used as the result of the lung volume task assessment.

8. The method for constructing the dual-task lung collapse assessment model according to claim 7, characterized in that, The segmentation results of the lung color task segmentation module include the segmentation results of collapsed areas and non-collapsed areas; The lung color task evaluation result is obtained based on the segmentation result of the lung color task segmentation module, including: The segmentation results of the collapsed and non-collapsed areas are converted from RGB to grayscale images, and the average grayscale value is extracted for each. The average gray values ​​are normalized to obtain the color weights corresponding to the two regions. The area ratio of each region is obtained based on the number of pixels occupied by the collapsed and non-collapsed regions, and the area ratio is used as the area weight of each region. Based on the color weight and area weight, the lung color index is obtained using the following formula: ； in, As an indicator of lung color, Color weights for the collapsed areas. Color weights for the non-collapsed areas. The area weight of the collapsed area, The area weight of the non-collapsed area; The second lung collapse level is obtained based on the lung color index and used as the assessment result of the lung color task.

9. The method for constructing the dual-task lung collapse assessment model according to claim 8, characterized in that, The proportion of the collapsed lung area to the total thoracic cavity area is divided into n levels in descending order to obtain the first lung collapse degree level; The second degree of lung collapse was obtained by the following method: Lung color indices were obtained from multiple lung images with different degrees of lung collapse. The maximum and minimum values ​​of these lung color indices were then represented as follows: and ; by The result of +0.1 is used as the upper bound for evaluation. ,like If +0.1>1, then take 1; The result of -0.1 is used as the lower bound for evaluation. ,like If -0.1 < 0, then the value is 0; based on - The value is used to construct n equally divided intervals, and then arranged in descending order to obtain n levels, which serve as the second lung collapse degree level.

10. An electronic device, characterized in that, It includes at least one processor and at least one memory communicatively connected to the processor; The memory stores instructions that can be executed by the processor to implement the method for constructing the dual-task lung collapse assessment model according to any one of claims 1-9.

Images