A thoracoscope lung collapse intelligent evaluation system

Abstract

The present application provides a kind of intelligent evaluation system of thoracoscope under lung collapse, the system includes: data acquisition unit, for acquiring lung image by thoracoscope probe;Dual-task lung collapse evaluation unit, for based on lung volume attention mechanism and lung color attention mechanism, lung image is extracted and region segmentation respectively in lung space and lung color feature;And based on feature extraction result and region segmentation result, lung volume task evaluation result and lung color task evaluation result are obtained;Dual-task fusion unit, for fusing lung volume task evaluation result and lung color task evaluation result, obtain the final lung collapse evaluation result.The present application solves the problem that lung collapse evaluation in the prior art mainly relies on subjective experience of doctors during operation, lacks artificial intelligence assistance, resulting in large differences in scoring results, which can easily cause misjudgment.

Description

Technical Field

[0001] This invention belongs to the field of clinical medical technology, and in particular relates to an intelligent assessment system for lung collapse under thoracoscopic surgery. Background Technology

[0002] Pulmonary collapse is a common and important physiological phenomenon in thoracic surgery anesthesia and thoracoscopic surgery. It can reflect the effectiveness of anesthetic management such as one-lung ventilation, lung recruitment, and ventilation strategies, and is also one of the key indicators of postoperative lung function recovery.

[0003] Current lung collapse assessments primarily rely on the subjective judgment of anesthesiologists or surgeons during surgery, such as visually estimating lung volume changes via thoracoscopic examination to determine the degree of collapse. However, this method is highly subjective, lacks repeatability, and inconsistencies in experience and subjective standards among different doctors lead to significant differences in scoring results. Furthermore, existing lung collapse assessment systems are mostly qualitative descriptions, lacking standardized quantitative indicators, making them unsuitable for quantification and automation, and thus difficult to use for research and multi-center data comparisons. Intraoperative observations are not recorded by the system, data traceability is impossible, and this hinders postoperative analysis and algorithm improvement.

[0004] With the development of artificial intelligence algorithms, the probability of misjudging lung collapse can be greatly reduced by using pre-trained models for auxiliary assessment. However, there is currently no reliable lung collapse assessment system under thoracoscopic surgery that can be used for clinical assistance. Therefore, there is an urgent need for a reliable, standardized, objective, and traceable lung collapse assessment system for real-time intraoperative evaluation and postoperative data analysis, so as to provide a scientific basis for optimizing clinical anesthesia strategies and postoperative rehabilitation. Summary of the Invention

[0005] Based on the above analysis, the present invention aims to provide an intelligent assessment system for lung collapse under thoracoscopic surgery, which solves the problem that the assessment of lung collapse in the prior art mainly relies on the subjective experience of the surgeon during the operation, lacks artificial intelligence assistance, and leads to large differences in scoring results, which is prone to misjudgment.

[0006] The objective of this invention is mainly achieved through the following technical solutions:

[0007] On one hand, the present invention provides an intelligent assessment system for lung collapse under thoracoscopic guidance, comprising:

[0008] The data acquisition unit is used to acquire lung images through a thoracoscope probe;

[0009] The dual-task lung collapse assessment unit is used to extract lung spatial and lung color features and segment regions from the lung images based on the lung volume attention mechanism and the lung color attention mechanism, respectively; and to obtain lung volume task assessment results and lung color task assessment results based on the feature extraction results and region segmentation results.

[0010] The dual-task fusion unit is used to fuse the lung volume task assessment results and the lung color task assessment results to obtain the final lung collapse assessment result.

[0011] Furthermore, the dual-task lung collapse assessment unit includes a lung volume assessment module and a lung color assessment module arranged sequentially; both the lung volume assessment module and the lung color assessment module are based on the Unet model and are trained separately.

[0012] The lung volume assessment module is used to extract features and segment collapsed lung regions based on the lung images by fusing a lung volume attention mechanism, to obtain collapsed lung regions and pleural background regions, and to obtain lung volume task assessment results based on collapsed lung regions and pleural background regions.

[0013] The lung color assessment module is used to extract lung color features and segment regions based on the segmented image of the collapsed lung region, to obtain segmentation results of collapsed and non-collapsed regions, and to obtain lung color task assessment results based on the segmentation results.

[0014] Furthermore, the lung volume assessment module includes a first encoder, a first multi-scale feature fusion layer, a lung volume attention layer, a first decoder, and a first classification layer;

[0015] The first encoder adopts the encoder structure of the Unet model and is used to perform multiple feature extractions and feature compressions based on the lung image to obtain feature maps at multiple scales.

[0016] The first multi-scale feature fusion layer extracts and fuses features from the feature map output by the last layer of the first encoder through convolutional kernels of different scales set in parallel, thereby obtaining the first multi-scale fused feature map.

[0017] The lung volume attention layer is used to extract features from the first multi-scale fusion feature map through a lung volume attention mechanism that includes channel attention and spatial attention mechanisms, to obtain channel attention feature maps and spatial attention feature maps; and to fuse and output the channel attention feature maps and spatial attention feature maps to enhance the features of important channels and important spaces;

[0018] The first decoder adopts the decoder structure of the Unet model, which is used to perform multiple upsampling on the feature map output by the lung volume attention layer, and to perform skip connections with the feature map of the corresponding scale output by the first encoder to obtain the segmentation results of the collapsed lung region and the pleural background region.

[0019] The first classification layer is used to obtain lung volume task assessment results based on the segmentation results of the collapsed lung region and the pleural background region.

[0020] Furthermore, the first multi-scale feature fusion layer includes:

[0021] Standard convolution module: It uses a 3×3 receptive field to perform convolution operations on the received feature map in order to extract detailed features and maintain the consistency of local features;

[0022] Medium dilation rate convolution module: The dilation rate is set to 2, and a 5×5 receptive field is used to capture the structural relationships and contextual dependencies of the region;

[0023] High dilation rate convolution module: The dilation rate is set to 4, and a 9×9 receptive field is used to model global semantic constraints and long-distance spatial relationships;

[0024] Feature fusion module: This module is used to weight the feature maps obtained from each convolution module and then concatenate them along the channel dimension; and then perform dimensionality reduction through 1×1 convolutional channels to obtain the first multi-scale fused feature map.

[0025] When performing multi-view feature extraction through each convolutional module, padding is used to pre-fill the feature map with different amounts of padding to ensure that the extracted feature map is the same size as the original feature map.

[0026] Furthermore, feature extraction via the channel attention mechanism includes:

[0027] Global average pooling is performed on the input first multi-scale fusion feature map to compress all spatial information of each channel into a global feature;

[0028] The global features are learned through two fully connected layers set sequentially to learn the non-linear relationship between channels; the first fully connected layer reduces the number of channels, and the second fully connected layer restores the original number of channels.

[0029] The output of the second fully connected layer is restricted to between 0 and 1 by using the Sigmoid activation function, which serves as the weight for the corresponding channel.

[0030] After multiplying each channel of the first multi-scale fusion feature map by its corresponding weight, the two channels are concatenated along the channel dimension to obtain the channel attention feature map.

[0031] Furthermore, feature extraction via the spatial attention mechanism includes:

[0032] The first multi-scale fusion feature map input is subjected to average pooling and max pooling along the channel dimension to obtain two spatial feature maps;

[0033] The two spatial feature maps are spliced ​​together to obtain a feature map with a size of H×W×2;

[0034] The concatenated feature map is convolved to learn the importance of spatial location, and a weight map between 0 and 1 is generated by the Sigmoid function.

[0035] The weight map is multiplied position-by-position with the input first multi-scale fusion feature map to obtain the spatial attention feature map.

[0036] Furthermore, the lung color assessment module includes a second encoder, a second multi-scale feature fusion layer, a lung color attention layer, a second decoder, and a second classification layer arranged sequentially.

[0037] The second encoder, the second multi-scale feature fusion layer, and the second decoder have the same structure as the corresponding first encoder, the first multi-scale feature fusion layer, and the first decoder.

[0038] The lung color attention mechanism includes a color statistical attention mechanism and a local color distribution attention mechanism; the lung color attention layer includes a color statistical attention module, a local color distribution attention module, and a fusion module.

[0039] The color statistics attention module is used to extract the macroscopic color statistics features of the image;

[0040] The local color distribution attention module is used to extract local spatial patterns between different color channel groups;

[0041] The fusion module is used to perform weighted fusion of the outputs of the color statistics attention module and the local color distribution attention module to obtain the lung color feature extraction result;

[0042] Based on the lung color feature extraction results, the second decoder performs multiple upsampling operations and performs skip connections with the feature maps of the corresponding scale output by the second encoder to obtain the segmentation results of the collapsed and non-collapsed lung regions.

[0043] The second classification layer obtains the lung color task assessment results based on the segmentation results of the collapsed and non-collapsed regions.

[0044] Furthermore, the color statistical attention module extracts features using the following method:

[0045] Macroscopic color statistical features are extracted from the second multi-scale fusion feature map output by the second multi-scale feature fusion layer through both global average pooling and max pooling paths.

[0046] The two macroscopic color statistical feature maps are concatenated along the channel dimension, and the concatenated feature maps are then used to generate channel attention weights for each channel via the sigmoid activation function.

[0047] The second multi-scale fusion feature map is multiplied channel by channel with the corresponding channel attention weight to obtain the output of the color statistical attention module;

[0048] The local color distribution attention module extracts features using the following method:

[0049] The second multi-scale fusion feature map is grouped along the channel dimension, and a convolution operation is performed on each group of features. The convolution results are then connected within the group to obtain the feature map corresponding to each group.

[0050] Each set of corresponding feature maps is converted to a single-channel attention map by 1x1 convolution to reduce the number of channels to 1.

[0051] Each group of the second multi-scale fused feature map is multiplied by the corresponding single-channel attention map to obtain the output of the local color distribution attention module.

[0052] Furthermore, the thoracoscopic probe can be adapted to various surgical approaches, such as multi-port thoracoscopic probes or single-port thoracoscopic probes.

[0053] To obtain the optimal field of view for intelligent assessment, it is preferable to point the thoracoscopic probe towards the patient's head when acquiring images. This orientation helps to obtain panoramic images of the lung surface, reduces anatomical obstruction, and thus improves the accuracy of subsequent image analysis and assessment of the degree of lung collapse.

[0054] The beneficial effects of this technical solution are:

[0055] 1. In order to achieve a comprehensive assessment of lung collapse, this invention designs two assessment tasks for lung volume and lung color. Through two corresponding segmentation models (assessment modules), it accurately calculates the ratio of lung area to pleural cavity area after lung collapse and accurately analyzes the lung surface color. Finally, it completes a comprehensive assessment of the patient's lung collapse and integrates the assessment results of lung volume and lung color to obtain the final lung collapse assessment result. This avoids the misjudgment problem caused by a single assessment indicator and improves the accuracy of lung collapse assessment.

[0056] 2. This invention considers that the segmentation of collapsed lung regions and pleural cavity background regions relies on their spatial features in lung images, and introduces a lung volume attention mechanism to accurately locate key segmentation structures. This mechanism is based on a deep convolutional neural network and integrates two core technology modules: channel attention and spatial attention. The channel attention mechanism adjusts the contribution of each channel to enhance useful feature channels and suppress those with smaller contributions, allowing the network to focus on channels with greater information content. The spatial attention mechanism learns the importance of spatial locations, making the algorithm focus more on important spatial locations in the image, thereby enhancing the features of important spatial locations.

[0057] 3. This invention proposes an enhanced color attention mechanism for lung color scoring tasks, specifically optimizing the detection capability of light-colored regions caused by poor lung collapse. It extracts macroscopic color statistical features of lung images through color statistical attention; and analyzes fine-grained color texture changes through local color distribution attention, enabling the network to learn local spatial patterns between different color channel groups. This helps capture local texture changes on different color components, improving the segmentation accuracy of collapsed and non-collapsed lung regions. Attached Figure Description

[0058] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0059] Figure 1 This is a schematic diagram of the intelligent assessment system for lung collapse under thoracoscopic guidance according to an embodiment of the present invention. Detailed Implementation

[0060] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0061] One embodiment of the present invention provides an intelligent assessment system for lung collapse under thoracoscopic guidance, such as... Figure 1 As shown, it includes:

[0062] The data acquisition unit is used to acquire lung images through a thoracoscope probe;

[0063] The dual-task lung collapse assessment unit is used to extract lung spatial and lung color features and segment regions from the lung images based on the lung volume attention mechanism and the lung color attention mechanism, respectively; and to obtain lung volume task assessment results and lung color task assessment results based on the feature extraction results and region segmentation results.

[0064] The dual-task fusion unit is used to fuse the lung volume task assessment results and the lung color task assessment results to obtain the final lung collapse assessment result.

[0065] In some specific implementations, this application can use a clinically standard thoracoscopic probe to acquire lung images. Depending on the selected surgical procedure, either multi-port or single-port thoracoscopic techniques can be adaptively employed. To obtain the optimal field of view suitable for subsequent intelligent assessment, the lens can be oriented towards the patient's head for image acquisition. This orientation helps to obtain a panoramic image of the lung surface and reduces obstruction of key anatomical structures. After image acquisition, the lung images are first preprocessed by a preprocessing unit, including image stabilization, white balance, and noise reduction. To meet the input dimension requirements of the general image data model of convolutional neural networks (e.g., ResNet), the lung images are uniformly scaled to 224×224 pixels for input into the dual-task lung collapse assessment unit for subsequent processing.

[0066] Specifically, the dual-task lung collapse assessment unit includes a lung volume assessment module and a lung color assessment module arranged sequentially; both the lung volume assessment module and the lung color assessment module are based on the Unet model and are trained separately.

[0067] The lung volume assessment module is used to extract features and segment collapsed lung regions based on the lung images by fusing a lung volume attention mechanism, to obtain collapsed lung regions and pleural background regions, and to obtain lung volume task assessment results based on collapsed lung regions and pleural background regions.

[0068] The lung color assessment module is used to extract lung color features and segment regions based on the segmented image of the collapsed lung region, to obtain segmentation results of collapsed and non-collapsed regions, and to obtain lung color task assessment results based on the segmentation results.

[0069] In order to achieve a comprehensive assessment of lung collapse and improve the accuracy of lung collapse assessment, this embodiment designs two assessment tasks: lung volume and lung color. Through two corresponding segmentation models (assessment modules), it accurately calculates the ratio of lung area to pleural cavity area after lung collapse and accurately analyzes the lung surface color, ultimately completing a comprehensive assessment of the patient's lung collapse condition. The two segmentation models are based on the U-Net framework, combining multi-scale feature fusion and attention mechanisms designed for different tasks to achieve the fusion of the two assessment tasks of lung volume and lung color, avoiding misjudgment problems caused by a single assessment indicator.

[0070] Preferably, the lung volume assessment module includes a first encoder, a first multi-scale feature fusion layer, a lung volume attention layer, a first decoder, and a first classification layer;

[0071] The first encoder adopts the encoder structure of the Unet model and is used to perform multiple feature extractions and feature compressions based on the lung image to obtain feature maps at multiple scales.

[0072] The first multi-scale feature fusion layer extracts and fuses features from the feature map output by the last layer of the first encoder through convolutional kernels of different scales set in parallel, thereby obtaining the first multi-scale fused feature map.

[0073] The lung volume attention layer is used to extract features from the first multi-scale fusion feature map through a lung volume attention mechanism that includes channel attention and spatial attention mechanisms, to obtain channel attention feature maps and spatial attention feature maps; and to fuse and output the channel attention feature maps and spatial attention feature maps to enhance the features of important channels and important spaces;

[0074] The first decoder adopts the decoder structure of the Unet model, which is used to perform multiple upsampling on the feature map output by the lung volume attention layer, and to perform skip connections with the feature map of the corresponding scale output by the first encoder to obtain the segmentation results of the collapsed lung region and the pleural background region.

[0075] The first classification layer is used to obtain lung volume task assessment results based on the segmentation results of the collapsed lung region and the pleural background region.

[0076] The first multi-scale feature fusion layer includes:

[0077] Standard convolution module: It uses a 3×3 receptive field to perform convolution operations on the received feature map in order to extract detailed features and maintain the consistency of local features;

[0078] Medium dilation rate convolution module: The dilation rate is set to 2, and a 5×5 receptive field is used to capture the structural relationships and contextual dependencies of the region;

[0079] High dilation rate convolution module: The dilation rate is set to 4, and a 9×9 receptive field is used to model global semantic constraints and long-distance spatial relationships;

[0080] Feature fusion module: This module is used to weight the feature maps obtained from each convolution module and then concatenate them along the channel dimension; and then perform dimensionality reduction through 1×1 convolutional channels to obtain the first multi-scale fused feature map.

[0081] When performing multi-view feature extraction through each convolutional module, padding is used to pre-fill the feature map with different amounts of padding to ensure that the extracted feature map is the same size as the original feature map.

[0082] Specifically, this embodiment proposes a multi-scale feature fusion module. By applying convolutional kernels with different dilation rates in parallel, it extracts high-level semantic feature maps from the output of the last layer of the U-Net encoding layer from multiple perspectives. The standard convolutional kernel uses a 3×3 receptive field to focus on local feature consistency and detail preservation; the medium dilation rate convolutional kernel has a dilation rate of 2 and uses a 5×5 receptive field to capture regional structural relationships and contextual dependencies; and the large dilation rate convolutional kernel has a dilation rate of 4 and uses a 9×9 receptive field to model global semantic constraints and long-range spatial relationships. When using multi-view feature extraction, padding is used to pre-fill the feature map with different amounts of padding to ensure that the extracted feature map is the same size as the original feature map. The three different scale feature representations are fused through weighted summation, where the weight parameters are learned. The learned weight parameters are multiplied by the corresponding multi-scale features, concatenated along the channel direction, and then subjected to 1×1 convolutional channel dimensionality reduction to finally obtain a multi-scale fused feature map with the same size as the last layer of the original U-Net encoding layer.

[0083] Furthermore, feature extraction in the lung volume assessment model using the aforementioned channel attention mechanism includes:

[0084] Global average pooling is performed on the input first multi-scale fusion feature map to compress all spatial information of each channel into a global feature;

[0085] The global features are learned through two fully connected layers set sequentially to learn the non-linear relationship between channels; the first fully connected layer reduces the number of channels, and the second fully connected layer restores the original number of channels.

[0086] The output of the second fully connected layer is restricted to between 0 and 1 by using the Sigmoid activation function, which serves as the weight for the corresponding channel.

[0087] After multiplying each channel of the first multi-scale fusion feature map by its corresponding weight, the two channels are concatenated along the channel dimension to obtain the channel attention feature map.

[0088] Feature extraction via the spatial attention mechanism includes:

[0089] The first multi-scale fusion feature map input is subjected to average pooling and max pooling along the channel dimension to obtain two spatial feature maps;

[0090] The two spatial feature maps are spliced ​​together to obtain a feature map with a size of H×W×2;

[0091] The concatenated feature map is convolved to learn the importance of spatial location, and a weight map between 0 and 1 is generated by the Sigmoid function.

[0092] The weight map is multiplied position-by-position with the input first multi-scale fusion feature map to obtain the spatial attention feature map.

[0093] This embodiment considers that the segmentation of collapsed lung regions and pleural cavity background regions is based on their spatial features in lung images, and introduces a lung volume attention mechanism to accurately locate key segmentation structures. This mechanism is based on a deep convolutional neural network and integrates two core technology modules: channel attention and spatial attention. The channel attention mechanism adjusts the contribution of each channel to enhance useful feature channels and suppress those with smaller contributions, allowing the network to focus on channels with greater information content. The spatial attention mechanism learns the importance of spatial locations, making the algorithm focus more on important spatial locations in the image, thereby enhancing the features of important spatial locations.

[0094] After extracting features from the multi-scale fused feature map through channel attention and spatial attention branches respectively, the output weights of the two branches are multiplied by the original input feature map to obtain two weighted feature maps. The two weighted feature maps are then added together to obtain the output of the lung volume attention layer.

[0095] Furthermore, the lung color assessment module includes a second encoder, a second multi-scale feature fusion layer, a lung color attention layer, a second decoder, and a second classification layer arranged sequentially.

[0096] The second encoder, the second multi-scale feature fusion layer, and the second decoder have the same structure as the corresponding first encoder, the first multi-scale feature fusion layer, and the first decoder.

[0097] The lung color attention mechanism includes a color statistical attention mechanism and a local color distribution attention mechanism; the lung color attention layer includes a color statistical attention module, a local color distribution attention module, and a fusion module.

[0098] The color statistics attention module is used to extract the macroscopic color statistics features of the image;

[0099] The local color distribution attention module is used to extract local spatial patterns between different color channel groups;

[0100] The fusion module is used to perform weighted fusion of the outputs of the color statistics attention module and the local color distribution attention module to obtain the lung color feature extraction result;

[0101] Based on the lung color feature extraction results, the second decoder performs multiple upsampling operations and performs skip connections with the feature maps of the corresponding scale output by the second encoder to obtain the segmentation results of the collapsed and non-collapsed lung regions.

[0102] The second classification layer obtains the lung color task assessment results based on the segmentation results of the collapsed and non-collapsed regions.

[0103] The color statistics attention module extracts features using the following method:

[0104] Macroscopic color statistical features are extracted from the second multi-scale fusion feature map output by the second multi-scale feature fusion layer through both global average pooling and max pooling paths.

[0105] The two macroscopic color statistical feature maps are concatenated along the channel dimension, and the concatenated feature maps are then used to generate channel attention weights for each channel via the sigmoid activation function.

[0106] The second multi-scale fusion feature map is multiplied channel by channel with the corresponding channel attention weight to obtain the output of the color statistical attention module;

[0107] The local color distribution attention module extracts features using the following method:

[0108] The second multi-scale fusion feature map is grouped along the channel dimension, and a convolution operation is performed on each group of features. The convolution results are then connected within the group to obtain the feature map corresponding to each group.

[0109] Each set of corresponding feature maps is converted to a single-channel attention map by 1x1 convolution to reduce the number of channels to 1.

[0110] Each group of the second multi-scale fused feature map is multiplied by the corresponding single-channel attention map to obtain the output of the local color distribution attention module.

[0111] This embodiment proposes an enhanced color attention mechanism for lung color scoring tasks, specifically optimizing the detection capability of light-colored regions caused by poor lung collapse. Macroscopic color statistical features of lung images are extracted through color statistical attention; fine-grained color texture changes are analyzed based on grouped convolutions (e.g., group=4) through local color distribution attention. Specifically, each group is convolved separately, with the output channel number of each group equal to the input channel number divided by the group number. The outputs of all groups are then concatenated to obtain C / 4 channels. Finally, a 1x1 convolution reduces the number of channels in each group to 1, generating a single-channel attention map. This approach allows the network to learn local spatial patterns between different color channel groups, rather than mixing all color channels together. This helps capture local texture changes in different color components, improving the segmentation accuracy of collapsed and non-collapsed lung regions.

[0112] Furthermore, the first classification layer obtains the lung volume task assessment results using the following method:

[0113] Based on the segmentation results of the collapsed lung region and the pleural background region, lung masks and pleural background region masks are obtained;

[0114] Based on the lung mask and the pleural cavity background region mask, the number of pixels occupied by the collapsed lung region and the pleural cavity background region are obtained respectively, and the proportion of the collapsed lung region to the entire pleural cavity is obtained based on the number of pixels.

[0115] The first lung collapse level is obtained based on the ratio and used as the result of the lung volume task assessment.

[0116] Specifically, for the lung area task, this embodiment adopts the U-Net main framework, adding a multi-scale feature fusion module and a lung volume attention mechanism after its last encoding layer. The output is used as the input to the first decoding layer, and the final model output is a mask of the collapsed lung region and a mask of the pleural cavity background region (excluding the lung). By calculating the number of pixels occupied by each mask, the final ratio of the collapsed lung region to the pleural cavity area is: lung pixels / (lung pixels + pure pleural cavity pixels).

[0117] The second classification layer obtained the lung color task assessment results using the following method:

[0118] The segmentation results of the collapsed and non-collapsed areas are converted from RGB to grayscale images, and the average grayscale values ​​are extracted for each.

[0119] The average gray values ​​are normalized to obtain the corresponding color weights. and ;

[0120] The area proportion of each region is obtained based on the number of pixels occupied by the collapsed and non-collapsed regions, and this area proportion is used as the area weight of each region. , ;

[0121] Based on the color weight and area weight, the lung color index is obtained using the following formula:

[0122] ;

[0123] The second lung collapse level is obtained based on the lung color index and used as the assessment result of the lung color task.

[0124] It should be noted that during the lung collapse procedure, some areas of the lung may collapse well, while others may not collapse as well as expected.

[0125] For example, the surface features of lung tissue with good lung collapse include: fine or reticular folds on the surface of the visceral pleura; dark red or purplish-red color: due to slowed blood flow and accumulation of venous blood, it is in a state of relative ischemia;

[0126] The surface characteristics of lung tissue with poor lung collapse include: a smooth and full visceral pleura surface without obvious wrinkles; and a pink or bright red color, indicating that there is still a lot of arterial blood perfusion and oxygen exchange.

[0127] In this embodiment, the segmented lung region from the lung volume task is used as the input to the lung color evaluation module. After feature extraction and region segmentation, the lung color evaluation module outputs masks for the darker, less collapsed areas (collapsed regions) and masks for the lighter, less collapsed areas (non-collapsed regions). Based on these masks, images of the collapsed and non-collapsed lung regions are obtained. The OpenCV cvtColor function is used to convert the two segmented images from RGB to grayscale, and the average grayscale value is extracted for each. The average grayscale value is then normalized by dividing by 255 to obtain the color weights. , Calculate the area percentage of each entity to obtain their respective area weight. , For lung collapse, a darker color indicates better collapse, while poor collapse will appear too white. For grayscale images, a darker color corresponds to 0, while a lighter color corresponds to 255. To maintain this correspondence and facilitate comparison, color weight and area weight are used to obtain the lung color index using the following formula. A larger lung color index y indicates better lung collapse.

[0128] To facilitate the assessment of lung collapse, a unified assessment standard needs to be established. For example, five levels—A, B, C, D, and E—can be used to correspond to five levels of lung collapse: excellent, good, fair, poor, and very poor, respectively. For lung volume assessment, based on physician experience and previous research, and considering the proportion of the collapsed lung area to the thoracic cavity area, lung area assessment results are divided into five levels: A, B, C, and E. The area proportion corresponding to each level can be adaptively set based on actual application. For lung color assessment after collapse, samples of excellent and very poor lung collapse selected by the physician are used as references. Lung color indices are obtained using the aforementioned methods, and the maximum and minimum values ​​of the lung color indices are obtained, denoted as follows: and ;right Add 0.1 (if) If +0.1 > 1, then take 1) as the final upper bound for evaluation. ,right Subtract 0.1 (if) If -0.1 < 0, then 0 is taken as the final lower bound for evaluation. .right - The five equal intervals are constructed, and the five intervals from largest to smallest correspond one-to-one with the five levels of AE, to obtain the second lung collapse degree level, that is, the lung color task assessment result.

[0129] By fusing the lung volume task assessment results and the lung color task assessment results through a dual-task fusion unit, in this embodiment, the lung collapse condition of the patient is considered qualified only if both the lung volume task assessment results and the lung color task assessment results reach grade B or above.

[0130] In practical applications, when training the lung volume assessment module and the lung color assessment module, experienced doctors can annotate the sample images to construct lung volume assessment samples and lung color assessment samples respectively. The lung volume assessment samples and lung color assessment samples are all lung images acquired under thoracoscopy, which are preprocessed and labeled respectively.

[0131] Specifically, after acquiring lung images via a thoracoscope probe and performing preprocessing such as noise reduction, the lung images are further standardized to meet the input dimension requirements of general image data models for convolutional neural networks (such as ResNet). This standardization process, such as scaling the images to a uniform 224×224 pixel size, offers two advantages: First, compared to the high resolution of the original images, the standardized images save training memory and improve training efficiency. Second, the standardized image size matches the original input size of the model, facilitating the use of pre-trained model parameters and fully leveraging transfer learning capabilities. Following the standardization of the lung images, various data augmentation techniques can be employed, including random rotation, horizontal and vertical flipping, random translation, and photometric enhancement. This allows the model to be exposed to diverse image transformations, resisting common interferences from image orientation and lighting conditions, enhancing its generalization ability to unseen data, and reducing the risk of overfitting. In this embodiment, the range of random rotation is set to (-15°, 15°), the probability of horizontal and vertical flipping is set to 0.5, the maximum displacement of random translation is set to 0.2 (20% of the original image's horizontal size), and the luminance enhancement includes adjusting the image's brightness, contrast, and saturation, all of which are adjusted within a range of (0.8, 1.2), i.e., randomly adjusted between 80% and 120% of the original values.

[0132] The preprocessed images are labeled, including: labeling collapsed lung regions in lung images to construct lung volume assessment samples; and labeling collapsed and non-collapsed regions in collapsed lung regions of lung images to construct lung color assessment samples.

[0133] The corresponding lung volume assessment module and lung color assessment module were trained using lung volume assessment samples and lung color assessment samples, respectively. During training, the following loss function was used for both the lung volume assessment module and the lung color assessment module to improve the accuracy and precision of image segmentation:

[0134] ;

[0135] ;

[0136] Where N is the number of pixels, Let be the predicted probability of the i-th pixel for its true class. As a focal regulation factor, The focus is adjusted by weights to identify and prioritize difficult-to-classify boundary regions. To predict the probability in Gradient of direction; To predict the probability in Gradient of direction, Boundary enhancement factor; For the first Gradient of each pixel; These are gradient-aware weights used to accurately identify segmentation boundaries.

[0137] The experiments in this embodiment were conducted on an NVIDIA A100 GPU equipped with 24GB of memory, with Python as the primary programming language and PyTorch library used for data processing and deep learning model building.

[0138] All lung image data used for training were extracted from clear images captured by thoracoscopy or surgical camera systems. Annotators used polygon tools to precisely delineate the outline of each target and assigned corresponding label names to different categories, ultimately generating a single-channel mask image of the same size as the original image. In this mask image, the value of each pixel represents its category ID (e.g., 0 for background, 1 for foreground). The labeled data was converted to a model-readable digital format and subjected to identical data augmentation processing simultaneously with the original images, including random rotation (±15°), random flipping (both horizontal and vertical flip probabilities are 0.5), and random translation (maximum displacement 20%). This ensured that the geometric and spatial correspondence between the images and labels remained strictly consistent, providing the model with high-quality training samples.

[0139] To evaluate the model's robustness and generalization ability, a rigorous five-fold cross-validation strategy was employed. In each fold, the dataset was randomly divided into training and test sets in an 8:2 ratio. This process was repeated five times to generate five distinct training-test splits. Each split was used independently to train and evaluate the network, and the final performance metric was obtained by taking the median of the results from all folds. The final model parameters were selected based on the intermediate model. This approach mitigates the impact of a single split and ensures reliable model performance on diverse and unseen datasets.

[0140] During training, the batch size was set to 32, the model ran for 150 epochs, and the training dataset was randomly sampled. The Adam optimizer was used with an initial learning rate of 0.001, fine-tuning was performed on the pre-trained model, and the OneCycleLR learning rate scheduler was used to adjust the learning rate during training.

[0141] In summary, the intelligent assessment system for thoracoscopic lung collapse of the present invention, in order to achieve a comprehensive assessment of the lung collapse state, is designed with two assessment tasks: lung volume and lung color. Through two corresponding segmentation models (assessment modules), it accurately calculates the ratio of lung area to pleural cavity area after lung collapse and accurately analyzes the lung surface color, respectively. Finally, it completes a comprehensive assessment of the patient's lung collapse and integrates the assessment results of lung volume and lung color to obtain the final lung collapse assessment result. This avoids the misjudgment problem caused by a single assessment indicator and improves the accuracy of lung collapse assessment.

[0142] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0143] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A thoracoscopic lung collapse intelligent assessment system, characterized in that, include: The data acquisition unit is used to acquire lung images through a thoracoscope probe; A dual-task lung collapse assessment unit is used to extract lung spatial and lung color features and segment regions from the lung images based on lung volume attention mechanism and lung color attention mechanism, respectively. Based on the feature extraction and region segmentation results, lung volume task assessment results and lung color task assessment results are obtained; The lung volume attention mechanism includes a channel attention mechanism and a spatial attention mechanism, which are used to extract features from the received feature maps to obtain channel attention feature maps and spatial attention feature maps. The lung color attention mechanism includes a color statistical attention mechanism and a local color distribution attention mechanism; The macroscopic color statistical feature extraction of the image through the color statistical attention mechanism includes: extracting macroscopic color statistical features from the received feature maps using both global average pooling and max pooling paths; concatenating the two macroscopic color statistical feature maps along the channel dimension, and generating channel attention weights for each channel using the sigmoid activation function; multiplying the received feature maps with the corresponding channel attention weights channel by channel. The local spatial pattern extraction between different color channel groups through the local color distribution attention mechanism includes: grouping the received feature maps along the channel dimension, performing convolution operations on each group of features, and concatenating the convolution results within the group to obtain a feature map corresponding to each group; reducing the number of channels to 1 using a 1x1 convolution on each group of feature maps to generate a single-channel attention map; and multiplying each group of the received feature maps with the corresponding single-channel attention map. The dual-task fusion unit is used to fuse the lung volume task assessment results and the lung color task assessment results to obtain the final lung collapse assessment result.

2. The intelligent assessment system for lung collapse under thoracoscopic guidance according to claim 1, characterized in that, The dual-task lung collapse assessment unit includes a lung volume assessment module and a lung color assessment module arranged sequentially; both the lung volume assessment module and the lung color assessment module are based on the Unet model and are trained separately. The lung volume assessment module is used to extract features and segment collapsed lung regions based on the lung images by fusing a lung volume attention mechanism, to obtain collapsed lung regions and pleural background regions, and to obtain lung volume task assessment results based on collapsed lung regions and pleural background regions. The lung color assessment module is used to extract lung color features and segment regions based on the segmented image of collapsed lung regions, to obtain segmentation results of collapsed and non-collapsed regions, and to obtain lung color task assessment results based on the segmentation results.

3. The intelligent assessment system for lung collapse under thoracoscopic guidance according to claim 2, characterized in that, The lung volume assessment module includes a first encoder, a first multi-scale feature fusion layer, a lung volume attention layer, a first decoder, and a first classification layer; The first encoder adopts the encoder structure of the Unet model and is used to perform multiple feature extractions and feature compressions based on the lung image to obtain feature maps at multiple scales. The first multi-scale feature fusion layer extracts and fuses features from the feature map output by the last layer of the first encoder through convolutional kernels of different scales set in parallel, thereby obtaining the first multi-scale fused feature map. The lung volume attention layer is used to extract features from the first multi-scale fusion feature map through the lung volume attention mechanism to obtain channel attention feature map and spatial attention feature map; The channel attention feature map and spatial attention feature map are fused and output to enhance the features of important channels and important spaces; The first decoder adopts the decoder structure of the Unet model, which is used to perform multiple upsampling and skip connections based on the feature map output by the lung volume attention layer to obtain the segmentation results of the collapsed lung region and the pleural background region; The first classification layer is used to obtain lung volume task assessment results based on the segmentation results of the collapsed lung region and the pleural background region.

4. The intelligent assessment system for lung collapse under thoracoscopic surgery according to claim 3, characterized in that, The first multi-scale feature fusion layer includes: Standard convolution module: It uses a 3×3 receptive field to perform convolution operations on the received feature map in order to extract detailed features and maintain the consistency of local features; Medium dilation rate convolution module: The dilation rate is set to 2, and a 5×5 receptive field is used to capture the structural relationships and contextual dependencies of the region; High dilation rate convolution module: The dilation rate is set to 4, and a 9×9 receptive field is used to model global semantic constraints and long-distance spatial relationships; Feature fusion module: This module is used to weight the feature maps obtained from each convolution module and then concatenate them along the channel dimension; and then perform dimensionality reduction through 1×1 convolutional channels to obtain the first multi-scale fused feature map. When performing multi-view feature extraction through each convolutional module, padding is used to pre-fill the feature map with different amounts of padding to ensure that the extracted feature map is the same size as the original feature map.

5. The intelligent assessment system for lung collapse under thoracoscopic guidance according to claim 3, characterized in that, Feature extraction via the channel attention mechanism includes: Global average pooling is performed on the input first multi-scale fusion feature map to compress all spatial information of each channel into a global feature; The global features are learned through two fully connected layers set sequentially to learn the non-linear relationship between channels; the first fully connected layer reduces the number of channels, and the second fully connected layer restores the original number of channels. The output of the second fully connected layer is restricted to between 0 and 1 by using the Sigmoid activation function, which serves as the weight for the corresponding channel. After multiplying each channel of the first multi-scale fusion feature map by its corresponding weight, the two channels are concatenated along the channel dimension to obtain the channel attention feature map.

6. The intelligent assessment system for lung collapse under thoracoscopic surgery according to claim 5, characterized in that, Feature extraction via the spatial attention mechanism includes: The first multi-scale fusion feature map input is subjected to average pooling and max pooling along the channel dimension to obtain two spatial feature maps; The two spatial feature maps are spliced ​​together to obtain a feature map with a size of H×W×2; The concatenated feature map is convolved to learn the importance of spatial location, and a weight map between 0 and 1 is generated by the Sigmoid function. The weight map is multiplied position-by-position with the input first multi-scale fusion feature map to obtain the spatial attention feature map.

7. The intelligent assessment system for lung collapse under thoracoscopic surgery according to claim 3, characterized in that, The lung color assessment module includes a second encoder, a second multi-scale feature fusion layer, a lung color attention layer, a second decoder, and a second classification layer arranged sequentially. The second encoder, the second multi-scale feature fusion layer, and the second decoder have the same structure as the corresponding first encoder, the first multi-scale feature fusion layer, and the first decoder. The lung color attention layer includes a color statistics attention module, a local color distribution attention module, and a fusion module; The color statistics attention module is used to extract the macroscopic color statistics features of the image; The local color distribution attention module is used to extract local spatial patterns between different color channel groups; The fusion module is used to perform weighted fusion of the outputs of the color statistics attention module and the local color distribution attention module to obtain the lung color feature extraction result; Based on the lung color feature extraction results, the second decoder performs multiple upsampling operations and performs skip connections with the feature maps of the corresponding scale output by the second encoder to obtain the segmentation results of the collapsed and non-collapsed lung regions. The second classification layer obtains the lung color task assessment results based on the segmentation results of the collapsed and non-collapsed regions.

8. The intelligent assessment system for lung collapse under thoracoscopic surgery according to claim 2, characterized in that, The thoracoscopy probe is either a multi-port thoracoscopy probe or a single-port thoracoscopy probe.

9. The intelligent assessment system for lung collapse under thoracoscopic surgery according to claim 1, characterized in that, When acquiring lung images, the lens of the thoracoscope probe is oriented towards the head for image acquisition.

Images