Microcirculation blood vessel automatic segmentation and quantification method based on enhanced feature fusion

Abstract

The invention discloses a microcirculation blood vessel automatic segmentation and quantification method based on enhanced feature fusion. The method comprises the following steps: step 1, data preparation and preprocessing; 2, designing an enhanced feature fusion network model; step 3, automatically segmenting microcirculation blood vessels; and 4, quantifying the diameter of the blood vessel based on the segmentation result. According to the enhanced feature fusion network model provided by the invention, accurate segmentation of microvessels is realized through fusion and enhancement of multi-scale features. The model shows good generalization ability on diversified clinical image data through effective data enhancement, and can adapt to image data with different illumination conditions, contrast ratios and resolutions. Therefore, the model has high practicability and reliability in practical application. According to the method, the diameter of the blood vessel is quantified through the segmentation result, the diameter of the microvessel can be accurately measured, and powerful support is provided for evaluation of the clinical microcirculation function.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and relates to a microcirculation evaluation method, and specifically to an automatic segmentation and quantification method of microcirculation blood vessels based on enhanced feature fusion. Background Art

[0002] Microcirculatory function is essential for the maintenance of tissue health. Especially in acute conditions such as sepsis, microcirculatory dysfunction is common and closely related to the severity of the disease and patient prognosis. Traditional microcirculatory assessment methods mainly rely on direct microscopic observation, which requires high operator skills, and the observation results are easily affected by subjective factors, making it difficult to standardize and automate. In addition, these methods usually cannot provide fast and accurate quantitative data, limiting their application in clinical decision support. In recent years, deep learning technology has made significant progress in medical image analysis, especially in image segmentation, lesion detection, and tissue feature quantification. The introduction of deep learning models provides new possibilities for microvascular assessment. However, in microvascular image segmentation and quantification, there is still the problem of how to effectively fuse multi-scale features and accurately extract micro details. Summary of the invention

[0003] In order to solve the problems of insufficient feature fusion effect, low segmentation accuracy and difficulty in processing complex tissue structures in the process of microvascular image segmentation and quantification in the prior art, the present invention provides a method for automatic segmentation and quantification of microcirculatory vessels based on enhanced feature fusion. The method designs an improved deep learning network model for the segmentation and quantification of sublingual microcirculatory vessels. The model adopts residual connection in the convolution operation of the encoder and the deconvolution operation of the decoder to ensure the integrity of information transmission and the stability of the gradient. At the same time, the model adopts the design of enhanced jump connection, and the features of the encoder of the next level are pixel reorganized and upsampled through the self-attention mechanism, and feature fused with the encoder features and decoder features of the current level, thereby effectively improving the model's ability to capture microvascular detail features. In addition, the model performs loss calculation on the feature map of each decoder with the target segmentation map to ensure the accuracy and consistency of segmentation. This design significantly improves the accuracy of image segmentation and the accuracy of microvascular quantification, and provides a more efficient and reliable tool for the automated evaluation of microcirculatory function in clinical practice, especially in the case of complex tissue structures and poor signal quality.

[0004] The objective of the present invention is achieved through the following technical solutions:

[0005] A method for automatic segmentation and quantification of microcirculatory vessels based on enhanced feature fusion, comprising the following steps:

[0006] Step 1: Data preparation and preprocessing

[0007] Step 1, collecting microcirculation blood vessel images with different illumination conditions, contrasts and resolutions, wherein the microcirculation images include static images and video images of microcirculation under different health conditions and pathological conditions;

[0008] Step 1 and 2: extract the region of interest (ROI) of the microcirculatory blood vessel image, select and mark the microvascular area in the image, and crop these areas to form a new ROI image (an image containing the microvascular target area);

[0009] Step 13: Create a corresponding label image for each ROI image and accurately mark the microcirculatory blood vessels;

[0010] Step 14: Save the annotation result as a binary label image, where the microvascular area is marked as white with a pixel value of 1, and the background area is marked as black with a pixel value of 0. The generated label image corresponds to the original ROI image one by one to form a complete training sample pair;

[0011] Step 15: The dataset images are processed by data enhancement operation before being input into the enhanced feature fusion network model;

[0012] Step 2: Strengthen the design of feature fusion network model

[0013] The enhanced feature fusion network consists of an encoder and a decoder, where:

[0014] In the encoding stage, the input image first passes through a series of residual convolution modules for feature extraction. Each layer of residual convolution modules performs a pooling operation after convolution to gradually reduce the spatial resolution of the image and extract high-level feature information.

[0015] In the decoding stage, the spatial resolution of the image is gradually restored. The decoder features of each level are upsampled through deconvolution operations to restore low-resolution features to high resolution.

[0016] In the process of upsampling feature fusion, the design of enhanced jump connection is adopted. Specifically, the encoder features of the next level are deconvoluted and upsampled by the self-attention mechanism module, and feature fused with the encoder features and decoder features of the current level. After feature fusion, the encoder feature x1 and the decoder feature x2 are channel-fused by the channel space attention module, and then feature extraction is performed through the channel attention mechanism. Then, the feature map is added with the decoder feature x3 upsampled by the self-attention mechanism module. Finally, after spatial attention feature extraction, the feature map x4 is output. The fused feature map x4 is processed by a double-layer convolution module with residual connection. At the same time, the features of the decoders at all levels are upsampled by pixel reorganization, feature fusion is performed in the channel direction, and features are extracted by a double-layer convolution module. The segmentation result is output through the output layer.

[0017] Step 3: Automatic segmentation of microcirculatory vessels

[0018] Step 31: In the training phase, the preprocessed ROI image and its corresponding binary label image are input, and the enhanced feature fusion network model is used for training. The network parameters are continuously adjusted through the optimizer to minimize the error between the predicted result and the real label image. The joint loss function L combining Dice loss and cross entropy loss is used for monitoring to ensure that the model can effectively learn microvascular features.

[0019] Step 32: In the test phase, the microcirculation ROI image is input and automatically segmented through the enhanced feature fusion network model. The model accurately identifies and segments the microvascular area in the image through feature extraction and multi-scale feature fusion, and outputs the binary segmentation result;

[0020] Step 4: Quantification of blood vessel diameter based on segmentation results

[0021] Step 41: Obtain a binary vascular image from the segmentation result output by the enhanced feature fusion network model, and perform skeleton processing on the binary vascular image to obtain the center line of the blood vessel;

[0022] Step 42: Apply distance transform on the binarized vascular image to generate a distance transform map. In the skeletonized image, extract the distance values ​​on the distance transform map along the center line and multiply these distances by 2 to obtain the local diameter.

[0023] Step 43: By performing statistical analysis on the local diameters, the diameter distribution of blood vessels is obtained, thereby achieving accurate quantification of the diameters of microcirculatory blood vessels.

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] 1. The enhanced feature fusion network model proposed in this paper achieves accurate segmentation of microvessels by fusing and enhancing multi-scale features. The model can still accurately identify and segment microvessels in complex backgrounds and low-contrast images, showing excellent robustness and adaptability.

[0026] 2. The model of the present invention has good generalization ability on diverse clinical image data through effective data enhancement, and can adapt to image data with different lighting conditions, contrasts and resolutions. This makes the model highly practical and reliable in practical applications.

[0027] 3. The present invention can quantify the blood vessel diameter through segmentation results, accurately measure the diameter of microvessels, and provide strong support for the evaluation of clinical microcirculation function.

[0028] 4. The present invention can effectively improve the segmentation accuracy and quantification accuracy of microvascular images by introducing residual connections, self-attention mechanism and enhanced skip connection design. It not only performs well under the conditions of limited viewing angle and poor signal quality, but also can significantly improve the automation level of microcirculation function evaluation, providing reliable technical support for clinical diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of the network model for enhanced feature fusion proposed in the present invention;

[0030] Figure 2 The following is an example of the automatic segmentation application effect of the method proposed in the present invention in sublingual microcirculation blood vessel images. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0032] The present invention provides a method for automatic segmentation and quantification of microcirculatory blood vessels based on enhanced feature fusion, the method comprising the following steps:

[0033] Step 1: Data preparation and preprocessing

[0034] In order to ensure the effectiveness of model training and verification, the present invention carried out detailed microcirculatory vascular image data preparation and preprocessing. First, a large number of microcirculatory static images and video images were collected from various sources. These data mainly come from high-resolution microscopes and microcirculatory camera equipment, covering microcirculatory vascular images under different health conditions and pathological conditions. The diversity of the data includes images with different lighting conditions, contrasts, and resolutions to ensure that the model has stronger generalization ability.

[0035] In order to improve the learning efficiency and segmentation accuracy of the model, ROI (region of interest) is extracted from the image. Since the original image usually contains a lot of irrelevant information, the goal of extracting ROI is to focus on the key areas of the image containing microcirculatory blood vessels, thereby removing redundant background information. The present invention uses professional image processing software to manually select and mark the microvascular areas in the image, and cut out these areas to form a new ROI image. During the cutting process, ensure that the selected ROI area contains a complete microvascular structure and is of moderate size to facilitate subsequent model training.

[0036] In order to conduct supervised deep learning training, a corresponding label image is created for each ROI image, and professional labeling tools such as LabelMe are used to accurately label the microcirculatory vessels. Specifically, each ROI image is first imported into LabelMe, and then the polygon tool (PolygonTool) is used to manually outline the outline of each microvessel to ensure the accuracy and consistency of the labeling. After the labeling is completed, the labeling result is saved as a binary label image, in which the microvascular area is marked as white (pixel value is 1) and the background area is marked as black (pixel value is 0). Finally, the generated label image corresponds one-to-one with the original ROI image to form a complete training sample pair.

[0037] The labeled dataset is divided into training set, validation set and test set in a ratio of 8:1:1. The training set is used for parameter learning and optimization of the model to ensure that the training set covers all types and variations of microcirculatory blood vessels to enhance the learning ability of the model. The validation set is used to evaluate the performance of the model during the training process and guide the adjustment of model parameters to prevent overfitting. After the model training and validation are completed, the test set is used for the final evaluation of the model to detect its generalization ability and accuracy in practical applications. The data in the test set are also data that the model has not seen during training, thus ensuring the fairness of the evaluation.

[0038] The dataset images were subjected to a series of data enhancement operations such as contrast enhancement and filtering before being input into the network model. These preprocessing steps are designed to improve the quality of the image, enhance the contrast between blood vessels and background, and reduce the impact of noise, thus laying the foundation for subsequent feature extraction. The processed input image size is 256×256 pixels, ensuring that the model can capture sufficient detail information while maintaining computational efficiency.

[0039] Step 2: Strengthen the design of feature fusion network model

[0040] In this paper, in order to achieve accurate segmentation and quantification of microcirculatory blood vessels, a deep learning network model with enhanced feature fusion is designed. The model is based on the U-Net architecture and combines multi-scale feature enhancement processing to improve the accuracy and robustness of microvascular segmentation.

[0041] The architecture of the enhanced feature fusion network model is as follows Figure 1 As shown in the figure. In the encoding stage, the input image first passes through a series of residual convolution modules (Resblock) for feature extraction. Resblock uses double-layer convolution with residual connection to ensure effective feature transmission and gradient stability. After each layer of Resblock convolution, a pooling operation (Down) is performed to gradually reduce the spatial resolution of the image and extract high-level feature information.

[0042] In the decoding stage, the spatial resolution of the image is gradually restored. The decoder features of each level are upsampled (Up) through the deconvolution operation to restore the low-resolution features to high resolution. In the process of upsampling feature fusion, the design of enhanced jump connection is adopted. Specifically, the encoder features of the next level are deconvoluted and upsampled by the self-attention mechanism module (module A1), and feature fused with the encoder features and decoder features of the current level. In order to enhance the cross-dimensional interaction in channels and space and compensate for the loss of spatial information caused by downsampling, the channel spatial attention module (module A2) is used after feature fusion. In this module, the encoder features x1 and the decoder features x2 are channel-fused, and then feature extraction is performed through the channel attention mechanism. The feature map is then added with the decoder features x3 upsampled by the A1 module, and finally the feature map x4 is output after spatial attention feature extraction. In order to further enhance the expressiveness of the features, the fused feature map x4 is processed by a double-layer convolution module (Resblock) with residual connections to ensure the accuracy of the model in capturing complex vascular structures. At the same time, the features of the decoders at all levels are upsampled (Up) by pixel recombination, and feature fusion is performed in the channel direction. The features are extracted by the Resblock module, and the segmentation results are output by the output layer 1×1 convolution layer (1×1Conv).

[0043] By strengthening the feature fusion design, the network model proposed in this paper performs well in the task of microcirculation vessel segmentation. It not only effectively improves the accuracy of segmentation, but also greatly improves the accuracy of microvascular quantification, especially in complex backgrounds and low-contrast images, showing strong robustness and adaptability.

[0044] Step 3: Automatic segmentation of microcirculatory vessels

[0045] Step 31: In the training phase, input the preprocessed ROI image and its corresponding binary label image, and use the enhanced feature fusion network model for training. The optimizer continuously adjusts the network parameters to minimize the error between the predicted results and the true label image, and uses loss function and Dice coefficient and other evaluation indicators for monitoring to ensure that the model can effectively learn microvascular features.

[0046] Step 32: In the test phase, the microcirculation ROI image is input and automatically segmented through the encoder-decoder structure enhanced feature fusion network model. The model accurately identifies and segments the microvascular area in the image through feature extraction and multi-scale feature fusion, and outputs a binary segmentation result.

[0047] Step 3: Quantitatively evaluate the microvascular results segmented by the model, using image segmentation evaluation indicators, including Dice coefficient, average intersection-over-union ratio, pixel accuracy and other indicators to evaluate the segmentation accuracy, to ensure that the segmentation results are highly consistent with the actual microvascular area. At the same time, qualitatively analyze the segmentation results and verify the actual application effect of the model through visual display.

[0048] Step 4: Quantification of blood vessel diameter based on segmentation results

[0049] First, a binary vascular image is obtained from the segmentation results output by the deep learning model, where the vascular part is white (value 1) and the background part is black (value 0). Then, the binary image is skeletonized to obtain the centerline of the vascular. The centerline is the minimum path to maintain the topological structure of the vascular and retains the topological structure of the vascular.

[0050] Next, a distance transform is applied to the binarized image. The distance transform computes the distance to the nearest background pixel for each pixel in the image, generating a distance map. In the skeletonized image, the distance values ​​on the distance transform map are extracted along the centerline and multiplied by 2 to obtain the local diameter (since the distance from the centerline to the edge of the vessel is equivalent to the radius).

[0051] Finally, by performing statistical analysis on these local diameters (such as calculating the average value, median value, etc.), the diameter distribution of the blood vessels can be obtained, thereby achieving accurate quantification of the diameter of the microcirculatory blood vessels.

[0052] Step 4: Experimental verification and application

[0053] In order to verify the effectiveness of the enhanced feature fusion network model designed in the present invention, the model performance was evaluated on clinical sublingual microcirculation detection data.

[0054] In model training, Dice loss and cross entropy loss are combined, and the joint loss function L is expressed as:

[0055] L=αL c +βL d

[0056] Among them, α is the cross entropy loss L c The weight of β is the Dice loss L d In the embodiment, α and β are set to 0.4 and 0.6 respectively. The Adam optimizer is used to optimize and adjust the network parameters.

[0057] The performance of the model was comprehensively evaluated on the previously divided test set. The images in the test set come from different sources and conditions, covering microcirculatory vascular images under complex backgrounds, different contrasts, and various pathological conditions. The segmentation results of each image were obtained by reasoning on the test set. In order to evaluate the accuracy and consistency of the segmentation, commonly used image segmentation evaluation indicators were used, including the Dice coefficient (DSC), mean intersection over union (mIoU), and pixel accuracy (PixelAccuracy, PA). The deep learning model with enhanced feature fusion proposed in the present invention has a Dice coefficient of 0.87, a mean intersection over union of 0.81, and a pixel accuracy of 0.89 in the sublingual microcirculatory vascular segmentation. The experimental results show that the model proposed in the present invention performs well under various test conditions, especially when dealing with complex backgrounds and low-contrast images, the segmentation results are highly accurate and consistent.

[0058] Figure 2 The visualization results of the model's automatic segmentation in sublingual microcirculatory blood vessel images are shown. Experimental results show that the model of the present invention can still accurately identify and segment microvessels in complex backgrounds and low-contrast images.

[0059] In the quantification stage, the blood vessel diameters based on the model output were measured and analyzed. The diameter of each blood vessel was calculated by the method described in step 3, and a diameter distribution map was generated. The stability and reliability of these quantitative results under different experimental conditions were further verified. The experimental results show that the model proposed in the present invention has high accuracy in the quantitative analysis of microvascular diameters, and the error with the manual annotation results is within an acceptable range, which provides strong support for clinical applications.

Claims

1. A method for automatic segmentation and quantification of microcirculatory vessels based on enhanced feature fusion, characterized in that The method comprises the following steps: Step 1: Data preparation and preprocessing Step 1, collecting microcirculation blood vessel images with different illumination conditions, contrasts and resolutions, wherein the microcirculation images include static images and video images of microcirculation under different health conditions and pathological conditions; Step 1 and 2: extract ROI from the microcirculatory blood vessel image, select and mark the microvascular area in the image, and crop these areas to form a new ROI image; Step 13: Create a corresponding label image for each ROI image and accurately mark the microcirculatory blood vessels; Step 14: Save the annotation result as a binary label image, where the microvascular area is marked as white with a pixel value of 1, and the background area is marked as black with a pixel value of 0. The generated label image corresponds to the original ROI image one by one to form a complete training sample pair; Step 15: The dataset images are processed by data enhancement operation before being input into the enhanced feature fusion network model; Step 2: Strengthen the design of feature fusion network model The enhanced feature fusion network consists of an encoder and a decoder, where: In the encoding stage, the input image first passes through a series of residual convolution modules for feature extraction. Each layer of residual convolution modules performs a pooling operation after convolution to gradually reduce the spatial resolution of the image and extract high-level feature information. In the decoding stage, the spatial resolution of the image is gradually restored, and the decoder features of each level are upsampled through deconvolution operations to restore low-resolution features to high resolution; In the process of upsampling feature fusion, the design of enhanced jump connection is adopted. Specifically, the encoder features of the next level are deconvoluted and upsampled by the self-attention mechanism module, and feature fused with the encoder features and decoder features of the current level. After feature fusion, the encoder feature x1 and the decoder feature x2 are channel-fused by the channel space attention module, and then feature extraction is performed through the channel attention mechanism. Then, the feature map is added with the decoder feature x3 upsampled by the self-attention mechanism module. Finally, after spatial attention feature extraction, the feature map x4 is output. The fused feature map x4 is processed by a double-layer convolution module with residual connection. At the same time, the features of the decoders at all levels are upsampled by pixel reorganization, feature fusion is performed in the channel direction, and features are extracted by a double-layer convolution module. The segmentation result is output through the output layer. Step 3: Automatic segmentation of microcirculatory vessels Step 31: In the training phase, the preprocessed ROI image and its corresponding binary label image are input, and the enhanced feature fusion network model is used for training. The network parameters are continuously adjusted through the optimizer to minimize the error between the predicted result and the real label image. The joint loss function L combining Dice loss and cross entropy loss is used for monitoring to ensure that the model can effectively learn microvascular features. Step 32: In the test phase, the microcirculation ROI image is input and automatically segmented through the enhanced feature fusion network model. The model accurately identifies and segments the microvascular area in the image through feature extraction and multi-scale feature fusion, and outputs the binary segmentation result; Step 4: Quantification of blood vessel diameter based on segmentation results Step 41: Obtain a binary vascular image from the segmentation result output by the enhanced feature fusion network model, and perform skeleton processing on the binary vascular image to obtain the center line of the blood vessel; Step 42: Apply distance transform on the binarized vascular image to generate a distance transform map. In the skeletonized image, extract the distance values ​​on the distance transform map along the center line and multiply these distances by 2 to obtain the local diameter. Step 43: By performing statistical analysis on the local diameters, the diameter distribution of blood vessels is obtained, thereby achieving accurate quantification of the diameters of microcirculatory blood vessels.

2. The method for automatic segmentation and quantification of microcirculatory vessels based on enhanced feature fusion according to claim 1, characterized in that The joint loss function L is expressed as: L=αL c +βL d Among them, α is the cross entropy loss L c The weight of β is the Dice loss L d The weight of .

Images