A forearm superficial blood vessel segmentation method and device based on an NSVA-NET deep learning network

Abstract

The application discloses a forearm shallow blood vessel segmentation method and device based on an NSVA-Net deep learning network, and relates to the technical field of medical image processing. The method comprises the following steps: collecting near-infrared images of the forearm shallow blood vessels of a subject by using a near-infrared light source and a near-infrared camera; performing denoising, contrast enhancement, cutting of the forearm region and normalization on the near-infrared images to obtain preprocessed images; performing two-stage image enhancement of RCAE and CLAHE on the preprocessed images to obtain enhanced images; inputting the enhanced images into a pre-trained NSVA-Net deep learning network to obtain a segmentation probability map of the forearm shallow blood vessels; performing threshold segmentation, connected domain analysis and skeletonization processing on the segmentation probability map to remove artifacts and isolated noise, and obtaining a blood vessel segmentation result and a center line; and superimposing and displaying the blood vessel segmentation result and the center line on the near-infrared images to output a final segmentation image. By using the application, the segmentation recall rate and the structural continuity of the forearm shallow small blood vessels of dialysis patients can be significantly improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method and apparatus for segmenting superficial blood vessels in the forearm based on the NSVA-NET deep learning network. Background Technology

[0002] In clinical procedures such as venipuncture, hemodialysis, and interventional therapy, visualization of superficial blood vessels is particularly important. Traditional methods of vein localization, relying on visual inspection and palpation, often suffer from low success rates, long operation times, and increased pain and complication risks associated with repeated punctures, especially in obese patients, those with darker skin pigmentation, or those with small veins. Near-infrared venography utilizes the optical properties of deoxyhemoglobin's strong absorption of specific wavelengths of near-infrared light and its strong reflection from surrounding tissues. By using a near-infrared light source and imaging sensor to obtain the contrast between the superficial vein and the background, and then processing the image to achieve real-time visualization of the vein, it helps improve the success rate of venipuncture, reduce the number of attempts, and decrease related complications. Existing research and products often employ dedicated cameras, FPGA / SoCs, and image enhancement and segmentation algorithms to display the processed vein images in real time. These devices offer advantages such as being non-invasive, easy to operate, and applicable to various skin tones and body surface locations, and are considered a promising auxiliary method for puncture and vascular assessment.

[0003] However, existing vein segmentation and imaging techniques still have shortcomings in terms of fine segmentation and structural continuity of superficial veins. On the one hand, Chinese invention patent CN112288739B, "A Deep Learning-Based Vein Segmentation Method," proposes to enhance and annotate vein images, and then use a U-Net model with an attention mechanism for training and inference to improve the accuracy of vein segmentation, mainly addressing the problems of poor imaging quality and insufficient robustness of traditional image processing methods. Although this method introduces an attention mechanism, which helps improve the distinction between vein regions and background, its network structure is still a single-path encoder-decoder framework. It is not specifically designed for the characteristics of near-infrared images of superficial forearm vessels, such as low contrast, high noise, and numerous thin branches. When processing superficial forearm veins, it is prone to problems such as small vessel defects and broken vessel courses, and lacks explicit constraints on the geometric continuity of the vessel centerline.

[0004] On the other hand, Chinese invention patent CN116363149A, "A Medical Image Segmentation Method Based on U-Net Improvement," proposes the OA-UNet model. This model introduces octave convolution and CBAM attention modules into the classic U-Net framework to enhance multi-scale features and improve the segmentation accuracy of pathological regions. This approach is primarily aimed at lesion region segmentation tasks in various publicly available medical image datasets. Its data preprocessing mainly involves conventional rotation, mirroring, brightness, and contrast adjustments. Input images are uniformly scaled to a fixed size, and the segmentation targets are mostly blocky lesion regions. While this type of improved U-Net method performs well in general medical image segmentation, it does not design specific network paths and loss constraints for the slender, tree-like topology of superficial veins in the forearm. It lacks focused modeling of the vessel centerline and minute branches, and it does not consider the special noise patterns and real-time requirements of near-infrared imaging in dialysis scenarios. When directly applied to forearm near-infrared superficial vessel segmentation, its recall rate for small vessels and structural continuity remain insufficient.

[0005] In summary, existing technologies still suffer from problems such as low contrast in near-infrared images of superficial veins in the forearm, easy breakage of small blood vessels, severe interference from background texture and noise, and the lack of continuity constraints for superficial vascular structures in existing deep learning segmentation networks. Summary of the Invention

[0006] To address the technical problems existing in the prior art, this invention provides a method and apparatus for segmenting superficial forearm vessels based on the NSVA-NET deep learning network. The technical solution is as follows:

[0007] On the one hand, a method for segmenting superficial forearm vessels based on the NSVA-NET deep learning network is provided. This method is implemented by a forearm superficial vessel segmentation device based on the NSVA-NET deep learning network, and includes: S1. Near-infrared images of superficial blood vessels in the forearm of the subject are acquired using a near-infrared light source and a near-infrared camera; S2. The near-infrared image is denoised, contrast enhanced, the forearm region is cropped and normalized to obtain a preprocessed image. S3. Perform two-stage image enhancement, RCAE and CLAHE, on the preprocessed image to obtain the enhanced image; S4. Input the enhanced image into the pre-trained NSVA-NET deep learning network to obtain the segmentation probability map of superficial blood vessels in the forearm; S5. Perform threshold segmentation, connected component analysis and skeletonization on the segmentation probability map to remove artifacts and isolated noise, and obtain the blood vessel segmentation result and centerline. S6. Overlay the blood vessel segmentation results and centerline with the near-infrared image to display the final segmented image.

[0008] On the other hand, a forearm superficial vessel segmentation device based on the NSVA-NET deep learning network is provided. This device is applied to the forearm superficial vessel segmentation method based on the NSVA-NET deep learning network. The device includes: The acquisition unit is used to acquire near-infrared images of superficial blood vessels in the subject's forearm using a near-infrared light source and a near-infrared camera; The preprocessing unit is used to denoise, enhance contrast, crop the forearm region and normalize the near-infrared image to obtain a preprocessed image. The enhancement unit is used to perform two-stage image enhancement, RCAE and CLAHE, on the preprocessed image to obtain the enhanced image; The segmentation unit is used to input the enhanced image into the pre-trained NSVA-NET deep learning network to obtain a segmentation probability map of superficial blood vessels in the forearm; The post-processing unit is used to perform threshold segmentation, connected component analysis and skeletonization on the segmentation probability map, remove artifacts and isolated noise, and obtain the blood vessel segmentation result and centerline. The display unit is used to overlay the blood vessel segmentation results and centerline with the near-infrared image and output the final segmented image.

[0009] On the other hand, a forearm superficial vessel segmentation device based on the NSVA-NET deep learning network is provided. The forearm superficial vessel segmentation device based on the NSVA-NET deep learning network includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, they implement any of the methods described above forearm superficial vessel segmentation methods based on the NSVA-NET deep learning network.

[0010] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for forearm superficial vessel segmentation based on the NSVA-NET deep learning network.

[0011] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: A two-stage image enhancement strategy, RCAE and CLAHE, is employed. On one hand, a learned enhancement module is used to suppress imaging noise and enhance superficial vessels; on the other hand, adaptive histogram equalization improves local contrast, making superficial forearm vessels more prominent in near-infrared images. In terms of network structure, an NSVA deep learning segmentation network incorporating contextual and detail paths is constructed. A forearm superficial vessel attention (NSVA) module is introduced into the multi-scale feature layer to specifically enhance the features of slender, low-contrast vessels and suppress skin texture and background interference. At the decoding end, a region segmentation branch and a centerline branch are designed, and the topological integrity and continuity of vessel course are explicitly constrained through joint optimization of centerline constraint loss and region segmentation loss. Through these improvements, this invention can significantly improve the segmentation recall and structural continuity of small superficial forearm vessels in dialysis patients while maintaining a moderate number of network parameters and a fast inference speed, better meeting the application needs of real-time imaging, thrombosis detection, and puncture navigation in clinical scenarios. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a method for segmenting superficial blood vessels in the forearm based on the NSVA-NET deep learning network, provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of a two-stage image enhancement method using RCAE and CLAHE provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the overall structure of an NSVA-Net deep learning network provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an NSVA attention module structure provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of an NSVA-Net network training structure based on joint loss provided by an embodiment of the present invention; Figure 6 This is a block diagram of a forearm superficial blood vessel segmentation device based on the NSVA-NET deep learning network provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a forearm superficial blood vessel segmentation device based on the NSVA-NET deep learning network provided in an embodiment of the present invention. Detailed Implementation

[0014] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0015] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0016] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0017] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0018] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0019] This invention provides a method for segmenting superficial blood vessels in the forearm based on the NSVA-NET deep learning network. This method can be deployed in near-infrared vascular imaging equipment in dialysis centers. Before dialysis, medical staff place the patient's forearm under the imaging device, and the image acquisition and storage can be completed automatically through simple operation. After subsequent image processing, the image can assist medical staff in selecting the puncture point and puncture direction.

[0020] like Figure 1 The flowchart shown is for a forearm superficial vessel segmentation method based on the NSVA-NET deep learning network. The processing flow of this method may include the following steps: S1. Near-infrared images of superficial blood vessels in the forearm of the subject are acquired using a near-infrared light source and a near-infrared camera.

[0021] In one feasible implementation, the dialysis patient (i.e. the subject) places the forearm to be tested flat on the support platform, and adjusts the position and orientation of the near-infrared light source and the near-infrared camera so that the superficial vein area of ​​the forearm is located in the center of the camera's field of view.

[0022] Optionally, the near-infrared light source operates at a wavelength of 850 nm to enhance the difference in light absorption characteristics between deoxygenated hemoglobin in the blood and surrounding tissues, thereby improving the contrast of superficial blood vessels in near-infrared images. The exposure time, gain, and frame rate of the near-infrared camera are set according to ambient lighting conditions and forearm surface reflectivity to obtain a raw near-infrared image with uniform grayscale distribution and no obvious saturation or overly dark areas. The acquired near-infrared images of superficial forearm blood vessels are transmitted to an image processing terminal via a wired connection as input data for subsequent processing.

[0023] S2. Denoise the near-infrared image, enhance its contrast, crop the forearm region, and normalize it to obtain a preprocessed image.

[0024] In one feasible implementation, the near-infrared image of the original forearm is preprocessed, specifically including grayscale normalization, noise reduction, and region of interest (ROI) extraction.

[0025] Optionally, the original image is first normalized to grayscale, linearly mapping the original grayscale values ​​to the standard range of [0, 255] to reduce the impact of different acquisition batches and changes in equipment parameters; then, bilateral filtering is used to denoise the image, smoothing random noise while preserving blood vessel edges and local structural information as much as possible; further, the forearm region is initially located based on the brightness threshold, and then the forearm contour is accurately delineated through edge detection, contour tracking and other methods to obtain the forearm region mask.

[0026] Based on this, the forearm region is appropriately shrunk according to a preset safety boundary, and a Region of Interest (ROI) containing the main superficial blood vessel distribution is cropped from the original image. To adapt to the input requirements of subsequent deep learning networks, the ROI region is scaled to a uniform network input size, preferably 1024×768 pixels, and the scaled image is normalized again to obtain a preprocessed image, denoted as . .

[0027] S3. Perform two-stage image enhancement, RCAE and CLAHE, on the preprocessed image to obtain the enhanced image.

[0028] In one feasible implementation, the preprocessed image Two-stage image enhancement was performed to obtain enhanced images that were more conducive to the segmentation of superficial vessels.

[0029] Optionally, the specific operation of S3 may include the following steps S31-S32: S31. Input the preprocessed image into the RCAE image enhancement network to obtain the first-stage enhanced image. The RCAE image enhancement network employs an encoder-decoder structure. The encoder layer extracts texture and structural features at different scales through multi-layer convolution and downsampling, while the decoder layer recovers spatial details through upsampling and skip connections. Residual connections are introduced in several layers at both the encoder and decoder ends.

[0030] In one feasible implementation, in the first stage, the preprocessed image is input into a residual convolutional autoencoder (RCAE) for learned enhancement, which can be expressed as: (1) in, This represents the first-stage enhanced image after RCAE processing. This represents a residual convolutional autoencoder. Its network parameters.

[0031] Specifically, RCAE employs an encoder-decoder structure. The encoder consists of multiple convolutional layers and downsampling to progressively extract texture and structural features at different scales. The decoder consists of multiple upsampling layers and convolutional layers to progressively restore spatial resolution and local details. Residual connections are incorporated at both the encoder and decoder ends to mitigate the vanishing gradient problem during deep network training and improve image reconstruction capabilities. Through offline training on a dataset containing a large number of forearm near-infrared images, RCAE learns a nonlinear transformation that maps noisy, low-contrast images to noise-suppressed, vascular structure-enhanced images. The first-stage enhanced image is obtained after RCAE processing.

[0032] S32. Input the first-stage enhanced image into the CLAHE image enhancement network, divide the first-stage enhanced image into several independent small regions, crop and normalize the gray-level histogram of each small region, perform histogram equalization, and then use interpolation to smoothly transition at the boundaries of the small regions to obtain the enhanced image.

[0033] In one feasible implementation, in the second stage, based on the enhanced results of the first stage, [the following is done / is done / etc.]. Perform CLAHE (Contrast Limited Adaptive Histogram Equalization) processing to obtain the second-stage enhanced image. , can be represented as: (2) in, This indicates the CLAHE operation. The contrast clipping threshold. The number of small blocks to divide the image into.

[0034] This function enhances local contrast and suppresses excessive noise amplification by cropping and equalizing the histogram of each small block, and then performing interpolation smoothing between blocks. While limiting excessive local contrast enhancement and noise amplification, it significantly enhances the grayscale difference between superficial blood vessels and background tissue. Figure 2 As shown. The result is... This image serves as the input for the subsequent NSVA-NET segmentation network.

[0035] S4. Input the enhanced image into the pre-trained NSVA-NET deep learning network to obtain the segmentation probability map of superficial blood vessels in the forearm.

[0036] Among them, such as Figure 3 As shown, the NSVA-NET deep learning network includes an encoder and a decoder. The encoder includes a detail path, a context path, a multi-scale forearm shallow vascular attention module (NSVA), and a feature fusion unit. The decoder includes an upsampling unit, a skip connection feature alignment unit, a skip connection fusion convolutional unit, and a prediction head.

[0037] The enhanced image from the second stage is input into the NSVA-NET network. In the detail path, several convolutional layers and downsampling operations with small strides are used to extract and preserve the vascular edge information and local texture details with high spatial resolution. In the context path, the receptive field is gradually expanded through stride convolution and dilated convolution structures to extract global semantic features related to the overall course of superficial blood vessels and complex background.

[0038] Optionally, the specific operation of S4 may include the following steps S41-S45: S41. Input the enhanced image into the detail path and the context path respectively.

[0039] The detailed path includes a three-layer structure, namely D1, D2 and D3. D1 includes a convolutional layer, batch normalization and ReLU activation function, while D2 and D3 are convolutional layers. The context path consists of a three-layer structure, namely C1, C2 and C3. C1 includes a convolutional layer, batch normalization and ReLU activation function, while C2 and C3 are convolutional layers.

[0040] S42. Input the second-layer output of the detail path and the second-layer output of the context path into the NSVA module to obtain the attention-weighted features.

[0041] Optionally, the specific operation of S42 may include the following steps S421-S425: S421. Combine the second-layer output of the detail path with the second-layer output of the context path into a single feature map.

[0042] S422. Input the combined feature maps into the channel attention branch and the spatial attention branch respectively.

[0043] S423. In the channel attention branch, global average pooling and global max pooling are performed on the input feature map to obtain two channel description vectors. The two channel description vectors are input into a shared multilayer perceptron, which outputs channel weight vectors. These vectors are then normalized using the sigmoid function to obtain the channel attention weights.

[0044] One feasible implementation method is, for example Figure 4 As shown, channel attention weights The channel description vectors obtained from global average pooling and global max pooling are computed by a shared multilayer perceptron and can be represented as: (3) in, and They represent respectively to The results obtained by global average pooling and global max pooling dimensional vector, and For learnable weight matrix, It is a non-linear activation function. This is the Sigmoid function.

[0045] S424. In the spatial attention branch, the spatial feature map is obtained by calculating the local contrast and gradient of the input feature map. The spatial feature map is concatenated with the channel pooling result and then processed by convolution and the Sigmoid function to obtain the spatial attention weights.

[0046] In one feasible implementation, spatial attention weights Determined jointly by local statistical features and channel pooling results, it can be expressed as: (4) in, and These represent the average pooling and max pooling feature maps along the channel dimension, respectively. This represents the feature map calculated based on local contrast. This represents the feature map calculated based on the gradient. This indicates splicing along the channel dimension. Indicates the kernel size as The convolution operation.

[0047] S425. Apply the channel attention weights and spatial attention weights to the input feature map respectively to obtain the attention-weighted features.

[0048] In one feasible implementation, the input feature map at a certain scale is... The NSVA module supports Modeling channel attention and spatial attention can be represented as follows: (5) in, For the enhanced feature map, This represents element-wise multiplication. This is a channel attention weight map. This is a spatial attention weight map.

[0049] S43. Input the third-layer output of the detail path, the third-layer output of the context path, and the attention-weighted features into the feature fusion unit. Based on the third-layer output of the detail path, the third-layer output of the context path, and the attention-weighted features, obtain the first fused feature. Based on the third-layer output of the detail path and the third-layer output of the context path, obtain the second fused feature.

[0050] Optionally, the specific operation steps of S43 may include the following steps S431-S432: S431. After concatenating the third-layer output of the detail path, the third-layer output of the context path, and the attention-weighted features, channel compression is performed by 1×1 convolution, and the proportion of different path features in the fusion result is controlled by a gating mechanism to obtain the first fused feature.

[0051] Specifically, let the third-level detailed path features be... The third-level context path features are Attention-weighted features are Then the first fusion feature It can be represented as: (6) in For different paths, the gating coefficients are... The parameters are 1×1 convolution kernel parameters. This indicates subsequent convolution and non-linear activation operations.

[0052] S432. After concatenating the third-layer output of the detail path and the third-layer output of the context path, the channels are compressed by 1×1 convolution, and the proportion of different path features in the fusion result is controlled by a gating mechanism to obtain the second fusion feature.

[0053] In one feasible implementation, let the detailed path features at a certain scale be... Context path features are The features after fusion Preferably, it can be expressed as: (7) in, The gating coefficient for this scale can be a learnable parameter or preset based on experience; The parameters are 1×1 convolution kernel parameters; This indicates subsequent convolution and non-linear activation operations.

[0054] S44. Input the first fused feature into the upsampling unit of the decoding end to obtain the sampled feature. Input the second fused feature into the skip connection feature alignment unit to obtain the aligned feature. Input the sampled feature and the aligned feature together into the skip connection fusion convolution unit to obtain the fused convolution feature. Input the fused convolution feature into the prediction head to obtain the blood vessel prominence feature map.

[0055] In one feasible implementation, the decoding end adopts a structure of progressive upsampling and skip connections to restore the fused feature map to a spatial resolution close to that of the input image layer by layer, and continues to embed NSVA attention modules in each decoding layer to further highlight the features of the vascular region.

[0056] S45. Perform main branch convolution mapping and auxiliary branch convolution mapping on the vascular salience feature map to obtain the vascular region probability map and the vascular centerline and vascular boundary probability map, respectively.

[0057] In one feasible implementation, a blood vessel region segmentation probability map is output through the main branch. The auxiliary branch outputs a probability map of the vessel centerline and vessel boundary. , can be represented as: (8) (9) in, This represents the vascular prominence feature map output by the decoding end. and These represent the convolution mappings of the main branch and the auxiliary branch, respectively.

[0058] It should be noted that this training method includes steps such as training dataset construction, network initialization and forward inference, joint loss function construction, and network parameter optimization. By training on a labeled dataset containing near-infrared images of the forearms of multiple dialysis patients, the NSVA-NET network achieved the expected performance indicators in terms of region segmentation accuracy and continuity of vascular centerline structure. Figure 5 As shown, the specific operation process includes:

[0059] The training images are input into the NSVA deep learning network to be trained. Following the execution process of the model described above, the images sequentially pass through the encoder, the NSVA module, and the decoder to obtain the blood vessel region prediction probability map and the centerline prediction probability map. A region segmentation loss function is calculated based on the blood vessel region prediction probability map and its annotations. A centerline constraint loss function is calculated based on the centerline prediction probability map and its annotations. A joint loss is then calculated based on the region segmentation loss function and the centerline constraint loss function. The parameters of the NSVA deep learning network to be trained are iteratively updated using the joint loss. When the required number of iterations is reached, the iterative training stops, resulting in the trained NSVA deep learning network.

[0060] Specifically, suppose the image has a total of 1 pixel and They represent the first 1 pixel in and Given the foreground probability and label value, the Dice loss for a region can be expressed as: (10) in, To prevent smoothing terms with a denominator of zero, and to further control the balance between false negatives and false positives, a Tversky-type loss can be introduced, assigning higher weight to the vascular prospect. Preferably, this can be expressed as:

[0061] (11) in, and It is an adjustable hyperparameter used to balance false positives and false negatives.

[0062] To explicitly constrain the vascular centerline structure, embodiments of the present invention utilize the centerline probability map output from auxiliary branches. With center line label Introduce the centerline Dice loss between them, and assume and They represent the first 1 pixel in and If the value in the equation is given, then the centerline loss can be expressed as: (12) Combining the above-mentioned region segmentation loss and centerline constraint loss, the joint loss function of the present invention can preferably be expressed as: (13) in, , and These are non-negative weighting coefficients used to balance optimization objectives related to region overlap, class imbalance, and centerline structure. By minimizing... The parameters of the NSVA-Net network are iteratively updated until convergence on the training and validation sets, thereby obtaining a trained model that achieves the expected performance in terms of segmentation accuracy of the superficial vascular region of the forearm and continuity of the central line structure.

[0063] S5. Threshold segmentation, connected component analysis, and skeletonization are performed on the segmentation probability map to remove artifacts and isolated noise, and the blood vessel segmentation results and centerline are obtained.

[0064] Optionally, the specific operation of S5 may include the following steps S51-S54: S51. Binarize the probability map of the blood vessel region using a preset threshold to obtain a binary blood vessel mask.

[0065] In one feasible implementation, binarization using a preset threshold can be expressed as follows: (14) S52. Perform connected component analysis on the binary blood vessel mask, count the number of pixels in each connected region, and remove isolated small connected regions with an area smaller than a preset threshold.

[0066] In one feasible implementation, isolated small connected regions with an area smaller than a preset threshold are removed to eliminate false positive regions caused by noise or local artifacts.

[0067] S53. Based on the preserved vascular region, a morphological refinement algorithm is used for skeletonization.

[0068] S54. Perform morphological opening and closing operations and local curvature-based filtering operations on the skeletonization results to remove pseudo-branches and sharp burrs with lengths less than a preset threshold, smooth the shape of the vessel centerline, and obtain the vessel segmentation results and centerline.

[0069] In one feasible implementation, smoothing the vascular centerline can make the segmentation results more visually continuous and geometrically more consistent with the actual vascular orientation. After the above processing, a superficial vascular segmentation result with continuous structure and clear branch topology, along with its corresponding centerline, is obtained.

[0070] Optionally, when performing the above morphological opening and closing operations and filtering operations based on local curvature, the centerline probability map of the auxiliary branch output can be combined to filter and refine the skeleton pixels according to the probability threshold or weighting strategy, so as to further suppress the false centerline caused by noise and improve the geometric accuracy and topological reliability of the final centerline.

[0071] S6. Overlay the blood vessel segmentation results and centerline with the near-infrared image to display the final segmented image.

[0072] In one feasible implementation, the obtained vessel segmentation results and centerlines are overlaid on the original forearm near-infrared image in a pseudo-color manner for display. Different colors highlight the vessel region outline and centerline, allowing medical personnel to visually observe the spatial distribution and course characteristics of superficial forearm vessels. Simultaneously, the image data and vectorized data of the vessel segmentation results and centerlines are output to the hospital information system via an interface for clinical applications such as thrombosis detection, hemodynamic analysis, and puncture path planning.

[0073] In this embodiment of the invention, a two-stage image enhancement strategy of RCAE and CLAHE is adopted. On the one hand, a learned enhancement module is used to suppress imaging noise and enhance superficial vessels; on the other hand, adaptive histogram equalization is used to improve local contrast, making the superficial vessels of the forearm more prominent in near-infrared images. In terms of network structure, an NSVA deep learning segmentation network containing contextual paths and detail paths is constructed. A forearm superficial vessel attention (NSVA) module is introduced into the multi-scale feature layer to specifically enhance the features of slender, low-contrast vessels and suppress skin texture and background interference. At the decoding end, a region segmentation branch and a centerline branch are designed, and the topological integrity and continuity of the vessel course are explicitly constrained through joint optimization of the centerline constraint loss and the region segmentation loss. Through these improvements, this invention can significantly improve the segmentation recall and structural continuity of small superficial vessels in the forearm of dialysis patients while maintaining a moderate number of network parameters and a fast inference speed, better meeting the application needs of real-time imaging, thrombosis detection, and puncture navigation in clinical scenarios.

[0074] Figure 6 This is a block diagram of a forearm superficial vessel segmentation device based on the NSVA-NET deep learning network, provided by an embodiment of the present invention. This device is used for forearm superficial vessel segmentation based on the NSVA-NET deep learning network. (Refer to...) Figure 6 The device includes a data acquisition unit 610, a preprocessing unit 620, an enhancement unit 630, a segmentation unit 640, a post-processing unit 650, and a display unit 660. Wherein:

[0075] The acquisition unit 610 is used to acquire near-infrared images of superficial blood vessels in the forearm of the subject using a near-infrared light source and a near-infrared camera. Preprocessing unit 620 is used to denoise, enhance contrast, crop the forearm region and normalize the near-infrared image to obtain a preprocessed image. Enhancement unit 630 is used to perform two-stage image enhancement, RCAE and CLAHE, on the preprocessed image to obtain an enhanced image; The segmentation unit 640 is used to input the enhanced image into the pre-trained NSVA-NET deep learning network to obtain a segmentation probability map of superficial blood vessels in the forearm. Post-processing unit 650 is used to perform threshold segmentation, connected component analysis and skeletonization on the segmentation probability map, remove artifacts and isolated noise, and obtain blood vessel segmentation results and center lines. Display unit 660 is used to overlay the blood vessel segmentation results and centerline with the near-infrared image for display, and output the final segmentation image.

[0076] Figure 7 This is a schematic diagram of a forearm superficial blood vessel segmentation device based on the NSVA-NET deep learning network provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the forearm superficial vessel segmentation device based on the NSVA-NET deep learning network may include the above-mentioned... Figure 6 The forearm superficial vessel segmentation device 710 based on the NSVA-NET deep learning network is shown. Optionally, the forearm superficial vessel segmentation device 710 based on the NSVA-NET deep learning network may include a first processor 2001.

[0077] Optionally, the forearm superficial blood vessel segmentation device 710 based on the NSVA-NET deep learning network may also include a memory 2002 and a transceiver 2003.

[0078] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0079] The following is combined with Figure 7 A detailed introduction to each component of the forearm superficial vessel segmentation device 710 based on the NSVA-NET deep learning network is provided below: The first processor 2001 is the control center of the forearm superficial blood vessel segmentation device 710 based on the NSVA-NET deep learning network. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0080] Optionally, the first processor 2001 can execute various functions of the forearm superficial blood vessel segmentation device 710 based on the NSVA-NET deep learning network by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0081] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 7 CPU0 and CPU1 are shown in the diagram.

[0082] In a specific implementation, as one example, the forearm superficial vessel segmentation device 710 based on the NSVA-NET deep learning network may also include multiple processors, for example... Figure 7 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0083] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0084] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the forearm superficial vessel segmentation device 710 based on the NSVA-NET deep learning network. Figure 7 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0085] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0086] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 7 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0087] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the forearm superficial blood vessel segmentation device 710 based on the NSVA-NET deep learning network. Figure 7 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0088] It should be noted that, Figure 7 The structure of the forearm superficial vessel segmentation device 710 based on the NSVA-NET deep learning network shown does not constitute a limitation on the router. Actual forearm superficial vessel segmentation devices based on the NSVA-NET deep learning network may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0089] Furthermore, the technical effect of the forearm superficial vessel segmentation device 710 based on the NSVA-NET deep learning network can be referred to the technical effect of the forearm superficial vessel segmentation method based on the NSVA-NET deep learning network described in the above method embodiments, and will not be repeated here.

[0090] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.

[0091] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0092] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0093] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0094] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0095] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0098] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0099] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0100] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0101] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for segmenting superficial blood vessels in the forearm based on the NSVA-NET deep learning network, characterized in that, The method includes: S1. Near-infrared images of superficial blood vessels in the forearm of the subject are acquired using a near-infrared light source and a near-infrared camera; S2. The near-infrared image is denoised, contrast enhanced, the forearm region is cropped and normalized to obtain a preprocessed image. S3. Perform two-stage image enhancement, RCAE and CLAHE, on the preprocessed image to obtain the enhanced image; S4. Input the enhanced image into the pre-trained NSVA-NET deep learning network to obtain the segmentation probability map of superficial blood vessels in the forearm; S5. Perform threshold segmentation, connected component analysis and skeletonization on the segmentation probability map to remove artifacts and isolated noise, and obtain the blood vessel segmentation result and centerline. S6. Overlay the blood vessel segmentation results and centerline with the near-infrared image to display the final segmented image.

2. The forearm superficial vessel segmentation method based on NSVA-NET deep learning network according to claim 1, characterized in that, The S3 process performs two-stage image enhancement, RCAE and CLAHE, on the preprocessed image to obtain an enhanced image, including: S31. Input the preprocessed image into the RCAE image enhancement network to obtain the first-stage enhanced image; wherein, the RCAE image enhancement network adopts an encoder-decoder structure, the encoder end gradually extracts texture and structural features at different scales through multi-layer convolution and downsampling, and the decoder end gradually restores spatial details through upsampling and skip connections; residual connections are introduced in several layers of the encoder and decoder ends; S32. Input the first-stage enhanced image into the CLAHE image enhancement network, divide the first-stage enhanced image into several independent small regions, crop and normalize the gray-level histogram of each small region, perform histogram equalization, and then use interpolation to smoothly transition at the boundaries of the small regions to obtain the enhanced image.

3. The forearm superficial vessel segmentation method based on NSVA-NET deep learning network according to claim 1, characterized in that, The NSVA-NET deep learning network includes an encoder and a decoder. The encoding end includes detail path, context path, multi-scale NSVA module, and feature fusion unit; The decoding end includes an upsampling unit, a skip connection feature alignment unit, a skip connection fusion convolution unit, and a prediction head.

4. The forearm superficial vessel segmentation method based on NSVA-NET deep learning network according to claim 3, characterized in that, S4 inputs the enhanced image into the pre-trained NSVA-NET deep learning network to obtain a segmentation probability map of superficial blood vessels in the forearm, including: S41. The enhanced image is input into the detail path and the context path respectively; the detail path and the context path each include a three-layer structure; S42. Input the second-layer output of the detail path and the second-layer output of the context path into the NSVA module to obtain the attention-weighted features. S43. Input the third-layer output of the detail path, the third-layer output of the context path, and the attention-weighted features into the feature fusion unit. Based on the third-layer output of the detail path, the third-layer output of the context path, and the attention-weighted features, obtain the first fused feature. Based on the third-layer output of the detail path and the third-layer output of the context path, obtain the second fused feature. S44. Input the first fusion feature into the upsampling unit of the decoding end to obtain the sampled feature. Input the second fusion feature into the skip connection feature alignment unit to obtain the aligned feature. Input the sampled feature and the aligned feature together into the skip connection fusion convolution unit to obtain the fusion convolution feature. Input the fusion convolution feature into the prediction head to obtain the blood vessel prominence feature map. S45. Perform main branch convolution mapping and auxiliary branch convolution mapping on the vascular salience feature map to obtain the vascular region probability map and the vascular centerline and vascular boundary probability map, respectively.

5. The forearm superficial vessel segmentation method based on NSVA-NET deep learning network according to claim 4, characterized in that, S42 inputs the second-layer output of the detail path and the second-layer output of the context path into the NSVA module to obtain attention-weighted features, including: S421. Combine the second-layer output of the detail path with the second-layer output of the context path into a single feature map; S422. Input the combined feature maps into the channel attention branch and the spatial attention branch respectively; S423. In the channel attention branch, two channel description vectors are obtained by performing global average pooling and global max pooling on the input feature map. The two channel description vectors are input into a shared multilayer perceptron, which outputs a channel weight vector and normalizes it through the Sigmoid function to obtain the channel attention weights. S424. In the spatial attention branch, the spatial feature map is obtained by calculating the local contrast and gradient of the input feature map. The spatial feature map is concatenated with the channel pooling result and then processed by convolution and the Sigmoid function to obtain the spatial attention weights. S425. Apply the channel attention weights and spatial attention weights to the input feature map respectively to obtain the attention-weighted features.

6. The forearm superficial vessel segmentation method based on NSVA-NET deep learning network according to claim 4, characterized in that, S43 obtains the first fused feature based on the third-layer output of the detail path, the third-layer output of the context path, and the attention-weighted features, including: S431. After concatenating the third-layer output of the detail path, the third-layer output of the context path, and the attention-weighted features, channel compression is performed by 1×1 convolution, and the proportion of different path features in the fusion result is controlled by a gating mechanism to obtain the first fused feature. S43 obtains a second fused feature based on the third-layer output of the detail path and the third-layer output of the context path, including: S432. After concatenating the third-layer output of the detail path and the third-layer output of the context path, the channels are compressed by 1×1 convolution, and the proportion of different path features in the fusion result is controlled by a gating mechanism to obtain the second fusion feature.

7. The forearm superficial vessel segmentation method based on NSVA-NET deep learning network according to claim 4, characterized in that, S5 performs threshold segmentation, connected component analysis, and skeletonization on the segmentation probability map to remove artifacts and isolated noise, obtaining the vessel segmentation result and centerline, including: S51. Binarize the probability map of the blood vessel region using a preset threshold to obtain a binary blood vessel mask. S52. Perform connected component analysis on the binary blood vessel mask, count the number of pixels in each connected region, and remove isolated small connected regions with an area smaller than a preset threshold. S53. Based on the preserved vascular region, a morphological refinement algorithm is used for skeletonization processing; S54. Perform morphological opening and closing operations and local curvature-based filtering operations on the skeletonization results to remove pseudo-branches and sharp burrs with lengths less than a preset threshold, smooth the shape of the vessel centerline, and obtain the vessel segmentation results and centerline.

8. A forearm superficial vessel segmentation device based on an NSVA-NET deep learning network, wherein the forearm superficial vessel segmentation device based on an NSVA-NET deep learning network is used to implement the forearm superficial vessel segmentation method based on an NSVA-NET deep learning network as described in any one of claims 1-7, characterized in that, The device includes: The acquisition unit is used to acquire near-infrared images of superficial blood vessels in the subject's forearm using a near-infrared light source and a near-infrared camera; The preprocessing unit is used to denoise, enhance contrast, crop the forearm region and normalize the near-infrared image to obtain a preprocessed image. The enhancement unit is used to perform two-stage image enhancement, RCAE and CLAHE, on the preprocessed image to obtain the enhanced image; The segmentation unit is used to input the enhanced image into the pre-trained NSVA-NET deep learning network to obtain a segmentation probability map of superficial blood vessels in the forearm; The post-processing unit is used to perform threshold segmentation, connected component analysis and skeletonization on the segmentation probability map, remove artifacts and isolated noise, and obtain the blood vessel segmentation result and centerline. The display unit is used to overlay the blood vessel segmentation results and centerline with the near-infrared image and output the final segmented image.

9. A forearm superficial blood vessel segmentation device based on the NSVA-NET deep learning network, characterized in that, The forearm superficial vessel segmentation device based on the NSVA-NET deep learning network includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images