Hand vein segmentation method based on deep snakelike high-resolution network

Abstract

The invention relates to the technical field of hand vein image segmentation, in particular to a hand vein segmentation method based on a deep snakelike high-resolution network. The method comprises the following steps: designing a hand vein image collected by a device for collecting the hand vein image to construct a data set; marking images in the data set, and dividing the data set into a training set and a test set; a deep snakelike high-resolution network is constructed, the deep snakelike high-resolution network comprises a high-resolution path, an SE attention mechanism module, an encoder and a decoder, and the hand vein image in the training set is input into the constructed network for hand vein segmentation to obtain a segmentation feature map; constructing a loss function, and optimizing the deep snakelike high-resolution network; and inputting the hand vein image in the test set into the optimized deep snakelike high-resolution network to obtain a final hand vein image segmentation result. According to the method, the complex slender structure of the hand vein can be better captured, and then the slender bent blood vessel in the hand vein image can be more accurately segmented.

Description

Technical Field

[0001] The present invention relates to the technical field of hand back vein image segmentation, and in particular to a hand back vein segmentation method based on a deep serpentine high-resolution network. Background Art

[0002] Venipuncture is a common clinical procedure for intravenous treatment and blood sample collection, which relies on the precise judgment and experience of medical professionals to locate and puncture veins. However, factors such as skin color and obesity may make vein detection more complicated, often leading to puncture failure. Accurate and efficient segmentation of veins in dorsal hand images can significantly reduce these risks. In addition, in our increasingly digital world, biometric features such as facial expressions, fingerprints, and dorsal hand veins are widely used for identity authentication. Identity recognition based on dorsal hand veins is particularly promising in applications such as mobile payment and access control due to its high detection accuracy, strong anti-counterfeiting ability, and wide acceptance. However, dorsal hand vein segmentation faces many challenges in data collection, such as collection difficulty, complex and time-consuming annotation process, and scarcity of relevant public datasets. Existing datasets still have problems such as low image contrast and drastic brightness changes, which make it difficult to annotate dorsal hand veins. These challenges have caused a serious data shortage and hindered research progress in this field.

[0003] Early studies on vessel segmentation mainly used traditional methods such as threshold-based segmentation, edge detection techniques, mathematical morphology, and region-based segmentation. Although these traditional methods are effective in some cases, they often have difficulties in processing complex images, resulting in inaccurate segmentation of vessel details and curved parts. For example, threshold-based methods are very sensitive to changes in illumination and skin texture, while edge detection methods may not accurately capture the continuity of blood vessels under noise interference. In addition, region-based methods usually rely on thresholds, are sensitive to noise, and have challenges in the selection of initial seeds and lack the ability to handle complex boundaries, which may lead to incomplete capture of important vascular information.

[0004] The emergence of convolutional neural networks (CNNs), known for their feature learning capabilities, has significantly improved the accuracy and efficiency of various vascular segmentation tasks and promoted the development of medical image analysis. However, due to changes in imaging angles and lighting conditions, traditional fixed-shape convolution kernels often have difficulty effectively capturing the local features of slender and curved blood vessels.

[0005] The introduction of the U-Net architecture marked an important milestone, leading to the widespread adoption of encoder-decoder structures with skip connections, which have become the basis of current state-of-the-art methods. These architectures use downsampling and convolution operations to extract higher-level semantic information, while using skip connections to retain low-level features from the encoder and combine them with high-level features in the decoder. However, despite its remarkable performance in many applications, this method is still prone to discontinuities or omissions in the segmentation of thin vessels in low-contrast images, and factors such as skin texture or hair may also introduce segmentation errors. Current encoder-decoder architectures face two major problems in vessel segmentation: (1) Skip connections may cause the loss of positional information of thin vessels. In the case of low-contrast and various artifacts in the back of the hand vein image, it often causes discontinuities or omissions of blood vessels, making it difficult for shallow networks to accurately depict thin vessels, resulting in a large number of false positives. (2) Although the downsampling process in these architectures can reduce computational costs and extract higher-level semantic information, it also leads to a reduction in spatial resolution, thereby losing high-resolution spatial details that are critical for the segmentation of low-pixel ratio and low-contrast vessels.

[0006] Therefore, the present invention provides a dorsal hand vein segmentation method based on a deep serpentine high-resolution network to avoid spatial information loss, and needs to improve the flexibility and feature extraction ability of the convolution kernel, so as to better capture the complex and slender structure of the dorsal hand vein and accurately segment the slender and curved blood vessels in the dorsal hand vein image. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention develops a dorsal hand vein segmentation method based on a deep serpentine high-resolution network, the main purpose of which is to better capture the complex and slender structure of the dorsal hand vein and accurately segment the slender and curved blood vessels in the dorsal hand vein image.

[0008] The technical solution of the present invention to solve the technical problem is: S1. Design a device to collect hand vein images and construct a dataset using the hand vein images collected by the hand vein image device. ; S2. Dataset The hand back vein images in the dataset are manually annotated to obtain the true labels of the hand back vein images. The hand vein images in the image are divided into training sets according to the proportion and test set ; S3. Build a deep snake-shaped high-resolution network. The deep snake-shaped high-resolution network includes a high-resolution path, a SE attention mechanism module, an encoder, and a decoder. The dorsal hand vein image is input into a deep snake-shaped high-resolution network for dorsal hand vein segmentation to obtain a segmentation feature map; S4. Construct a loss function, which is composed of binary cross entropy loss and weighted focal loss. Continuously adjust the parameters in the deep snake high-resolution network to gradually reduce the loss function, thereby optimizing the deep snake high-resolution network. S5. Test set The dorsal hand vein image in is input into the optimized deep snake-shaped high-resolution network to obtain the final dorsal hand vein image segmentation result.

[0009] S1 is as follows: The hand back vein imaging device includes a CMOS camera, a data acquisition box, a hand placement table and two LED light boards. The hand placement table is arranged at the bottom of the data acquisition box, the CMOS camera is arranged at the top of the data acquisition box facing the hand placement table, the LED light boards are arranged at both sides of the data acquisition box facing the hand placement table, and an opening is arranged at the bottom of one side of the data acquisition box. The process of obtaining the back of hand vein image by the back of hand vein image device is as follows: insert the hand into the opening on one side of the data acquisition box, place the back of the hand facing up on the hand placement table, then illuminate the back of the hand through the LED light board, and take a picture of the back of the hand with the CMOS camera to obtain the back of hand vein image; All the obtained hand back vein images are aggregated to construct a dataset .

[0010] S3 is as follows: S3.1. Build a deep snake-like high-resolution network: (1) Both the encoder and decoder are divided into four layers, and the encoder and decoder share the fourth layer. Each layer includes a dynamic snake convolution DSConv and an optimized residual block; Among them, the optimized residual module includes 3×3 convolution, DropBlock regularization module, GN normalization layer, ReLU activation function, 3×3 convolution, DropBlock regularization module, GN normalization layer, ReLU activation function in sequence; (2) The high-resolution path is divided into three layers. The first layer contains five MAM multi-resolution aggregation modules, the second layer contains three MAM multi-resolution aggregation modules, and the third layer contains one MAM multi-resolution aggregation module. Among them, the MAM multi-resolution aggregation module includes horizontal DSConv, vertical DSConv, 3×3 convolution, three groups of GN normalization layers and ReLU and BottleNeck bottleneck modules; The BottleNeck bottleneck module includes 1×1 convolution, GN normalization layer, ReLU activation function, 3×3 convolution, GN normalization layer, ReLU activation function, 1×1 convolution, GN normalization layer, ReLU activation function; Image the veins on the back of your hand Input into a deep snake high-resolution network, Represents the training set any image in .

[0011] S3.2. Image Input to the encoder, image First, it passes through the first layer of the encoder, and is processed by the dynamic snake convolution and optimized residual module in the first layer of the encoder to obtain the feature map output by the first layer. ; The feature map After the Maxpooling operation, the image is input to the second layer of the encoder. After the dynamic snake convolution and optimized residual module in the second layer of the encoder, the feature map of the second layer output is obtained. ; Similarly, the feature map After the Maxpooling operation, it is input to the third layer of the encoder to obtain the feature map output by the third layer. , the feature map After the Maxpooling operation, the fourth layer of the encoder is input to obtain the feature map output by the fourth layer. .

[0012] The dynamic snake convolution and optimized residual module are as follows: (1) Dynamic Snake Convolution DSConv: Dynamic snake convolution DSConv captures feature maps from both the x-axis and y-axis, i.e., the input feature maps , on the x-axis, a 3×3 convolution kernel is used to linearize along the x-axis to form a straight line consisting of nine grid points. The position of each grid point on the x-axis is expressed as ,in , Indicates the horizontal distance from the center grid. Represents the coordinate set of the dynamic snake convolution kernel on the x-axis, Represents the coordinate point of the dynamic snake convolution kernel on the x-axis, Represents the index of the number of grid points on the x-axis. Similarly, linearization on the y-axis also forms a straight line consisting of nine grid points, thus obtaining a 9×9 area. The position of each grid point on the y-axis is represented by ,in , Indicates the horizontal distance from the center grid. Represents the coordinate set of the dynamic snake convolution kernel on the y-axis, Represents the coordinate point of the dynamic snake convolution kernel on the y-axis, Indicates the index of the grid points on the y-axis; Introduce stepwise offsets for the x-axis and y-axis respectively and To determine the position of the next grid point, the change along the x-axis obtains the convolution kernel The representation is as follows: , The change along the y-axis gets the convolution kernel The representation is as follows: , in, and Represents the center grid of the convolution kernel coordinates on the x-axis, and Represents the center grid of the convolution kernel coordinates on the y-axis, It represents the point on the x-axis that is c away from the center grid point after the convolution kernel is linearized. It means that the distance from the center grid point on the x-axis after linearizing the convolution kernel is point, It means that the distance from the center grid point on the x-axis after linearizing the convolution kernel is point, It represents the point on the y-axis that is c away from the center grid point after the convolution kernel is linearized. It means that the distance from the center grid point on the y-axis after linearizing the convolution kernel is point, It means that the distance from the center grid point on the y-axis after linearizing the convolution kernel is point; The position of the convolution kernel is adjusted by bilinear interpolation. The calculation formula is as follows: , in, express and The decimal place of express and The integer position of Represents a bilinear interpolation operation; (2) Optimizing the residual module: The feature map is input into the optimized residual module. The result of the feature map after 1×1 convolution, GN normalization layer, ReLU operation, 3×3 convolution, GN normalization layer, ReLU operation, 1×1 convolution, and GN normalization layer is added to the input of the optimized residual module directly input through the shortcut connection, and then undergoes the Relu operation to obtain the output of the optimized residual module.

[0013] S3.3. Feature map , and Input to the first, second and third layers of the high-resolution path respectively, and output the corresponding feature maps respectively , and , Represents the feature map output by the fifth MAM module in the first layer of the high-resolution path, Represents the feature map output by the third MAM module in the second layer of the high-resolution path, Represents the feature map output by the first MAM module in the third layer of the high-resolution path; Among them, the MAM module combines the high-resolution feature map obtained by maximum pooling, the low-resolution feature map obtained by upsampling, and the medium-resolution feature map from the previous stage. The MAM module specifically includes the following four input situations: (1) The high-resolution feature map obtained by maximum pooling, the low-resolution feature map obtained by upsampling, and the medium-resolution feature map from the previous stage are fused and input into the MAM module; (2) The high-resolution feature map obtained by maximum pooling and the medium-resolution feature map from the previous stage are fused and input into the MAM module; (3) The low-resolution feature map obtained by upsampling and the medium-resolution feature map from the previous stage are fused and input into the MAM module; (4) Input the medium-resolution feature map from the previous stage into the MAM module; The input and output of each MAM module in each layer of the high-resolution path are as follows: The input of the first MAM module in the first layer is the feature map , the output feature map is ; The input of the second MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is ; The input of the first MAM module in the second layer is the feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the second MAM module in the second layer is the feature map , feature map after upsampling operation And the feature map after the maximum pooling operation , the output feature map is ; The input of the first MAM module in the third layer is the feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the third MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is ; The input of the fourth MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is ; The input of the third MAM module in the second layer is the feature map , feature map after upsampling operation And the feature map after the maximum pooling operation , the output feature map is ; The input of the fifth MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is .

[0014] The MAM module operations are as follows: The inputs of the MAM module in four cases are input into the MAM module, and then go through 3×3 convolution, horizontal DSConv, and vertical DSConv in parallel. The output results are input into a set of GN normalization layers and ReLU activation function layers respectively. The three sets of outputs are fused and input into the BottleNeck module, and finally the feature map is output.

[0015] S3.4 combines the output in the high-resolution path with the feature map output by the fourth layer of the encoder Input to the decoder, the input and output of each layer in the decoder are as follows: The input of the fourth layer of the decoder is the feature map output by the fourth layer of the encoder , the output feature map is ; The input of the third layer of the decoder is the feature map after upsampling operation , feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the second layer of the decoder is the feature map after upsampling operation , feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the first layer of the decoder is the feature map after upsampling operation and feature map , the output feature map is .

[0016] S3.5. Feature map , , and are input into the SE module respectively, and each SE module outputs the corresponding segmented image, which are , , and , and then perform weighted calculation on the four segmented images to obtain the final predicted segmented image , the specific calculation is as follows: , in, represents the number of segmented images, , Indicates Segmented image, Indicates The weight of the segmented image, , , represents the feature extraction module, represents the Sigmoid activation function, Represents a multiplication operation.

[0017] S4 is as follows: The segmentation feature map and the true label Input to the constructed loss function The calculation formula is as follows: , , , in, represents the binary cross entropy loss, represents the weighted focal loss, represents the weight of the hand back vein category, Represents the focus coefficient used to adjust the emphasis on difficult and easy samples, the true label It is divided into veins and background, with veins being recorded as 1 and background being recorded as 0. Represents the weight parameter.

[0018] The effects provided in the content of the invention are only the effects of the embodiments, rather than all the effects of the invention. The above technical solution has the following advantages or beneficial effects: The present invention proposes a dorsal hand vein segmentation method based on a deep serpentine high-resolution network. This method can avoid the loss of spatial information of the dorsal hand vein, further enhance the focusing ability of the model in dorsal hand vein segmentation, enable it to adaptively focus on the most critical channel features, and improve the flexibility and feature extraction ability of the convolution kernel, so as to better capture the complex and slender structure of the dorsal hand vein, providing new ideas for subsequent vascular segmentation research and engineering applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0020] Figure 1 It is a device for collecting images of veins on the back of the hand.

[0021] Figure 2 The back of the hand vein image and its corresponding true label.

[0022] Figure 3 This is a high-resolution network structure diagram based on deep snake.

[0023] Figure 4 Comparison of segmentation results of different methods. DETAILED DESCRIPTION

[0024] In order to clearly illustrate the technical features of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below.

[0025] Example 1 like Figures 1 to 3 As shown in FIG. 1 , a hand back vein segmentation method based on a deep snake-shaped high-resolution network is shown as follows: S1. Design a device to collect hand vein images and construct a dataset using the hand vein images collected by the hand vein image device. ; S2. Dataset The hand back vein images in the dataset are manually annotated to obtain the true labels of the hand back vein images. The hand vein images in the image are divided into training sets according to the proportion and test set ; S3. Build a deep snake-shaped high-resolution network. The deep snake-shaped high-resolution network includes a high-resolution path, a SE attention mechanism module, an encoder, and a decoder. The dorsal hand vein image is input into a deep snake-shaped high-resolution network for dorsal hand vein segmentation to obtain a segmentation feature map; S4. Construct a loss function, which is composed of binary cross entropy loss and weighted focal loss. Continuously adjust the parameters in the deep snake high-resolution network to gradually reduce the loss function, thereby optimizing the deep snake high-resolution network. S5. Test set The dorsal hand vein image in is input into the optimized deep snake-shaped high-resolution network to obtain the final dorsal hand vein image segmentation result.

[0026] S1 is as follows: Medical imaging studies have shown that when the dorsal veins of the hand are exposed to near-infrared light with a wavelength of 700nm to 1100nm, the light can penetrate about 3mm thick skin tissue and reach the dorsal veins. Within this wavelength range, the hemoglobin in the veins absorbs light waves very strongly, while the absorption of adjacent tissues is relatively weak. Therefore, the equipment used in the present invention's research includes a dual CMOS camera with a resolution of 5 million pixels, which can reach a maximum of pixels and can operate at a maximum frame rate of 30 frames per second. The camera is equipped with an infrared long-pass filter. The data acquisition box was designed using SolidWorks. Figure 1 As shown, the dimensions of the box are 25 cm in height, 17 cm in length, and 15 cm in width. It is manufactured using 3D printing technology. The inner wall of the box is coated with black paint to absorb infrared light, thereby forming a dark background and enhancing the visibility of the back of the hand image. Each side of the box is equipped with an infrared light panel consisting of 24 1W infrared LEDs, which can penetrate the surface of the hand and reveal the vascular structure of the entire back of the hand. The infrared LEDs used are standard 850nm bulbs, which are commonly used in monitoring applications and are installed at an angle of approximately 50° to the plane of the hand. Such a layout ensures uniform coverage of infrared light on the entire back of the hand surface, thereby achieving consistent vein texture in the captured image. The SDNU dataset created by the present invention team contains 102 participants, including 55 males and 47 females. Three samples were collected for each hand, so a total of 612 back of hand vein images were obtained.

[0027] S2 is as follows: After data collection, the present invention uses Labelme software for manual annotation. The data set is divided into 504 images for training and 108 images for testing. The hand back vein images obtained by the acquisition device of the present invention and the manually annotated real labels are as follows: Figure 2 As shown, eight groups of hand back vein images and the corresponding true labels of the hand back vein images are provided.

[0028] S3 is as follows: S3.1. Build a deep snake-like high-resolution network: (1) Both the encoder and decoder are divided into four layers, and the encoder and decoder share the fourth layer. Each layer includes a dynamic snake convolution DSConv and an optimized residual block; Among them, the optimized residual module includes 3×3 convolution, DropBlock regularization module, GN normalization layer, ReLU activation function, 3×3 convolution, DropBlock regularization module, GN normalization layer, ReLU activation function in sequence; (2) The high-resolution path is divided into three layers. The first layer contains five MAM multi-resolution aggregation modules, the second layer contains three MAM multi-resolution aggregation modules, and the third layer contains one MAM multi-resolution aggregation module. Among them, the MAM multi-resolution aggregation module includes horizontal DSConv, vertical DSConv, 3×3 convolution, three groups of GN normalization layers and ReLU and BottleNeck bottleneck modules; The BottleNeck bottleneck module includes 1×1 convolution, GN normalization layer, ReLU activation function, 3×3 convolution, GN normalization layer, ReLU activation function, 1×1 convolution, GN normalization layer, ReLU activation function; Image the veins on the back of your hand Input into a deep snake high-resolution network, Represents the training set any image in .

[0029] S3.2. Image Input to the encoder, image First, it passes through the first layer of the encoder, and is processed by the dynamic snake convolution and optimized residual module in the first layer of the encoder to obtain the feature map output by the first layer. ; The feature map After the Maxpooling operation, the image is input to the second layer of the encoder. After the dynamic snake convolution and optimized residual module in the second layer of the encoder, the feature map of the second layer output is obtained. ; Similarly, the feature map After the Maxpooling operation, it is input to the third layer of the encoder to obtain the feature map output by the third layer. , the feature map After the Maxpooling operation, the fourth layer of the encoder is input to obtain the feature map output by the fourth layer. .

[0030] The dynamic snake convolution and optimized residual module are as follows: (1) Dynamic Snake Convolution DSConv: Dynamic snake convolution DSConv captures feature maps from both the x-axis and y-axis, i.e., the input feature maps , on the x-axis, a 3×3 convolution kernel is used to linearize along the x-axis to form a straight line consisting of nine grid points. The position of each grid point on the x-axis is expressed as ,in , Indicates the horizontal distance from the center grid. Represents the coordinate set of the dynamic snake convolution kernel on the x-axis, Represents the coordinate point of the dynamic snake convolution kernel on the x-axis, Represents the index of the number of grid points on the x-axis. Similarly, linearization on the y-axis also forms a straight line consisting of nine grid points, thus obtaining a 9×9 area. The position of each grid point on the y-axis is represented by ,in , Indicates the horizontal distance from the center grid. Represents the coordinate set of the dynamic snake convolution kernel on the y-axis, Represents the coordinate point of the dynamic snake convolution kernel on the y-axis, Indicates the index of the grid points on the y-axis; Introduce stepwise offsets for the x-axis and y-axis respectively and To determine the position of the next grid point, the change along the x-axis obtains the convolution kernel The representation is as follows: , The change along the y-axis gets the convolution kernel The representation is as follows: , in, and Represents the center grid of the convolution kernel coordinates on the x-axis, and Represents the center grid of the convolution kernel coordinates on the y-axis, It represents the point on the x-axis that is c away from the center grid point after the convolution kernel is linearized. It means that the distance from the center grid point on the x-axis after linearizing the convolution kernel is point, It means that the distance from the center grid point on the x-axis after linearizing the convolution kernel is point, It represents the point on the y-axis that is c away from the center grid point after the convolution kernel is linearized. It means that the distance from the center grid point on the y-axis after linearizing the convolution kernel is point, It means that the distance from the center grid point on the y-axis after linearizing the convolution kernel is point; The position of the convolution kernel is adjusted by bilinear interpolation. The calculation formula is as follows: , in, express and The decimal place of express and The integer position of Represents a bilinear interpolation operation; (2) Optimizing the residual module: The feature map is input into the optimized residual module. The result of the feature map after 1×1 convolution, GN normalization layer, ReLU operation, 3×3 convolution, GN normalization layer, ReLU operation, 1×1 convolution, and GN normalization layer is added to the input of the optimized residual module directly input through the shortcut connection, and then undergoes the Relu operation to obtain the output of the optimized residual module.

[0031] S3.3. Feature map , and Input to the first, second and third layers of the high-resolution path respectively, and output the corresponding feature maps respectively , and , Represents the feature map output by the fifth MAM module in the first layer of the high-resolution path, Represents the feature map output by the third MAM module in the second layer of the high-resolution path, Represents the feature map output by the first MAM module in the third layer of the high-resolution path; Among them, the MAM module combines the high-resolution feature map obtained by maximum pooling, the low-resolution feature map obtained by upsampling, and the medium-resolution feature map from the previous stage. The MAM module specifically includes the following four input situations: (1) The high-resolution feature map obtained by maximum pooling, the low-resolution feature map obtained by upsampling, and the medium-resolution feature map from the previous stage are fused and input into the MAM module; (2) The high-resolution feature map obtained by maximum pooling and the medium-resolution feature map from the previous stage are fused and input into the MAM module; (3) The low-resolution feature map obtained by upsampling and the medium-resolution feature map from the previous stage are fused and input into the MAM module; (4) Input the medium-resolution feature map from the previous stage into the MAM module; The input and output of each MAM module in each layer of the high-resolution path are as follows: The input of the first MAM module in the first layer is the feature map , the output feature map is ; The input of the second MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is ; The input of the first MAM module in the second layer is the feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the second MAM module in the second layer is the feature map , feature map after upsampling operation And the feature map after the maximum pooling operation , the output feature map is ; The input of the first MAM module in the third layer is the feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the third MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is ; The input of the fourth MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is ; The input of the third MAM module in the second layer is the feature map , feature map after upsampling operation And the feature map after the maximum pooling operation , the output feature map is ; The input of the fifth MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is .

[0032] The MAM module operations are as follows: The inputs of the MAM module in four cases are input into the MAM module, and then go through 3×3 convolution, horizontal DSConv, and vertical DSConv in parallel. The output results are input into a set of GN normalization layers and ReLU activation function layers respectively. The three sets of outputs are fused and input into the BottleNeck module, and finally the feature map is output.

[0033] S3.4 combines the output in the high-resolution path with the feature map output by the fourth layer of the encoder Input to the decoder, the input and output of each layer in the decoder are as follows: The input of the fourth layer of the decoder is the feature map output by the fourth layer of the encoder , the output feature map is ; The input of the third layer of the decoder is the feature map after upsampling operation , feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the second layer of the decoder is the feature map after upsampling operation , feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the first layer of the decoder is the feature map after upsampling operation and feature map , the output feature map is .

[0034] S3.5. Feature map , , and are input into the SE module respectively, and each SE module outputs the corresponding segmented image, which are , , and , and then perform weighted calculation on the four segmented images to obtain the final predicted segmented image , the specific calculation is as follows: , in, represents the number of segmented images, , Indicates Segmented image, Indicates The weight of the segmented image, , , represents the feature extraction module, represents the Sigmoid activation function, Represents a multiplication operation.

[0035] S4 is as follows: The segmentation feature map and the true label Input to the constructed loss function The calculation formula is as follows: , , , in, represents the binary cross entropy loss, represents the weighted focal loss, represents the weight of the hand back vein category, Represents the focus coefficient used to adjust the emphasis on difficult and easy samples, the true label It is divided into veins and background, with veins being recorded as 1 and background being recorded as 0. Represents the weight parameter.

[0036] The model was trained using the PyTorch framework on an NVIDIA GeForce RTX 3090 GPU with 24GB of video memory. The deep snake high-resolution network framework was designed to handle basic operations, including data tensor input, loss calculation, backpropagation, and model parameter updates. During training, the images were uniformly resized to To adapt to memory limitations, the Adam algorithm was used in the optimization process. The initial learning rate was set to 0.0001, the batch size was 1, the total training cycle was 600 times, and a 10\% learning rate decay strategy was adopted when the training loss did not change within 10 iterations.

[0037] In order to evaluate the segmentation performance of the deep snake high-resolution network, the present invention has conducted an evaluation on the collected data set, using indicators such as accuracy, recall, specificity, Dice coefficient and intersection over union. The present invention compares the deep snake high-resolution network with a variety of advanced segmentation methods, including U-net, U-net's improved network U-net++, attention U-net, residual U-net, FCN8s, channel attention network CENet, weighted multimodal feature fusion network WS-DMF, full-resolution U-net FR-Unet and snake convolution network DSCNet. The performance comparison results are shown in Table 1, where the best result is shown in bold and the second best result is underlined.

[0038] Table 1 Performance comparison of different algorithms on the data set collected by the present invention As shown in Table 1, on the dataset collected by the present invention, DSHR-Net outperforms other models in recall rate, Dice coefficient and IoU index, reaching 0.8294, 0.8436 and 0.7297 respectively. Compared with the U-Net baseline model, these indicators increased by 0.0329, 0.0345 and 0.0489 respectively, highlighting the effectiveness of the deep snake high-resolution network. It is worth noting that the deep snake high-resolution network achieves a higher recall rate, which shows that it has a strong ability in extracting subtle blood vessels and boundary pixels. However, the specificity score is relatively low, indicating that some non-vascular pixels are misclassified. Although the deep snake high-resolution network enhances the extraction of subtle vascular structures, it inevitably increases the tolerance to false positives. Although its accuracy is only 0.0034 lower than the best performing network, DSHR-Net still shows strong performance relative to the baseline network. Overall, the deep snake high-resolution network shows excellent performance in the dorsal hand vein segmentation task.

[0039] Example 2 In order to demonstrate the beneficial effects of the present invention, the segmentation results obtained by the method of the present invention and the existing methods are experimentally compared on the basis of the data set collected by the present invention. In order to evaluate the segmentation performance of the deep snake-shaped high-resolution network, the segmentation performance evaluation indicators are accuracy, recall, specificity, Dice coefficient and intersection over union. The existing methods include U-Net, Atten U-Net, U-Net++, DSCNet and FR-Unet. The comparison results are shown in Figure 2. Figure 4 As shown, Figure 4 Five groups of control results are given in the figure, which compares the hand back vein image segmentation results obtained by the method of the present invention and various existing methods with the real label Label of the input image Image. It can be seen intuitively that there are most differences between the segmentation results obtained by the existing methods and the real labels, which can be seen in the red box marks. The segmentation results obtained by the method of the present invention are almost consistent with the real labels. It can be seen that the accuracy of hand back vein image segmentation can be effectively improved by the method of the present invention.

[0040] Although the above describes the specific implementation mode of the invention in conjunction with the drawings, it is not intended to limit the scope of protection of the invention. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. A hand back vein segmentation method based on deep snake-like high-resolution network, characterized in that: The following steps are involved: S1. Design a device to collect hand vein images and construct a dataset using the hand vein images collected by the hand vein image device. ; S2. Dataset The hand back vein images in the dataset are manually annotated to obtain the true labels of the hand back vein images. The hand vein images in the image are divided into training sets according to the proportion and test set ; S3. Build a deep snake-shaped high-resolution network. The deep snake-shaped high-resolution network includes a high-resolution path, a SE attention mechanism module, an encoder, and a decoder. The dorsal hand vein image is input into a deep snake-shaped high-resolution network for dorsal hand vein segmentation to obtain a segmentation feature map; S4. Construct a loss function, which is composed of binary cross entropy loss and weighted focal loss. Continuously adjust the parameters in the deep snake high-resolution network to gradually reduce the loss function, thereby optimizing the deep snake high-resolution network. S5. Test set The dorsal hand vein image in is input into the optimized deep snake-shaped high-resolution network to obtain the final dorsal hand vein image segmentation result.

2. According to claim 1, a method for hand back vein segmentation based on deep snake-like high-resolution network is characterized in that: S1 is as follows: The hand back vein imaging device includes a CMOS camera, a data acquisition box, a hand placement table and two LED light boards. The hand placement table is arranged at the bottom of the data acquisition box, the CMOS camera is arranged at the top of the data acquisition box facing the hand placement table, the LED light boards are arranged at both sides of the data acquisition box facing the hand placement table, and an opening is arranged at the bottom of one side of the data acquisition box. The process of obtaining the back of hand vein image by the back of hand vein image device is as follows: insert the hand into the opening on one side of the data acquisition box, place the back of the hand facing up on the hand placement table, then illuminate the back of the hand through the LED light board, and take a picture of the back of the hand with the CMOS camera to obtain the back of hand vein image; All the obtained hand back vein images are aggregated to construct a dataset .

3. The method for hand vein segmentation based on deep serpentine high-resolution network according to claim 2, characterized in that S3 The details are as follows: S3.

1. Build a deep snake-like high-resolution network: (1) Both the encoder and decoder are divided into four layers, and the encoder and decoder share the fourth layer. Each layer includes a dynamic snake convolution DSConv and an optimized residual block; Among them, the optimized residual module includes 3×3 convolution, DropBlock regularization module, GN normalization layer, ReLU activation function, 3×3 convolution, DropBlock regularization module, GN normalization layer, ReLU activation function in sequence; (2) The high-resolution path is divided into three layers. The first layer contains five MAM multi-resolution aggregation modules, the second layer contains three MAM multi-resolution aggregation modules, and the third layer contains one MAM multi-resolution aggregation module. Among them, the MAM multi-resolution aggregation module includes horizontal DSConv, vertical DSConv, 3×3 convolution, three groups of GN normalization layers and ReLU and BottleNeck bottleneck modules; The BottleNeck bottleneck module includes 1×1 convolution, GN normalization layer, ReLU activation function, 3×3 convolution, GN normalization layer, ReLU activation function, 1×1 convolution, GN normalization layer, ReLU activation function; Image the veins on the back of your hand Input into a deep snake high-resolution network, Represents the training set any image in .

4. The method for hand vein segmentation based on deep serpentine high-resolution network according to claim 3 is characterized by: S3.

2. Image Input to the encoder, image First, it passes through the first layer of the encoder, and is processed by the dynamic snake convolution and optimized residual module in the first layer of the encoder to obtain the feature map output by the first layer. ; The feature map After the Maxpooling operation, the image is input to the second layer of the encoder. After the dynamic snake convolution and optimized residual module in the second layer of the encoder, the feature map of the second layer output is obtained. ; Similarly, the feature map After the Maxpooling operation, it is input to the third layer of the encoder to obtain the feature map output by the third layer. , the feature map After the Maxpooling operation, the fourth layer of the encoder is input to obtain the feature map output by the fourth layer. .

5. The method for hand back vein segmentation based on deep serpentine high-resolution network according to claim 4 is characterized in that: The dynamic snake convolution and optimized residual module are as follows: (1) Dynamic Snake Convolution DSConv: Dynamic snake convolution DSConv captures feature maps from both the x-axis and y-axis, i.e., the input feature maps , on the x-axis, a 3×3 convolution kernel is used to linearize along the x-axis to form a straight line consisting of nine grid points. The position of each grid point on the x-axis is expressed as ,in , Indicates the horizontal distance from the center grid. Represents the coordinate set of the dynamic snake convolution kernel on the x-axis, Represents the coordinate point of the dynamic snake convolution kernel on the x-axis, Represents the index of the number of grid points on the x-axis. Similarly, linearization on the y-axis also forms a straight line consisting of nine grid points, thus obtaining a 9×9 area. The position of each grid point on the y-axis is represented by ,in , Indicates the horizontal distance from the center grid. Represents the coordinate set of the dynamic snake convolution kernel on the y-axis, Represents the coordinate point of the dynamic snake convolution kernel on the y-axis, Indicates the index of the grid points on the y-axis; Introduce stepwise offsets for the x-axis and y-axis respectively and To determine the position of the next grid point, the change along the x-axis obtains the convolution kernel The representation is as follows: ， The change along the y-axis gets the convolution kernel The representation is as follows: ， in, and Represents the center grid of the convolution kernel coordinates on the x-axis, and Represents the center grid of the convolution kernel coordinates on the y-axis, It represents the point on the x-axis that is c away from the center grid point after the convolution kernel is linearized. It means that the distance from the center grid point on the x-axis after linearizing the convolution kernel is point, It means that the distance from the center grid point on the x-axis after linearizing the convolution kernel is point, It represents the point on the y-axis that is c away from the center grid point after the convolution kernel is linearized. It means that the distance from the center grid point on the y-axis after linearizing the convolution kernel is point, It means that the distance from the center grid point on the y-axis after linearizing the convolution kernel is point; The position of the convolution kernel is adjusted by bilinear interpolation. The calculation formula is as follows: ， in, express and The decimal place of express and The integer position of Represents a bilinear interpolation operation; (2) Optimizing the residual module: The feature map is input into the optimized residual module. The result of the feature map after 1×1 convolution, GN normalization layer, ReLU operation, 3×3 convolution, GN normalization layer, ReLU operation, 1×1 convolution, and GN normalization layer is added to the input of the optimized residual module directly input through the shortcut connection, and then undergoes the Relu operation to obtain the output of the optimized residual module.

6. The method for hand back vein segmentation based on deep serpentine high-resolution network according to claim 5 is characterized by: S3.

3. Feature map , and Input to the first, second and third layers of the high-resolution path respectively, and output the corresponding feature maps respectively , and , Represents the feature map output by the fifth MAM module in the first layer of the high-resolution path, Represents the feature map output by the third MAM module in the second layer of the high-resolution path, Represents the feature map output by the first MAM module in the third layer of the high-resolution path; Among them, the MAM module combines the high-resolution feature map obtained by maximum pooling, the low-resolution feature map obtained by upsampling, and the medium-resolution feature map from the previous stage. The MAM module specifically includes the following four input situations: (1) The high-resolution feature map obtained by maximum pooling, the low-resolution feature map obtained by upsampling, and the medium-resolution feature map from the previous stage are fused and input into the MAM module; (2) The high-resolution feature map obtained by maximum pooling and the medium-resolution feature map from the previous stage are fused and input into the MAM module; (3) The low-resolution feature map obtained by upsampling and the medium-resolution feature map from the previous stage are fused and input into the MAM module; (4) Input the medium-resolution feature map from the previous stage into the MAM module; The input and output of each MAM module in each layer of the high-resolution path are as follows: The input of the first MAM module in the first layer is the feature map , the output feature map is ; The input of the second MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is ; The input of the first MAM module in the second layer is the feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the second MAM module in the second layer is the feature map , feature map after upsampling operation And the feature map after the maximum pooling operation , the output feature map is ; The input of the first MAM module in the third layer is the feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the third MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is ; The input of the fourth MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is ; The input of the third MAM module in the second layer is the feature map , feature map after upsampling operation And the feature map after the maximum pooling operation , the output feature map is ; The input of the fifth MAM module in the first layer is the feature map And the feature map after upsampling operation , the output feature map is .

7. The method for hand back vein segmentation based on deep serpentine high-resolution network according to claim 6, characterized in that: The MAM module operations are as follows: The inputs of the MAM module in four cases are input into the MAM module, and then go through 3×3 convolution, horizontal DSConv, and vertical DSConv in parallel. The output results are input into a set of GN normalization layers and ReLU activation function layers respectively. The three sets of outputs are fused and input into the BottleNeck module, and finally the feature map is output.

8. The method for hand back vein segmentation based on deep serpentine high-resolution network according to claim 7 is characterized by: S3.4 combines the output in the high-resolution path with the feature map output by the fourth layer of the encoder Input to the decoder, the input and output of each layer in the decoder are as follows: The input of the fourth layer of the decoder is the feature map output by the fourth layer of the encoder , the output feature map is ; The input of the third layer of the decoder is the feature map after upsampling operation , feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the second layer of the decoder is the feature map after upsampling operation , feature map And the feature map after the maximum pooling operation , the output feature map is ; The input of the first layer of the decoder is the feature map after upsampling operation and feature map , the output feature map is .

9. The method for hand back vein segmentation based on deep serpentine high-resolution network according to claim 8, characterized in that: S3.

5. Feature map , , and are input into the SE module respectively, and each SE module outputs the corresponding segmented image, which are , , and , and then perform weighted calculation on the four segmented images to obtain the final predicted segmented image , the specific calculation is as follows: ， in, represents the number of segmented images, , Indicates Segmented image, Indicates The weight of the segmented image, , , represents the feature extraction module, represents the Sigmoid activation function, Represents a multiplication operation.

10. The method for hand back vein segmentation based on deep serpentine high-resolution network according to claim 9, characterized in that: S4 is as follows: The segmentation feature map and the true label Input to the constructed loss function The calculation formula is as follows: ， ， ， in, represents the binary cross entropy loss, represents the weighted focal loss, represents the weight of the hand back vein category, Represents the focus coefficient used to adjust the emphasis on difficult and easy samples, the true label It is divided into veins and background, with veins being recorded as 1 and background being recorded as 0. Represents the weight parameter.

Images