An automatic subcutaneous venous blood sampling method based on infrared imaging

Abstract

The present invention belongs to the field of image segmentation technology, and specifically relates to an automatic subcutaneous venous blood sampling method based on infrared imaging, the method comprising: deploying a first camera and a second camera; calibrating and positioning the two cameras; using the first camera to obtain a near-infrared image of the back of a user's hand; using a trained multi-dimensional decoupling network to perform vein segmentation on the near-infrared image of the back of the hand; using a positioning algorithm to process the segmented vein image to obtain the two-dimensional coordinates of the optimal blood sampling point; using the second camera to obtain a spatial position image of the back of the user's hand, and calculating the three-dimensional coordinates of the optimal blood sampling point based on the two-dimensional coordinates of the optimal blood sampling point; and sampling blood from the user based on the three-dimensional coordinates of the optimal blood sampling point. The present invention uses a multi-dimensional decoupling network to accurately segment the veins in the near-infrared image of the back of the hand, thereby improving the accuracy of blood sampling.

Description

Technical Field

[0001] The invention belongs to the technical field of image segmentation, and in particular relates to an automatic subcutaneous venous blood sampling method based on infrared imaging. Background Art

[0002] As a fundamental laboratory test method, venous blood sampling plays a crucial role in emergency care, treatment, and rehabilitation. Currently, clinical practice primarily relies on manual needle insertion by medical staff. Even with the continuous advancement of medical technology, manual injections remain a significant challenge for medical professionals. Even experienced nurses may not be able to immediately and accurately locate the patient's vein. When blood is drawn from young children or patients with other special medical conditions, venous blood collection can be stressful for both the patient and the patient. Repeated needle insertions inevitably increase the physical and emotional trauma of the patient. Failures are primarily due to improper operation caused by inexperienced medical staff and unexpected situations caused by complex procedures. With the rise of artificial intelligence, researchers have begun exploring intelligent and unmanned blood collection. This approach aims to effectively avoid direct contact between patients and medical staff, reducing the risk of secondary infection, while also improving the success rate of venous blood collection and reducing the pain of the patient during the blood collection process.

[0003] The first step in intelligentizing the blood collection process is to accurately determine the location of veins on the back of the hand. This requires using a binocular near-infrared camera to capture near-infrared images of the back of the hand. Stereo vision technology utilizes binocular cameras combined with appropriate algorithms to obtain depth information. The performance of traditional stereo vision algorithms depends on the choice of cost function, which describes the similarity between two pixels in a stereo image pair. Convolutional neural networks (CNNs) learn stereo image similarity directly from data. Although many researchers have made progress in this area, accurately finding stereo correspondences in stereo image pairs remains a significant challenge due to suboptimal lighting conditions, occlusions, and textureless areas. These issues often lead to inaccurate disparity calculations around object edges, which directly impacts segmentation results. Secondly, vein identification is required. Mainstream vein identification algorithms primarily extract features from three perspectives: shape, texture, and depth. Mathematical methods and vein image segmentation algorithms are methods for obtaining vein image shape information. These algorithms primarily mine and segment vein shape features from raw grayscale vein images. Therefore, obtaining a segmentation map of the grayscale shape features of the raw grayscale vein images is a key factor limiting their accuracy. Vein texture feature models mainly include the Local Grayscale Coding Model (LBP) and the Scale-Invariant Feature (SIFT) model. The Local Grayscale Coding Model (LBP) is widely used in the field of vein recognition, but its ability to extract sparse network structure features is limited, limiting its application in this field. Summary of the Invention

[0004] In order to solve the problems existing in the above-mentioned prior art, the present invention proposes an automatic subcutaneous venous blood sampling method based on infrared imaging, which includes: deploying a first camera and a second camera; calibrating and positioning the two cameras; using the first camera to obtain a near-infrared image of the back of the user's hand; using a trained multi-dimensional decoupling network to perform vein segmentation on the near-infrared image of the back of the hand; using a positioning algorithm to process the segmented vein image to obtain the two-dimensional coordinates of the best blood sampling point; using the second camera to obtain the spatial position image of the back of the user's hand, and calculating the three-dimensional coordinates of the best blood sampling point based on the two-dimensional coordinates of the best blood sampling point; and sampling blood from the user according to the three-dimensional coordinates of the best blood sampling point.

[0005] Beneficial effects of the present invention:

[0006] The present invention uses a multi-dimensional decoupling network to accurately segment veins in near-infrared images of the back of the hand; uses a positioning algorithm to accurately locate the optimal venous blood collection position; the imaging system uses a combination of a binocular near-infrared camera and a high-speed infrared camera, which can accurately determine the three-dimensional coordinates of the optimal venous blood collection point while clearly obtaining near-infrared images of the back of the hand; and an automatic venous blood collection arm is set to realize automatic and accurate venous blood collection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a flow chart of the method of the present invention;

[0008] Figure 2 Setting parameters for the camera of the present invention and a high-speed infrared camera deployment diagram;

[0009] Figure 3 The camera-corrected chessboard image of the present invention and the chessboard images before and after camera deformity correction;

[0010] Figure 4 The multi-dimensional decoupling network framework for hand back near-infrared image vein segmentation of the present invention;

[0011] Figure 5 It is the feature decomposition network subnetwork diagram of the present invention;

[0012] Figure 6 This is the network structure diagram of the multi-dimensional attention residual block MSARB of the present invention;

[0013] Figure 7 This is a structural diagram of the vein depth feature learning module of the present invention;

[0014] Figure 8 This is a structural diagram of the multi-dimensional parallel decoder of the present invention. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0016] An automatic subcutaneous venous blood sampling method based on infrared imaging, such as Figure 1 As shown, the method includes: deploying a first camera and a second camera; calibrating and positioning the two cameras; using the first camera to obtain a near-infrared image of the back of the user's hand; using a trained multi-dimensional decoupling network to perform vein segmentation on the near-infrared image of the back of the hand; using a positioning algorithm to process the segmented vein image to obtain the two-dimensional coordinates of the best blood collection point; using the second camera to obtain a spatial position image of the back of the user's hand, and calculating the three-dimensional coordinates of the best blood collection point based on the two-dimensional coordinates of the best blood collection point; and using a robotic arm to collect blood from the user based on the three-dimensional coordinates of the best blood collection point.

[0017] Optionally, the deployed first camera and the second camera are a binocular near-infrared camera and a high-speed infrared camera, respectively.

[0018] Preferably, the high-speed infrared camera is deployed as follows Figure 2 As shown in (a), a Hikvision DS network camera is used, and its deployment parameters are as follows: Figure 2 As shown in (b), the camera's focal length determines the size of the object imaged in the captured image. The camera's field of view represents the range of the image captured from the camera's optical center. The optimal shooting distance is 12 meters or less. Images captured beyond this range will be blurred. Therefore, when collecting data, the human body should be within the camera's field of view and shooting distance. The binocular infrared camera used is the ISG LW-1.3-S-1394, which has good response within the infrared frequency range. During acquisition, 10 frames are captured continuously after the control system stabilizes, and the final captured image is averaged.

[0019] Calibration and pose positioning of two cameras include:

[0020] Step 1: Use the first camera and the second camera to shoot 15 calibration chessboard images;

[0021] Step 2: Extract the corner points of each chessboard image;

[0022] Step 3: Calculate the relative position of the camera and each chessboard image based on the corner points of the chessboard image;

[0023] Step 4: Set the camera intrinsic parameters according to the relative position of the chessboard image, and construct the camera intrinsic parameter matrix based on the camera intrinsic parameters;

[0024] Step 5: Repeat steps 1 to 4 to complete the camera calibration.

[0025] Step 6: All cameras are paired together to shoot 15 calibration chessboard images;

[0026] Step 7: Calculate the rotation matrix R and translation matrix t between each camera;

[0027] Step 8: Get the position relationship of all cameras based on the rotation matrix R and translation matrix.

[0028] Preferably, the chessboard used for camera correction is as follows Figure 3 As shown in (a), the chessboard array is 9×12, and each grid size is 40×40mm. The chessboard corners are clear. It is made of alumina, which has a flat surface, is non-reflective, has a low thermal expansion coefficient, is lightweight, and is not easily deformed, meeting the calibration requirements. The effects before and after correction are shown in the figure. Figure 3 As shown in (b) and (c).

[0029] like Figure 4 As shown in the figure, the multi-dimensional decoupling network includes: a multi-dimensional feature decomposition network, a vein deep feature learning module and a multi-dimensional parallel decoder.

[0030] like Figure 5 As shown, the feature decomposition network includes: five-dimensional vein encoders F0, F1, F2, F3, F4, four-dimensional generator Y T 、Y S 、Y R 、Y G And an image reconstruction generator G; wherein the vein encoder F0 is a low-dimensional feature extractor, the vein encoder F1 is a shape encoder, the vein encoder F2 is a texture encoder, the vein encoder F3 is a spatial encoder and the vein encoder F4 is a grayscale encoder; the generator Y T For vein shape generator, generator Y S For texture generator, generator Y R Grayscale generator, generator Y G A space generator.

[0031] The multi-dimensional attention residual block MSARB network structure is as follows Figure 6 As shown in the figure, its network structure mainly consists of a multi-dimensional module MSB, a ReLU activation function and a spatial attention module.

[0032] like Figure 7 As shown in the figure, the deep vein feature learning module is a dilated spatial pyramid pooling (ASPP) structure. After decomposing and weighting the four deep vein features, this paper uses a parallel decoder to decode the previously extracted features. By constructing different receptive fields to obtain more features, while increasing the receptive field, it also minimizes problems such as pooling information loss. As a new decoder, it can extract high-dimensional data from low-dimensional features and better analyze multi-dimensional features. ASPP introduces dilated convolution based on SPP, using one 1×1 convolution and three 3×3 dilated convolutions with dilated rates of rates = {6, 12, 18}. This allows neurons in the output feature layer to contain multiple receptive field sizes, encoding the multi-dimensional feature information of veins, ultimately improving segmentation performance.

[0033] The process of training a multi-dimensional decoupled network includes:

[0034] Step 1: Obtain a near-infrared image dataset of the back of the hand and the corresponding vein labels;

[0035] Step 2: Input the hand back near infrared image I into the multi-dimensional vein encoder to obtain the feature set of the hand back near infrared image I, which includes the shape feature matrix Z T , texture feature matrix Z S , spatial feature matrix Z R , grayscale feature matrix Z G and the low-dimensional initial decomposition feature matrix Z1;

[0036] Step 3: Input the feature set into the multi-dimensional generator to obtain the prediction matrix information; the prediction matrix information includes: shape prediction matrix P T ′, texture prediction matrix P S ′, spatial prediction matrix P R ′, grayscale prediction matrix P G '; shape prediction matrix P T ′, texture prediction matrix P S ′, spatial prediction matrix P R ′, grayscale prediction matrix P G 'Generate the corresponding shape label information L T , texture label information L S , spatial label information L R , grayscale label information L G ;

[0037] Step 4: Construct the first loss function based on the prediction matrix information;

[0038] Step 5: Decompose the low-dimensional feature matrix Z1 and shape feature matrix Z T and texture feature matrix Z S Splicing to generate the first joint feature matrix; the shape feature matrix Z T , texture feature matrix Z S , spatial feature matrix Z R Splicing to generate the second joint feature matrix; the texture feature matrix Z S , spatial feature matrix Z R , grayscale feature matrix Z G Splicing to generate the third joint feature matrix;

[0039] Step 6: Input the first joint feature matrix, the second joint feature matrix, and the third joint feature matrix into the image reconstruction generator G respectively to obtain reconstructed images I1′, I2′, and I3′; use the reconstructed images I1′, I2′, and I3′ and the corresponding back-of-hand near-infrared image I to construct a second loss function;

[0040] Step 7: Weight each feature in the feature set and prediction matrix information, and convert the weighted low-dimensional initial decomposition feature matrix Z1 and the weighted shape feature matrix Z T And the weighted texture feature matrix Z S Input the vein depth feature learning module respectively for feature learning, and splice to generate the fourth joint feature matrix; the weighted shape feature matrix Z T , weighted texture feature matrix Z S , weighted spatial feature matrix Z RInput them into the vein depth feature learning module for feature learning, and splice them to generate the fifth joint feature matrix; the texture feature matrix Z S , spatial feature matrix Z R , grayscale feature matrix Z G Input them into the vein deep feature learning module respectively and concatenate them to generate the sixth joint feature matrix;

[0041] Step 8: Input the fourth joint feature matrix, the fifth joint feature matrix, and the sixth joint feature matrix into the multi-dimensional parallel decoder to obtain the vein prediction matrix P F ; According to the vein prediction matrix P F And the corresponding label constructs the third loss function;

[0042] Step 9: Weight the first loss function, the second loss function, and the third loss function respectively, and construct the total loss function based on the weighted first loss function, the second loss function, and the third loss function;

[0043] Step 10: Update all parameters of the multi-dimensional decoupling network through the back-propagation mechanism according to the total loss function, and complete the network training when the total loss function converges.

[0044] Optionally, the first loss function is a binary cross entropy loss function, which is expressed as:

[0045]

[0046] Among them, P′ T is the shape prediction matrix, P′ S is the texture prediction matrix, P′ R is the spatial prediction matrix, P′ G is the grayscale prediction matrix and L T is the shape label information, L S is the texture label information, L R is the spatial label information, L G Grayscale label information.

[0047] Optionally, the second loss is L1 loss:

[0048] Loss2=|II′1|+|II′2|+|II′3|

[0049] Among them, I′1, I′2, and I′3 are reconstructed near-infrared back of hand images, and I is the original near-infrared back of hand image.

[0050] Optionally, the third loss is the binary cross entropy loss expressed as:

[0051]

[0052] Among them, P seg is the vein prediction matrix, L seg For intravenous labeling.

[0053] The expression of the total loss function is:

[0054] Loss 总 =β1·Loss1+β2·Loss2+β3·Los3

[0055] Among them, β1, β2, and β3 are balance parameters, and their values ​​in the experiment are 1, 1, and 0.5; Loss1, Loss2, and Loss3 are the first, second, and third losses, respectively.

[0056] The fourth, fifth and sixth joint feature matrices are obtained by weighted concatenation of feature matrices, and their weights are as follows: Figure 4 As shown, its formula is expressed as:

[0057] The fourth joint characteristic matrix = Concate[a·Z1,b·Z T ,c·Z S ]

[0058] The fifth joint characteristic matrix = Concate[a·Z T ,b·Z S ,c·Z R ]

[0059] The sixth joint characteristic matrix = Concate[a·Z S ,b·Z R ,c·Z G ]

[0060] Among them, a, b, c are balance parameters, Z1 is the low-dimensional initial decomposition feature matrix, Z T is the weighted shape feature matrix, Z S is the texture feature matrix, Z R is the spatial feature matrix.

[0061] Optionally, during the network training process, the Adam optimizer is used to optimize the network parameters, where the learning rate of the Adam optimizer is 0.0001 and the weight decay coefficient is 0.5. The total number of network training times is 100.

[0062] In deep learning, one fundamental approach to avoiding overfitting is to obtain more data from the data source. To obtain a larger dataset to meet the algorithm's adaptability requirements, data augmentation is necessary to transform the original data and generate new data to expand the training set. Before training, each image was flipped, rotated, scaled, cropped, and shifted, ultimately resulting in 82×2×3×5 images of the dorsal hand veins.

[0063] Obtaining the two-dimensional coordinates of the optimal blood collection point includes: using a centerline extraction method to detect the centerline of the vein in the predicted image; and using an edge intersection method to obtain the center point of the maximum diameter as the optimal blood collection point.

[0064] Preferably, the centerline extraction method includes:

[0065] Step 1: Use the improved region growing method to perform initial segmentation on the binary image, and finally use the level sum algorithm to process the initial segmentation result to obtain the isosurface and level value information along the blood vessels;

[0066] Step 2: Use the fast matching algorithm to process the image after level set segmentation, calculate the shortest distance between all pixels in the blood vessel and the edge of the blood vessel, and obtain the shortest distance distribution map;

[0067] Step 3: Using the information of the shortest distance distribution map, the fast matching algorithm is used to further calculate the blood vessel pixel point farthest from the blood vessel edge, that is, the global maximum distance point A;

[0068] Step 4: In the local blood vessel area, starting from point A, the blood vessel pixel point B farthest from point A is calculated using a fast matching algorithm;

[0069] Step 5: In the local blood vessel area, starting from point B, the blood vessel pixel point C farthest from point B is calculated again using the fast matching algorithm;

[0070] Step 6: Finally, point C is selected as the starting point of the algorithm. Then, the gradient descent backtracking algorithm is used to traverse along the direction of the blood vessel to find all the blood vessel center points, and finally the complete blood vessel centerline is obtained.

[0071] Preferably, the edge intersection method includes:

[0072] Step 1: Get the edge of the image and obtain its edge point coordinates;

[0073] Step 2: Calculate the minimum circumscribed rectangle of the edge, then translate the left edge of the rectangle and find the intersection with the edge.

[0074] Step 3: Calculate the average value of the obtained diameter array, and the maximum value is the maximum diameter of the vein;

[0075] Step 4: Select the coordinates of the midpoint of the maximum diameter as the optimal blood collection coordinates.

[0076] The process of calculating three-dimensional coordinates by a high-speed infrared camera includes:

[0077] Step 1: Take the camera coordinate system of camera C1 as the world coordinate system. It is known that the rotation and translation matrices of camera C2 compared to camera C1 are R 12 , t 12 , the pixel coordinates of point J in the image I acquired by the two cameras are represents the pixel coordinates of point j in the i-th frame of the video acquired by camera C1, Represents the pixel coordinates of point j in the i-th frame of the video acquired by camera C2; the coordinate relationship is:

[0078]

[0079] Among them, d1 and d2 are two pixel points and Depth relative to the respective camera coordinate system.

[0080] Step 2: Use the camera coordinate system of camera C1 as the world coordinate system to calculate the depth d1; multiply both sides of the above equation by Its expression is:

[0081]

[0082] in, Pixel The observed values ​​of the coordinates.

[0083] Step 3: Simplify it into an equation about d1, calculate the depth d1, and then solve the three-dimensional coordinates of the joint point in the specified world coordinate system

[0084] Multi-dimensional parallel decoder (MPPD) Figure 8 As shown in (a), it consists of a parallel decoder (PPD) and a 1x1 convolution. The network structure of the parallel decoder is as follows: Figure 8 (b) shown.

[0085] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation plans of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An automatic subcutaneous venous blood sampling method based on infrared imaging, characterized in that: The method includes: deploying a first camera and a second camera; calibrating and positioning the two cameras; using the first camera to obtain a near-infrared image of the back of a user's hand; using a trained multi-dimensional decoupling network to perform vein segmentation on the near-infrared image of the back of the hand; using a positioning algorithm to process the segmented vein image to obtain the two-dimensional coordinates of the optimal blood sampling point; using the second camera to obtain a spatial position image of the back of the user's hand, and calculating the three-dimensional coordinates of the optimal blood sampling point based on the two-dimensional coordinates of the optimal blood sampling point; and sampling blood from the user based on the three-dimensional coordinates of the optimal blood sampling point. The process of training a multi-dimensional decoupled network includes: Step 1: Obtain a near-infrared image dataset of the back of the hand and the corresponding vein labels; Step 2: Input the hand back near infrared image I into the multi-dimensional vein encoder to obtain the feature set of the hand back near infrared image I, which includes the shape feature matrix Z T , texture feature matrix Z S , spatial feature matrix Z R , grayscale feature matrix Z G and the low-dimensional initial decomposition feature matrix Z1; Step 3: Input the feature set into the multi-dimensional generator to obtain the prediction matrix information; the prediction matrix information includes: shape prediction matrix P′ T , texture prediction matrix P′ S , spatial prediction matrix P′ R , grayscale prediction matrix P′ G ; For the shape prediction matrix P′ T , texture prediction matrix P′ S , spatial prediction matrix P′ R , grayscale prediction matrix P′ G , generate the corresponding shape label information L T , texture label information L S , spatial label information L R , grayscale label information L G ; Step 4: Construct the first loss function based on the prediction matrix information; Step 5: Decompose the low-dimensional feature matrix Z1 and shape feature matrix Z T and texture feature matrix Z S Splicing to generate the first joint feature matrix; the shape feature matrix Z T , texture feature matrix Z S , spatial feature matrix Z R Splicing to generate the second joint feature matrix; the texture feature matrix Z S , spatial feature matrix Z R , grayscale feature matrix Z G Splicing to generate the third joint feature matrix; Step 6: Input the first joint feature matrix, the second joint feature matrix, and the third joint feature matrix into the image reconstruction generator G respectively to obtain reconstructed images I′1, I′2, and I′3; use the reconstructed images I′1, I′2, and I′3 and the corresponding back-of-hand near-infrared image I to construct the second loss function; Step 7: Weight each feature in the feature set and prediction matrix information, and convert the weighted low-dimensional initial decomposition feature matrix Z1 and the weighted shape feature matrix Z T And the weighted texture feature matrix Z S Input the vein depth feature learning module respectively for feature learning, and splice to generate the fourth joint feature matrix; the weighted shape feature matrix Z T , weighted texture feature matrix Z S , weighted spatial feature matrix Z R Input them into the vein depth feature learning module for feature learning, and splice them to generate the fifth joint feature matrix; the texture feature matrix Z S , spatial feature matrix Z R , grayscale feature matrix Z G Input them into the vein deep feature learning module respectively and concatenate them to generate the sixth joint feature matrix; Step 8: Input the fourth joint feature matrix, the fifth joint feature matrix, and the sixth joint feature matrix into the multi-dimensional parallel decoder to obtain the vein prediction matrix P F ; According to the vein prediction matrix P F And the corresponding label constructs the third loss function; Step 9: Weight the first loss function, the second loss function, and the third loss function respectively, and construct the total loss function based on the weighted first loss function, the second loss function, and the third loss function; Step 10: Update all parameters of the multi-dimensional decoupling network through the back-propagation mechanism according to the total loss function, and complete the network training when the total loss function converges.

2. The automatic subcutaneous venous blood sampling method based on infrared imaging according to claim 1, characterized in that: Calibration and pose positioning of two cameras include: Step 1: Use the first camera and the second camera to capture a calibration chessboard image; Step 2: Extract the corner points of each chessboard image; Step 3: Calculate the relative position of the camera and each chessboard image based on the corner points of the chessboard image; Step 4: Set the camera intrinsic parameters according to the relative position of the chessboard image, and construct the camera intrinsic parameter matrix based on the camera intrinsic parameters; Step 5: Repeat steps 1 to 4 to complete the camera calibration. Step 6: All cameras are paired together to shoot 15 calibration chessboard images; Step 7: Calculate the rotation matrix R and translation matrix t between each camera; Step 8: Get the position relationship of all cameras based on the rotation matrix R and translation matrix.

3. The automatic subcutaneous venous blood sampling method based on infrared imaging according to claim 1, characterized in that: The multi-dimensional decoupling network includes: a multi-dimensional feature decomposition network, a vein deep feature learning module and a multi-dimensional parallel decoder; wherein the multi-dimensional feature decomposition network includes five-dimensional vein encoders F0, F1, F2, F3, F4, four-dimensional generators Y T 、Y S 、Y R 、Y G And an image reconstruction generator G; wherein the vein encoder F0 is a low-dimensional feature extractor, the vein encoder F1 is a shape encoder, the vein encoder F2 is a texture encoder, the vein encoder F3 is a spatial encoder and the vein encoder F4 is a grayscale encoder; the generator Y T For vein shape generator, generator Y S For texture generator, generator Y R Grayscale generator, generator Y G It is a spatial generator; the vein encoder, generator, and image reconstruction generator contain a multidimensional attention residual block MSARB; the multidimensional attention residual block consists of a multidimensional module MSB, a ReLU activation function, and a spatial attention module; the vein deep feature learning module is a hollow spatial pyramid pooling structure.

4. The automatic subcutaneous venous blood sampling method based on infrared imaging according to claim 1, characterized in that: The expression of the first loss function is: Among them, P′ T is the shape prediction matrix, P′ S is the texture prediction matrix, P′ R is the spatial prediction matrix, P′ G is the grayscale prediction matrix and L T is the shape label information, L S is the texture label information, L R is the spatial label information, L G Grayscale label information.

5. The automatic subcutaneous venous blood sampling method based on infrared imaging according to claim 1, characterized in that: The second loss function expression is: Loss2=|II′1|+|II′2|+|II′3| Among them, I′1, I′2, and I′3 are reconstructed near-infrared back of hand images, and I is the original near-infrared back of hand image.

6. The automatic subcutaneous venous blood sampling method based on infrared imaging according to claim 1, characterized in that: The third loss function expression is: Among them, P seg is the vein prediction matrix, L seg For intravenous labeling.

7. The automatic subcutaneous venous blood sampling method based on infrared imaging according to claim 1, characterized in that: Obtaining the two-dimensional coordinates of the optimal blood collection point includes: using a centerline extraction method to detect the centerline of the vein in the predicted image; and using an edge intersection method to obtain the center point of the maximum diameter as the optimal blood collection point.

8. The automatic subcutaneous venous blood sampling method based on infrared imaging according to claim 1, characterized in that: Calculating the three-dimensional coordinates of the optimal blood collection point includes: Step 1: Take the camera coordinate system of camera C1 as the world coordinate system. It is known that the rotation and translation matrices of camera C2 compared to camera C1 are R 12 , t 12 , the pixel coordinates of the joint point J in the image I obtained by the two cameras are represents the pixel coordinates of joint j in the i-th frame of the video acquired by camera C1, represents the pixel coordinates of joint j in the i-th frame of the video acquired by camera C2; Step 2: Calculate the depth d1 using the camera coordinate system of camera C1 as the world coordinate system; Step 3: Simplify it into an equation about d1, calculate the depth d1, and then solve the three-dimensional coordinates of the joint point in the specified world coordinate system

Images