Automatic vein and artery puncture control method, device and system

Abstract

The invention discloses an automatic vein and artery puncture control method, which is used for realizing high-precision puncture through precise control, and comprises the following steps of: after determining to start automatic vein and artery puncture control, collecting a blood vessel ultrasonic image; aiming at the target blood vessel ultrasonic image data, establishing a static artery blood vessel detection model based on a lightweight deep network and a target detection algorithm; the blood vessel detection model is used for detecting and positioning blood vessels; collecting a blood vessel image to be detected and positioned in real time, and obtaining the position and category of the blood vessel based on the trained blood vessel detection model; determining a target blood vessel position based on blood vessel positioning and classification, wherein the target blood vessel position is a puncture target position; planning a puncture path based on the target blood vessel position, including vein puncture path planning and artery puncture path planning; and converting the path and the control parameters based on puncture path planning. The invention further discloses a device and a system, which have low requirements on working environment, can adapt to dynamic environment, are convenient to popularize, reduce puncture difficulty and labor intensity and are portable.

Description

Technical Field

[0001] The present invention relates to the technical field of medical instruments, venous puncture and arterial puncture control, and in particular to an automatic venous and arterial puncture control method, device and system. Background Art

[0002] For some patients, especially those receiving long-term infusions, it may be difficult to determine the location of the vein during venipuncture, and repeated punctures are often required. In addition, a small number of patients have lesions or injuries on the skin surface. Although blood sampling and infusion can be performed, it is very difficult to determine the location of the blood vessels and perform punctures by human eyes.

[0003] Arteries are located deeper than veins. It is usually difficult to directly observe the artery on the surface of the skin during arterial puncture. Arterial puncture requires touching the arterial pulse to help determine the location of the artery for puncture. Arterial puncture is more difficult than venous puncture. Failure of puncture may lead to subcutaneous hematoma and other problems. Generally speaking, arterial puncture requires higher experience of the operator than venous puncture.

[0004] Automatic vascular puncture equipment based on ultrasonic probe blood vessel acquisition positioning and application of mechanical force can significantly reduce the pain of repeated punctures for patients, reduce the risk of puncture failure, and reduce the requirements for operator experience. The venous puncture function can effectively avoid the pain caused by repeated punctures of superficial veins, and can also effectively avoid the damage of anti-tumor drugs to blood vessels and local tissue necrosis caused by drug extravasation. The arterial puncture function can effectively improve the puncture success rate and reduce the pain and risk of repeated punctures caused by puncture failure. In recent years, robot-assisted puncture of soft tissue puncture has received more and more attention. However, traditional large-scale puncture robots have problems such as high cost, high requirements for the working environment, and limited puncture location. Robot-assisted puncture is still difficult to achieve widespread application. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the present invention provides the following technical solutions, namely, an automatic venous and arterial puncture control method, device and system, wherein the automatic venous and arterial puncture control method, and the portable automatic venous and arterial puncture device and system can be used by the operator to perform puncture by hand, so as to realize puncture of different parts of the human body, have low requirements on the working environment, can adapt to dynamic environments, and are easy to promote and apply; they can achieve high-precision positioning of veins or arteries, and automatically control the puncture device to perform needle insertion, catheter placement and needle withdrawal, reducing the difficulty and labor intensity of the operator's judgment of the blood vessel position based on experience and performing puncture, and the designed puncture device is easy to carry and adapt to dynamic scene applications.

[0006] On one hand, the present invention provides an automatic vein and artery puncture control method for achieving high-precision puncture through precise control, comprising:

[0007] S1, after determining to start the automatic vein and artery puncture control, collecting a blood vessel ultrasonic image;

[0008] S2, for the target blood vessel ultrasound image data, establishing a venous-arterial blood vessel detection model based on a lightweight deep network and a target detection algorithm; the blood vessel detection model is used to detect and locate the blood vessel;

[0009] S3, collecting the blood vessel image to be detected and located in real time, and obtaining the blood vessel position and category based on the trained blood vessel detection model;

[0010] S4, confirming the target blood vessel position based on the blood vessel positioning and classification, wherein the target blood vessel position is the puncture target position;

[0011] S5, performing puncture path planning based on the target blood vessel position, including: venous puncture path planning and arterial puncture path planning;

[0012] S6, converting the path and control parameters based on the puncture path planning.

[0013] Preferably, the lightweight deep network includes YOLOv8, NanoDet or NanoDe-Plus.

[0014] Preferably, S2 includes:

[0015] S21, establishing a basic blood vessel detection model, manually marking the blood vessel positions in the collected blood vessel ultrasound image to obtain a manually marked image;

[0016] S22, performing model training on the basic blood vessel detection model based on the manually annotated image to obtain a trained blood vessel detection model.

[0017] Preferably, S3 includes:

[0018] S31, collecting an image of the blood vessel to be detected and located in real time by an ultrasound device, and inputting the image of the blood vessel to be detected and located into the trained blood vessel detection model to obtain all blood vessels in the image as a first detection result;

[0019] S32, integrating the first detection result with the color ultrasound venous and arterial blood vessel classification method to obtain high-precision blood vessel positioning and classification, wherein the high-precision blood vessel positioning and classification are used for subsequent target blood vessel selection and puncture calculation, including:

[0020] (1) selecting a detection target area based on the first detection result: selecting a location where a blood vessel may exist in the B-ultrasound image according to the puncture target, and intercepting a portion of the entire image as the detection target area;

[0021] (2) performing blood vessel positioning and rough classification on the first detection result in the detection target area using a blood vessel positioning and rough classification method based on deep learning, including: obtaining a positioning detection result based on the blood vessel positioning method as a circumscribed rectangle or square of the blood vessel; classifying veins and arteries as a relatively rough classification result without precise positioning of the blood vessel edge; and using the classification result as reference information to further perform blood vessel edge fitting within the detection frame;

[0022] (3) performing high-precision classification of veins and arteries based on a color Doppler method or a spectral Doppler method, thereby obtaining high-precision distinction between veins and arteries corresponding to the rough classification;

[0023] (4) Venous and arterial classification and vascular edge fitting, including:

[0024] A. If the rough classification based on veins or arteries is consistent with the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method, the blood vessel type is determined; if the rough classification based on veins or arteries is inconsistent with the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method, and the confidence level of the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method is high, the results of high-precision vein and artery classification are directly adopted; if the rough classification based on veins or arteries is inconsistent with the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method, and the confidence level of the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method is low, further confirmation is prompted;

[0025] B. After the blood vessel detection results are determined and accurate vein and artery classification is obtained, high-precision blood vessel edge fitting is performed, including: within the detected blood vessel rectangular area, the ultrasound image and the colored image are binarized and fused respectively, and then Hough transform circle or ellipse detection is performed to obtain a high-precision circle or ellipse fitting result of the blood vessel edge; wherein, the method for binarizing and fusing the ultrasound image and the colored image is: both the ultrasound image and the colored image are changed into a consistent binary or grayscale image, the blood vessel area is represented by 0, and other areas are represented by 1; the two images are added to make the blood vessel area more obvious.

[0026] Preferably, the prompt further confirms that:

[0027] Perform mutual human-machine pressure-based venous vessel confirmation, including: when the blood vessel type cannot be automatically determined with high precision, prompt for manual pressure confirmation, apply appropriate pressure to the probe to observe the image, and if the blood vessel does not change with the pressure, it is judged to be an artery; if it gradually closes with the increase of pressure, it is judged to be a vein; determine the target blood vessel by manually clicking on the screen, that is, input the system vein and artery information.

[0028] Preferably, S4 includes:

[0029] S41, select arteriovenous targets according to working status, including:

[0030] (1) If the current working mode is venipuncture, select an available venous vessel with a suitable distance from the skin as the target vessel. If there are multiple available vessels, they constitute a to-be-selected set. The suitable distance from the skin means that the vessel type needs to be selected according to the puncture task;

[0031] (2) If the current working mode is arterial puncture, the arterial vessel closest to the skin is selected as the target vessel. If there are multiple available vessels, they constitute a candidate set;

[0032] S42, selecting the optimal target vessel based on the location and correlation of the vessels;

[0033] · Venous target vessel selection strategy: select the thickest vessel for puncture; there are no other vessels between the punctured vessel and the skin; there is no overlap of vessels; the venous target vessel selection strategy is completed through geometric calculation and comparison;

[0034] · Optimal strategy for arterial target vessel selection: When the thickness meets the requirements, the vessel should be as close to the skin as possible; there should be no other blood vessels between the punctured vessel and the skin; there should be no overlapping of blood vessels.

[0035] Preferably, the venipuncture path planning includes:

[0036] (1) Connect the center of the target vessel with the center of the needle direction to determine whether there are other veins in the connection path;

[0037] (2) If there are no other veins, the planning is successful and the angle between the steel needle and the skin and the insertion length are calculated;

[0038] (3) if there are other veins, determine whether the veins in the path are other veins that meet the puncture requirements, replace the other veins that meet the puncture requirements with the target veins and re-plan the puncture path; if the puncture requirements are not met, return to select other veins that meet the puncture requirements as the target veins; the puncture requirements include: the veins and the thickness of the veins meet the puncture requirements;

[0039] (4) If all other venous vessels fail to meet the puncture requirements, it indicates that the position is inappropriate and the operator needs to make adjustments;

[0040] The arterial puncture path planning is a planning strategy for searching for feasible paths from the vertical direction to both sides, including:

[0041] (1) Connect the center of the target vessel with the center of the needle direction to determine whether there are other veins or arteries in the connection path;

[0042] (2) If there are no other veins or arteries, the planning is successful and the angle between the steel needle and the skin and the insertion length are calculated;

[0043] (3) If there are other veins or arteries, it will be prompted that safe puncture is not possible and the fixed position of the needle needs to be readjusted; if the needle has a lateral motion control mechanism, the needle position is adjusted laterally at a fixed step length for searching; if the needle does not have a lateral motion control mechanism, it will be prompted that the position is inappropriate and the operator needs to make adjustments.

[0044] Preferably, S6 includes:

[0045] S61, after the puncture path planning is successful, the control device compares the current angle and needle tip position information of the needle body to be punctured with the planned puncture path to form displacement vector information; the displacement vector information includes axial displacement and angular displacement;

[0046] S62, the control device converts the displacement vector information into a control signal and transmits it to the driving device;

[0047] S63, the driving device receives the displacement vector information, outputs an electrical signal, and controls the motor movement of the automatic venous and arterial puncture device to perform puncture.

[0048] The second aspect of the present invention is to provide an automatic venous artery puncture device, which performs automatic venous artery puncture based on the control method described in the first aspect; comprising: a fixing device, a guide device and a driving device; the fixing device is slidably connected to the guide device, and the fixing device is fixedly connected to the driving device; wherein:

[0049] The fixing device is used to fix the needle body to be punctured and the hose to be inserted; the hose to be inserted is fixed to the fixing device through the needle body to be punctured;

[0050] The guide device is used to determine the target blood vessel position, that is, the puncture target position;

[0051] The driving device is used to drive the fixing device to move along the direction guided by the guiding device;

[0052] The fixing device comprises: a syringe, an inner cylinder, an outer cylinder, a first connecting piece, a fixing plate, a needle clamp and a needle withdrawal button; the hose to be inserted is fixedly arranged on the syringe through the needle body to be punctured; the syringe is arranged in the inner cylinder and fixedly connected to the inner cylinder; the inner cylinder is arranged in the outer cylinder through the first connecting piece, and the outer cylinder can be slidably connected to the inner cylinder; the outer cylinder is fixedly arranged on the fixing plate through the first connecting piece; the needle clamp cooperates with the needle withdrawal button to trigger the needle withdrawal button under the action of external force, so that the needle body to be punctured and the hose to be inserted are separated from the fixing device; after the needle body to be punctured is fixed on the fixing device, its initial position and the relative spatial position between the ultrasonic probe are determined and no longer change, and the fixing device is calibrated and used for the control algorithm to realize the puncture control based on ultrasonic image positioning;

[0053] The guide device comprises three parts: an ultrasonic probe, a needle guide and an outer tube guide, which is used to guide the fixing device and the needle to be punctured to move in a fixed direction based on ultrasound positioning of the blood vessel; the ultrasonic probe is used to collect data information of the preselected target blood vessel through ultrasound and send the data information to the control device; the needle guide is fixedly arranged on the ultrasonic probe, and the needle to be punctured is mounted on the needle guide; the outer tube guide is fixedly arranged on the fixing plate, and the outer tube is mounted on the outer tube guide;

[0054] The ultrasonic probe is used in conjunction with the ultrasonic coupling sleeve, and the ultrasonic coupling sleeve is installed at the front end of the ultrasonic probe in an installation manner that can be disassembled multiple times and is easy to disassemble;

[0055] The driving device is connected to the control device, and is used for driving the fixing device to move along the direction guided by the guide device; the driving device is divided into a needle control part driving device and a tube placement part driving device; the needle control part driving device comprises: a needle control motor and a second connecting piece; the needle control motor is fixedly arranged on the fixing plate, and is fixedly connected to the inner cylinder through the second connecting piece; the tube placement part driving device comprises: a tube placement motor, a third connecting piece, an outer cylinder connecting rod and a motor connecting rod; the tube placement motor is fixedly arranged on the fixing plate through the motor connecting rod, the tube placement motor is fixedly connected to the third connecting piece, and the third connecting piece is connected to the outer cylinder through the outer cylinder connecting rod.

[0056] The third aspect of the present invention is used to provide a venous artery puncture system, which is completed by the control device and the automatic venous artery puncture device according to the second aspect, and the automatic venous artery puncture device is connected to the control device, wherein:

[0057] The automatic venous arterial puncture device collects data information of a preselected target blood vessel through an ultrasonic probe, and sends the data information of the preselected target blood vessel to the control device;

[0058] The control device is used to receive the data information of the preselected target blood vessel, calculate the position information of the target blood vessel according to the data information, and generate a control signal for controlling the operation of the automatic venous artery puncture device according to the position information; the automatic venous artery puncture device controls the fixed needle body to be punctured and the hose to be inserted to puncture the skin according to the control signal;

[0059] The driving device accurately controls the movement step lengths of the needle control motor and the tube motor; the control device converts the blood vessel information collected by the ultrasonic probe into the position information and puncture path of the target blood vessel based on algorithm recognition and optimization; the control device compares the current coordinate information of the puncture point and the needle body to be punctured, calculates the displacement vector, and transmits it to the driving device in the form of a control signal; the driving device receives the control signal and moves to the puncture point according to the control signal.

[0060] A fourth aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is used to read the instructions and execute the method as described in the first aspect.

[0061] A fifth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor and execute the method described in the first aspect.

[0062] The control method, device and system provided by the present invention have the following beneficial effects:

[0063] Compared with manual puncture or mechanical assisted medical robot puncture in the prior art, the venous puncture device of this patent has the following advantages:

[0064] 1. The overall cost is lower than that of the robot medical system, it is more practical, all-in-one, portable, and can adapt to dynamic environments.

[0065] 2. The intravenous puncture device forms a mechanical connection with the ultrasonic probe and cooperates with the control device to realize the automatic control of the puncture operation. The operator does not need to participate in the puncture action, which improves efficiency, reduces dependence on the operator's experience, and is not affected by the operator's operating habits.

[0066] 3. The blood vessel positioning algorithm based on ultrasound images is divided into venous and arterial control. The guide device in the venous puncture device uses an ultrasonic probe to locate the veins and arteries. Compared with manual puncture, which relies on visual and hand touch to determine the position, the blood vessel positioning accuracy is greatly improved, which can ensure the accuracy of puncture.

[0067] 4. The ultrasonic probe is connected to the ultrasonic coupling sleeve. The ultrasonic coupling sleeve is a disposable item, and the connection is easy to disassemble and can be disassembled and assembled multiple times. The ultrasonic coupling sleeve replaces the application of ordinary ultrasonic coupling agent on the skin to avoid contamination of the acupuncture site. Adding a coupling sleeve meets the requirements of ultrasonic probe imaging and ensures that it does not affect the sterility requirements of the puncture site.

[0068] 5. The ultrasonic coupling sleeve is a disposable sterile product, and the outer surface can be treated by disinfection. The skin position that has been in contact with it can still be punctured, which is convenient for use. The ultrasonic probe has no direct contact with the patient's skin and can be kept clean, which is conducive to extending the service life of the ultrasonic probe.

[0069] 6. Error compensation between ultrasound probe viewing angle and needle viewing angle. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a flow chart of the automatic vein and artery puncture control method described in the present invention.

[0071] Figure 2 This is a schematic diagram of the YOLOv8 network structure described in the present invention.

[0072] Figure 3 This is a schematic diagram of the YOLOv8 network structure described in the present invention.

[0073] Figure 4 This is a schematic diagram of the NanoDet-Plus network structure described in the present invention.

[0074] Figure 5 This is a schematic diagram of the manually labeled veins and arteries in the figure as described in the present invention.

[0075] Figure 6 It is a schematic diagram of the positioning and classification process of veins and arteries according to the present invention.

[0076] Figure 7 It is a schematic diagram of the classification results of veins and arteries described in the present invention.

[0077] Figure 8 This is a schematic diagram of the venous blood vessel positioning described in the present invention.

[0078] Fig. 9 It is a schematic diagram of the basilic vein described in the present invention.

[0079] Fig.10 It is the great saphenous vein map and the small saphenous vein map described in the present invention.

[0080] Fig.11 It is a schematic diagram of the structure of the ultrasonic coupling sleeve described in the present invention.

[0081] Fig.12 The figure is a schematic structural diagram of an embodiment of the electronic device described in the present invention. DETAILED DESCRIPTION

[0082] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0083] like Figure 1 As shown, the present invention provides an automatic vein and artery puncture control method on the one hand, which is used to achieve high-precision puncture through precise control, including:

[0084] S1, after determining to start the automatic vein and artery puncture control, collecting a blood vessel ultrasonic image;

[0085] As a preferred embodiment, S1 includes: acquiring ultrasound images of blood vessels of multiple body parts that may be punctured, such as the back of the hand, forearm, upper arm, etc., and the acquisition is performed based on multiple people and multiple adjustments.

[0086] S2, for the target blood vessel ultrasound image data, establishing a venous-arterial blood vessel detection model based on a lightweight deep network and a target detection algorithm; the blood vessel detection model is used to detect and locate the blood vessel;

[0087] In this embodiment, in order to realize the deployment of portable devices and ensure the puncture efficiency, a lightweight deep network is used to detect and locate blood vessels, such as YOLOv8, NanoDet or NanoDe-Plus, in which the lightweight deep network is the basis of the venous and arterial blood vessel detection and positioning algorithm.

[0088] (1) Figure 2 The figure shows a schematic diagram of the YOLOv8 network structure used in this embodiment.

[0089] YOLOv8 is an improvement and upgrade based on the previous 7 versions of the YOLO detection algorithm. The main improvements include: the backbone uses the C2f module, the detection head uses the anchor-free+Decoup led-head, the loss function uses the classification BCELoss, the regression CIoU and VFL combination, the box matching strategy is changed from static matching to Task-AlignedAssigner matching, and the Mosaic operation is turned off in the last 10 epochs.

[0090] (2) Figure 3 Shown is a schematic diagram of the NanoDet network structure used in this embodiment.

[0091] NanoDet is a FCOS-style single-stage anchor-free target detection model that uses ATSS for target sampling and Generate Focus Loss loss function for classification and box regression. The network structure diagram is shown in the figure. Figure 3 As shown in the figure, NanoDet uses the Generated Focal Loss function. This function can remove the Centerness branch of FCOS, eliminating a large number of convolutions on this branch, thereby reducing the computational overhead of the detection head, which is very suitable for lightweight deployment on mobile terminals.

[0092] (3) Figure 4 Shown is a schematic diagram of the NanoDet-Plus network structure used in this embodiment.

[0093] A simpler and lighter training auxiliary module AssignGuidanceModule (AGM) is designed in the NanoDet-Plus detection model, which is combined with a dynamic soft label assignment strategy.

[0094] Dynamic SoftLabelAss igner (DSLA) is used to solve the optimal label matching problem in lightweight models. The overall architecture of NanoDet-Plus is as follows Figure 4 As shown in the figure, it improves Ghost-PAN in feature fusion, improves the convolution kernel size and downsampling layer in the detection head, and modifies the training optimization strategy.

[0095] As a preferred embodiment, S2 includes:

[0096] S21, establishing a basic blood vessel detection model, manually marking the blood vessel positions in the collected blood vessel ultrasound image to obtain a manually marked image;

[0097] In this embodiment, the veins and arteries in the figure are manually marked. Figure 5 As shown in Figure 2, it is difficult to distinguish between venous vessels and arterial vessels in a single static short-axis B-ultrasound image frame.

[0098] S22, performing model training on the basic blood vessel detection model based on the manually annotated image to obtain a trained blood vessel detection model.

[0099] S3, collecting the blood vessel image to be detected and located in real time, and obtaining the blood vessel position and category based on the trained blood vessel detection model.

[0100] As a preferred implementation, S3 includes:

[0101] S31, collecting an image of the blood vessel to be detected and located in real time by an ultrasound device, and inputting the image of the blood vessel to be detected and located into the trained blood vessel detection model to obtain all blood vessels in the image as a first detection result;

[0102] S32, integrating the first detection result with the color ultrasound venous and arterial blood vessel classification method to obtain high-precision blood vessel positioning and classification, and the high-precision blood vessel positioning and classification are used for subsequent target blood vessel selection and puncture calculation.

[0103] In this embodiment, in actual application, the step S32 can also be combined with the operator's pressing of the venous blood vessel to further ensure the safety of the operation. The process design of this part is shown in the figure below.

[0104] like Figure 6 As shown, as a preferred implementation, the S32 includes:

[0105] (1) Selecting the detection target area based on the first detection result: According to the puncture target, select the location where the blood vessel may exist in the B-ultrasound image, and intercept a part of the entire image as the detection target area. The advantage of this is to reduce false detection and reduce the complexity of post-processing;

[0106] (2) The blood vessel positioning and rough classification method based on deep learning performs blood vessel positioning and rough classification on the first detection result in the detection target area, including: the positioning detection result obtained by the blood vessel positioning method based on deep learning is a circumscribed rectangle or square of the blood vessel; the classification of veins and arteries is a relatively rough classification result, without precise positioning of the blood vessel edge. The classification result is used as reference information, and further blood vessel edge fitting is performed within the detection frame;

[0107] (3) High-precision vein and artery classification is performed based on the color Doppler method or the spectral Doppler method, thereby obtaining high-precision vein and artery distinction corresponding to the rough classification; in this embodiment, as follows Figure 6 As shown in the figure, different regions generally present irregular edges. In addition to real blood vessels, some non-vascular regions are also detected. Combining the results of deep learning vascular localization, real veins and arteries can be screened out, such as Figure 7 shown.

[0108] (4) Venous and arterial classification and vascular edge fitting, including:

[0109] A. If the rough classification based on veins or arteries is consistent with the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method, the blood vessel type is determined; if the rough classification based on veins or arteries is inconsistent with the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method, and the confidence level of the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method is high, the results of high-precision vein and artery classification are directly adopted; if the rough classification based on veins or arteries is inconsistent with the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method, and the confidence level of the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method is low, further confirmation is prompted;

[0110] B. After the vascular detection results are determined and accurate vein and artery classification is obtained, high-precision vascular edge fitting is performed. Figure 8 For example, in the detected rectangular area of ​​the blood vessel, the ultrasound image and the colored image are binarized and fused respectively, and then the Hough transform circle or ellipse detection is performed to obtain a high-precision circle or ellipse fitting result for the edge of the blood vessel. The Hough transform circle or ellipse detection method is relatively mature and can be used in common methods. The fitting result is used in subsequent steps.

[0111] The method of binarization and fusion of the ultrasound image and the colored image is as follows: both the ultrasound image and the colored image are changed into a consistent binary or grayscale image, where the blood vessel area is represented by 0 and other areas are represented by 1. The two images are added to make the blood vessel area more obvious.

[0112] Hough transform is a method for detecting specific shapes in an image. It can detect a variety of geometric shapes such as lines, circles, and ellipses. Its principle is based on the mapping relationship between image space and parameter space. For Hough circle or ellipse detection, the principle can be summarized as the following steps:

[0113] Edge detection: Edge detection is required for the image in order to determine all potential circular or elliptical edge points in the image.

[0114] · Parameter mapping: Map edge points in image space to parameter space. For a circle, this means mapping each edge point to a point (x, y, r) in 3D space, where (x, y) are the coordinates of the edge point and r is the radius from the center of the image to that point. For an ellipse, the parameter space is more complicated because additional parameters need to be considered, such as the length of the major and minor axes and the rotation angle of the ellipse.

[0115] Voting mechanism: In parameter space, for each possible center position, the gradient vectors from all edge points to the center are calculated, and the intersection of these vectors is the potential center position. For ellipses, this process is more complicated because more parameters need to be considered to determine the specific position and shape of the ellipse.

[0116] Peak detection: Find the points in parameter space with the most votes, which correspond to the centers of the most likely circular or elliptical shapes in the image.

[0117] Shape Verification: Finally, the detected shapes are verified to conform to the expected circular or elliptical properties by checking for peaks in the parameter space.

[0118] As a preferred embodiment, the prompt further confirms that:

[0119] Perform mutual human-machine pressure-based venous vessel confirmation, including: when the blood vessel type cannot be automatically determined with high precision, prompt for manual pressure confirmation, apply appropriate pressure to the probe to observe the image, and if the blood vessel does not change with the pressure, it is judged to be an artery; if it gradually closes with the increase of pressure, it is judged to be a vein; determine the target blood vessel by manually clicking on the screen, that is, input the system vein and artery information.

[0120] After the above steps are completed, the detailed information of the blood vessels in the ultrasound image at the current position can be obtained, and the process proceeds to step S4.

[0121] S4, confirming the target blood vessel position based on the blood vessel positioning and classification, wherein the target blood vessel position is the puncture target position;

[0122] In this embodiment, multiple different types of blood vessels may be detected in the image. It is necessary to first determine the target blood vessel range and then select the target blood vessel.

[0123] As a preferred implementation, S4 includes:

[0124] S41, select arteriovenous target according to working status:

[0125] In this embodiment:

[0126] (1) If the current working mode is venipuncture, select an available vein with a suitable distance from the skin (displayed as a circle with the largest radius in the image) as the target vessel. If there are multiple available vessels, they constitute a candidate set. The suitable distance from the skin mainly requires the selection of the vessel type according to the puncture task. Currently, venipuncture is mainly performed on the basilic vein, such as Fig. 9 Generally, the great saphenous vein or the small saphenous vein that is very close to the skin is not punctured. Fig.10 As shown. Fig.10 The left side shows the greater saphenous vein, and the right side shows the lesser saphenous vein.

[0127] Since the distribution positions of the basilic vein and the great saphenous and small saphenous veins on B-ultrasound images are quite different, a simple threshold range can be used. For example, if the coordinates on the image are greater than a threshold (this threshold is related to the puncture site, color ultrasound equipment characteristics and parameter adjustment, and can be preset after the equipment and site are determined), the great saphenous vein and small saphenous vein can be excluded.

[0128] (2) If the current working mode is arterial puncture, the artery closest to the skin (the image shows the artery target closest to the top) is selected as the target vessel. If there are multiple available vessels, they constitute a to-be-selected set.

[0129] S42, selecting the optimal target blood vessel according to the position and correlation of the blood vessel in the graph.

[0130] · Optimal selection of target venous vessels: Generally, the thickest vessel is selected for puncture; there are no other vessels between the punctured vessel and the skin; there is no overlap of vessels. These optimization strategies can be completed through geometric calculation and comparison.

[0131] · Target arterial vessel selection: The target artery should be as close to the skin as possible while meeting the requirements; there should be no other blood vessels between the punctured blood vessel and the skin; and there should be no overlapping of blood vessels.

[0132] S5, performing puncture path planning based on the target blood vessel position, including: venous puncture path planning and arterial puncture path planning.

[0133] In this embodiment, the steel needle is a rigid object and is planned as a line. The objectives that need to be planned for the puncture needle path planning include: the skin puncture position, the angle between the steel needle and the skin, and the puncture length (determine the depth of entry).

[0134] As a preferred embodiment, the venipuncture path planning includes:

[0135] (1) Connect the center of the target vessel with the center of the needle direction to determine whether there are other veins in the connection path;

[0136] (2) If there are no other veins, the planning is successful and the angle between the steel needle and the skin and the insertion length are calculated;

[0137] (3) If there are other veins, determine whether the veins in the path meet the puncture requirements (the two necessary points of the puncture requirements are: the vein and the thickness of the vein meet the puncture requirements). Replace the other veins that meet the puncture requirements with the target vessels and re-plan the puncture path. If the puncture requirements are not met, return to select other veins that meet the puncture requirements as the target vessels.

[0138] (4) If all other veins fail to meet the puncture requirements, it indicates that the position is inappropriate and the operator needs to make adjustments.

[0139] As a preferred embodiment, the arterial puncture path planning: Since the artery is located deeper, there are often veins between it and the skin, and the puncture needle must avoid these veins during the puncture process. Generally, the puncture process tries to shorten the length of the needle passing through the body. Therefore, this embodiment designs a planning strategy for searching for feasible paths from the vertical direction to the two sides:

[0140] (1) Connect the center of the target vessel with the center of the needle direction selection to determine whether there are other veins or arteries in the connection path.

[0141] (2) If there are no other veins or arteries, the planning is successful and the angle between the steel needle and the skin and the insertion length are calculated;

[0142] (3) If there are other veins or arteries, it will be prompted that safe puncture is not possible and the fixed position of the needle needs to be readjusted; if the needle has a lateral motion control mechanism, the needle position is adjusted laterally at a fixed step length for searching; if the needle does not have a lateral motion control mechanism, it will be prompted that the position is inappropriate and the operator needs to make adjustments.

[0143] S6, converting the path and control parameters based on the puncture path planning.

[0144] As a preferred implementation, S6 includes:

[0145] S61, after the puncture path planning is successful, the control device compares the current angle and needle tip position information of the needle body to be punctured with the planned puncture path to form displacement vector information; the displacement vector information includes axial displacement and angular displacement;

[0146] S62, the control device converts the displacement vector information into a control signal and transmits it to the driving device;

[0147] S63, the driving device receives the displacement vector information, outputs an electrical signal, and controls the motor movement of the automatic venous and arterial puncture device to perform puncture.

[0148] The second aspect of the present invention is to provide an automatic venous artery puncture device, which performs automatic venous artery puncture based on the control method described in the first aspect; comprising: a fixing device, a guide device and a driving device; the fixing device is slidably connected to the guide device, and the fixing device is fixedly connected to the driving device; wherein:

[0149] The fixing device is used to fix the needle body to be punctured and the hose to be inserted; the hose to be inserted is fixed to the fixing device through the needle body to be punctured;

[0150] The guide device is used to determine the target blood vessel position, that is, the puncture target position;

[0151] The driving device is used to drive the fixing device to move along the direction guided by the guiding device.

[0152] (I) Fixing device

[0153] The fixing device comprises: a needle cylinder, an inner cylinder, an outer cylinder, a first connecting piece, a fixing plate, a needle clamp and a needle withdrawal button.

[0154] Structural description: The hose to be inserted is fixedly arranged on the syringe through the needle body to be punctured; the syringe is arranged in the inner cylinder and fixedly connected to the inner cylinder; the inner cylinder is arranged in the outer cylinder through the first connecting piece, and the outer cylinder can be slidably connected to the inner cylinder; the outer cylinder is fixedly arranged on the fixed plate through the first connecting piece; the needle card cooperates with the needle withdrawal button, which is used to trigger the needle withdrawal button under the action of external force, so that the needle body to be punctured and the hose to be inserted are separated from the fixing device; after the needle body to be punctured is fixed on the fixing device, its initial position and the relative spatial position between the ultrasonic probe are determined and no longer change. After the fixing device is calibrated and used for the control algorithm, the puncture control based on ultrasonic image positioning can be realized.

[0155] (ii) Guide device

[0156] The guiding device comprises three parts: an ultrasonic probe, a needle body guiding member and an outer cylinder guiding member, and is used for guiding the fixing device and the needle body to be punctured to move in a fixed direction based on ultrasound positioning of the blood vessel.

[0157] 1. Ultrasonic probe, used to collect data information of pre-selected target blood vessels through ultrasound and send the data information to the control device.

[0158] 2. A needle guide is fixedly arranged on the ultrasonic probe, and the needle to be punctured is mounted on the needle guide.

[0159] 3. An outer cylinder guide, which is fixedly arranged on the fixing plate, and the outer cylinder is mounted on the outer cylinder guide.

[0160] As a preferred embodiment, the ultrasonic probe is used in conjunction with the ultrasonic coupling sleeve, and the ultrasonic coupling sleeve is installed at the front end of the ultrasonic probe in an installation manner that can be disassembled multiple times and is easy to disassemble. Fig.11 Shown is a schematic diagram of the structure of an ultrasound coupling sleeve, which is mainly used to ensure that the artery is sterile.

[0161] (III) Driving device

[0162] The driving device is connected to the control device and is used to drive the fixing device to move along the direction guided by the guide device; the driving device is divided into a needle control part driving device and a tube placement part driving device.

[0163] 1. The needle control part driving device comprises: a needle control motor and a second connecting member; the needle control motor is fixedly arranged on the fixing plate and is fixedly connected to the inner cylinder through the second connecting member.

[0164] 2. The driving device of the catheter placement part includes: a catheter placement motor, a third connecting piece, an outer tube connecting rod and a motor connecting rod; the catheter placement motor is fixedly arranged on the fixed plate through the motor connecting rod, the catheter placement motor is fixedly connected to the third connecting piece, and the third connecting piece is connected to the outer tube through the outer tube connecting rod.

[0165] The third aspect of the present invention is used to provide a venous artery puncture system, which is completed by the control device and the automatic venous artery puncture device according to the second aspect, and the automatic venous artery puncture device is connected to the control device, wherein:

[0166] The automatic venous arterial puncture device collects data information of a preselected target blood vessel through an ultrasonic probe, and sends the data information of the preselected target blood vessel to the control device;

[0167] The control device is used to receive the data information of the preselected target blood vessel, calculate the position information of the target blood vessel based on the data information, and generate a control signal for controlling the operation of the automatic venous arterial puncture device based on the position information; the automatic venous arterial puncture device controls the needle body to be punctured and the hose to be inserted fixed on itself to puncture the skin according to the control signal.

[0168] The driving device accurately controls the movement step lengths of the needle control motor and the tube motor; the control device converts the blood vessel information collected by the ultrasonic probe into the position information and puncture path of the target blood vessel based on algorithm recognition and optimization; the control device compares the current coordinate information of the puncture point and the needle body to be punctured, calculates the displacement vector, and transmits it to the driving device in the form of a control signal; the driving device receives the control signal and moves to the puncture point according to the control signal.

[0169] As a preferred embodiment, the venous arterial puncture system also includes a portable industrial computer or an all-in-one machine, and the portable industrial computer or the all-in-one machine and the B-ultrasound machine are placed together to achieve portability; the AI ​​in the portable industrial computer is used to calculate the computing resources of the board / pad.

[0170] The present invention also provides a memory storing a plurality of instructions, wherein the instructions are used to implement the method as in the first embodiment.

[0171] like Fig.12 As shown, the present invention also provides an electronic device, including a processor 301 and a memory 302 connected to the processor 301, the memory 302 stores a plurality of instructions, and the instructions can be loaded and executed by the processor so that the processor can execute the method as in the first embodiment.

[0172] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An automatic vein and artery puncture control method, characterized in that: For high-precision puncture with precise control, including: S1, after determining to start the automatic vein and artery puncture control, collecting a blood vessel ultrasonic image; S2, for the target blood vessel ultrasound image data, establishing a venous-arterial blood vessel detection model based on a lightweight deep network and a target detection algorithm; the blood vessel detection model is used to detect and locate the blood vessel; S3, collecting the blood vessel image to be detected and located in real time, and obtaining the blood vessel position and category based on the trained blood vessel detection model; S4, confirming the target blood vessel position based on the blood vessel positioning and classification, wherein the target blood vessel position is the puncture target position; S5, performing puncture path planning based on the target blood vessel position, including: venous puncture path planning and arterial puncture path planning; S6, converting the path and control parameters based on the puncture path planning.

2. The automatic vein and artery puncture control method according to claim 1, characterized in that: The lightweight deep network includes YOLOv8, NanoDet or NanoDe-Plus.

3. The automatic vein and artery puncture control method according to claim 2, characterized in that: The S2 includes: S21, establishing a basic blood vessel detection model, manually marking the blood vessel positions in the collected blood vessel ultrasound image to obtain a manually marked image; S22, performing model training on the basic blood vessel detection model based on the manually annotated image to obtain a trained blood vessel detection model.

4. The automatic vein and artery puncture control method according to claim 3, characterized in that: The S3 includes: S31, collecting an image of the blood vessel to be detected and located in real time by an ultrasound device, and inputting the image of the blood vessel to be detected and located into the trained blood vessel detection model to obtain all blood vessels in the image as a first detection result; S32, integrating the first detection result with the color ultrasound venous and arterial blood vessel classification method to obtain high-precision blood vessel positioning and classification, wherein the high-precision blood vessel positioning and classification are used for subsequent target blood vessel selection and puncture calculation, including: (1) selecting a detection target area based on the first detection result: selecting a location where a blood vessel may exist in the B-ultrasound image according to the puncture target, and intercepting a portion of the entire image as the detection target area; (2) performing blood vessel positioning and rough classification on the first detection result in the detection target area using a blood vessel positioning and rough classification method based on deep learning, including: obtaining a positioning detection result based on the blood vessel positioning method as a circumscribed rectangle or square of the blood vessel; classifying veins and arteries as a relatively rough classification result without precise positioning of the blood vessel edge; and using the classification result as reference information to further perform blood vessel edge fitting within the detection frame; (3) performing high-precision classification of veins and arteries based on a color Doppler method or a spectral Doppler method, thereby obtaining high-precision distinction between veins and arteries corresponding to the rough classification; (4) Venous and arterial classification and vascular edge fitting, including: A. If the rough classification based on veins or arteries is consistent with the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method, the blood vessel type is determined; if the rough classification based on veins or arteries is inconsistent with the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method, and the confidence level of the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method is high, the results of high-precision vein and artery classification are directly adopted; if the rough classification based on veins or arteries is inconsistent with the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method, and the confidence level of the results of high-precision vein and artery classification based on the color Doppler effect or spectral Doppler method is low, further confirmation is prompted; B. After the blood vessel detection results are determined and accurate vein and artery classification is obtained, high-precision blood vessel edge fitting is performed, including: within the detected blood vessel rectangular area, the ultrasound image and the colored image are binarized and fused respectively, and then Hough transform circle or ellipse detection is performed to obtain a high-precision circle or ellipse fitting result of the blood vessel edge; wherein, the method for binarizing and fusing the ultrasound image and the colored image is: both the ultrasound image and the colored image are changed into a consistent binary or grayscale image, the blood vessel area is represented by 0, and other areas are represented by 1; the two images are added to make the blood vessel area more obvious.

5. The automatic vein and artery puncture control method according to claim 4, characterized in that: The prompt further confirms that: Perform mutual human-machine pressure-based venous vessel confirmation, including: when the blood vessel type cannot be automatically determined with high precision, prompt for manual pressure confirmation, apply appropriate pressure to the probe to observe the image, and if the blood vessel does not change with the pressure, it is judged to be an artery; if it gradually closes with the increase of pressure, it is judged to be a vein; determine the target blood vessel by manually clicking on the screen, that is, input the system vein and artery information.

6. The automatic vein and artery puncture control method according to claim 5, characterized in that: The S4 includes: S41, select arteriovenous targets according to working status, including: (1) If the current working mode is venipuncture, select an available venous vessel with a suitable distance from the skin as the target vessel. If there are multiple available vessels, they constitute a to-be-selected set. The suitable distance from the skin means that the vessel type needs to be selected according to the puncture task; (2) If the current working mode is arterial puncture, the arterial vessel closest to the skin is selected as the target vessel. If there are multiple available vessels, they constitute a candidate set; S42, selecting the optimal target vessel based on the location and correlation of the vessels; · Venous target vessel selection strategy: select the thickest vessel for puncture; there are no other vessels between the punctured vessel and the skin; there is no overlap of vessels; the venous target vessel selection strategy is completed through geometric calculation and comparison; · Optimal strategy for arterial target vessel selection: When the thickness meets the requirements, the vessel should be as close to the skin as possible; there should be no other blood vessels between the punctured vessel and the skin; there should be no overlapping of blood vessels.

7. The automatic vein and artery puncture control method according to claim 6, characterized in that: The venipuncture path planning includes: (1) Connect the center of the target vessel with the center of the needle direction to determine whether there are other veins in the connection path; (2) If there are no other veins, the planning is successful and the angle between the steel needle and the skin and the insertion length are calculated; (3) if there are other veins, determine whether the veins in the path are other veins that meet the puncture requirements, replace the other veins that meet the puncture requirements with the target veins and re-plan the puncture path; if the puncture requirements are not met, return to select other veins that meet the puncture requirements as the target veins; the puncture requirements include: the veins and the thickness of the veins meet the puncture requirements; (4) If all other venous vessels fail to meet the puncture requirements, it indicates that the position is inappropriate and the operator needs to make adjustments; The arterial puncture path planning is a planning strategy for searching for feasible paths from the vertical direction to both sides, including: (1) Connect the center of the target vessel with the center of the needle direction to determine whether there are other veins or arteries in the connection path; (2) If there are no other veins or arteries, the planning is successful and the angle between the steel needle and the skin and the insertion length are calculated; (3) If there are other veins or arteries, it will be prompted that safe puncture is not possible and the fixed position of the needle needs to be readjusted; if the needle has a lateral motion control mechanism, the needle position is adjusted laterally at a fixed step length for searching; if the needle does not have a lateral motion control mechanism, it will be prompted that the position is inappropriate and the operator needs to make adjustments.

8. The automatic vein and artery puncture control method according to claim 7, characterized in that: The S6 includes: S61, after the puncture path planning is successful, the control device compares the current angle and needle tip position information of the needle body to be punctured with the planned puncture path to form displacement vector information; the displacement vector information includes axial displacement and angular displacement; S62, the control device converts the displacement vector information into a control signal and transmits it to the driving device; S63, the driving device receives the displacement vector information, outputs an electrical signal, and controls the motor movement of the automatic venous and arterial puncture device to perform puncture.

9. An automatic venous arterial puncture device, which performs automatic venous arterial puncture based on the control method according to any one of claims 1 to 8, characterized in that: It includes: a fixing device, a guiding device and a driving device; The fixing device is slidably connected to the guide device, and the fixing device is fixedly connected to the driving device; wherein: The fixing device is used to fix the needle body to be punctured and the hose to be inserted; the hose to be inserted is fixed to the fixing device through the needle body to be punctured; The guide device is used to determine the target blood vessel position, that is, the puncture target position; The driving device is used to drive the fixing device to move along the direction guided by the guiding device; The fixing device comprises: a syringe, an inner cylinder, an outer cylinder, a first connecting piece, a fixing plate, a needle clamp and a needle withdrawal button; the hose to be inserted is fixedly arranged on the syringe through the needle body to be punctured; the syringe is arranged in the inner cylinder and fixedly connected to the inner cylinder; the inner cylinder is arranged in the outer cylinder through the first connecting piece, and the outer cylinder can be slidably connected to the inner cylinder; the outer cylinder is fixedly arranged on the fixing plate through the first connecting piece; the needle clamp cooperates with the needle withdrawal button to trigger the needle withdrawal button under the action of external force, so that the needle body to be punctured and the hose to be inserted are separated from the fixing device; after the needle body to be punctured is fixed on the fixing device, its initial position and the relative spatial position between the ultrasonic probe are determined and no longer change, and the fixing device is calibrated and used for the control algorithm to realize the puncture control based on ultrasonic image positioning; The guide device comprises three parts: an ultrasonic probe, a needle guide and an outer tube guide, which is used to guide the fixing device and the needle to be punctured to move in a fixed direction based on ultrasound positioning of the blood vessel; the ultrasonic probe is used to collect data information of the preselected target blood vessel through ultrasound and send the data information to the control device; the needle guide is fixedly arranged on the ultrasonic probe, and the needle to be punctured is mounted on the needle guide; the outer tube guide is fixedly arranged on the fixing plate, and the outer tube is mounted on the outer tube guide; The ultrasonic probe is used in conjunction with the ultrasonic coupling sleeve, and the ultrasonic coupling sleeve is installed at the front end of the ultrasonic probe in an installation manner that can be disassembled multiple times and is easy to disassemble; The driving device is connected to the control device, and is used for driving the fixing device to move along the direction guided by the guide device; the driving device is divided into a needle control part driving device and a tube placement part driving device; the needle control part driving device comprises: a needle control motor and a second connecting piece; the needle control motor is fixedly arranged on the fixing plate, and is fixedly connected to the inner cylinder through the second connecting piece; the tube placement part driving device comprises: a tube placement motor, a third connecting piece, an outer cylinder connecting rod and a motor connecting rod; the tube placement motor is fixedly arranged on the fixing plate through the motor connecting rod, the tube placement motor is fixedly connected to the third connecting piece, and the third connecting piece is connected to the outer cylinder through the outer cylinder connecting rod.

10. A venous arterial puncture system, characterized in that: The automatic venous arterial puncture device is completed by the cooperation of the control device and the automatic venous arterial puncture device according to claim 9, wherein the automatic venous arterial puncture device is connected to the control device, wherein: The automatic venous arterial puncture device collects data information of a preselected target blood vessel through an ultrasonic probe, and sends the data information of the preselected target blood vessel to the control device; The control device is used to receive the data information of the preselected target blood vessel, calculate the position information of the target blood vessel according to the data information, and generate a control signal for controlling the operation of the automatic venous artery puncture device according to the position information; the automatic venous artery puncture device controls the fixed needle body to be punctured and the hose to be inserted to puncture the skin according to the control signal; The driving device accurately controls the movement step lengths of the needle control motor and the tube motor; the control device converts the blood vessel information collected by the ultrasonic probe into the position information and puncture path of the target blood vessel based on algorithm recognition and optimization; the control device compares the current coordinate information of the puncture point and the needle body to be punctured, calculates the displacement vector, and transmits it to the driving device in the form of a control signal; the driving device receives the control signal and moves to the puncture point according to the control signal.

Images