An AI image-based full-automatic intravenous puncture guiding method and system

Abstract

The application discloses a kind of based on AI image full-automatic intravenous puncture guiding method and system, it is related to medical technical field, the method includes: using ultrasonic sensor as imager, real-time image of real-time image of vein blood vessel of the intravenous puncture area of the arm of patient is collected;Real-time image of vein blood vessel is sent to image processing system;Using image processing system, real-time image of vein blood vessel is handled using first AI image processing model, to determine multiple transverse section vein characteristics and multiple longitudinal section vein characteristics;Using multi-axis robot, drive ultrasonic sensor to move to target puncture vein position;Real-time transverse section image of real-time vein of target puncture vein position corresponding target vein segment is collected;Using image processing system, real-time transverse section image is rendered into real-time vein visualization image capable of simulating blood flow dynamics using third AI image processing algorithm;Using auxiliary puncture display, real-time vein visualization image is displayed in real time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical treatment, in particular to an AI image-based full-automatic venipuncture guiding method and system. BACKGROUND

[0002] In the existing blood transfusion and collection industry, when performing venipuncture for blood transfusion and collection on the median cubital vein, experienced nurses use traditional hand feeling to feel the position of the vein, and then, after predicting the position and direction of the vein, perform blind puncture on the predicted vein. The existing blood transfusion and collection puncture process is as follows: puncture preparation, hand washing and disinfection, laying of a pad, bandaging, hand feeling to confirm the position of the vein, disinfection of the puncture point, asking the patient to clench the fist, re-disinfection, blood collection needle puncture, and fixing of the needle head by a blood collection patch.

[0003] In the existing blood transfusion and collection process, at least the following two problems exist: first, the existing traditional method of predicting the position of the vein based on the experience of nurses is not necessarily 100% accurate, and the failure rate is high when using traditional blind puncture of the blood collection needle, because the failure of blind puncture requires re-puncture, which causes pain to the blood transfusion and collection personnel. Second, with the gradual improvement of the domestic living standard, the number of obese people is also increasing year by year, and the depth of the vein of obese people is deeper due to muscle accumulation, so even experienced nurses have difficulty in correctly sensing the accurate position of the vein. Once-time puncture failure of such people may cause phlebemphraxis and swelling of the vein area, and cause pain to the patient.

[0004] In view of the above problems, no effective solution has been proposed so far. SUMMARY

[0005] The embodiments of the present application provide an AI image-based full-automatic venipuncture guiding method and system to solve the above technical problems.

[0006] The application provides an AI image-based full-automatic intravenous puncture guiding method, comprising the following steps: S101, using an ultrasonic sensor as an imaging device to collect real-time vein vessel images of an intravenous puncture area of a patient's arm in real time; S102, sending the real-time vein vessel images to an image processing system; S103, using the image processing system to process the real-time vein vessel images by using a first AI image processing model to determine a plurality of cross-sectional vein features and a plurality of longitudinal vein features; S104, using the image processing system to analyze positions corresponding to the plurality of cross-sectional vein features and positions corresponding to the plurality of longitudinal vein features by using a second AI image processing model to determine a target puncture vein position; wherein the target puncture vein position comprises a needle insertion position, a needle insertion direction and a needle insertion angle of intravenous puncture; S105, using a multi-axis robot to drive the ultrasonic sensor to move to the target puncture vein position; S106, using the ultrasonic sensor as an imaging device to collect real-time intravenous cross-sectional images of a target vein segment corresponding to the target puncture vein position in real time; S107, using the image processing system to render the real-time intravenous cross-sectional images into real-time intravenous visualization images capable of simulating blood flow dynamics by using a third AI image processing algorithm; S108, using an auxiliary puncture display to display the real-time intravenous visualization images in real time; wherein the real-time intravenous visualization images are used to determine a real-time needle insertion state of intravenous puncture and real-time image features of the target vein segment.

[0007] The application provides an AI image-based full-automatic intravenous puncture guiding system, comprising a multi-axis robot, an auxiliary puncture display, a control system and an ultrasonic sensor; wherein the ultrasonic sensor is used for real-time collection of real-time images of a vein blood vessel of an intravenous puncture area of a patient's arm and real-time collection of real-time transverse section images of a target vein segment corresponding to a target puncture vein position; the control system is used for sending the real-time images of the vein blood vessel to the image processing system and driving the ultrasonic sensor to move to the target puncture vein position by using the multi-axis robot; the control system comprises the image processing system; the image processing system is used for processing the real-time images of the vein blood vessel by using a first AI image processing model to determine a plurality of transverse section vein characteristics and a plurality of longitudinal section vein characteristics, analyzing positions corresponding to the plurality of transverse section vein characteristics and positions corresponding to the plurality of longitudinal section vein characteristics by using a second AI image processing model, and determining the target puncture vein position; wherein the target puncture vein position comprises a needle insertion position, a needle insertion direction and a needle insertion angle of intravenous puncture; the real-time transverse section images are rendered into real-time vein visualization images capable of simulating blood flow dynamics by using a third AI image processing algorithm; and the auxiliary puncture display is used for real-time display of the real-time vein visualization images; wherein the real-time vein visualization images are used for determination of a real-time needle insertion state of intravenous puncture and real-time image characteristics of the target vein segment.

[0008] Based on the embodiments provided in the present application, the ultrasonic sensor is used to collect the venous vessel image of the patient's arm in real time, the first AI image processing model is used to determine the multiple cross-sectional venous features and multiple longitudinal-sectional venous features, and then the second AI image processing model is used to comprehensively analyze the positions corresponding to the features to accurately determine the optimal target puncture venous position. Compared with the traditional method of relying on the experience of nurses to perceive and predict the venous position, the multi-dimensional feature analysis and positioning method based on AI can greatly improve the accuracy of puncture and reduce the puncture failure caused by inaccurate positioning. In the puncture process, the third AI image processing algorithm renders the real-time venous cross-sectional image into a real-time venous visualization image that can simulate the dynamic blood flow, and displays the image on the auxiliary puncture display in real time to guide the optimal needle insertion direction and angle. Medical staff can adjust the puncture state in real time according to the dynamic image to correct the deviation in time and further improve the success rate of puncture. Due to the improvement of the puncture success rate, the patient does not need to experience the process of multiple puncture failures and re-puncture. This can effectively avoid the problems such as venous region congestion and swelling caused by repeated puncture, and reduce the pain and discomfort of the patient. The multi-axis robot moves the ultrasonic sensor to the target puncture venous position according to the determined target puncture venous position. The accurate operation and control of the robot system can control the positioning error of the puncture needle within a very small range, realize precise puncture, reduce the damage and stimulation to the patient's vein, and thus reduce the pain of the patient. The whole venous puncture process is highly automated, and the collection of venous vessel image, feature analysis, target position determination, and accurate positioning and puncture of the puncture needle are all completed automatically by the system. This greatly reduces the operation steps and workload of medical staff in the puncture process and improves the work efficiency. The automatic puncture system can quickly and accurately complete the puncture task, avoiding the time-consuming process of searching for veins and multiple attempts to puncture in the traditional puncture process. This saves the waiting and treatment time of the patient and improves the work efficiency of the medical institution. BRIEF DESCRIPTION OF DRAWINGS

[0009] The accompanying drawings, which are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and serve to explain the present application and, to appropriate, do not limit the present application. In the drawings:

[0010] Figure 1 A flowchart of an optional AI image-based full-automatic venous puncture guiding method according to an embodiment of the present application;

[0011] Figure 2 A flowchart of another optional AI image-based full-automatic venous puncture guiding method according to an embodiment of the present application;

[0012] Figure 3 A schematic diagram of an AI image-based full-automatic venous puncture guiding system according to an embodiment of the present application;

[0013] Figure 4 A schematic view of a patient's arm being secured on an arm contouring platform;

[0014] Figure 5 A schematic view of a multi-axis collaborative robot, an ultrasonic sensor, and an ultrasonic sensor mounting bracket assembly;

[0015] Figure 6 A schematic view of a multi-axis collaborative robot, an ultrasonic sensor, an ultrasonic sensor mounting bracket assembly, and an ultrasonic sensor horizontal angle compensation shaft;

[0016] Figure 7 A schematic view of another AI image-based full-automatic intravenous puncture guiding system according to an embodiment of the present application;

[0017] Figure 8 A schematic view of a constituent element of an AI image-based full-automatic intravenous puncture guiding system;

[0018] Figure 9 A control flowchart of an AI image-based full-automatic intravenous puncture guiding system.

[0019] The implementation, functional features, and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0021] Optionally, as shown in the present application, an AI image-based full-automatic intravenous puncture guiding method is provided, comprising: Figure 1

[0022] S101, using an ultrasonic sensor as an imaging device to collect real-time images of a vein in a vein puncture area of a patient's arm in real time;

[0023] S102, sending the real-time images of the vein to an image processing system;

[0024] S103, using the image processing system to process the real-time images of the vein by using a first AI image processing model to determine a plurality of cross-sectional vein features and a plurality of longitudinal vein features;

[0025] ​S104, using the image processing system, analyzing the positions corresponding to the multiple cross-section vein features and the positions corresponding to the multiple longitudinal-section vein features by using the second AI image processing model to determine a target puncture vein position; wherein the target puncture vein position includes a needle insertion position, a needle insertion direction, and a needle insertion angle of the vein puncture;

[0026] In the embodiments of the present application, the needle insertion direction of the vein puncture can include but is not limited to the needle tip pointing direction and the needle tip bevel direction. The needle insertion direction is to select the best direction to puncture, and the needle insertion direction is the direction parallel to the probe.

[0027] The needle tip pointing direction is used to describe the advancing direction of the puncture needle tip; and the needle tip bevel direction is used to describe the orientation of the puncture needle bevel.

[0028] The needle insertion angle is used to describe the angle between the puncture needle and the skin, which is a definition of the depth from the skin to the blood vessel. The second AI image processing model can be used to determine a target vein segment, and then determine the target puncture vein position according to the target vein segment; wherein the target vein segment can be determined based on the following method:

[0029] A weight learning model based on machine learning or deep learning, i.e., a second AI image processing model, is constructed. The goal of this model is to learn the importance weight of different features in determining the target vein segment. One of the following models can be used:

[0030] Multi-layer perception: using multi-layer perception to learn feature weights, the input layer of the model corresponds to each feature, the hidden layer is used for nonlinear mapping of features, and the output layer outputs the weight of each feature;

[0031] Support vector machine: using the kernel method of support vector machine to learn feature weights, by selecting a suitable kernel function, the nonlinear relationship between features can be effectively handled;

[0032] Deep neural network model: a deep neural network model is constructed, which learns feature weights through multiple layers of nonlinear transformation. Each layer of the model can abstract and combine features at a deeper level.

[0033] The second AI image processing model is trained using a labeled training data set. The training data set should include the feature values of multiple vein segments and their corresponding target vein segment labels (i.e., whether they are target vein segments). During training, the model adjusts the weight parameters by optimizing the loss function, so that the model can accurately predict the target vein segment.

[0034] S105, using a multi-axis robot to drive the ultrasonic sensor to move to the target puncture vein position;

[0035] S106, using an ultrasonic sensor as an imaging device, real-time acquisition of the real-time vein cross-section image of the target vein segment corresponding to the target puncture vein position;

[0036] S107, using an image processing system, rendering the real-time vein cross-section image into a real-time vein visualization image capable of simulating blood flow dynamics using a third AI image processing algorithm;

[0037] S108, using an auxiliary puncture display to display the real-time vein visualization image in real time; wherein the real-time vein visualization image is used to determine the real-time needle insertion state of the vein puncture and the real-time image characteristics of the target vein segment.

[0038] The real-time vein visualization image can help the puncture personnel to observe the needle insertion state of the vein puncture in real time; the puncture personnel puncture according to the displayed vein position, and the display displays the state of the puncture needle entering the vein in real time until the puncture personnel judges that the puncture is successful.

[0039] Based on the embodiments provided in the present application, the ultrasonic sensor is used to collect the real-time image of the venous vessel of the patient's arm in real time, the first AI image processing model is used to determine the multiple cross-sectional venous features and multiple longitudinal-sectional venous features, and then the second AI image processing model is used to comprehensively analyze the positions corresponding to the features to accurately determine the optimal target puncture venous position. Compared with the traditional method of relying on the experience of nurses to perceive and predict the venous position, the multi-dimensional feature analysis and positioning method based on AI greatly improves the accuracy of puncture and reduces the failure of puncture caused by inaccurate positioning. In the puncture process, the third AI image processing algorithm renders the real-time venous cross-sectional image into a real-time venous visualization image that can simulate the dynamic blood flow, and displays the real-time venous visualization image on the auxiliary puncture display to guide the optimal needle insertion direction and angle. Medical staff can adjust the puncture state in real time according to the dynamic image to correct the deviation in time and further improve the success rate of puncture. Due to the improvement of the success rate of puncture, the patient does not need to experience the process of multiple puncture failures and re-puncture. This can effectively avoid the problems such as venous region congestion and swelling caused by repeated puncture, and reduce the pain and discomfort of the patient. The multi-axis robot drives the ultrasonic sensor to move to the target puncture venous position. The accurate operation and control of the robot system can control the positioning error of the puncture needle within a very small range, realize precise puncture, reduce the damage and stimulation to the patient's vein, and thus reduce the pain of the patient. The whole venous puncture process is highly automated, from the collection of venous vessel image, feature analysis, target position determination to the accurate positioning and puncture of the puncture needle, which are all completed automatically by the system. This greatly reduces the operation steps and workload of medical staff in the puncture process, and improves the work efficiency. The automatic puncture system can quickly and accurately complete the puncture task, avoiding the time-consuming process of searching for veins and multiple attempts in the traditional puncture process. This saves the waiting and treatment time of the patient, and also improves the work efficiency of the medical institution.

[0040] Further, before S101, the method further comprises the following steps:

[0041] S100, a multi-axis robot is used to drive the ultrasonic sensor to move to the venous puncture area of the patient's arm; wherein the patient's arm is fixed on the arm shaping platform by the wrist fixing sleeve and the arm end fixing sleeve; the arm end fixing sleeve has an inflation function; the multi-axis robot drives the ultrasonic sensor based on multi-joint force control sensing.

[0042] An auxiliary puncture display is used to display the collected real-time image of the venous vessel of the venous puncture area of the patient's arm in real time; this step can be performed after S101;

[0043] The auxiliary puncture display is used to display the real-time vein cross-section image of the target vein segment in real time; this step can be performed after S106.

[0044] For example, Figure 2 A flowchart of another AI image-based full-automatic vein puncture guidance method.

[0045] Further, the first AI image processing model is trained and optimized based on the following steps:

[0046] S301, based on the improved U-Net model combined with attention mechanism and Transformer architecture, a first AI image processing model is established; the improved U-Net model is responsible for image segmentation, and the Transformer architecture is used for global correlation and fusion of features;

[0047] S302, a vein vessel image sample set is obtained, and the cross-section vein features and longitudinal section vein features corresponding to each vein vessel image sample in the vein vessel image sample set are labeled; wherein the cross-section vein features and longitudinal section vein features have corresponding positions;

[0048] S303, according to the labeled vein vessel image sample, a first training sample set and a first test sample set are established;

[0049] S304, for the plurality of vein vessel image samples included in the first training sample set, an enhanced algorithm based on Hessian matrix is applied for preprocessing;

[0050] S305, using the structure of the improved U-Net model, combined with the attention mechanism, the plurality of vein vessel image samples after preprocessing are subjected to image segmentation to obtain cross-section vein images and longitudinal section vein images;

[0051] S306, the morphological features of the blood vessels are extracted from the segmented cross-section vein images; wherein the morphological features are used to analyze the symmetry and shape features of the blood vessels;

[0052] S307, using a Gabor filter, the texture features of the blood vessel wall are extracted from the segmented cross-section vein images; wherein the texture features include texture density and texture direction;

[0053] S308, using a Frangi filter, the centerline and branch structure of the blood vessels are extracted from the segmented longitudinal section vein images to analyze the features of the path and branch of the blood vessels;

[0054] S309, input the extracted cross-section features and longitudinal-section features into the Transformer architecture; wherein the cross-section features include morphological features of the blood vessels and texture features of the blood vessel walls; the longitudinal-section features include features of the paths and branching conditions of the blood vessels;

[0055] S310, utilize the self-attention mechanism of the Transformer architecture to globally correlate and fuse the cross-section features and the longitudinal-section features to obtain a fused feature matrix;

[0056] S311, match the fused feature matrix output by the Transformer architecture with the labeled cross-section venous features and longitudinal-section venous features to train the first AI image processing model;

[0057] S312, apply the first AI image processing model trained and optimized to the first test sample set to evaluate the multiple venous blood vessel image samples included in the first test sample set.

[0058] Further, the cross-section venous features include the blood vessel wall thickness, the blood vessel lumen size, the blood vessel cross-sectional shape, and the texture density of the blood vessel wall; the longitudinal-section venous features include the blood vessel direction, the branching structure, the blood vessel length, the blood vessel curvature, and the venous valve position;

[0059] In the structure of the improved U-Net model, the encoder gradually reduces the spatial dimension of the image while extracting the feature information of the image; the decoder gradually restores the spatial dimension of the image and uses the feature information extracted by the encoder for image reconstruction and segmentation;

[0060] The attention mechanism is used to enhance the attention degree of the improved U-Net model to key features, including: paying attention to the thickness change of the blood vessel wall when extracting cross-section features; paying attention to the branching points of the blood vessels when extracting longitudinal-section features.

[0061] Further, ;

[0062] Wherein, is the fused feature matrix; is a query matrix used for matching and correlating with other features in the self-attention mechanism; is a key matrix used for matching with the query vector in the self-attention mechanism; is the transpose of the key matrix; in the self-attention mechanism, the query matrix performs dot product operation with the transpose of the key matrix to obtain an attention score matrix; is the dimension of the key vector; is a value matrix used for weighted summation according to the attention weight in the self-attention mechanism; The function is used for normalizing the attention score matrix; is a layer normalization operation, which is used for normalizing the result after feature fusion; layer normalization is usually performed on each feature dimension, so that the mean of each feature is 0 and the variance is 1; The original cross-sectional features and longitudinal features input into the Transformer architecture are connected in residual connection with , so as to retain the information of the original features.

[0063] When training the improved U-Net model, the following loss function can be used to optimize the segmentation effect:

[0064] ;

[0065] wherein, is a Dice loss function, which is used to measure the overlap between the segmentation result and the true label; is a binary cross-entropy loss function, which is used to measure the accuracy of the segmentation result; is a weight parameter for balancing the Dice loss and the binary cross-entropy loss.

[0066] Further, while using an ultrasonic sensor as an imaging device to collect real-time vein cross-sectional images of a target vein segment corresponding to a target puncture vein position in real time, Doppler signals are collected; the Doppler signals include blood flow feature information;

[0067] S104, using a third AI image processing algorithm to render the real-time vein cross-sectional images into real-time vein visualization images that can simulate blood flow dynamics, including:

[0068] S402, adaptively selecting a wavelet basis and a decomposition layer number according to the complexity and noise characteristics of each sub-region included in the real-time vein cross-sectional image; for sub-regions in the noise feature set, select Daubechies wavelet; for sub-regions with blood flow detail complexity greater than a complexity threshold, select Symlets wavelet;

[0069] On different scales after wavelet decomposition, adaptively set the threshold according to the local features of each sub-region, and process each sub-region; for low-frequency sub-images, the threshold is small, and more structural information is retained; for high-frequency sub-images, the threshold is large, effectively removing noise while retaining blood flow detail features;

[0070] S403, inverse wavelet transform the processed sub-regions to obtain enhanced sub-regions; use a Laplacian pyramid fusion algorithm to fuse the enhanced sub-regions with the original sub-regions to obtain an enhanced real-time vein cross-sectional image;

[0071] S404. For each frame in the image sequence of the enhanced vein cross-section image, calculate the dynamic speckle contrast image sequence corresponding to each frame to analyze the dynamic change characteristics of the pixels in each frame in continuous frames. Unlike traditional speckle contrast, dynamic speckle contrast not only considers the intensity change of pixels, but also introduces the time dimension to analyze the dynamic change characteristics of the pixels in each frame in continuous frames, thereby more accurately reflecting blood flow information.

[0072] S405, The trained deep learning segmentation model is used to segment the dynamic speckle contrast image sequence to obtain the blood flow region segmentation result; wherein, the deep learning segmentation model is learned by the labeled training dynamic speckle contrast image sequence to learn the feature differences between the blood flow region and the static tissue region.

[0073] S406 uses image morphological operations to optimize the blood flow region segmentation results, fills the holes in the blood flow region, and obtains the blood flow region image; among which, image morphological operations include erosion, dilation, opening operation and closing operation;

[0074] S407 uses an adaptive filter to remove noise and interference signals from the acquired Doppler signal to obtain an optimized Doppler signal;

[0075] S408 introduces a particle system to simulate the dynamic process of blood flow based on the blood flow characteristic information included in the optimized Doppler signal, and obtains the blood flow simulation results. Each particle represents a blood flow unit, and the motion state of each particle is updated according to the blood flow characteristic information. The particle system is used to simulate the flow, diffusion, and convergence of blood flow. The motion state of each particle includes position, velocity, and direction.

[0076] S409 combines the blood flow simulation results of the particle system with the blood flow region image for visualization rendering;

[0077] Among its features, advanced rendering technologies (such as shadow rendering and texture mapping) are employed to enhance the visual effects of blood flow, making its dynamic changes more clearly visible. Simultaneously, user interaction functions are provided, such as adjusting the viewing angle, zooming in and out of the image, pausing and replaying the blood flow animation, allowing users to more intuitively observe and analyze the blood flow dynamics.

[0078] The velocity update formula for particles in the particle system is as follows:

[0079] ;

[0080] in, Indicates the first The blood flow unit corresponding to each particle is subjected to the superposition of multiple forces, including the resistance of the blood vessel wall and the viscous force of the blood flow. For the first mass of the particle; is the first acceleration generated by the superposition of various forces on the blood flow unit corresponding to the particle, according to Newton's second law, the acceleration is the total force divided by the mass; is the acceleration calculated according to the blood flow characteristic information, used to simulate the influence of the dynamic change of the blood flow on the particle motion; is the time step, indicating the time interval for updating the particle state each time in the simulation process.

[0081] The formula complexly updates the velocity of the particle by comprehensively considering the various forces on the particle and the blood flow characteristic information, thereby more realistically simulating the dynamic motion process of the blood flow unit in the blood flow, including flow, diffusion, and convergence phenomena.

[0082] Optionally, the present application provides an AI image-based full-automatic venipuncture guiding system, comprising a multi-axis robot, an auxiliary puncture display, a control system, and an ultrasonic sensor;

[0083] The ultrasonic sensor is configured to collect real-time images of a venous vessel in a venipuncture region of a patient's arm in real time, and collect real-time transverse section images of a target venous segment corresponding to a target puncture venous position.

[0084] The control system is configured to send the real-time images of the venous vessel to an image processing system, and drive the ultrasonic sensor to move to the target puncture venous position by using the multi-axis robot.

[0085] The control system comprises an image processing system, which is configured to process the real-time images of the venous vessel by using a first AI image processing model to determine a plurality of transverse section venous characteristics and a plurality of longitudinal section venous characteristics, analyze positions corresponding to the plurality of transverse section venous characteristics and positions corresponding to the plurality of longitudinal section venous characteristics by using a second AI image processing model to determine the target puncture venous position, and render the real-time transverse section images into real-time venous visualization images capable of simulating blood flow dynamics by using a third AI image processing algorithm.

[0086] The auxiliary puncture display is configured to display the real-time venous visualization images in real time, wherein the real-time venous visualization images are used to determine a real-time needle insertion state of the venipuncture and real-time image characteristics of the target venous segment.

[0087] Further, the AI image-based full-automatic venipuncture guiding system further comprises:

[0088] The arm profiling platform, the trolley assembly, the arm bearing table board, the main control cabinet, the ultrasonic sensor fixing support assembly, the ultrasonic sensor horizontal angle compensation shaft, the power module, the wrist fixing sleeve, the arm end fixing sleeve and the fixing belt inflation control module; wherein the fixing belt is the wrist fixing sleeve and / or the arm end fixing sleeve.

[0089] Further, the multi-axis robot drives the ultrasonic sensor based on the multi-joint force control sensing mode;

[0090] The multi-axis robot is also used to drive the ultrasonic sensor to move to the venipuncture area of the patient's arm before using the ultrasonic sensor as an imaging device to collect the real-time image of the venous vessel of the venipuncture area of the patient's arm in real time; wherein the patient's arm is fixed on the arm profiling platform by the wrist fixing sleeve and the arm end fixing sleeve; the arm end fixing sleeve has an inflation function;

[0091] The auxiliary puncture display is also used to display the collected real-time image of the venous vessel of the venipuncture area of the patient's arm and the real-time venous cross-section image of the target vein segment in real time.

[0092] Taking the multi-axis robot as an example, Figure 3 It is a schematic diagram of an AI image-based full-automatic venipuncture guiding system.

[0093] It should be understood that the multi-axis robot can also be a collaborative four-axis robot, a collaborative eight-axis robot, etc., and the present application does not limit this. In the following examples of the present application, the collaborative six-axis robot is mainly taken as an example for illustration.

[0094] Figure 4 It is a schematic diagram of a patient's arm fixed on an arm profiling platform.

[0095] Figure 5 It is a schematic diagram of a multi-axis collaborative robot, an ultrasonic sensor and an ultrasonic sensor fixing support assembly.

[0096] Figure 6 It is a schematic diagram of a multi-axis collaborative robot, an ultrasonic sensor, an ultrasonic sensor fixing support assembly and an ultrasonic sensor horizontal angle compensation shaft.

[0097] Figure 7 It is a schematic diagram of an AI image-based full-automatic venipuncture guiding system.

[0098] Figure 8 It is a schematic diagram of the constitutive elements of an AI image-based full-automatic venipuncture guiding system.

[0099] Figure 9 It is a control flowchart of an AI image-based full-automatic venipuncture guiding system.

[0100] It should be noted that, in the present application, the embodiments implemented based on the AI image full-automatic venipuncture guiding system can be mutually referenced with the embodiments implemented based on the AI image full-automatic venipuncture guiding method, and the present application will not be described one by one.

[0101] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An AI image-based full-automatic intravenous puncture guiding system, characterized in that, The system comprises a multi-axis robot, an auxiliary puncture display, a control system and an ultrasonic sensor. The ultrasonic sensor is configured to collect real-time images of a venous vessel in a venipuncture area of a patient's arm and real-time cross-sectional images of a target venous segment corresponding to a target puncture position. The control system is configured to send the real-time images of the venous vessel to an image processing system and drive the ultrasonic sensor to move to the target puncture position by using the multi-axis robot. The control system comprises the image processing system, which is configured to determine a plurality of cross-sectional venous features and a plurality of longitudinal venous features by processing the real-time images of the venous vessel using a first AI image processing model, analyze positions corresponding to the cross-sectional venous features and positions corresponding to the longitudinal venous features using a second AI image processing model, determine the target puncture position, wherein the target puncture position comprises a needle insertion position, a needle insertion direction and a needle insertion angle, and render the real-time cross-sectional images of the venous vessel into real-time visualized images of the venous vessel that can simulate blood flow dynamics using a third AI image processing algorithm. The auxiliary puncture display is configured to display the real-time visualized images of the venous vessel in real time, which are used to determine a real-time needle insertion state of the venipuncture and real-time image features of the target venous segment. 2.The AI-based image full-automatic intravenous puncture guiding system according to claim 1, characterized in that, The AI image-based full-automatic venipuncture guiding system further comprises: an arm profiling platform, a trolley assembly, an arm bearing table, a main control cabinet, an ultrasonic sensor fixing support assembly, an ultrasonic sensor horizontal angle compensation shaft, a power module, a wrist fixing cuff, an arm end fixing cuff and a fixing belt inflation control module, wherein the fixing belt is the wrist fixing cuff and / or the arm end fixing cuff.

3. The AI image-based full-automatic venipuncture guiding system according to claim 2, wherein the multi-axis robot drives the ultrasonic sensor based on a multi-joint force control sensing mode; the multi-axis robot is further configured to drive the ultrasonic sensor to move to a venipuncture area of a patient's arm before collecting real-time images of a venous vessel in the venipuncture area of the patient's arm using the ultrasonic sensor as an imaging device, wherein the patient's arm is fixed on the arm profiling platform by the wrist fixing cuff and the arm end fixing cuff, and the arm end fixing cuff has an inflation function; the auxiliary puncture display is further configured to display the collected real-time images of the venous vessel in the venipuncture area of the patient's arm and the real-time cross-sectional images of the target venous segment in real time. 4.The AI-based image full-automatic intravenous puncture guiding system according to claim 1, characterized in that, The first AI image processing model is trained and optimized based on the following steps: S301, based on an improved U-Net model combined with attention mechanism and Transformer architecture, the first AI image processing model is established. S302, obtain a set of venous vessel image samples, and label the cross-sectional venous features and longitudinal-sectional venous features corresponding to each venous vessel image sample in the set of venous vessel image samples; wherein the cross-sectional venous features and the longitudinal-sectional venous features each have a corresponding position; S303, establish a first training sample set and a first test sample set according to the labeled venous vessel image samples; S304, apply a Hessian matrix-based enhancement algorithm to pre-process a plurality of venous vessel image samples included in the first training sample set; S305, use the structure of the improved U-Net model and combine an attention mechanism to perform image segmentation on the pre-processed plurality of venous vessel image samples, to obtain cross-sectional venous images and longitudinal-sectional venous images; S306, extract morphological features of the blood vessels from the segmented cross-sectional venous images; wherein the morphological features are used to analyze the symmetry and shape features of the blood vessels; S307, use a Gabor filter to extract texture features of the blood vessel walls from the segmented cross-sectional venous images; wherein the texture features include texture density and texture direction; S308, use a Frangi filter to extract the centerline and branch structure of the blood vessels from the segmented longitudinal-sectional venous images, to analyze the features of the path and branch conditions of the blood vessels; S309, input the extracted cross-sectional features and longitudinal-sectional features into a Transformer architecture; wherein the cross-sectional features include the morphological features of the blood vessels and the texture features of the blood vessel walls; and the longitudinal-sectional features include the features of the path and branch conditions of the blood vessels; S310, use the self-attention mechanism of the Transformer architecture to globally correlate and fuse the cross-sectional features and the longitudinal-sectional features, to obtain a fusion feature matrix; S311, match the fusion feature matrix output by the Transformer architecture with the labeled cross-sectional venous features and longitudinal-sectional venous features, and train the first AI image processing model; S312, apply the first AI image processing model that has been trained and optimized to the first test sample set, and evaluate a plurality of venous vessel image samples included in the first test sample set.

5. The AI image-based full-automatic venous puncture guiding system according to claim 4, wherein the cross-sectional venous features include the blood vessel wall thickness, the blood vessel lumen size, the blood vessel cross-sectional shape, and the texture density of the blood vessel wall; and the longitudinal-sectional venous features include the blood vessel direction, the branch structure, the blood vessel length, the blood vessel curvature, and the venous valve position; in the structure of the improved U-Net model, the encoder gradually reduces the spatial dimension of the image while extracting the feature information of the image; the decoder gradually recovers the spatial dimension of the image, and uses the feature information extracted by the encoder for image reconstruction and segmentation; wherein the attention mechanism is used to enhance the attention degree of the improved U-Net model to the key features, including paying attention to the thickness change of the blood vessel wall when extracting the cross-sectional features, and paying attention to the branch points of the blood vessels when extracting the longitudinal-sectional features. ​ 6.The AI-based image full-automatic intravenous puncture guiding system according to claim 1, characterized in that, The ultrasonic sensor is used as an imaging device to collect real-time vein cross-section images of a target vein segment corresponding to a target puncture vein position of the patient in real time, and Doppler signals are collected at the same time; the Doppler signals include blood flow characteristic information; The real-time vein cross-section images are rendered into real-time vein visualization images capable of simulating blood flow dynamics by using a third AI image processing algorithm, including: S401, a wavelet basis and a decomposition layer number are adaptively selected according to the complexity and noise characteristics of each sub-region included in the real-time vein cross-section images; S402, on different scales after wavelet decomposition, a threshold is adaptively set according to the local characteristics of each sub-region, and each sub-region is processed; S403, each processed sub-region is inverse wavelet transformed to obtain each enhanced sub-region; the enhanced sub-regions and the original sub-regions are fused by using a Laplacian pyramid fusion algorithm to obtain an enhanced real-time vein cross-section image; S404, for each frame of image in the image sequence of the enhanced vein cross-section image, a dynamic speckle contrast image sequence corresponding to each frame of image is calculated to analyze the dynamic change characteristics of the pixel points in each frame of image in the continuous frames. 7.The AI image-based full-automatic intravenous puncture guiding system according to claim 6, characterized in that, The real-time vein cross-section images are rendered into real-time vein visualization images capable of simulating blood flow dynamics by using a third AI image processing algorithm, and the rendering further includes: S405, a trained deep learning segmentation model is used to segment the dynamic speckle contrast image sequence to obtain a blood flow region segmentation result; wherein the deep learning segmentation model is learned by using a labeled training dynamic speckle contrast image sequence to learn the feature differences between the blood flow region and the static tissue region; S406, an image morphological operation is used to optimize the blood flow region segmentation result to fill the holes of the blood flow region to obtain a blood flow region image; wherein the image morphological operation includes erosion, dilation, opening operation and closing operation; S407, an adaptive filter is used to remove noise and interference signals of the collected Doppler signals to obtain an optimized Doppler signal; S408, a particle system is introduced to simulate the dynamic process of blood flow according to the blood flow characteristic information included in the optimized Doppler signal to obtain a blood flow simulation result; wherein each particle represents a blood flow unit, and the motion state of each particle is updated according to the blood flow characteristic information; the particle system is used to simulate the flow, diffusion and convergence of blood flow; the motion state of each particle includes position, speed and direction; S409, the blood flow simulation result of the particle system is combined with the blood flow region image for visualization rendering.

Images