Full-automatic venipuncture guiding method and system based on AI image

Abstract

The invention discloses a full-automatic venipuncture guiding method and system based on an AI image, and relates to the technical field of medical treatment.The method comprises the steps that an ultrasonic sensor is adopted as an imaging device, and real-time vein blood vessel images of a venipuncture area of an arm of a patient are collected in real time; sending the vein real-time image to an image processing system; processing the vein real-time image by adopting an image processing system and utilizing a first AI image processing model so as to determine a plurality of cross section vein features and a plurality of longitudinal section vein features; using a multi-axis robot to drive an ultrasonic sensor to move to a target puncture vein position; acquiring a real-time vein cross section image of a target vein segment corresponding to the target puncture vein position in real time; rendering the real-time vein cross section image into a real-time vein visual image capable of simulating the dynamic state of blood flow by adopting an image processing system and utilizing a third AI image processing algorithm; and an auxiliary puncture display is adopted to display a real-time vein visual image in real time.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and more specifically, to a fully automatic venous puncture guidance method and system based on AI images. Background Art

[0002] In the existing blood transfusion and collection industry, when inserting a blood transfusion needle into the cubital vein, experienced nurses use traditional hand palpation to feel the location of the vein, and then use blind puncture to puncture the predicted vein after predicting the location and direction of the vein based on experience. The existing blood transfusion and collection puncture process is: puncture preparation, hand washing and disinfection, laying a blanket, tying a tourniquet, confirming the location of the vein by hand, disinfecting the puncture point, asking the patient to make a fist, disinfecting again, puncturing with a blood collection needle, and fixing the needle with a blood collection patch.

[0003] In the existing blood transfusion and blood collection process, there are at least two shortcomings: First, the existing traditional method of predicting the location of the vein based on the experience of nurses is not necessarily 100% accurate. When using the traditional blind puncture blood collection needle, the probability of failure is high, because the failure of blind puncture causes pain to the blood transfusion personnel when re-puncturing. Second, with the gradual improvement of domestic living standards, the number of obese people is also increasing year by year, and the veins of obese people are deeper due to muscle accumulation. Even nurses with certain puncture experience find it difficult to correctly perceive the exact location of the vein. Once such people fail to puncture once, it may cause congestion and swelling in the vein area, and bring pain to the patient.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a fully automatic venous puncture guidance method and system based on AI images to solve the above-mentioned technical problems.

[0006] The present application provides a fully automatic venous puncture guidance method based on AI images, including: S101, using an ultrasonic sensor as an imaging device to collect real-time images of the venous blood vessels in the venous puncture area of ​​the patient's arm in real time; S102, sending the real-time images of the venous blood vessels to an image processing system; S103, using the image processing system, using a first AI image processing model to process the real-time images of the venous blood vessels to determine multiple cross-sectional vein features and multiple longitudinal vein features; S104, using the image processing system, using a second AI image processing model to analyze the positions corresponding to the multiple cross-sectional vein features and the positions corresponding to the multiple longitudinal vein features to determine the target puncture vein position; wherein, the target puncture vein The position includes the needle insertion position, needle insertion direction and needle insertion angle of the venous puncture; S105, using a multi-axis robot to drive the ultrasonic sensor to move to the target puncture vein position; S106, using the ultrasonic sensor as an imaging device, real-time acquisition of the real-time venous cross-section image of the target vein segment corresponding to the target puncture vein position; S107, using the image processing system, using the third AI image processing algorithm to render the real-time venous cross-section image into a real-time venous visualization image that can simulate blood flow dynamics; S108, using an auxiliary puncture display to display the real-time venous visualization image in real time; wherein, the real-time venous visualization image is used to determine the real-time needle insertion state of the venous puncture and the real-time imaging characteristics of the target vein segment.

[0007] The present application provides a fully automatic venous puncture guidance system based on AI images, comprising: a multi-axis robot, an auxiliary puncture display, a control system and an ultrasonic sensor; wherein the ultrasonic sensor is used to collect real-time images of the venous blood vessels in the venous puncture area of ​​the patient's arm; and to collect real-time venous cross-sectional images of the target vein segment corresponding to the target puncture vein position; the control system is used to send the real-time images of the venous blood vessels to the image processing system; and to use the multi-axis robot to drive the ultrasonic sensor to move to the target puncture vein position; the control system includes the image processing system; the image processing system is used to use the first AI image processing model to process the real-time images of the venous blood vessels The image is processed to determine multiple cross-section vein features and multiple longitudinal section vein features; the positions corresponding to the multiple cross-section vein features and the positions corresponding to the multiple longitudinal section vein features are analyzed by a second AI image processing model to determine the target puncture vein position; wherein the target puncture vein position includes the needle insertion position, needle insertion direction and needle insertion angle of the venous puncture; the real-time vein cross-section image is rendered into a real-time vein visualization image that can simulate blood flow dynamics by a third AI image processing algorithm; the auxiliary puncture display is used to display the real-time vein visualization image in real time; wherein the real-time vein visualization image is used to determine the real-time needle insertion state of the venous puncture and the real-time image features of the target vein segment.

[0008] Based on the embodiment provided by the present application, an ultrasonic sensor is used to collect real-time venous blood vessel images of the patient's arm, and a plurality of cross-section venous features and a plurality of longitudinal venous features are determined by a first AI image processing model, and then a second AI image processing model comprehensively analyzes the positions corresponding to these features to accurately determine the optimal target puncture vein position; this AI-based multi-dimensional feature analysis and positioning method greatly improves the accuracy of puncture and reduces puncture failures caused by inaccurate positioning compared to the traditional method in which nurses rely on experience to perceive and predict the location of veins; during the puncture process, the third AI image processing algorithm renders the real-time venous cross-section image into a real-time venous visualization image that can simulate the blood flow dynamics, and displays it in real time through an auxiliary puncture display to guide the optimal needle insertion direction and angle; medical staff can adjust the puncture status in real time according to these dynamic images, correct deviations in a timely manner, and further improve the success rate of puncture: due to the improvement in the success rate of puncture, patients do not need to experience multiple puncture failures. The robot can effectively avoid the problem of congestion and swelling in the vein area caused by repeated punctures, and relieve the pain and discomfort of patients. The multi-axis robot drives the ultrasonic sensor to move to the target puncture vein according to the determined target puncture vein position. The precise operation and control of the robot system can control the positioning error of the puncture needle within a very small range, achieve accurate puncture, reduce damage and stimulation to the patient's vein, and thus reduce the patient's pain. The entire venous puncture process is highly automated, from venous blood vessel image acquisition, feature analysis, target position determination to accurate positioning and puncture of the puncture needle, all of which are automatically completed by the system. This greatly reduces the operating steps and workload of medical staff during the puncture process and improves work efficiency. The automated puncture system can complete the puncture task quickly and accurately, avoiding the time spent on finding veins and multiple attempts at puncture in the traditional puncture process. It saves patients waiting and treatment time, and also improves the work efficiency of medical institutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flowchart of an optional AI image-based fully automatic venous puncture guidance method according to an embodiment of the present application; Figure 2 is a flowchart of another optional AI image-based fully automatic venipuncture guidance method according to an embodiment of the present application; Figure 3 is a schematic diagram of an AI image-based fully automatic venous puncture guidance system according to an embodiment of the present application; Figure 4A schematic diagram of a patient's arm being fixed on an arm contouring platform; Figure 5 It is a schematic diagram of a multi-axis collaborative robot, an ultrasonic sensor and an ultrasonic sensor fixing bracket assembly; Figure 6 It is a schematic diagram of a multi-axis collaborative robot, an ultrasonic sensor and an ultrasonic sensor fixing bracket assembly, and an ultrasonic sensor horizontal angle compensation axis; Figure 7 is a schematic diagram of another AI image-based fully automatic venipuncture guidance system according to an embodiment of the present application; Figure 8 A schematic diagram illustrating the components of an AI-based fully automatic venipuncture guidance system; Fig. 9 This is a control flow chart of a fully automatic venipuncture guidance system based on AI images.

[0010] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] Alternatively, if Figure 1 As shown, the present application provides a fully automatic venous puncture guidance method based on AI images, comprising: S101, using an ultrasonic sensor as an imaging device to collect real-time images of the venous blood vessels in the venipuncture area of ​​the patient's arm; S102, sending the real-time image of the venous blood vessels to the image processing system; S103, using an image processing system to process the real-time image of the venous blood vessels using a first AI image processing model to determine a plurality of cross-sectional venous features and a plurality of longitudinal venous features; S104, using an image processing system and a second AI image processing model to analyze the positions corresponding to the multiple cross-sectional vein features and the positions corresponding to the multiple longitudinal vein features to determine the target puncture vein position; wherein the target puncture vein position includes the needle insertion position, needle insertion direction, and needle insertion angle of the venous puncture; In the embodiment of the present application, the needle insertion direction of the venous puncture may include but is not limited to the needle tip direction and the needle tip bevel direction. The needle insertion direction is to select the best direction for puncture, and the needle insertion direction is the direction parallel to the probe.

[0013] Among them, the needle tip direction is used to describe the forward direction of the puncture needle tip; the needle tip bevel direction is used to describe the direction of the bevel of the puncture needle.

[0014] The needle insertion angle is used to describe the angle between the puncture needle and the skin, and is a definition of the depth from the skin to the blood vessel. The second AI image processing model can be used to determine the target vein segment, and then determine the target puncture vein position based on the target vein segment; the target vein segment can be determined based on the following methods: Build a weight learning model based on machine learning or deep learning, i.e. the second AI image processing model. The goal of this model is to learn the importance weights of different features in determining the target vein segment. One of the following models can be used: Multilayer Perceptron: Use a multilayer perceptron to learn feature weights. The input layer of the model corresponds to each feature, the hidden layer is used for nonlinear mapping of features, and the output layer outputs the weight of each feature. Support vector machine: Use the kernel method of support vector machine to learn feature weights. By selecting a suitable kernel function, the nonlinear relationship between features can be effectively handled. Deep neural network model: Build a deep neural network model to learn feature weights through multiple layers of nonlinear transformations. Each layer of the model can abstract and combine features at a deeper level.

[0015] The second AI image processing model is trained using the labeled training data set. The training data set should contain the feature values ​​of multiple vein segments and their corresponding target vein segment labels (i.e., whether it is a target vein segment). During the training process, the model adjusts the weight parameters by optimizing the loss function so that the model can accurately predict the target vein segment.

[0016] S105, using a multi-axis robot to drive the ultrasonic sensor to move to the target puncture vein position; S106, using an ultrasonic sensor as an imaging device to collect a real-time vein cross-section image of a target vein segment corresponding to the target puncture vein position in real time; S107, using an image processing system to render the real-time vein cross-section image into a real-time vein visualization image capable of simulating blood flow dynamics using a third AI image processing algorithm; S108, using an auxiliary puncture display to display a real-time vein visualization image in real time; wherein the real-time vein visualization image is used to determine the real-time needle insertion status of the venous puncture and the real-time image characteristics of the target vein segment.

[0017] The real-time vein visualization image can help the puncture personnel to observe the needle insertion status of the vein puncture in real time; the puncture personnel perform the puncture according to the displayed vein position, and the display shows the real-time status of the puncture needle entering the vein until the puncture personnel determines that the puncture is successful.

[0018] Based on the embodiment provided by the present application, an ultrasonic sensor is used to collect real-time venous blood vessel images of the patient's arm, and a plurality of cross-section venous features and a plurality of longitudinal venous features are determined by a first AI image processing model, and then a second AI image processing model comprehensively analyzes the positions corresponding to these features to accurately determine the optimal target puncture vein position; this AI-based multi-dimensional feature analysis and positioning method greatly improves the accuracy of puncture and reduces puncture failures caused by inaccurate positioning compared to the traditional method in which nurses rely on experience to perceive and predict the location of veins; during the puncture process, the third AI image processing algorithm renders the real-time venous cross-section image into a real-time venous visualization image that can simulate the blood flow dynamics, and displays it in real time through an auxiliary puncture display to guide the optimal needle insertion direction and angle; medical staff can adjust the puncture status in real time according to these dynamic images, correct deviations in a timely manner, and further improve the success rate of puncture: due to the improvement in the success rate of puncture, patients do not need to experience multiple puncture failures. The robot can effectively avoid the problem of congestion and swelling in the vein area caused by repeated punctures, and relieve the pain and discomfort of patients. The multi-axis robot drives the ultrasonic sensor to move to the target puncture vein according to the determined target puncture vein position. The precise operation and control of the robot system can control the positioning error of the puncture needle within a very small range, achieve accurate puncture, reduce damage and stimulation to the patient's vein, and thus reduce the patient's pain. The entire venous puncture process is highly automated, from venous blood vessel image acquisition, feature analysis, target position determination to accurate positioning and puncture of the puncture needle, all of which are automatically completed by the system. This greatly reduces the operating steps and workload of medical staff during the puncture process and improves work efficiency. The automated puncture system can complete the puncture task quickly and accurately, avoiding the time spent on finding veins and multiple attempts at puncture in the traditional puncture process. It saves patients waiting and treatment time, and also improves the work efficiency of medical institutions.

[0019] Furthermore, before S101, using an ultrasonic sensor as an imaging device to collect real-time images of venous blood vessels in the venous puncture area of ​​the patient's arm, the method further includes: S100 uses a multi-axis robot to drive the ultrasonic sensor to move to the venous puncture area of ​​the patient's arm; wherein the patient's arm is fixed to the arm contouring platform by a wrist fixing cuff and an arm end fixing cuff; the arm end fixing cuff has an inflation function; the multi-axis robot drives the ultrasonic sensor based on multi-joint force control sensing.

[0020] Using an auxiliary puncture display to display in real time the acquired real-time image of the venous blood vessels in the venous puncture area of ​​the patient's arm; this step may be performed after S101; The auxiliary puncture display is used to display the real-time venous cross-section image of the target venous segment in real time; this step can be performed after S106.

[0021] For example, Figure 2 This is a flow chart of another fully automatic venipuncture guidance method based on AI images.

[0022] Furthermore, the first AI image processing model is trained and optimized based on the following steps: S301, based on the improved U-Net model combined with the attention mechanism and Transformer architecture, the first AI image processing model was established; the improved U-Net model is responsible for image segmentation, while the Transformer architecture is used for global association and fusion of features; S302, obtaining a venous blood vessel image sample set, and marking a cross-section vein feature and a longitudinal section vein feature corresponding to each venous blood vessel image sample in the venous blood vessel image sample set; wherein the cross-section vein feature and the longitudinal section vein feature both have corresponding positions; S303, establishing a first training sample set and a first test sample set according to the labeled venous blood vessel image samples; S304, preprocessing the plurality of venous blood vessel image samples included in the first training sample set by applying an enhancement algorithm based on a Hessian matrix; S305, using the structure of the improved U-Net model and combining the attention mechanism, performing image segmentation on the preprocessed multiple venous blood vessel image samples to obtain a cross-section venous image and a longitudinal venous image; S306, extracting morphological features of the blood vessels from the segmented cross-section venous image; wherein the morphological features are used to analyze the symmetry and shape features of the blood vessels; S307, using a Gabor filter to extract the texture features of the blood vessel wall from the segmented cross-section vein image; wherein the texture features include texture density and texture direction; S308, using a Frangi filter, extracting the centerline and branch structure of the blood vessel from the segmented longitudinal section venous image to analyze the characteristics of the path and branching of the blood vessel; S309, inputting the extracted cross-section features and longitudinal section features into the Transformer architecture; wherein the cross-section features include the morphological features of the blood vessels and the texture features of the blood vessel walls; and the longitudinal section features include the features of the blood vessel paths and branches; S310, using the self-attention mechanism of the Transformer architecture, globally associate and fuse the cross-section features and the longitudinal section features to obtain a fused feature matrix; S311, matching the fusion feature matrix output by the Transformer architecture with the annotated cross-section vein features and longitudinal section vein features, and training the first AI image processing model; S312, applying the first AI image processing model that has been trained and optimized to the first test sample set, and evaluating the multiple venous vessel image samples included in the first test sample set.

[0023] Furthermore, the cross-sectional vein characteristics include the vessel wall thickness, vessel lumen size, vessel cross-sectional shape, and vessel wall texture density; the longitudinal vein characteristics include vessel direction, branching structure, vessel length, vessel curvature, and venous valve position; In the structure of the improved U-Net model, the encoder gradually reduces the spatial dimension of the image and extracts the feature information of the image; the decoder gradually restores the spatial dimension of the image and uses the feature information extracted by the encoder for image reconstruction and segmentation; Among them, the attention mechanism is used to enhance the improved U-Net model's attention to key features, including: paying attention to the thickness changes of the blood vessel wall when extracting cross-section features; paying attention to the branch points of the blood vessels when extracting longitudinal section features.

[0024] Further, ; in, is the fusion feature matrix; is the query matrix, which is used to match and associate with other features in the self-attention mechanism; is the key matrix, used to match the query vector in the self-attention mechanism; is the transpose of the key matrix; in the self-attention mechanism, the query matrix with the transpose of the key matrix Perform dot product operation to obtain the attention score matrix; is the dimension of the key vector; is a value matrix used for weighted summation according to the attention weights in the self-attention mechanism; The function is used to normalize the attention score matrix; It is a layer normalization operation, which is used to normalize the result after feature fusion. Layer normalization is usually performed on each feature dimension so that the mean of each feature is 0 and the variance is 1. The original cross-section features and longitudinal features input into the Transformer architecture are obtained by Perform residual connection to retain the information of the original features.

[0025] When training the improved U-Net model, the following loss function can be used to optimize the segmentation effect: ; in, It is the Dice loss function, which is used to measure the overlap between the segmentation result and the true annotation; is the binary cross entropy loss function, which is used to measure the accuracy of the segmentation results; is the weight parameter used to balance the Dice loss and the binary cross entropy loss.

[0026] Furthermore, while using an ultrasonic sensor as an imaging device to collect a real-time venous cross-section image of a target venous segment corresponding to the target punctured vein position, a Doppler signal is collected; the Doppler signal includes blood flow characteristic information; S104, using a third AI image processing algorithm to render the real-time vein cross-section image into a real-time vein visualization image capable of simulating blood flow dynamics, including: S402, adaptively selecting a wavelet basis and a decomposition layer number according to the complexity and noise characteristics of each sub-region included in the real-time venous cross-section image; for a sub-region with concentrated noise characteristics, selecting a Daubechies wavelet; for a sub-region with a blood flow detail complexity greater than a complexity threshold, selecting a Symlets wavelet; At different scales after wavelet decomposition, thresholds are adaptively set according to the local features of each sub-region to process each sub-region; for low-frequency sub-images, the threshold is small to retain more structural information; for high-frequency sub-images, the threshold is large to effectively remove noise while retaining blood flow detail features; S403, performing inverse wavelet transform on each processed sub-region to obtain each enhanced sub-region; using Laplace pyramid fusion algorithm, fusing each enhanced sub-region with each original sub-region to obtain an enhanced real-time vein cross-section image; S404, for each frame of the image sequence of the enhanced venous cross-section image, a dynamic speckle contrast image sequence corresponding to each frame of the image is calculated to analyze the dynamic change characteristics of the pixel points of each frame of the image in the continuous frames; different from the traditional speckle contrast, the dynamic speckle contrast not only considers the intensity change of the pixel points, but also introduces the time dimension to analyze the dynamic change characteristics of the pixel points of each frame of the image in the continuous frames, so as to more accurately reflect the blood flow information; S405, segmenting the dynamic speckle contrast image sequence using the trained deep learning segmentation model to obtain a blood flow region segmentation result; wherein the deep learning segmentation model is learned by the labeled training dynamic speckle contrast image sequence to obtain the characteristic difference between the blood flow region and the static tissue region; S406, optimizing the blood flow region segmentation result by using image morphological operation, filling the holes in the blood flow region, and obtaining a blood flow region image; wherein the image morphological operation includes erosion, dilation, opening operation, and closing operation; S407, using an adaptive filter to remove noise and interference signals from the collected Doppler signal to obtain an optimized Doppler signal; S408, introducing a particle system, simulating the dynamic process of blood flow according to the blood flow characteristic information included in the optimized Doppler signal, and obtaining a blood flow simulation result; wherein each particle represents a blood flow unit, and the motion state of each particle is updated according to the blood flow characteristic information; the particle system is used to simulate the flow, diffusion and convergence of blood flow; the motion state of each particle includes position, speed and direction; S409, combining the blood flow simulation result of the particle system with the blood flow area image to perform visual rendering; Among them, advanced rendering technology (such as shadow rendering and texture mapping) is used to enhance the visual effect of blood flow, making the dynamic changes of blood flow more clearly visible. At the same time, user interaction functions are provided, such as adjusting the observation angle, zooming in and out of the image, pausing and playing the blood flow dynamics, etc., so that users can observe and analyze the blood flow dynamics more intuitively; Among them, the speed update formula of particles in the particle system is: ; in, Indicates The blood flow unit corresponding to each particle is subjected to multiple forces, including the resistance of the blood vessel wall and the viscosity of the blood flow. For the The mass of a particle; For the The acceleration generated by the superposition of multiple forces on the blood flow unit corresponding to each particle. According to Newton's second law, the resultant force divided by the mass is the acceleration. The impact acceleration is calculated based on the blood flow characteristic information and is used to simulate the impact of the dynamic changes of blood flow on particle motion; The time step is the time interval between each particle state update during the simulation.

[0027] This formula comprehensively considers the various forces acting on the particles and the blood flow characteristic information, and updates the particle velocity in a complex manner, thereby more realistically simulating the dynamic movement process of the blood flow unit in the blood flow, including flow, diffusion, and convergence.

[0028] Optionally, the present application provides an AI image-based fully automatic venous puncture guidance system, including a multi-axis robot, an auxiliary puncture display, a control system, and an ultrasonic sensor; The ultrasonic sensor is used to collect real-time images of the venous blood vessels in the venous puncture area of ​​the patient's arm; and to collect real-time venous cross-sectional images of the target venous segment corresponding to the target puncture vein position; A control system for sending real-time images of venous vessels to an image processing system; and a multi-axis robot for driving an ultrasonic sensor to move to a target puncture vein position; The control system includes an image processing system; the image processing system is used to process the real-time image of the venous blood vessels using a first AI image processing model to determine multiple cross-section vein features and multiple longitudinal section vein features; use a second AI image processing model to analyze the positions corresponding to the multiple cross-section vein features and the positions corresponding to the multiple longitudinal section vein features to determine the target puncture vein position; wherein the target puncture vein position includes the needle insertion position, needle insertion direction and needle insertion angle of the venous puncture; use a third AI image processing algorithm to render the real-time vein cross-section image into a real-time vein visualization image that can simulate blood flow dynamics; The auxiliary puncture display is used to display the real-time vein visualization image in real time; wherein, the real-time vein visualization image is used to determine the real-time needle insertion status of the venous puncture and the real-time imaging characteristics of the target vein segment.

[0029] Furthermore, the AI ​​image-based fully automatic venipuncture guidance system also includes: An arm contouring platform, a trolley assembly, an arm bearing table, a host control cabinet, an ultrasonic sensor fixing bracket assembly, an ultrasonic sensor horizontal angle compensation axis, a power module, a wrist fixing cuff, an arm end fixing cuff and a fixing belt inflation control module; wherein the fixing belt is a wrist fixing cuff and / or an arm end fixing cuff.

[0030] Furthermore, the multi-axis robot drives the ultrasonic sensor based on multi-joint force control sensing; The multi-axis robot is also used to drive the ultrasonic sensor to move to the venous puncture area of ​​the patient's arm before using the ultrasonic sensor as an imaging device to collect real-time images of the venous blood vessels in the venous puncture area of ​​the patient's arm; wherein the patient's arm is fixed to the arm profiling platform by a wrist fixing cuff and an arm end fixing cuff; the arm end fixing cuff has an inflation function; The auxiliary puncture display is also used to display the real-time image of the venous blood vessels in the venous puncture area of ​​the patient's arm and the real-time venous cross-section image of the target vein segment.

[0031] Take the collaborative six-axis robot as an example. Figure 3 Schematic diagram of a fully automatic venipuncture guidance system based on AI images.

[0032] It should be understood that the multi-axis robot can also be a collaborative four-axis robot, a collaborative eight-axis robot, etc., and this application does not limit this. In the following examples of this application, collaborative six-axis robots are often used as examples for explanation.

[0033] Figure 4 Schematic diagram of a patient's arm being secured on an arm contouring platform.

[0034] Figure 5 It is a schematic diagram of a multi-axis collaborative robot, an ultrasonic sensor and an ultrasonic sensor fixing bracket assembly.

[0035] Figure 6 It is a schematic diagram of a multi-axis collaborative robot, an ultrasonic sensor and an ultrasonic sensor fixing bracket assembly, and an ultrasonic sensor horizontal angle compensation axis.

[0036] Figure 7 Schematic diagram of a fully automatic venipuncture guidance system based on AI images.

[0037] Figure 8 This is a schematic diagram illustrating the components of an AI image-based fully automatic venipuncture guidance system.

[0038] Fig. 9 This is a control flow chart of a fully automatic venipuncture guidance system based on AI images.

[0039] It should be noted that in the present application, the embodiments implemented on the AI ​​image-based fully automatic venous puncture guidance system side can be cross-referenced with the embodiments implemented on the AI ​​image-based fully automatic venous puncture guidance method side, and this application will not go into details one by one.

[0040] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A fully automatic venipuncture guidance method based on AI images, characterized in that: include: S101, using an ultrasonic sensor as an imaging device to collect real-time images of the venous blood vessels in the venipuncture area of ​​the patient's arm; S102, sending the real-time image of the venous blood vessels to an image processing system; S103, using the image processing system to process the real-time image of the venous blood vessels using a first AI image processing model to determine a plurality of cross-sectional venous features and a plurality of longitudinal venous features; S104, using the image processing system and a second AI image processing model to analyze the positions corresponding to the multiple cross-sectional vein features and the positions corresponding to the multiple longitudinal vein features to determine the target puncture vein position; wherein the target puncture vein position includes the needle insertion position, needle insertion direction, and needle insertion angle of the venous puncture; S105, using a multi-axis robot to drive the ultrasonic sensor to move to the target puncture vein position; S106, using the ultrasonic sensor as an imaging device to collect a real-time vein cross-section image of the target vein segment corresponding to the target puncture vein position in real time; S107, using the image processing system to render the real-time vein cross-section image into a real-time vein visualization image capable of simulating blood flow dynamics using a third AI image processing algorithm; S108, using an auxiliary puncture display to display the real-time vein visualization image in real time; wherein the real-time vein visualization image is used to determine the real-time needle insertion status of the venous puncture and the real-time image characteristics of the target vein segment.

2. The AI ​​image-based fully automatic venous puncture guidance method according to claim 1 is characterized in that: Before the step S101 of using an ultrasonic sensor as an imaging device to collect real-time images of the venous blood vessels in the venous puncture area of ​​the patient's arm, the method further includes: S100, using the multi-axis robot, drive the ultrasonic sensor to move to the venous puncture area of ​​the patient's arm; wherein the patient's arm is fixed to the arm contouring platform by a wrist fixing cuff and an arm end fixing cuff; the arm end fixing cuff has an inflation function; the multi-axis robot drives the ultrasonic sensor based on multi-joint force control sensing.

3. The AI ​​image-based fully automatic venous puncture guidance method according to claim 1 is characterized in that: The auxiliary puncture display is used to display in real time the real-time image of the venous blood vessels in the venous puncture area of ​​the patient's arm and the real-time venous cross-section image of the target venous segment.

4. The AI ​​image-based fully automatic venous puncture guidance method according to claim 1 is characterized in that: The first AI image processing model is trained and optimized based on the following steps: S301, establishing the first AI image processing model based on an improved U-Net model combined with an attention mechanism and a Transformer architecture; S302, obtaining a venous blood vessel image sample set, and marking a cross-section vein feature and a longitudinal section vein feature corresponding to each venous blood vessel image sample in the venous blood vessel image sample set; wherein the cross-section vein feature and the longitudinal section vein feature both have corresponding positions; S303, establishing a first training sample set and a first test sample set according to the labeled venous blood vessel image samples; S304, preprocessing the plurality of venous vessel image samples included in the first training sample set by applying an enhancement algorithm based on a Hessian matrix; S305, using the structure of the improved U-Net model and combining the attention mechanism, performing image segmentation on the preprocessed multiple venous blood vessel image samples to obtain a cross-section venous image and a longitudinal venous image; S306, extracting morphological features of the blood vessels from the segmented cross-section venous image; wherein the morphological features are used to analyze the symmetry and shape features of the blood vessels; S307, using a Gabor filter to extract the texture features of the blood vessel wall from the segmented cross-section vein image; wherein the texture features include texture density and texture direction; S308, using a Frangi filter, extracting the centerline and branch structure of the blood vessel from the segmented longitudinal section venous image to analyze the characteristics of the path and branching of the blood vessel; S309, inputting the extracted cross-section features and longitudinal section features into the Transformer architecture; wherein the cross-section features include morphological features of the blood vessels and texture features of the blood vessel walls; and the longitudinal section features include features of the blood vessel paths and branching conditions; S310, using the self-attention mechanism of the Transformer architecture, globally correlating and fusing the cross-section features and the longitudinal section features to obtain a fused feature matrix; S311, matching the fusion feature matrix output by the Transformer architecture with the annotated cross-section vein features and longitudinal section vein features, and training the first AI image processing model; S312: Apply the first AI image processing model that has been trained and optimized to the first test sample set, and evaluate the multiple venous vessel image samples included in the first test sample set.

5. The AI ​​image-based fully automatic venous puncture guidance method according to claim 4 is characterized in that: The cross-section vein characteristics include the vessel wall thickness, vessel lumen size, vessel cross-sectional shape and vessel wall texture density; the longitudinal section vein characteristics include vessel direction, branching structure, vessel length, vessel curvature and venous valve position; In the structure of the improved U-Net model, the encoder gradually reduces the spatial dimension of the image while extracting feature information of the image; The decoder gradually restores the spatial dimensions of the image and uses the feature information extracted by the encoder for image reconstruction and segmentation; The attention mechanism is used to enhance the improved U-Net model's attention to key features, including: paying attention to changes in the thickness of the blood vessel wall when extracting cross-section features; and paying attention to the branch points of the blood vessels when extracting longitudinal section features.

6. The AI ​​image-based fully automatic venous puncture guidance method according to claim 1 is characterized in that: While using the ultrasonic sensor as an imaging device to collect a real-time venous cross-section image of the target venous segment corresponding to the target punctured vein position in real time, a Doppler signal is collected; the Doppler signal includes blood flow characteristic information; The step S104, using a third AI image processing algorithm to render the real-time vein cross-section image into a real-time vein visualization image capable of simulating blood flow dynamics, includes: S401, adaptively selecting a wavelet basis and a decomposition layer number according to the complexity and noise characteristics of each sub-region included in the real-time vein cross-section image; S402, at different scales after wavelet decomposition, adaptively setting a threshold according to the local features of each sub-region, and processing each sub-region; S403, performing inverse wavelet transform on each processed sub-region to obtain each enhanced sub-region; using Laplace pyramid fusion algorithm, fusing each enhanced sub-region with each original sub-region to obtain an enhanced real-time vein cross-section image; S404, for each frame of the image in the image sequence of the enhanced venous cross-section image, a dynamic speckle contrast image sequence corresponding to each frame of the image is calculated to analyze the dynamic change characteristics of the pixel points of each frame of the image in continuous frames.

7. The AI ​​image-based fully automatic venous puncture guidance method according to claim 6 is characterized in that: While using the ultrasonic sensor as an imaging device to collect a real-time venous cross-section image of the target venous segment corresponding to the target punctured vein position in real time, a Doppler signal is collected; the Doppler signal includes blood flow characteristic information; The step S104, using a third AI image processing algorithm to render the real-time vein cross-section image into a real-time vein visualization image capable of simulating blood flow dynamics, further includes: S405, segmenting the dynamic speckle contrast image sequence using the trained deep learning segmentation model to obtain a blood flow region segmentation result; wherein the deep learning segmentation model is learned by the labeled training dynamic speckle contrast image sequence to obtain the characteristic difference between the blood flow region and the static tissue region; S406, optimizing the blood flow region segmentation result by using image morphological operation, filling the holes in the blood flow region, and obtaining a blood flow region image; wherein the image morphological operation includes erosion, dilation, opening operation and closing operation; S407, using an adaptive filter to remove noise and interference signals from the collected Doppler signal to obtain an optimized Doppler signal; S408, introducing a particle system, simulating the dynamic process of blood flow according to the blood flow characteristic information included in the optimized Doppler signal, and obtaining a blood flow simulation result; wherein each particle represents a blood flow unit, and the motion state of each particle is updated according to the blood flow characteristic information; the particle system is used to simulate the flow, diffusion and convergence of blood flow; the motion state of each particle includes position, speed and direction; S409, combining the blood flow simulation result of the particle system with the blood flow area image to perform visual rendering.

8. A fully automatic venous puncture guidance system based on AI images, the system implementing the method according to claim 1, characterized in that: It includes a multi-axis robot, an auxiliary puncture display, a control system, and an ultrasonic sensor; The ultrasonic sensor is used to collect real-time images of the venous blood vessels in the venous puncture area of ​​the patient's arm; and to collect real-time venous cross-section images of the target venous segment corresponding to the target puncture vein position; The control system is used to send the real-time image of the venous blood vessels to the image processing system; and use the multi-axis robot to drive the ultrasonic sensor to move to the target puncture vein position; The control system includes the image processing system; the image processing system is used to process the real-time image of the venous blood vessels using the first AI image processing model to determine multiple cross-section vein features and multiple longitudinal section vein features; use the second AI image processing model to analyze the positions corresponding to the multiple cross-section vein features and the positions corresponding to the multiple longitudinal section vein features to determine the target puncture vein position; wherein the target puncture vein position includes the needle insertion position, needle insertion direction and needle insertion angle of the venous puncture; use the third AI image processing algorithm to render the real-time vein cross-section image into a real-time vein visualization image that can simulate blood flow dynamics; The auxiliary puncture display is used to display the real-time vein visualization image in real time; wherein, the real-time vein visualization image is used to determine the real-time needle insertion status of the venous puncture and the real-time image characteristics of the target vein segment.

9. The AI ​​image-based fully automatic venous puncture guidance system according to claim 8 is characterized in that: The AI ​​image-based fully automatic venous puncture guidance system also includes: An arm contouring platform, a trolley assembly, an arm bearing table, a host control cabinet, an ultrasonic sensor fixing bracket assembly, an ultrasonic sensor horizontal angle compensation axis, a power module, a wrist fixing cuff, an arm end fixing cuff and a fixing belt inflation control module; wherein the fixing belt is a wrist fixing cuff and / or an arm end fixing cuff.

10. The AI ​​image-based fully automatic venous puncture guidance system according to claim 9 is characterized in that: The multi-axis robot drives the ultrasonic sensor based on multi-joint force control sensing; The multi-axis robot is also used to drive the ultrasonic sensor to move to the venous puncture area of ​​the patient's arm before using the ultrasonic sensor as an imaging device to collect real-time images of the venous blood vessels in the venous puncture area of ​​the patient's arm; wherein the patient's arm is fixed to the arm profiling platform by the wrist fixing cuff and the arm end fixing cuff; the arm end fixing cuff has an inflation function; The auxiliary puncture display is also used to display in real time the real-time image of the venous blood vessels in the venous puncture area of ​​the patient's arm and the real-time venous cross-section image of the target venous segment.

Images