A pre-venipuncture strategy analysis and puncture path planning method and system

Abstract

The application discloses a pre-venipuncture strategy analysis and puncture path planning method and system, a series of image frames are acquired by rapidly scanning the region through an infrared imaging device. Through an image quality evaluation function, a most clear image of a vein contour is automatically selected as a reference for subsequent analysis; for patients with poor imaging of blood vessels in the ultrasound image, the selection of the needle insertion position on the vein center line can also be accurately performed, the puncture success rate is improved, the joint parameters of the puncture equipment are accurately solved, the accurate control of the equipment is ensured, the equipment can accurately reach the target puncture point, the safety of medical staff is ensured when puncturing patients with highly infectious diseases, and the risk of puncture failure caused by inaccurate path planning is reduced.

Description

Technical Field

[0001] This invention belongs to the field of medical data processing technology, specifically relating to a method and system for preoperative strategy analysis and puncture path planning for intravenous puncture. Background Technology

[0002] Venous puncture is one of the most common procedures in clinical medicine, frequently used for drug administration, intravenous infusion, and other similar procedures. The challenge in performing deep vein puncture lies in finding the appropriate puncture path for needle insertion, and the quality of this insertion directly impacts the success of the procedure. Currently, medical personnel typically use blind puncture or ultrasound-guided needle insertion methods. They often rely on experience to select the needle path, which demands a high level of expertise and skill from the medical staff. The success rate of venipuncture depends on the experience of the medical personnel and the patient's vascular condition. However, individual patient differences are significant, with variations in vein visibility and elasticity. For patients with poor vascular conditions, even experienced medical personnel cannot guarantee a high success rate. Improper needle angle, force, or speed can lead to vein puncture. Repeated attempts inevitably cause significant pain for the patient and can easily result in vascular damage. Furthermore, the path chosen by medical personnel based on experience is often inaccurate, affecting the accuracy and outcome of the puncture procedure.

[0003] In recent years, ultrasound-guided puncture technology has attracted widespread attention. This technology has solved the problem of difficult puncture when blood vessels are invisible and intangible to some extent, but the following problems still exist:

[0004] (1) It is technically difficult, and medical staff performing the puncture need to undergo professional ultrasound image recognition training.

[0005] (2) For patients whose blood vessels are poorly imaged in ultrasound images, it is impossible to accurately select the needle insertion point on the center line of the vein for puncture, resulting in a lower success rate of puncture;

[0006] (3) When performing punctures on patients with highly infectious diseases, the safety of medical staff may be threatened. Summary of the Invention

[0007] To address the shortcomings of the existing technology, this application provides a method and system for preoperative strategy analysis and puncture path planning for intravenous puncture.

[0008] The first aspect of this application proposes a method for preoperative strategy analysis and puncture path planning for intravenous puncture, including the following steps:

[0009] Images of the patient's puncture site are acquired as images to be analyzed. Improved restricted median filtering and contrast enhancement are applied to the images to be analyzed to obtain a first processed image with clear contours.

[0010] The first processed image is subjected to threshold segmentation to extract a vein image with clear contours;

[0011] Skeleton extraction is performed on the vein image to determine the center line of the vein in the vein image;

[0012] The depth information of the vein image is obtained by local stereo matching, the location of the vein centerline in the three-dimensional coordinate system is determined, and the target puncture point is selected from the vein centerline.

[0013] Initialize the parameters of the multi-joint puncture device, including: number of joint segments, joint length, bending angle of bending unit, rotation angle, needle starting point, and target puncture point;

[0014] The ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device is uniformly discretized, and the multi-joint puncture device is controlled to step from the needle starting point. The joint parameters of the multi-joint puncture device during the puncture process are calculated, and the step size of the step forward is controlled by the joint parameters.

[0015] The puncture device is judged to have reached the target puncture point based on the step length of the next step. If the target puncture point has not been reached, the joint parameters of each bending unit are recalculated when the device moves forward again to obtain a new step length. The puncture device is then controlled to move forward using the new step length. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded.

[0016] In some optional implementations of certain embodiments, the step of performing improved restricted median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with sharp contours includes:

[0017] Select a filter template size from the image to be analyzed, traverse the pixels in the selected filter template, obtain gray values ​​and sort them, compare the gray values ​​of neighboring pixels with the median of the gray value sequence, skip if they are the same, replace if they are different, move the template to the next pixel according to the filter template size, until all templates in the image have been filtered, and obtain the filtered image.

[0018] The probability density of gray values ​​in the filtered image is calculated, then the cumulative distribution function is used for gray-level mapping, and finally gray-level replacement is performed to obtain a first processed image with clear outlines.

[0019] In some optional implementations of certain embodiments, the step of performing threshold segmentation on the first processed image to extract a vein image with clear contours includes:

[0020] Analyze the grayscale histogram of the first processed image to determine the difference in grayscale distribution between the veins and the background, and obtain the grayscale analysis results;

[0021] Based on the grayscale analysis results, one or more segmentation thresholds are selected, and the selected segmentation thresholds are applied to segment the first processed image. Pixels larger than the segmentation threshold are classified as veins, and pixels smaller than or equal to the segmentation threshold are classified as background.

[0022] Canny edge detection is performed on the segmented vein image to extract the vein contour lines, resulting in a vein image with clear contours.

[0023] In some optional implementations of certain embodiments, the step of extracting the skeleton from the vein image to determine the vein centerline in the vein image includes:

[0024] The skeleton extraction includes image thinning, which is used to gradually peel off the image edges until the center line of the image is obtained. In this process, veins are thinned by erosion, and then pseudo skeleton lines are extracted by image thinning. From the pseudo skeleton lines of the veins, the real vein skeleton lines are extracted.

[0025] The centerline algorithm is used to determine the line segment representing the center of the vein from the real vein skeleton line, and the determined line segment is the centerline of the vein in the vein image.

[0026] In some optional implementations of certain embodiments, the step of obtaining depth information of the vein image through local stereo matching, determining the location of the vein centerline in a three-dimensional coordinate system, and selecting the target puncture point from the vein centerline includes:

[0027] Depth information of vein images is obtained through local stereo matching;

[0028] A disparity map is generated based on the depth information, and the disparity map is converted into a three-dimensional coordinate map. The spatial coordinates of each pixel are calculated using the principle of triangulation, and the coordinate information of the vein centerline is determined based on the spatial coordinates.

[0029] Analyze the geometric features of the vein centerline, set thresholds and rules based on clinical experience to select the target puncture point, and obtain the coordinate information of the target puncture point.

[0030] In some optional implementations of certain embodiments, the step of uniformly discretizing the ideal path curve between the needle initiation point and the target puncture point of the multi-joint puncture device, controlling the multi-joint puncture device to step from the needle initiation point, calculating the joint parameters of the multi-joint puncture device during the puncture process, and controlling the step size of each step forward through the joint parameters includes:

[0031] Based on the positions of the needle starting point and the target puncture point, an ideal puncture path curve is constructed;

[0032] The ideal puncture path curve is uniformly discretized into a series of discrete points and their position coordinates are determined.

[0033] Calculate the principal normal vector of the discrete point, which is perpendicular to the tangent plane of the ideal path curve at the discrete point; calculate the bi normal vector of the discrete point, which is perpendicular to both the principal normal and the tangent; calculate the tangent vector of the discrete point, which is tangent to the ideal path curve at the discrete point.

[0034] Construct the pose matrix based on the position coordinates of the discrete points, as well as the principal normal vector, the binormal vector, and the tangent vector;

[0035] The joint parameters of the bending unit at the end of the puncture device and the joint parameters of other bending units of the puncture device are solved based on the pose matrix.

[0036] The step length of the puncture device is controlled based on the joint parameters obtained from the solution.

[0037] In some optional implementations of certain embodiments, the step of determining whether the puncture device has reached the target puncture point based on the step length is performed. If the target puncture point has not been reached, the joint parameters of each bending unit are recalculated for the next step forward to obtain a new step length. The puncture device is then controlled to move forward using the new step length, and this operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded, including:

[0038] Position Update: Update the current position of the puncture device after it moves forward;

[0039] Target determination: Determine whether the puncture device has reached the target puncture point by comparing the distance between the current position of the puncture device and the target puncture point;

[0040] Path planning ends: If the puncture device has reached the target puncture point, the path planning ends and the current puncture path planning is recorded; otherwise, the joint parameters are solved and the puncture device is controlled to move forward.

[0041] Secondly, this application proposes a preoperative strategy analysis and puncture path planning system for venipuncture, including a first image processing module, a vein image processing module, a centerline image processing module, a puncture point positioning module, a device initialization module, a step analysis module, and a path planning module.

[0042] The first image processing module is used to acquire an image of the patient's puncture site as an image to be analyzed, and to perform improved restricted median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with clear contours.

[0043] The vein image processing module is used to perform threshold segmentation on the first processed image to extract a vein image with clear contours.

[0044] The centerline image processing module is used to extract the skeleton of the vein image and determine the centerline of the vein in the vein image;

[0045] The puncture point positioning module is used to obtain the depth information of the vein image through local stereo matching, determine the positioning of the vein centerline in the three-dimensional coordinate system, and select the target puncture point from the vein centerline.

[0046] The device initialization module is used to initialize and set the parameters of the multi-joint puncture device, including: the number of joint segments, the joint length, the bending angle of the bending unit, the rotation angle, the needle starting point, and the target puncture point.

[0047] The stepping analysis module is used to uniformly discretize the ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device, control the multi-joint puncture device to step from the needle starting point, calculate the joint parameters of the multi-joint puncture device during the puncture process, and control the step length of the next step through the joint parameters.

[0048] The path planning module is used to determine whether the puncture device has reached the target puncture point based on the step length of the next step. If the target puncture point has not been reached, the joint parameters of each bending unit are recalculated when the device moves forward another step to obtain a new step length. The new step length is then used to control the puncture device to move forward. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded.

[0049] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0050] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0051] The beneficial effects of this invention are:

[0052] The area is rapidly scanned using infrared imaging equipment, acquiring a series of image frames. An image quality evaluation function automatically selects the image with the clearest vein outline as the benchmark for subsequent analysis. For patients with poorly imaged blood vessels on ultrasound, the needle insertion point can be accurately selected along the vein's center line, improving the success rate of puncture. The joint parameters of the puncture equipment are precisely calculated, ensuring accurate control and guaranteeing that the equipment accurately reaches the target puncture point. When performing punctures on patients with highly infectious diseases, this ensures the safety of medical personnel and reduces the risk of puncture failure due to inaccurate path planning. Attached Figure Description

[0053] Figure 1 This is the overall flowchart of the present invention.

[0054] Figure 2 This is a system principle block diagram of the present invention. Detailed Implementation

[0055] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0056] The first aspect of this application proposes a method for preoperative strategy analysis and puncture path planning for intravenous puncture, such as... Figure 1 As shown, it includes the following steps:

[0057] S100: Acquire an image of the patient's puncture site as the image to be analyzed, and perform improved restricted median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with clear contours;

[0058] In some optional implementations of certain embodiments, the step of performing improved restricted median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with sharp contours includes:

[0059] Select a filter template size from the image to be analyzed, traverse the pixels in the selected filter template, obtain gray values ​​and sort them, compare the gray values ​​of neighboring pixels with the median of the gray value sequence, skip if they are the same, replace if they are different, move the template to the next pixel according to the filter template size, until all templates in the image have been filtered, and obtain the filtered image.

[0060] The probability density of gray values ​​in the filtered image is calculated, then the cumulative distribution function is used for gray-level mapping, and finally gray-level replacement is performed to obtain a first processed image with clear outlines.

[0061] Before performing venipuncture, infrared imaging technology is first used to acquire images of the puncture site. The key to this step is ensuring that the acquired images clearly reflect the structure and contour of the vein for subsequent processing and analysis. In this embodiment, a vein ultrasound image acquisition system based on the Mindray M9 ultrasound vascular image acquisition system and the Vision Image MV-VGA200E acquisition card is used for image acquisition. The image acquisition system receives the image output signal from the ultrasound system through the computer's VGA interface, thereby acquiring the vein images from the M9 ultrasound instrument and storing them in the computer. The components and functions of the system are described in detail below. The specific image acquisition process is as follows:

[0062] Choose a suitable location: Locate an appropriate area near the patient's skin surface where the puncture is to be performed, ensuring that the veins within that area are clearly visible, and avoiding vein branches and bends as much as possible to improve the accuracy of subsequent image processing.

[0063] Infrared imaging equipment preparation: Use a calibrated infrared imaging device and adjust its parameters, such as resolution, focal length, and aperture, to suit different patient skin types and vein depths.

[0064] Image acquisition: The infrared imaging device is activated to scan the puncture site, acquiring a series of consecutive image frames. The frame with the best image quality is selected as the image to be analyzed.

[0065] After image acquisition, preprocessing the images to be analyzed is a crucial step in improving image quality and highlighting vein contours. The following is a detailed image preprocessing procedure:

[0066] The original image contains noise and needs to be filtered to improve its quality. Mean filtering is simple in principle; it calculates the mean of the grayscale values ​​of pixels within a template and then performs grayscale replacement. Let the original grayscale value be... The grayscale value after filtering is The expression for the mean filter is as follows:

[0067]

[0068] In the formula, s is the neighborhood of the n×n template. The number of pixels in the neighborhood. These are the two-dimensional coordinates of the pixels in the neighborhood. The above filter has low computational cost and obvious smoothing effect. However, the filtering effect is greatly affected by the template. If the template is too large, the vein edges will become blurred, which is not conducive to extracting the vein contour from the image; the smaller the template, the worse the image filtering effect.

[0069] Median filtering typically involves two steps. First, the largest and smallest gray values ​​in the template are removed as noise. Then, the gray values ​​of the remaining pixels are arranged, the median value is calculated, and finally, gray value replacement is performed. The principle of the median filter is shown in the following formula:

[0070]

[0071] Compared to the mean filter, the median filter performs slightly better, but it still struggles to overcome the problem of blurring vein edges. This paper proposes an appropriate improvement to the median filtering algorithm to address this issue. Specifically, the improvement involves performing a judgment before grayscale replacement; pixels identified as edges are not subject to median replacement, thus ensuring edge preservation. The specific algorithm flow is as follows:

[0072] (1) Select a filter template size of 3×3;

[0073] (2) Traverse the pixels, obtain the grayscale values ​​and sort them;

[0074] (3) Compare the gray values ​​of neighboring pixels with the median of the gray value sequence. If they are the same, skip them; if they are different, replace them with the median.

[0075] (4) Move the template to the next pixel and repeat the above steps.

[0076] Furthermore, image enhancement can improve visual effects and grayscale contrast, facilitating subsequent image segmentation and extraction of vein contours. After contrast enhancement, vein images become more prominent, aiding in vein extraction. Contrast enhancement methods involve mapping and replacing the grayscale values ​​of pixels in the image, increasing the difference in grayscale values ​​between the vein location and the surrounding skin, making the vein more prominent and clear.

[0077] Gray-scale stretching enhancement amplifies the difference in gray-scale values ​​between veins and other parts of the image by enlarging the image's gray-scale range. The gray-scale stretching enhancement algorithm formula is as follows:

[0078]

[0079] In the formula, the grayscale range is expanded from [a,b] to [c,d], enhancing the contrast between veins and other regions. The image histogram reflects the grayscale distribution and the image's basic features. Histogram equalization first calculates the probability density of grayscale values, then uses the cumulative distribution function for grayscale mapping, and finally performs grayscale replacement. The specific algorithm flow is as follows:

[0080] (1) Calculate the probability density of the gray values ​​of the filtered image:

[0081]

[0082] in, Indicates the first grayscale value, This indicates the number of pixels at that gray level.

[0083] (2) The cumulative distribution function of the image:

[0084]

[0085] in, Indicates the first The grayscale value after mapping the grayscale values.

[0086] (3) Round the mapped gray values.

[0087] (4) Replace the grayscale value to obtain a new image.

[0088] The two image enhancement algorithms mentioned above were used to process the vein images. However, histogram equalization achieved better results, and the vein contours became clearer in the processed images.

[0089] After image preprocessing, thresholding segmentation is required to accurately extract vein images from the first processed image. Thresholding segmentation is a segmentation method based on image gray levels. By setting one or more thresholds, image pixels are divided into foreground (veins) and background. The following is a detailed vein image extraction process:

[0090] S200: Perform threshold segmentation on the first processed image to extract a vein image with clear contours;

[0091] In some optional implementations of certain embodiments, the step of performing threshold segmentation on the first processed image to extract a vein image with clear contours includes:

[0092] Analyze the grayscale histogram of the first processed image to determine the difference in grayscale distribution between the veins and the background, and obtain the grayscale analysis results;

[0093] Based on the grayscale analysis results, one or more segmentation thresholds are selected, and the selected segmentation thresholds are applied to segment the first processed image. Pixels larger than the segmentation threshold are classified as veins, and pixels smaller than or equal to the segmentation threshold are classified as background.

[0094] Canny edge detection is performed on the segmented vein image to extract the vein contour lines, resulting in a vein image with clear contours.

[0095] Specifically, this includes:

[0096] Grayscale image analysis: Analyze the grayscale histogram of the first processed image to determine the difference in grayscale distribution between the veins and the background.

[0097] Choosing an appropriate threshold: Based on the grayscale analysis results, select one or more appropriate thresholds. To improve the accuracy of segmentation, dynamic thresholding or region-growing-based methods can be used.

[0098] Threshold segmentation: The selected threshold is applied to segment the image, and pixels larger than the threshold are classified as veins, while pixels smaller than or equal to the threshold are classified as background.

[0099] Furthermore, the Otsu algorithm is applied for threshold segmentation:

[0100] A series of different thresholds are set to segment the image, and the inter-class variance is used to evaluate the quality of the threshold segmentation. The threshold is set to S; gray values ​​greater than S are considered targets, and those less than S are considered background. The number of target and background pixels is counted separately, and their respective proportions are calculated and denoted as . and The average gray level of the target is calculated as follows: The average gray level of the background is The average gray level of the image is The calculation method is as follows:

[0101]

[0102] Wherein, the inter-class variance is The calculation formula is:

[0103]

[0104] Contour extraction: The contour lines of the veins are extracted using edge detection algorithms, such as Canny edge detection.

[0105] S300: Perform skeleton extraction on the vein image to determine the center line of the vein in the vein image;

[0106] In some optional implementations of certain embodiments, the step of extracting the skeleton from the vein image to determine the vein centerline in the vein image includes:

[0107] The skeleton extraction includes image thinning, which is used to gradually peel off the image edges until the center line of the image is obtained. In this process, veins are thinned by erosion, and then pseudo skeleton lines are extracted by image thinning. From the pseudo skeleton lines of the veins, the real vein skeleton lines are extracted.

[0108] The centerline algorithm is used to determine the line segment representing the center of the vein from the real vein skeleton line, and the determined line segment is the centerline of the vein in the vein image.

[0109] After extracting the vein image, the centerline of the vein needs to be determined in order to accurately plan the puncture path. This step is achieved through morphological processing and skeleton extraction algorithms, and the detailed process is as follows:

[0110] 1. Morphological Processing: Morphological processing is a common method in image processing. It improves the structure of an image through a series of morphological operations, facilitating subsequent analysis. In this step, the extracted vein contours are first subjected to erosion and dilation operations to eliminate fine branches and protrusions in the image;

[0111] 2. Skeleton Extraction: Skeleton extraction is an image thinning algorithm that progressively peels away image edges until the center line of the image is obtained. In this process, an improved iterative thinning algorithm is used, which can adapt to vein structures of different widths and sizes, accurately extracting the center line of the veins.

[0112] 3. Centerline Determination: Based on the extracted skeleton, further analysis is used to determine the main skeleton line representing the vein center. This step can be achieved by finding the longest continuous line segment in the skeleton, or by using other algorithms such as minimum spanning tree.

[0113] Taking a preprocessed and thresholded vein image as an example, morphological processing is first performed on it. Erosion and dilation operations are used to remove small branches and noise from the image, resulting in a cleaner vein structure image. Next, a thinning algorithm is applied to extract the skeleton of the vein, obtaining its skeleton line. Based on this, a centerline algorithm, such as finding the longest continuous line segment, is used to determine the line segment representing the vein center. This line segment is the vein centerline in the puncture path planning and will serve as the reference for subsequent path planning.

[0114] S400: Obtain the depth information of the vein image through local stereo matching, determine the location of the vein centerline in the three-dimensional coordinate system, and select the target puncture point from the vein centerline;

[0115] In some optional implementations of certain embodiments, the step of obtaining depth information of the vein image through local stereo matching, determining the location of the vein centerline in a three-dimensional coordinate system, and selecting the target puncture point from the vein centerline includes:

[0116] Depth information of vein images is obtained through local stereo matching;

[0117] A disparity map is generated based on the depth information, and the disparity map is converted into a three-dimensional coordinate map. The spatial coordinates of each pixel are calculated using the principle of triangulation, and the coordinate information of the vein centerline is determined based on the spatial coordinates.

[0118] Analyze the geometric features of the vein centerline, set thresholds and rules based on clinical experience to select the target puncture point, and obtain the coordinate information of the target puncture point.

[0119] Local stereo matching obtains the 3D information of the photographed object by matching different image pixels. Specifically, it maps the 2D displacement difference of the matched pixels to 3D depth, thereby constructing a 3D model as a reproduction of the original scene. Because the position and depth of the vein need to be updated in real time during venipuncture, the algorithm has high real-time requirements. However, global matching algorithms have significant latency; therefore, this embodiment uses a local stereo matching algorithm. The specific implementation process is as follows:

[0120] Step 1: Obtain depth information of the vein image using a local stereo matching algorithm;

[0121] Feature extraction: Extracting key feature points from an image, such as SIFT and SURF.

[0122] Local stereo matching:

[0123] Matching cost calculation: For each pixel, calculate its matching cost between images from different viewpoints, usually using SSD (Sum of Squared Differences) or NCC (Normalized Cross-Correlation).

[0124] Cost aggregation: Perform cost aggregation within a local window to find the disparity value with the minimum matching cost.

[0125] Matching cost calculation:

[0126]

[0127] in, Parallax is The matching cost at that time and These are the pixel values ​​of the left and right images, respectively.

[0128] Cost aggregation:

[0129]

[0130] in, It is a point The optimal disparity, i.e., depth information.

[0131] Parameter explanation:

[0132] Parallax: The difference in the horizontal coordinate of the same object point in the left and right images.

[0133] Pixel coordinates in the image.

[0134] : Pixel values ​​of the left and right images.

[0135] Step 2: Determine the location of the vein centerline in the three-dimensional coordinate system;

[0136] Disparity map generation: Generate a disparity map based on the disparity information obtained from local stereo matching.

[0137] 3D reconstruction: Convert the disparity map into a 3D coordinate map and calculate the spatial coordinates of each pixel using the principle of triangulation.

[0138] Centerline extraction: The centerline of the vein is extracted using edge detection, morphological and other techniques.

[0139] Triangulation:

[0140]

[0141] in, It is depth information. It's the camera's focal length. It is the baseline distance between the two cameras.

[0142] Step 3: Select the target puncture point from the center line of the vein;

[0143] Central line analysis: Analyze the geometric characteristics of the vein's central line, such as curvature and width.

[0144] Target puncture point selection:

[0145] Based on clinical experience, points with smaller curvature, moderate width, and appropriate depth are selected as puncture points.

[0146] Automatic selection can be achieved by setting certain thresholds and rules.

[0147] Curvature calculation:

[0148]

[0149] in, It's curvature. and The center line is at The slope of the tangent at the location and the second derivative.

[0150] The above steps determine the location of the vein's centerline and the target puncture point.

[0151] S500: Initializes the parameters of the multi-joint puncture device, including: number of joint segments, joint length, bending angle of the bending unit, rotation angle, needle starting point, and target puncture point;

[0152] After determining the target puncture point, parameter initialization of the puncture equipment is necessary to achieve precise puncture. This step includes determining key parameters such as the number of joint segments, length, bending angle, and rotation angle of the equipment. The following is a detailed process for initializing the equipment parameters:

[0153] Determining the number of joint segments: Based on the mechanical structure of the puncture device, the number of joint segments, n, is determined. The number of joint segments determines the degree of bending and flexibility that the device can achieve.

[0154] Joint length setting: Based on the mechanical design of the equipment, the length of each joint segment is set to 1. The joint length affects the overall length and bending radius of the equipment.

[0155] Bending angle range setting: Based on the mechanical performance of the equipment, the maximum and minimum bending angles that each joint segment can achieve are set. The bending angle determines the degree of bending of the equipment in space. .

[0156] Rotation angle range setting: Sets the range of angles that the end effector (such as a needle) can rotate. The rotation angle determines the needle's ability to adjust its direction during puncture. .

[0157] Needle starting point setting: Based on the mechanical structure of the equipment and the location of the target puncture point, the starting position of the needle is set. The starting point determines the initial position of the puncture operation. .

[0158] Target puncture point setting: Based on the target puncture point determined in step S400, set the target location for the puncture operation. The target puncture point is... .

[0159] S600: The ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device is uniformly discretized, and the multi-joint puncture device is controlled to step from the needle starting point. The joint parameters of the multi-joint puncture device during the puncture process are calculated, and the step length of the step forward is controlled by the joint parameters.

[0160] In some optional implementations of certain embodiments, the step of uniformly discretizing the ideal path curve between the needle initiation point and the target puncture point of the multi-joint puncture device, controlling the multi-joint puncture device to step from the needle initiation point, calculating the joint parameters of the multi-joint puncture device during the puncture process, and controlling the step size of each step forward through the joint parameters includes:

[0161] Based on the positions of the needle starting point and the target puncture point, an ideal puncture path curve is constructed;

[0162] The ideal puncture path curve is uniformly discretized into a series of discrete points and their position coordinates are determined.

[0163] Calculate the principal normal vector of the discrete point, which is perpendicular to the tangent plane of the ideal path curve at the discrete point; calculate the bi normal vector of the discrete point, which is perpendicular to both the principal normal and the tangent; calculate the tangent vector of the discrete point, which is tangent to the ideal path curve at the discrete point.

[0164] Construct the pose matrix based on the position coordinates of the discrete points, as well as the principal normal vector, the binormal vector, and the tangent vector;

[0165] The joint parameters of the bending unit at the end of the puncture device and the joint parameters of other bending units of the puncture device are solved based on the pose matrix.

[0166] The step length of the puncture device is controlled based on the joint parameters obtained from the solution.

[0167] Specifically, the ideal puncture path curve is uniformly discretized, dividing the ideal path curve between the starting point and the target point of the puncture device into m points, i.e., points. Time Each ideal path point is represented in the base coordinate system as:

[0168]

[0169] Let M be the discrete point on the ideal path curve closest to the puncture device (for example, if the actual step point at the current moment is discrete point 3, then the closest discrete point is 4), and s be the arc length from the starting point of the ideal path to point M. Then the tangent vector of the curve at point M is:

[0170]

[0171] Where t is the tangent vector, and the radial vector at point M on the curve is... ,if ,in, and They are respectively the radial vectors With respect to the first and second derivatives of s, the tangent vector and position vector at point M uniquely determine an osculating plane, the equation of which is:

[0172]

[0173] in, The equation of the binormal is given by:

[0174]

[0175] Given the tangent vector and binormal vector In this case, the principal normal at point M can be solved using the right-hand rule as follows:

[0176]

[0177] Therefore, the puncture device advances along the ideal path curve for an arc length of... At that time, the homogeneous transformation matrix of the actual stepping point of the puncture device relative to the base coordinate system is:

[0178]

[0179] in, Let M be the position coordinates of point M in the base coordinate system. The ideal path curve is divided into m points at equal intervals. The tangent, principal normal, and secondary normal of each point are used as the x, y, and z axes of its coordinate system. The homogeneous transformation matrix of the point relative to the base coordinate system is solved, and then the pose matrix of the puncture device at all ideal path points is obtained.

[0180] Specifically, the bending angle and rotation angle that minimize the distance between the actual stepping point and the ideal path point of the puncture device are calculated using the distance function of fmincon. The distance function is:

[0181]

[0182] In the formula, The actual stepping point of the puncture device is obtained through a forward kinematics model. This represents the position of the ideal path point on the path;

[0183] According to the forward kinematics model, the actual stepping point position of the puncture device can be obtained from the homogeneous transformation matrix of the bending element from segment (i-1) to segment ith. for:

[0184]

[0185] Solving the above equation yields for:

[0186]

[0187] Furthermore, based on the reverse recursive method, the actual stepping point matrix of the puncture device is multiplied by the inverse matrix of the homogeneous transformation matrix of the nth bending unit relative to the (n-1)th bending unit to obtain the transformation matrix of the (n-1)th bending unit and its actual stepping point position. The bending angle and rotation angle of the other bending units in the (n-1)th bending unit are obtained through the above formula. By analogy, the bending angle and rotation angle of the other bending units can be solved. The other bending units are the part between the non-interventional bending unit and the terminal bending unit, thus obtaining the step length of the puncture device.

[0188] Taking a puncture device with multiple joint segments as an example, the joint parameters of the end-effector bending unit are first solved using a distance function. In this step, the distance between the current pose of the device and the target pose is obtained by calculating the Euclidean distance between them. Based on this distance, the distance function outputs a suitable joint parameter value.

[0189] Next, a reverse recursive method is used to solve for the parameters of other joint segments. Starting from the end effector, the parameters that each joint segment needs to achieve are calculated based on the kinematic model of the device and the reverse recursive formula. This process needs to take into account the device's structural parameters, such as joint length and bending angle range.

[0190] Finally, the joint parameters obtained by the reverse recursive method are fine-tuned. This adjustment process takes into account the stability and accuracy of the equipment during movement, ensuring that the equipment can smoothly move along the planned path.

[0191] S700: Determine whether the puncture device has reached the target puncture point based on the step length of the next step. If the target puncture point has not been reached, recalculate the joint parameters of each bending unit when the device moves forward to obtain a new step length. Then, control the puncture device to move forward using the new step length and repeat this operation until the target puncture point is reached. If the target puncture point is reached, record the current puncture path plan.

[0192] In some optional implementations of certain embodiments, the step of determining whether the puncture device has reached the target puncture point based on the step length is performed. If the target puncture point has not been reached, the joint parameters of each bending unit are recalculated for the next step forward to obtain a new step length. The puncture device is then controlled to move forward using the new step length, and this operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded, including:

[0193] Position Update: Update the current position of the puncture device after it moves forward;

[0194] Target determination: Determine whether the puncture device has reached the target puncture point by comparing the distance between the current position of the puncture device and the target puncture point;

[0195] Path planning ends: If the puncture device has reached the target puncture point, the path planning ends and the current puncture path planning is recorded; otherwise, the joint parameters are solved and the puncture device is controlled to move forward.

[0196] Taking a multi-joint puncture device as an example, the process begins by controlling each joint segment of the device to move along a planned path based on the calculated joint parameters. The determination of this step length must consider the device's mechanical characteristics and operational requirements to ensure puncture accuracy and efficiency. After the device moves forward, its current position is updated. This step can be achieved by recording the rotation angle and length changes of each joint. Next, it is determined whether the device has reached the target puncture point. This is done by comparing the distance between the device's current position and the target puncture point. If the distance is less than a preset threshold, the device is considered to have reached the target point. If the device has not reached the target point, the joint parameters are calculated again, and the device is controlled to move forward. This process is repeated until the device reaches the target puncture point, at which point path planning is complete.

[0197] Secondly, this application proposes a preoperative strategy analysis and puncture path planning system for intravenous puncture, such as... Figure 2 As shown, it includes a first image processing module, a vein image processing module, a centerline image processing module, a puncture point positioning module, a device initialization module, a step analysis module, and a path planning module.

[0198] The first image processing module is used to acquire an image of the patient's puncture site as an image to be analyzed, and to perform improved restricted median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with clear contours.

[0199] The vein image processing module is used to perform threshold segmentation on the first processed image to extract a vein image with clear contours.

[0200] The centerline image processing module is used to extract the skeleton of the vein image and determine the centerline of the vein in the vein image;

[0201] The puncture point positioning module is used to obtain the depth information of the vein image through local stereo matching, determine the positioning of the vein centerline in the three-dimensional coordinate system, and select the target puncture point from the vein centerline.

[0202] The device initialization module is used to initialize and set the parameters of the multi-joint puncture device, including: the number of joint segments, the joint length, the bending angle of the bending unit, the rotation angle, the needle starting point, and the target puncture point.

[0203] The stepping analysis module is used to uniformly discretize the ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device, control the multi-joint puncture device to step from the needle starting point, calculate the joint parameters of the multi-joint puncture device during the puncture process, and control the step length of the next step through the joint parameters.

[0204] The path planning module is used to determine whether the puncture device has reached the target puncture point based on the step length of the next step. If the target puncture point has not been reached, the joint parameters of each bending unit are recalculated when the device moves forward another step to obtain a new step length. The new step length is then used to control the puncture device to move forward. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded.

[0205] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0206] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0207] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0208] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0209] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0210] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0211] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0212] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0213] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0214] The above are merely preferred embodiments of the present invention. It should be noted that any modifications and improvements made by those skilled in the art without departing from the present technical solution should also be considered to fall within the scope of protection claimed in this claim.

Claims

1. A pre-venipuncture strategy analysis and puncture path planning system, characterized by: The operation method of the system comprises the following steps: An image of a part to be punctured of a patient is collected as an image to be analyzed, and improved limited median filtering and contrast enhancement are performed on the image to be analyzed to obtain a first processed image with clear contours; Threshold segmentation processing is performed on the first processed image to extract a vein image with clear contours; Skeleton extraction is performed on the vein image to determine a vein center line in the vein image; Depth information of the vein image is obtained through local stereo matching to determine the positioning of the vein center line in a three-dimensional coordinate system, and a target puncture point is selected from the vein center line; Parameters of a multi-joint puncture device are initialized, including the number of joint segments, the length of the joint, the bending angle of the bending unit, the rotation angle, the starting point of the needle, and the target puncture point; An ideal path curve between the starting point of the needle and the target puncture point of the multi-joint puncture device is uniformly dispersed, and the multi-joint puncture device is controlled to step from the starting point of the needle, the joint parameters of the multi-joint puncture device in the puncture process are calculated, and the step length of the next step is controlled through the joint parameters; Whether the puncture device reaches the target puncture point is determined according to the step length of the next step, if the target puncture point is not reached, the joint parameters of each bending unit for the next step are recalculated to obtain a new step length, and then the puncture device is controlled to step through the new step length, and the operation is repeatedly performed until the target puncture point is reached, and if the target puncture point is reached, the current puncture path planning is recorded.

2. The system of claim 1, wherein: The improved limited median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with clear contours comprises: A filter template size is selected from the image to be analyzed, the traversal pixels in the selected filter template are traversed, the gray values are obtained and sorted, the gray values of the adjacent pixels are compared with the median value in the gray sequence, if the same, the median value is skipped, if different, the median value is replaced, the template is moved to the next pixel according to the filter template size, until all templates in the image are filtered, and a filtered image is obtained; The probability density of the gray values of the filtered image is calculated, then the gray values are mapped using the cumulative distribution function, and finally the gray values are replaced to obtain a first processed image with clear contours.

3. The system of claim 2, wherein: The threshold segmentation processing on the first processed image to extract a vein image with clear contours comprises: The gray histogram of the first processed image is analyzed to determine the gray distribution difference between the vein and the background, and a gray analysis result is obtained; One or more segmentation threshold values are selected according to the gray analysis result, the selected segmentation threshold values are applied to segment the first processed image, the pixel points greater than the segmentation threshold value are divided into the vein part, and the pixel points less than or equal to the segmentation threshold value are divided into the background; Canny edge detection is performed on the vein image obtained by segmentation to extract the contour line of the vein, and a vein image with clear contours is obtained.

4. The system of claim 3, wherein: The skeleton extraction on the vein image to determine a vein center line in the vein image comprises: The skeleton extraction includes image thinning, which is used to peel the image edge step by step until the center line of the image is obtained, wherein the vein is thinned by an erosion operation, and then a pseudo-skeleton line is extracted by image thinning, and a real vein skeleton line is extracted from the pseudo-skeleton line of the vein; A line segment representing the center of the vein is determined from the real vein skeleton line by using a center line algorithm, and the determined line segment is the vein center line in the vein image.

5. The system of claim 4, wherein: The determination of the positioning of the vein center line in a three-dimensional coordinate system and the selection of the target puncture point from the vein center line by local stereo matching to obtain the depth information of the vein image, comprising: obtaining the depth information of the vein image by local stereo matching; generating a disparity map according to the depth information, converting the disparity map into a three-dimensional coordinate map, calculating the spatial coordinates of each pixel point by the principle of triangulation, and determining the coordinate information of the vein center line according to the spatial coordinates; analyzing the geometric characteristics of the vein center line, setting threshold values and rules according to clinical experience to select the target puncture point, and obtaining the coordinate information of the target puncture point.

6. The system of claim 5, wherein: The ideal path curve between the needle starting point of the multi-joint puncture device and the target puncture point is uniformly dispersed, and the multi-joint puncture device is controlled to step from the needle starting point, the joint parameters of the multi-joint puncture device in the puncture process are calculated, and the step length of the next step is controlled through the joint parameters, comprising: constructing an ideal puncture path curve according to the positions of the needle starting point and the target puncture point; uniformly dispersing the ideal puncture path curve into a series of discrete points and determining their position coordinates; calculating the main normal vector of the discrete point, the main normal vector being perpendicular to the tangent plane of the ideal path curve at the discrete point, calculating the secondary normal vector of the discrete point, the secondary normal vector being perpendicular to the main normal and the tangent, and calculating the tangent vector of the discrete point, the tangent vector being tangent to the ideal path curve at the discrete point; constructing a pose matrix according to the position coordinates of the discrete points, and the main normal vector, the secondary normal vector and the tangent vector; solving the joint parameters of the end bending unit of the puncture device and solving the joint parameters of other bending units of the puncture device according to the pose matrix; controlling the step length of the next step of the puncture device according to the solved joint parameters.

7. The system of claim 6, wherein: The step length of the next step of the puncture device is controlled through the solved joint parameters. If the target puncture point has not been reached, the joint parameters of each bending unit for the next step are recalculated to obtain a new step length, and then the puncture device is controlled to step through the new step length, and this operation is repeatedly performed until the target puncture point is reached, if the target puncture point is reached, the current puncture path planning is recorded, comprising: position update: updating the current position of the puncture device after the puncture device advances; target judgment: judging whether the puncture device has reached the target puncture point by comparing the distance between the current position of the puncture device and the target puncture point; path planning end: if the puncture device has reached the target puncture point, the path planning is ended, and the current puncture path planning is recorded, otherwise, the joint parameters are solved and the puncture device is controlled to advance.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor implements the steps of the operation method of the pre-venipuncture strategy analysis and puncture path planning system according to any one of claims 1-7 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the operation method of the pre-venipuncture strategy analysis and puncture path planning system according to any one of claims 1-7.

Images