Vein paracentesis preoperative strategy analysis and puncture path planning method and system

Abstract

The invention discloses a vein paracentesis preoperative strategy analysis and puncture path planning method and system, and the method comprises the steps: carrying out the quick scanning of an area through infrared imaging equipment, and obtaining a series of image frames; automatically selecting an image with the clearest vein contour as a reference for subsequent analysis through an image quality evaluation function; for a patient with poor blood vessel imaging in an ultrasonic image, the needle inserting position can be accurately selected on the vein center line, the puncture success rate is increased, joint parameters of puncture equipment are accurately solved, guarantee is provided for accurate control of the equipment, it is ensured that the equipment can accurately reach a target puncture point, and the puncture efficiency is improved. When a patient suffering from highly infectious diseases is punctured, the safety of medical staff is guaranteed, and the risk of puncture failure caused by inaccurate path planning is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical data processing, and in particular relates to a method and system for pre-operative strategy analysis and puncture path planning for venipuncture. Background Art

[0002] Venipuncture is one of the most common operations in the field of clinical medicine, and is often used in drug administration, infusion and other scenarios. The difficulty in performing deep vein puncture is to find a suitable puncture path for needle insertion, and the quality of needle insertion is directly related to the success or failure of the operation. At present, when performing venous puncture, medical staff usually use blind puncture or ultrasound-guided needle insertion. When performing puncture, medical staff often select the needle insertion path based on experience, which requires high requirements for medical staff and requires them to have rich experience and skilled skills. The success rate of venous puncture depends on the experience of medical staff and the patient's vascular conditions. However, there are large individual differences among patients in clinical practice, and the visibility and elasticity of veins are different. For patients with poor vascular conditions, even experienced medical staff can hardly guarantee the success rate of puncture for these patients. Improper needle insertion angle or improper force and speed may cause vein puncture. Repeated puncture attempts will inevitably bring great pain to patients and easily cause vascular damage. The path selected by medical staff based on experience is often not accurate, which will affect the accuracy and results of the puncture operation.

[0003] In recent years, the emergence of ultrasound-guided puncture technology has attracted widespread attention. This technology has solved the problem of difficult puncture when the blood vessels are invisible and untouchable to a certain extent, but the following problems still exist:

[0004] (1) The technique is technically difficult, and medical staff performing the puncture operation need to undergo professional training in ultrasound image recognition;

[0005] (2) For patients whose blood vessels are poorly imaged in ultrasound images, it is impossible to accurately select the needle insertion position on the center line of the vein for puncture, resulting in a lower success rate of puncture;

[0006] (3) When performing puncture on patients with highly infectious diseases, the personal safety of medical staff may be threatened. Summary of the invention

[0007] In view of the above-mentioned deficiencies in the prior art, the present application provides a method and system for pre-intravenous puncture strategy analysis and puncture path planning.

[0008] In a first aspect, the present application proposes a method for pre-venipuncture strategy analysis and puncture path planning, comprising the following steps:

[0009] An image of the patient's puncture site is collected as an image to be analyzed, and the image to be analyzed is subjected to improved restrictive median filtering and contrast enhancement to obtain a first processed image with clear contours;

[0010] Performing threshold segmentation processing on the first processed image to extract a vein image with clear contours;

[0011] Performing skeleton extraction on the vein image to determine the vein centerline in the vein image;

[0012] Acquire the depth information of the vein image through local stereo matching, determine the location of the vein centerline in the three-dimensional coordinate system, and select the target puncture point from the vein centerline;

[0013] Initialize and set the parameters of the multi-joint puncture device, including: the number of device joint segments, the length of the device joints, the bending angle of the device bending unit, the rotation angle of the device, the needle starting point and the target puncture point;

[0014] Uniformly discretizing an ideal path curve between a needle starting point and a target puncture point of the multi-joint puncture device, controlling the multi-joint puncture device to step from the needle starting point, calculating joint parameters of the multi-joint puncture device during the puncture process, and controlling the step length of one step forward by the joint parameters;

[0015] Whether the puncture device has reached the target puncture point is determined based on the step length of one step forward. If it has not reached the target puncture point, the joint parameters of each bending unit when it moves forward one step further are recalculated to obtain a new step length, and then the puncture device is controlled to step forward using the new step length. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded.

[0016] In some optional implementations of some embodiments, performing improved restrictive median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with clear contours includes:

[0017] Selecting a filter template size from the image to be analyzed, traversing the traversed pixels in the selected filter template, obtaining grayscale values ​​and sorting them, comparing the grayscale values ​​of neighboring pixels with the median of the grayscale sequence, skipping them if they are the same, and replacing them with the median if they are different, moving the template to the next pixel according to the filter template size until all templates in the image are filtered, and obtaining a filtered image;

[0018] The probability density of the grayscale value of the filtered image is calculated, and then the cumulative distribution function is used for grayscale mapping, and finally the grayscale replacement is performed to obtain a first processed image with a clear outline.

[0019] In some optional implementations of some embodiments, performing threshold segmentation processing on the first processed image to extract a vein image with a clear outline includes:

[0020] Analyzing the grayscale histogram of the first processed image to determine the grayscale distribution difference between the vein and the background, and obtaining a grayscale analysis result;

[0021] According to the grayscale analysis result, one or more segmentation thresholds are selected, and the first processed image is segmented by applying the selected segmentation thresholds, so that pixels greater than the segmentation thresholds are divided into vein parts, and pixels less than or equal to the segmentation thresholds are divided into backgrounds;

[0022] Canny edge detection is performed on the segmented vein image to extract the contour of the vein and obtain a vein image with clear contours.

[0023] In some optional implementations of some embodiments, performing skeleton extraction on the vein image to determine the vein centerline in the vein image includes:

[0024] The skeleton extraction includes image thinning, and the image thinning is used to gradually peel off the edge of the image until the center line of the image is obtained, wherein the vein is thinned by an erosion operation, and then a pseudo skeleton line is extracted by image thinning, and a real vein skeleton line is extracted from the pseudo skeleton line of the vein;

[0025] A centerline algorithm is used to determine a line segment representing the center of the vein from the real vein skeleton line, and the determined line segment is the vein centerline in the vein image.

[0026] In some optional implementations of some embodiments, acquiring the depth information of the vein image by local stereo matching, determining the location of the vein centerline in a three-dimensional coordinate system, and selecting a target puncture point from the vein centerline includes:

[0027] The depth information of the vein image is obtained through local stereo matching;

[0028] Generate a disparity map according to the depth information, convert the disparity map into a three-dimensional coordinate map, calculate the spatial coordinates of each pixel point by the triangulation principle, and determine the coordinate information of the vein centerline according to the spatial coordinates;

[0029] The geometric features of the vein centerline are analyzed, and the target puncture point is selected by setting thresholds and rules based on clinical experience to obtain the coordinate information of the target puncture point.

[0030] In some optional implementations of some embodiments, the ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device is uniformly discretized, and the multi-joint puncture device is controlled to step from the needle starting point, the joint parameters of the multi-joint puncture device during the puncture process are calculated, and the step length of one step forward is controlled by the joint parameters, including:

[0031] According to the positions of the needle starting point and the target puncture point, an ideal puncture path curve is constructed;

[0032] The ideal puncture path curve is evenly discretized into a series of discrete points and their position coordinates are determined;

[0033] Calculate the principal normal vector of the discrete point, the principal normal vector is perpendicular to the tangent plane of the ideal path curve at the discrete point, calculate the secondary normal vector of the discrete point, the secondary normal vector is perpendicular to the principal normal and the tangent, calculate the tangent vector of the discrete point, the tangent vector is tangent to the ideal path curve at the discrete point;

[0034] Construct the pose matrix according to the position coordinates of the discrete points, as well as the main normal vector, binormal vector and tangent vector;

[0035] Solving the joint parameters of the bending unit at the end of the puncture device and solving the joint parameters of other bending units of the puncture device according to the posture matrix;

[0036] The step length of the puncture device moving forward is controlled according to the joint parameters obtained by the solution.

[0037] In some optional implementations of some embodiments, the step length of one step forward is used to determine whether the puncture device has reached the target puncture point. If the target puncture point has not been reached, the joint parameters of each bending unit when the next step is taken are recalculated to obtain a new step length, and then the puncture device is controlled to step by the new step length. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded, including:

[0038] Position update: after the puncture device moves forward, the current position of the puncture device is updated;

[0039] Target judgment: judge whether the puncture device has reached the target puncture point by comparing the current position of the puncture device with the distance between the target puncture point;

[0040] End of path planning: If the puncture device has reached the target puncture point, end the path planning and record the current puncture path planning; otherwise, continue to solve the joint parameters and control the puncture device to move forward.

[0041] In the second aspect, the present application proposes a pre-operative strategy analysis and puncture path planning system for intravenous puncture, including a first image processing module, a vein image processing module, a centerline image processing module, a puncture point positioning module, an equipment initialization module, a step analysis module and a path planning module.

[0042] The first image processing module is used to collect an image of the patient's puncture site as an image to be analyzed, and perform improved restrictive median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with a clear outline;

[0043] The vein image processing module is used to perform threshold segmentation processing on the first processed image to extract a vein image with clear contour;

[0044] The centerline image processing module is used to perform skeleton extraction on the vein image to determine the vein centerline in the vein image;

[0045] The puncture point positioning module is used to obtain the depth information of the vein image through local stereo matching, determine the location of the vein centerline in the three-dimensional coordinate system, and select the target puncture point from the vein centerline;

[0046] The device initialization module is used to initialize and set the parameters of the multi-joint puncture device, including: the number of device joint segments, the length of device joints, the bending angle of the device bending unit, the rotation angle of the device, the needle starting point and the target puncture point;

[0047] The step analysis module is used to evenly discretize the ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device, control the multi-joint puncture device to step from the needle starting point, calculate the joint parameters of the multi-joint puncture device during the puncture process, and control the step length of one step forward through the joint parameters;

[0048] The path planning module is used to determine whether the puncture device has reached the target puncture point based on the step length of one step forward. If the target puncture point has not been reached, the joint parameters of each bending unit when the next step is forward are recalculated to obtain a new step length, and then the puncture device is controlled to step by the new step length. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded.

[0049] In a third aspect, the present application proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0050] In a fourth aspect, the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0051] Beneficial effects of the present invention:

[0052] The infrared imaging device is used to quickly scan the area and obtain a series of image frames. The image quality evaluation function automatically selects an image with the clearest vein contour as the basis for subsequent analysis; for patients whose blood vessels are poorly imaged in ultrasound images, the needle insertion position can also be accurately selected on the vein centerline to improve the puncture success rate, accurately solve the joint parameters of the puncture device, provide guarantees for the precise control of the device, and ensure that the device can accurately reach the target puncture point. When puncturing patients with highly infectious diseases, it protects the safety of medical staff themselves and reduces the risk of puncture failure due to inaccurate path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is the overall flow chart of the present invention.

[0054] Figure 2 It is a system principle block diagram of the present invention. DETAILED DESCRIPTION

[0055] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein; on the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0056] In the first aspect, the present application proposes a method for pre-operative strategy analysis and puncture path planning for venipuncture, such as Figure 1 As shown, the following steps are included:

[0057] S100: collecting an image of the patient's puncture site as an image to be analyzed, and performing improved restrictive median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with clear contours;

[0058] In some optional implementations of some embodiments, performing improved restrictive median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with clear contours includes:

[0059] Selecting a filter template size from the image to be analyzed, traversing the traversed pixels in the selected filter template, obtaining grayscale values ​​and sorting them, comparing the grayscale values ​​of neighboring pixels with the median of the grayscale sequence, skipping them if they are the same, and replacing them with the median if they are different, moving the template to the next pixel according to the filter template size until all templates in the image are filtered, and obtaining a filtered image;

[0060] The probability density of the grayscale value of the filtered image is calculated, and then the cumulative distribution function is used for grayscale mapping, and finally the grayscale replacement is performed to obtain a first processed image with a clear outline.

[0061] Among them, before implementing the venipuncture, infrared imaging technology is first used to collect images of the puncture site. The key to this step is to ensure that the collected image can clearly reflect the structure and contour of the vein for subsequent processing and analysis. Among them, this embodiment is based on the Mindray M9 ultrasonic vascular image acquisition system and the venous ultrasonic image acquisition system of the Vision Image MV-VGA200E model acquisition card for image acquisition. The image acquisition system receives the image output signal of the ultrasonic system through the VGA interface of the computer, thereby collecting the venous vascular image of the M9 ultrasound instrument into the computer. The components of the system and its functions are described in detail below. The specific image acquisition process is as follows:

[0062] Choose a suitable location: Locate an appropriate area near the patient's skin surface where the puncture is to be performed, ensure that the veins in this area are clearly visible, and try to avoid the venous branches and curved parts to improve the accuracy of subsequent image processing.

[0063] Preparation of infrared imaging equipment: Use a calibrated infrared imaging device and adjust its parameters, such as resolution, focal length, and aperture, to suit different patient skin types and vein depths.

[0064] Image acquisition: Start the infrared imaging device, scan the puncture site, and obtain a series of continuous image frames. Among these frames, select the frame with the best image quality as the image to be analyzed.

[0065] After image acquisition, preprocessing the image to be analyzed is a key step to improve image quality and highlight vein contours. The following is a detailed image preprocessing process:

[0066] The original image has noise, so it is necessary to filter the original image to improve the image quality. The principle of mean filtering is simple. The specific method is to find the mean of the grayscale values ​​of the pixels in the template and then replace the grayscale. Suppose the original grayscale value is , the gray value after filtering is , the expression of the mean filter is as follows:

[0067]

[0068] Where s is the neighborhood of the n×n template, is the number of pixels in the neighborhood, is the two-dimensional coordinate of the pixel in the neighborhood. The above filter has a small amount of calculation and an obvious smoothing effect. However, the filtering effect is greatly affected by the template. If the template is too large, the edge of the vein will become blurred, which is not conducive to extracting the vein contour from the image; the smaller the template, the worse the image filtering effect.

[0069] Median filtering is usually divided into two steps. First, the maximum and minimum grayscale in the template are removed as noise. Then the grayscale values ​​of the remaining pixels are arranged, the median of the grayscale values ​​is calculated, and finally the grayscale is replaced. The principle of the median filter is shown in the following formula:

[0070]

[0071] Compared with the mean filter, the median filter has a slightly better effect, but it is still difficult to overcome the problem of blurring the vein edge. In this paper, the median filter algorithm is appropriately improved to improve the above problem. The specific improvement is: before grayscale replacement, the pixel points judged as the edge are judged to not perform median replacement, so that the edge can be preserved. The specific algorithm flow is as follows:

[0072] (1) Select the filter template size as 3×3;

[0073] (2) Traverse the pixels, obtain the grayscale values ​​and sort them;

[0074] (3) Compare the grayscale value of the neighborhood pixel with the median of the grayscale sequence. If they are the same, skip them; if they are different, replace them with the median.

[0075] (4) Move the template to the next pixel and repeat the above steps.

[0076] Furthermore, image enhancement can enhance visual effects and grayscale contrast, making it easier to perform image segmentation and extract the outer contour of the vein. After contrast enhancement, the vein image can make the vein position more obvious, which is conducive to the next step of vein extraction. The method of graphic contrast enhancement is to increase the difference between the grayscale value of the vein position and the grayscale value of the normal skin by mapping and replacing the grayscale value of the pixels in the image, making the vein more prominent and clear.

[0077] Grayscale stretching enhancement is to enlarge the grayscale range of the image, thereby amplifying the grayscale value difference between veins and other parts. The grayscale stretching enhancement algorithm formula is as follows:

[0078]

[0079] In the formula, the grayscale range is expanded from [a, b] to [c, d], and the contrast between veins and other areas is enhanced. The histogram of an image can reflect the grayscale distribution of the image and the basic characteristics of the image. Histogram equalization first calculates the probability density of the grayscale, then uses the cumulative distribution function for grayscale mapping, and finally performs grayscale replacement. The specific algorithm flow is as follows:

[0080] (1) Calculate the probability density of the grayscale value of the filtered image:

[0081]

[0082] in, Indicates Gray value, Indicates the number of pixels at this gray level.

[0083] (2) Cumulative distribution function of the image:

[0084]

[0085] in, Indicates The gray value after the gray value mapping.

[0086] (3) Round the mapped grayscale value.

[0087] (4) Replace the grayscale value to obtain a new image.

[0088] Among them, the above two image enhancement algorithms were used to process the vein image respectively, but the use of histogram equalization achieved better results, and the vein contours in the processed images became clear.

[0089] After completing the image preprocessing, in order to accurately extract the vein image, the first processed image needs to be threshold segmented. Threshold segmentation is a segmentation method based on the gray level of the image. By setting one or more thresholds, the image pixels are divided into foreground (veins) and background. The following is a detailed vein image extraction process:

[0090] S200: performing threshold segmentation processing on the first processed image to extract a vein image with clear contour;

[0091] In some optional implementations of some embodiments, performing threshold segmentation processing on the first processed image to extract a vein image with a clear outline includes:

[0092] Analyzing the grayscale histogram of the first processed image to determine the grayscale distribution difference between the vein and the background, and obtaining a grayscale analysis result;

[0093] According to the grayscale analysis result, one or more segmentation thresholds are selected, and the first processed image is segmented by applying the selected segmentation thresholds, so that pixels greater than the segmentation thresholds are divided into vein parts, and pixels less than or equal to the segmentation thresholds are divided into backgrounds;

[0094] Canny edge detection is performed on the segmented vein image to extract the contour of the vein and obtain a vein image with clear contours.

[0095] Specifically, they include:

[0096] Grayscale image analysis: Analyze the grayscale histogram of the first processed image to determine the grayscale distribution difference between the vein and the background.

[0097] Select appropriate thresholds: According to the grayscale analysis results, select one or more appropriate thresholds. In order to improve the accuracy of segmentation, dynamic thresholding or region growing based methods can be used.

[0098] Threshold segmentation: Apply the selected threshold to segment the image, divide the pixels greater than the threshold into vein parts, and divide the pixels less than or equal to the threshold into background.

[0099] Furthermore, Otsu algorithm is applied for threshold segmentation:

[0100] Set a series of different thresholds to segment the image, and use the inter-class variance to evaluate the quality of threshold segmentation. Set the threshold to S, and the grayscale value greater than S is judged as the target, and less than S is judged as the background. Count the number of target and background pixels respectively, and calculate their respective proportions, recorded as and , calculate the average grayscale of the target as , the average grayscale of the background is The average grayscale of the image is , the calculation method is as follows:

[0101]

[0102] Among them, the between-class variance is , the calculation formula is:

[0103]

[0104] Contour extraction: The contour of the vein is extracted through edge detection algorithms, such as Canny edge detection.

[0105] S300: performing skeleton extraction on the vein image to determine the vein centerline in the vein image;

[0106] In some optional implementations of some embodiments, performing skeleton extraction on the vein image to determine the vein centerline in the vein image includes:

[0107] The skeleton extraction includes image thinning, and the image thinning is used to gradually peel off the edge of the image until the center line of the image is obtained, wherein the vein is thinned by an erosion operation, and then a pseudo skeleton line is extracted by image thinning, and a real vein skeleton line is extracted from the pseudo skeleton line of the vein;

[0108] A centerline algorithm is used to determine a line segment representing the center of the vein from the real vein skeleton line, and the determined line segment is the vein centerline in the vein image.

[0109] After the vein image is extracted, in order to accurately plan the puncture path, the center line of the vein needs to be determined. This step is achieved through morphological processing and skeleton extraction algorithms. The following is the detailed process:

[0110] 1. Morphological processing: Morphological processing is a common method in image processing. It improves the structure of the image through a series of morphological operations to facilitate subsequent analysis. In this step, the extracted vein contour is first eroded and expanded to eliminate small branches and protruding parts in the image;

[0111] 2. Skeleton extraction: Skeleton extraction is an image thinning algorithm that gradually peels off the edges of the image until the center line of the image is obtained. In this process, an improved iterative thinning algorithm is used, which can adapt to vein structures of different widths and sizes and accurately extract the center line of the vein;

[0112] 3. Centerline determination: Based on skeleton extraction, further analysis is performed to determine the main skeleton line representing the center of the vein. This step can be achieved by finding the longest continuous line segment in the skeleton, or by other algorithms such as the minimum spanning tree.

[0113] Taking the vein image that has been preprocessed and threshold segmented as an example, we first perform morphological processing on it, and remove the small branches and noise in the image through corrosion and expansion operations to obtain a cleaner vein structure image. Then, we apply the thinning algorithm to extract the skeleton of the image to obtain the skeleton line of the vein. On this basis, we use the centerline algorithm, such as the method of finding the longest continuous line segment, to determine the line segment representing the center of the vein. This line segment is the centerline of the vein in the puncture path planning, which will serve as the benchmark for subsequent path planning.

[0114] S400: acquiring depth information of the vein image through local stereo matching, determining the location of the vein centerline in a three-dimensional coordinate system, and selecting a target puncture point from the vein centerline;

[0115] In some optional implementations of some embodiments, acquiring the depth information of the vein image by local stereo matching, determining the location of the vein centerline in a three-dimensional coordinate system, and selecting a target puncture point from the vein centerline includes:

[0116] The depth information of the vein image is obtained through local stereo matching;

[0117] Generate a disparity map according to the depth information, convert the disparity map into a three-dimensional coordinate map, calculate the spatial coordinates of each pixel point by the triangulation principle, and determine the coordinate information of the vein centerline according to the spatial coordinates;

[0118] The geometric features of the vein centerline are analyzed, and the target puncture point is selected by setting thresholds and rules based on clinical experience to obtain the coordinate information of the target puncture point.

[0119] Local stereo matching obtains the three-dimensional information of the photographed object by matching different image pixels. The specific method is to map the 2D displacement difference of the matching pixels to 3D depth, thereby constructing a 3D model as a reproduction of the original scene. Because the position and depth of the vein need to be updated in real time during the venipuncture process, the real-time requirements of the algorithm are relatively high. However, the global matching algorithm has a large delay, so this embodiment uses a local stereo matching algorithm, and the specific implementation process is as follows:

[0120] Step 1: Obtain the depth information of the vein image through a local stereo matching algorithm;

[0121] Feature extraction: Extract key feature points in the image, such as SIFT, SURF, etc.

[0122] Local Stereo Matching:

[0123] Matching cost calculation: For each pixel, calculate its matching cost between images of different perspectives, usually using SSD (Sum of Squared Differences) or NCC (Normalized Cross-Correlation).

[0124] Cost aggregation: Cost aggregation is performed within a local window to find the disparity value with the minimum matching cost.

[0125] Matching cost calculation:

[0126]

[0127] in, The parallax is The matching cost when and are the pixel values ​​of the left and right images respectively.

[0128] Cost aggregation:

[0129]

[0130] in, Yes The optimal disparity, i.e. depth information.

[0131] Parameter explanation:

[0132] : Parallax, which means the difference in the horizontal coordinates of the same object point in the left and right images.

[0133] : Pixel coordinates in the image.

[0134] : Pixel values ​​of left and right images.

[0135] Step 2: Determine the location of the vein centerline in the three-dimensional coordinate system;

[0136] Disparity map generation: Generate a disparity map based on the disparity information obtained by local stereo matching.

[0137] 3D reconstruction: Convert the disparity map into a 3D coordinate map and calculate the spatial coordinates of each pixel through the principle of triangulation.

[0138] Centerline extraction: Use edge detection, morphology and other operations to extract the vein centerline.

[0139] Triangulation:

[0140]

[0141] in, is the depth information, is the focal length of the camera, is the baseline distance between the dual cameras.

[0142] Step 3: Select the target puncture point from the vein centerline;

[0143] Centerline analysis: Analyze the geometric characteristics of the vein centerline, such as curvature, width, etc.

[0144] Target puncture point selection:

[0145] Based on clinical experience, points with smaller curvature, moderate width, and appropriate depth are selected as puncture points.

[0146] Automatic selection can be performed by setting certain thresholds and rules.

[0147] Curvature calculation:

[0148]

[0149] in, is the curvature, and The center line is The slope and second derivative of the tangent line at position.

[0150] The above steps are used to determine the location of the vein centerline and target puncture point.

[0151] S500: Initializing and setting parameters of the multi-joint puncture device, including: the number of device joint segments, the length of the device joints, the bending angle of the device bending unit, the rotation angle of the device, the needle starting point and the target puncture point;

[0152] After determining the target puncture point, in order to achieve accurate puncture operation, the parameters of the puncture device need to be initialized. This step includes determining the key parameters of the device such as the number of joint segments, length, bending angle, rotation angle, etc. The following is a detailed device parameter initialization process:

[0153] Determination of the number of joint segments: According to the mechanical structure of the puncture device, the number of joint segments it has is determined as n. The number of joint segments determines the degree of bending and flexibility that the device can achieve.

[0154] Joint length setting: According to the mechanical design of the device, set the length of each joint segment to 1. The joint length affects the overall length and bending radius of the device.

[0155] Bending angle range setting: According to the mechanical properties of the device, set the maximum and minimum bending angles that each joint segment can reach. The bending angle determines the bending degree of the device in space. The bending angle is .

[0156] Rotation angle range setting: Set the angle range that the end effector of the device (such as a needle) can rotate. The rotation angle determines the direction adjustment ability of the needle during the puncture process. .

[0157] Needle starting point setting: Set the needle starting position according to the mechanical structure of the device and the location of the target puncture point. The starting point determines the initial position of the puncture operation. .

[0158] Target puncture point setting: According to the target puncture point determined in step S400, the target position of the puncture operation is set. The target puncture point is .

[0159] S600: evenly discretize the ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device, control the multi-joint puncture device to step from the needle starting point, calculate the joint parameters of the multi-joint puncture device during the puncture process, and control the step length of one step forward through the joint parameters;

[0160] In some optional implementations of some embodiments, the ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device is uniformly discretized, and the multi-joint puncture device is controlled to step from the needle starting point, the joint parameters of the multi-joint puncture device during the puncture process are calculated, and the step length of one step forward is controlled by the joint parameters, including:

[0161] According to the positions of the needle starting point and the target puncture point, an ideal puncture path curve is constructed;

[0162] The ideal puncture path curve is evenly discretized into a series of discrete points and their position coordinates are determined;

[0163] Calculate the principal normal vector of the discrete point, the principal normal vector is perpendicular to the tangent plane of the ideal path curve at the discrete point, calculate the secondary normal vector of the discrete point, the secondary normal vector is perpendicular to the principal normal and the tangent, calculate the tangent vector of the discrete point, the tangent vector is tangent to the ideal path curve at the discrete point;

[0164] Construct the pose matrix according to the position coordinates of the discrete points, as well as the main normal vector, binormal vector and tangent vector;

[0165] Solving the joint parameters of the bending unit at the end of the puncture device and solving the joint parameters of other bending units of the puncture device according to the posture matrix;

[0166] The step length of the puncture device moving forward is controlled according to the joint parameters obtained by the solution.

[0167] The ideal puncture path curve is evenly discretized, and the ideal path curve between the starting point and the target point of the puncture device is evenly discretized into m points, namely, point To point , each ideal path point is expressed in the base coordinate system as:

[0168]

[0169] The discrete point closest to the puncture device on the ideal path curve is set as M (for example, the actual step point at the current moment is discrete point 3, then the nearest discrete point is 4), s is the arc length from the starting point of the ideal path to point M, and the tangent vector of the curve at point M is:

[0170]

[0171] Among them, t is the tangent vector, and the radial vector at point M on the curve is ,if ,in, and Radial vector Relative to the first and second derivatives of s, the tangent vector and position vector at point M can uniquely determine an osculating plane, and the equation of the osculating plane is:

[0172]

[0173] in, is the binormal equation, and the standard binormal equation is expressed as:

[0174]

[0175] At a known tangent vector and the binormal vector In the case of , the principal normal at point M is solved by the right-hand rule:

[0176]

[0177] Therefore, the puncture device moves along the ideal path curve. When , the homogeneous transformation matrix of the actual step point of the puncture device relative to the base coordinate system is:

[0178]

[0179] in, is the position coordinate of point M in the base coordinate system. The ideal path curve is equally dispersed into m points, and the tangent, principal normal and binormal of each point are used as the x, y and z axes of its coordinate system. The homogeneous transformation matrix of the point relative to the base coordinate system is solved, and then the position matrix of the puncture device at all ideal path points is obtained.

[0180] Specifically, the distance function of fmincon is used to solve the bending angle and rotation angle when the distance between the actual step point of the puncture device and the ideal path point is the smallest, where the distance function is:

[0181]

[0182] In the formula, is the actual step point position of the puncture device, obtained through the forward kinematics model. is the position of the ideal path point on the path;

[0183] From the forward kinematics model, we can know that the actual stepping point position of the puncture device can be obtained according to the homogeneous transformation matrix from the i-1th segment to the i-th segment bending unit, that is, for:

[0184]

[0185] Solving the above formula, we can get for:

[0186]

[0187] Furthermore, based on the reverse recursive method, the actual step point matrix of the puncture device is multiplied by the inverse matrix of the homogeneous transformation matrix of the n-th bending unit relative to the n-1-th bending unit to obtain the n-1 bending unit transformation matrix and its actual step point position. The bending angle and rotation angle of the other n-1 bending units are obtained by the above formula. Similarly, the bending angle and rotation angle of other bending units can be solved. The other bending units are the parts between the unintervening bending units and the terminal bending units, thereby obtaining the step length of the puncture device moving forward one step.

[0188] Taking a puncture device with multiple joint segments as an example, the distance function is first used to solve the joint parameters of the terminal bending unit. In this step, the distance between the current posture of the device and the target posture is obtained by calculating the Euclidean distance between the two. Based on this distance, the distance function will output a suitable joint parameter value.

[0189] Next, the inverse recursion method is used to solve the parameters of other joint segments. Starting from the end, the parameters that need to be achieved for each joint segment are calculated based on the kinematic model of the device and the inverse recursion formula. This process needs to take into account the device structural parameters, such as joint length and bending angle range.

[0190] Finally, the joint parameters solved by the reverse recursion method are fine-tuned. This adjustment process takes into account the stability and accuracy of the device during movement to ensure that the device can step smoothly along the planned path.

[0191] S700: Determine whether the puncture device has reached the target puncture point based on the step length of one step forward. If it has not reached the target puncture point, recalculate the joint parameters of each bending unit when it moves forward one step further to obtain a new step length, and then control the stepping of the puncture device by the new step length. Repeat this operation until it reaches the target puncture point. If it reaches the target puncture point, record the current puncture path planning.

[0192] In some optional implementations of some embodiments, the step length of one step forward is used to determine whether the puncture device has reached the target puncture point. If the target puncture point has not been reached, the joint parameters of each bending unit when the next step is taken are recalculated to obtain a new step length, and then the puncture device is controlled to step by the new step length. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded, including:

[0193] Position update: after the puncture device moves forward, the current position of the puncture device is updated;

[0194] Target judgment: judge whether the puncture device has reached the target puncture point by comparing the current position of the puncture device with the distance between the target puncture point;

[0195] End of path planning: If the puncture device has reached the target puncture point, end the path planning and record the current puncture path planning; otherwise, continue to solve the joint parameters and control the puncture device to move forward.

[0196] Among them, taking a multi-joint puncture device as an example, first, according to the solved joint parameters, each joint segment of the device is controlled to perform corresponding movements, so that the device moves forward one step along the planned path. In this step, the determination of the step length needs to take into account the mechanical characteristics and operating requirements of the device to ensure the accuracy and efficiency of the puncture. After the device moves forward, the current position of the device is updated. This step can be achieved by recording the rotation angle and length change of each joint of the device, and then judging whether the device has reached the target puncture point. This can be achieved by comparing the distance between the current position of the device and the target puncture point. If the distance is less than the preset threshold, it is considered that the device has reached the target point. If the device has not reached the target point, the joint parameters will continue to be solved and the device will be controlled to move forward. This process will be repeated until the device reaches the target puncture point, at which time the path planning ends.

[0197] In the second aspect, the present application proposes a system for pre-operative strategy analysis and puncture path planning for venipuncture, such as Figure 2 As shown, it includes a first image processing module, a vein image processing module, a centerline image processing module, a puncture point positioning module, a device initialization module, a step analysis module and a path planning module.

[0198] The first image processing module is used to collect an image of the patient's puncture site as an image to be analyzed, and perform improved restrictive median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with a clear outline;

[0199] The vein image processing module is used to perform threshold segmentation processing on the first processed image to extract a vein image with clear contour;

[0200] The centerline image processing module is used to perform skeleton extraction on the vein image to determine the vein centerline in the vein image;

[0201] The puncture point positioning module is used to obtain the depth information of the vein image through local stereo matching, determine the location of the vein centerline in the three-dimensional coordinate system, and select the target puncture point from the vein centerline;

[0202] The device initialization module is used to initialize and set the parameters of the multi-joint puncture device, including: the number of device joint segments, the length of device joints, the bending angle of the device bending unit, the rotation angle of the device, the needle starting point and the target puncture point;

[0203] The step analysis module is used to evenly discretize the ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device, control the multi-joint puncture device to step from the needle starting point, calculate the joint parameters of the multi-joint puncture device during the puncture process, and control the step length of one step forward through the joint parameters;

[0204] The path planning module is used to determine whether the puncture device has reached the target puncture point based on the step length of one step forward. If the target puncture point has not been reached, the joint parameters of each bending unit when the next step is forward are recalculated to obtain a new step length, and then the puncture device is controlled to step by the new step length. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded.

[0205] In a third aspect, the present application proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0206] In a fourth aspect, the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0207] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0208] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0209] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0210] In the embodiments provided in the present disclosure, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of the apparatus or unit, which may be electrical, mechanical or other forms.

[0211] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0212] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0213] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0214] The above are only preferred implementations of the present invention. It should be pointed out that a number of modifications and improved technical solutions made by those skilled in the art without departing from the technical solution should also be deemed to fall within the scope of protection required by the claims.

Claims

1. A method for pre-venipuncture strategy analysis and puncture path planning, characterized by: The following steps are involved: An image of the patient's puncture site is collected as an image to be analyzed, and the image to be analyzed is subjected to improved restrictive median filtering and contrast enhancement to obtain a first processed image with clear contours; Performing threshold segmentation processing on the first processed image to extract a vein image with clear contours; Performing skeleton extraction on the vein image to determine the vein centerline in the vein image; Acquire the depth information of the vein image through local stereo matching, determine the location of the vein centerline in the three-dimensional coordinate system, and select the target puncture point from the vein centerline; Initialize and set the parameters of the multi-joint puncture device, including: the number of device joint segments, the length of the device joints, the bending angle of the device bending unit, the rotation angle of the device, the needle starting point and the target puncture point; Uniformly discretizing an ideal path curve between a needle starting point and a target puncture point of the multi-joint puncture device, controlling the multi-joint puncture device to step from the needle starting point, calculating joint parameters of the multi-joint puncture device during the puncture process, and controlling the step length of one step forward by the joint parameters; Whether the puncture device has reached the target puncture point is determined based on the step length of one step forward. If it has not reached the target puncture point, the joint parameters of each bending unit when it moves forward one step further are recalculated to obtain a new step length, and then the puncture device is controlled to step forward using the new step length. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded.

2. The method according to claim 1, characterized in that: The step of performing improved restrictive median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with clear contours comprises: Selecting a filter template size from the image to be analyzed, traversing the traversed pixels in the selected filter template, obtaining grayscale values ​​and sorting them, comparing the grayscale values ​​of neighboring pixels with the median of the grayscale sequence, skipping them if they are the same, and replacing them with the median if they are different, moving the template to the next pixel according to the filter template size until all templates in the image are filtered, and obtaining a filtered image; The probability density of the grayscale value of the filtered image is calculated, and then the cumulative distribution function is used for grayscale mapping, and finally the grayscale replacement is performed to obtain a first processed image with a clear outline.

3. The method according to claim 2, characterized in that: The step of performing threshold segmentation processing on the first processed image to extract a vein image with clear contours includes: Analyzing the grayscale histogram of the first processed image to determine the grayscale distribution difference between the vein and the background, and obtaining a grayscale analysis result; According to the grayscale analysis result, one or more segmentation thresholds are selected, and the first processed image is segmented by applying the selected segmentation thresholds, so that pixels greater than the segmentation thresholds are divided into vein parts, and pixels less than or equal to the segmentation thresholds are divided into backgrounds; Canny edge detection is performed on the segmented vein image to extract the contour of the vein and obtain a vein image with clear contours.

4. The method according to claim 3, characterized in that: The step of extracting a skeleton from the vein image to determine a vein centerline in the vein image includes: The skeleton extraction includes image thinning, and the image thinning is used to gradually peel off the edge of the image until the center line of the image is obtained, wherein the vein is thinned by an erosion operation, and then a pseudo skeleton line is extracted by image thinning, and a real vein skeleton line is extracted from the pseudo skeleton line of the vein; A centerline algorithm is used to determine a line segment representing the center of the vein from the real vein skeleton line, and the determined line segment is the vein centerline in the vein image.

5. The method according to claim 4, characterized in that: The method of acquiring the depth information of the vein image by local stereo matching, determining the location of the vein centerline in a three-dimensional coordinate system, and selecting a target puncture point from the vein centerline includes: The depth information of the vein image is obtained through local stereo matching; Generate a disparity map according to the depth information, convert the disparity map into a three-dimensional coordinate map, calculate the spatial coordinates of each pixel point by the triangulation principle, and determine the coordinate information of the vein centerline according to the spatial coordinates; The geometric features of the vein centerline are analyzed, and the target puncture point is selected by setting thresholds and rules based on clinical experience to obtain the coordinate information of the target puncture point.

6. The method according to claim 5, characterized in that: The method comprises the steps of uniformly discretizing the ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device, controlling the multi-joint puncture device to step from the needle starting point, calculating the joint parameters of the multi-joint puncture device during the puncture process, and controlling the step length of one step forward by the joint parameters, including: According to the positions of the needle starting point and the target puncture point, an ideal puncture path curve is constructed; The ideal puncture path curve is evenly discretized into a series of discrete points and their position coordinates are determined; Calculate the principal normal vector of the discrete point, the principal normal vector is perpendicular to the tangent plane of the ideal path curve at the discrete point, calculate the secondary normal vector of the discrete point, the secondary normal vector is perpendicular to the principal normal and the tangent, calculate the tangent vector of the discrete point, the tangent vector is tangent to the ideal path curve at the discrete point; Construct the pose matrix according to the position coordinates of the discrete points, as well as the main normal vector, binormal vector and tangent vector; Solving the joint parameters of the bending unit at the end of the puncture device and solving the joint parameters of other bending units of the puncture device according to the posture matrix; The step length of the puncture device moving forward is controlled according to the joint parameters obtained by the solution.

7. The method according to claim 6, characterized in that: The step length of the step forward is used to determine whether the puncture device has reached the target puncture point. If the target puncture point has not been reached, the joint parameters of each bending unit when the step forward is recalculated to obtain a new step length, and then the puncture device is controlled to step by the new step length. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded, including: Position update: after the puncture device moves forward, the current position of the puncture device is updated; Target judgment: judge whether the puncture device has reached the target puncture point by comparing the current position of the puncture device with the distance between the target puncture point; End of path planning: If the puncture device has reached the target puncture point, end the path planning and record the current puncture path planning; otherwise, continue to solve the joint parameters and control the puncture device to move forward.

8. A system for pre-operative strategy analysis and puncture path planning for venipuncture, characterized in that: It includes a first image processing module, a vein image processing module, a centerline image processing module, a puncture point positioning module, an equipment initialization module, a step analysis module and a path planning module; The first image processing module is used to collect an image of the patient's puncture site as an image to be analyzed, and perform improved restrictive median filtering and contrast enhancement on the image to be analyzed to obtain a first processed image with a clear outline; The vein image processing module is used to perform threshold segmentation processing on the first processed image to extract a vein image with clear contour; The centerline image processing module is used to perform skeleton extraction on the vein image to determine the vein centerline in the vein image; The puncture point positioning module is used to obtain the depth information of the vein image through local stereo matching, determine the location of the vein centerline in the three-dimensional coordinate system, and select the target puncture point from the vein centerline; The device initialization module is used to initialize and set the parameters of the multi-joint puncture device, including: the number of device joint segments, the length of device joints, the bending angle of the device bending unit, the rotation angle of the device, the needle starting point and the target puncture point; The step analysis module is used to evenly discretize the ideal path curve between the needle starting point and the target puncture point of the multi-joint puncture device, control the multi-joint puncture device to step from the needle starting point, calculate the joint parameters of the multi-joint puncture device during the puncture process, and control the step length of one step forward through the joint parameters; The path planning module is used to determine whether the puncture device has reached the target puncture point based on the step length of one step forward. If the target puncture point has not been reached, the joint parameters of each bending unit when the next step is forward are recalculated to obtain a new step length, and then the puncture device is controlled to step by the new step length. This operation is repeated until the target puncture point is reached. If the target puncture point is reached, the current puncture path planning is recorded.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images