Tumor needle biopsy robot and image real-time navigation system thereof

Abstract

The invention discloses a tumor needle biopsy robot and an image real-time navigation system thereof, and relates to the technical field of medical instruments, and the method comprises the following steps: extracting historical data, and constructing a segmentation model according to the historical data; acquiring a plurality of medical images of the patient, and constructing a three-dimensional model of the patient according to the plurality of medical images; constructing a visual puncture path according to the three-dimensional model; the method comprises the following steps: acquiring optical tracking data, acquiring a needle point position according to the optical tracking data, correcting a puncture path according to the needle point position, acquiring a path point sequence according to a puncture starting point, a target point, a cost function and a graph structure when an optimal path is planned, and continuously adjusting the path through a path optimization algorithm. The path is far away from key tissues such as important blood vessels and nerves, the path is prevented from excessively approaching or directly penetrating through a dangerous area, so that the rationality of optimal path planning is ensured, the precision controllability of the puncture process is enhanced, the puncture path is corrected in real time through the position of the needle tip, and the possibility of the scheme is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to a tumor puncture biopsy robot and a real-time imaging navigation system thereof. Background Art

[0002] Tumor biopsy, also known as tumor puncture, is a minimally invasive diagnostic technique that uses a fine or coarse needle to puncture diseased tissue to obtain cell or tissue samples for pathological examination. It is an important clinical tool for confirming the nature of a tumor (benign or malignant), determining its pathological type, and guiding subsequent treatment.

[0003] Tumor biopsy involves using a needle to precisely locate the tumor site under the guidance of imaging modalities (such as ultrasound, CT, and MRI) and extract a small sample of cells or tissue. Pathologists use a microscope to observe the sample's cellular morphology, structure, and molecular characteristics to determine the tumor's benign or malignant nature, type, and grade, providing a basis for clinical treatment.

[0004] Most current planning methods tend to simply focus on the target tumor area, but seriously lack comprehensive consideration of potential danger zones during the puncture process and the boundaries of complex anatomical structures of the human body. Danger zones may cover important blood vessels, nerves and other key tissues, while anatomical boundaries involve the precise demarcation between different tissues and organs. If these factors are ignored, the planned puncture path is likely to be unreasonable. This unreasonableness is reflected in the fact that the path may be too close to or even directly pass through the danger zone, or there may be unnecessary detours and turns in complex anatomical structures, making it difficult to accurately control the puncture process. Therefore, the present invention proposes a tumor puncture biopsy robot and its real-time imaging navigation system. Summary of the Invention

[0005] In order to solve the above technical problems, a tumor puncture biopsy robot and its real-time imaging navigation system are provided to solve the problem that the planned puncture path is likely to be unreasonable.

[0006] In order to achieve the above objects, the technical solution adopted by the present invention is:

[0007] Tumor biopsy robot and its real-time imaging navigation system include:

[0008] S100, extracting historical data and building a segmentation model based on the historical data;

[0009] S200, obtaining multiple medical images of the patient, and constructing a three-dimensional model of the patient based on the multiple medical images;

[0010] S300, constructing a visualized optimal path based on the three-dimensional model;

[0011] S400 , obtaining optical tracking data, obtaining the needle tip position based on the optical tracking data, and correcting the puncture path based on the needle tip position.

[0012] Preferably, extracting historical data and constructing a segmentation model based on the historical data includes the following steps:

[0013] S101, extracting multiple images from historical data and obtaining their corresponding labels;

[0014] S102, dividing the multiple images into a training set and a validation set;

[0015] S103, inputting the image in the training set into the segmentation model, and obtaining the prediction result through the forward propagation calculation of the segmentation model, wherein the segmentation model is based on U-Net;

[0016] S104. Calculate the loss function value based on the prediction result and the corresponding label;

[0017] S105, performing a back propagation algorithm on the loss function value to obtain a gradient, and using the gradient to update the parameters of the segmentation model;

[0018] S106, repeat the above forward propagation, loss calculation and back propagation steps until the segmentation model converges or reaches a preset number of iterations;

[0019] S107. Verify the iterated segmentation model based on the verification set.

[0020] Preferably, the acquiring of multiple medical images of the patient and constructing a three-dimensional model of the patient based on the multiple medical images comprises the following steps:

[0021] S201, preprocessing each medical image to obtain a preprocessed image;

[0022] S202, inputting the pre-processed image into the segmentation model to obtain the category of each region in the pre-processed image;

[0023] S203, performing SIFT feature extraction on the multiple pre-processed images to obtain key point sets and feature descriptors of the pre-processed images;

[0024] S204, obtaining the Euclidean distance between the feature descriptors in two adjacent pre-processed images, and obtaining the matching key point pairs in the two pre-processed images;

[0025] S205, obtaining two pre-processed image transformation parameters using a random sampling consensus algorithm based on the key point pairs;

[0026] S206, obtaining a registered paired image based on the pre-processed image and the corresponding transformation parameters;

[0027] S207: Generate a three-dimensional model based on the paired images using surface rendering technology.

[0028] Preferably, the preprocessing of each medical image to obtain a preprocessed image comprises the following steps:

[0029] S2011, performing Gaussian filtering on each medical image to obtain a filtered, denoised image;

[0030] S2012, performing sharpening processing on each noise reduction image to obtain a sharpened enhanced image;

[0031] S2013, normalizing the pixel values ​​of each pixel in the enhanced image to obtain multiple normalized images;

[0032] S2014. Adjust each normalized image to the same resolution to obtain an adjusted pre-processed image.

[0033] Preferably, the step of constructing a visualized optimal path based on the three-dimensional model includes the following steps:

[0034] S301, rendering the three-dimensional model according to OpenGL, and obtaining the rendered visual model;

[0035] S302, extracting region categories and marking corresponding regions in the visual model, and determining puncture target points based on the markings;

[0036] S303, dividing the three-dimensional model into discrete voxels according to the Dijkstra algorithm, and constructing an adjacency relationship between each voxel, where one voxel represents a node;

[0037] S304, obtaining and constructing the adjacency relationship between each voxel, and constructing a graph structure based on the adjacency relationship and the voxels;

[0038] S305: Obtain the dangerous area in the visual model, and construct a cost function for each node based on the graph structure and the dangerous area;

[0039] S306: Obtain the puncture starting point, and obtain a path point sequence based on the puncture starting point, the puncture target point, the cost function, and the graph structure;

[0040] S307: Optimize the path point sequence according to the path optimization algorithm to obtain the optimized optimal path.

[0041] Preferably, the step of optimizing the path point sequence according to the path optimization algorithm to obtain the optimized optimal path comprises the following steps:

[0042] S3071. Obtain the distance between each path point in the path point sequence and the dangerous area, and obtain the gradient based on the distance;

[0043] S3072. Obtaining a step length according to the specifications of the biopsy robot;

[0044] S3073, updating the path point position according to the step size and the gradient of each path point to obtain an updated new path point;

[0045] S3074. Replace the original path point in the path point sequence with the new path point to obtain a new path sequence;

[0046] S3075, obtaining an anatomical structure boundary according to the visual model, and determining a new path sequence according to the anatomical structure boundary;

[0047] S3076, looping steps S3071-S3077, and obtaining the path length and number of loops for each loop;

[0048] S3077. Extract the preset maximum number of cycles. If the path length does not reach the shortest, or the number of cycles reaches the maximum number of cycles, output the path as the optimal path.

[0049] Preferably, the step of obtaining the anatomical structure boundary based on the visual model and determining the new path sequence based on the anatomical structure boundary comprises the following steps:

[0050] S30751. Obtaining anatomical structure boundaries based on a visual model;

[0051] S30752, and judging the new path sequence based on the anatomical structure boundary;

[0052] S30753: If the new path point in the new path sequence is within the boundary of the anatomical structure, the new path point is determined to be unqualified and the original path point does not need to be replaced;

[0053] S30754: If the new path point in the new path sequence does not fall within the boundary of the anatomical structure, the new path point is determined to be qualified and replaces the original path point.

[0054] Preferably, the obtaining of optical tracking data, obtaining the needle tip position according to the optical tracking data, and correcting the puncture path according to the needle tip position comprises the following steps:

[0055] S401, obtaining optical tracking data, and obtaining the three-dimensional coordinates of the marking position according to the optical tracking data;

[0056] S402, obtaining the needle tip coordinates, and obtaining the needle tip position by the least square method based on the optical tracking data and the needle tip coordinates;

[0057] S403, obtaining a smooth trajectory of the needle tip according to the optimal path;

[0058] S404: Correct the puncture path according to the smooth trajectory.

[0059] Preferably, the correction of the puncture path according to the smooth trajectory includes the following steps:

[0060] S4041. Obtain the Euclidean distance between the smooth trajectory and the optimal path, and obtain the real-time error based on the Euclidean distance;

[0061] S4042. Extract the PID parameters of the biopsy robot, and obtain the control increment of the robotic arm based on the PID parameters and the real-time error;

[0062] S4043. Obtain the current state of the robotic arm, and obtain the joint angle increment by controlling the increment and the current state of the robotic arm according to inverse kinematics;

[0063] S4044, generating a robot arm adjustment instruction based on the joint angle increment;

[0064] S4045. Correct the puncture path according to the robot arm adjustment instruction.

[0065] Preferably, a tumor puncture biopsy robot and a real-time imaging navigation system thereof are proposed, which are used to implement the above-mentioned tumor puncture biopsy robot and real-time imaging navigation method thereof, including:

[0066] Control module: The control module is used to control the biopsy robot;

[0067] Image acquisition module: The image acquisition module is used to acquire medical images of patients;

[0068] Image processing module: The image acquisition module is used to pre-process medical images;

[0069] Image analysis module: The image analysis module is used to analyze medical images and build visual models;

[0070] Data processing module: The data processing module is used to generate a visual model and a path point sequence;

[0071] Data analysis module: The data analysis module is used to analyze the path point sequence, generate the optimal path, and correct the puncture path in real time.

[0072] Compared with the existing technology, the advantages of the present invention are: based on the three-dimensional model, the present invention uses the Dijkstra algorithm to divide it into discrete voxels, and constructs the adjacency relationship and graph structure between voxels, clearly outlining the precise boundaries of different tissues and organs, so that the complex anatomical structure areas can be bypassed when planning the puncture path, avoiding unnecessary detours and turns in the path; for dangerous areas, the system first determines the dangerous areas in the visual model, and then constructs the cost function of each node based on these dangerous areas. When planning the optimal path, the path point sequence is obtained according to the puncture starting point, target point, cost function and graph structure, and the path is continuously adjusted through the path optimization algorithm to keep the path away from important blood vessels, nerves and other key tissues, preventing the path from being too close to or directly passing through the dangerous area, thereby ensuring the rationality of the optimal path planning, enhancing the precise controllability of the puncture process, and correcting the puncture path in real time through the needle tip position, further improving the possibility of the solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 This is a flow chart of steps S100-S400 in the tumor puncture biopsy robot and real-time imaging navigation system proposed in the present invention;

[0074] Figure 2 This is a flow chart of steps S101-S108 in the tumor puncture biopsy robot and real-time imaging navigation system proposed in the present invention;

[0075] Figure 3 This is a flow chart of steps S201-S207 in the tumor puncture biopsy robot and real-time imaging navigation system proposed in the present invention;

[0076] Figure 4 This is a flow chart of steps S2011-S2014 in the tumor puncture biopsy robot and real-time imaging navigation system proposed in the present invention;

[0077] Figure 5 This is a flow chart of steps S301-S307 in the tumor puncture biopsy robot and real-time imaging navigation system proposed in the present invention;

[0078] Figure 6 This is a flow chart of steps S3071-S3077 in the tumor puncture biopsy robot and real-time imaging navigation system proposed in the present invention;

[0079] Figure 7 This is a flow chart of steps S30751-S30754 in the tumor puncture biopsy robot and real-time imaging navigation system proposed in the present invention;

[0080] Figure 8This is a flow chart of steps S401-S404 in the tumor puncture biopsy robot and real-time imaging navigation system proposed in the present invention;

[0081] Figure 9 This is a flow chart of steps S4041-S4045 in the tumor puncture biopsy robot and real-time imaging navigation system proposed in the present invention;

[0082] Figure 10 This is a structural block diagram of the tumor puncture biopsy robot and its real-time imaging navigation system proposed in the present invention. DETAILED DESCRIPTION

[0083] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0084] Reference Figure 1-10 As shown, the tumor puncture biopsy robot and its real-time imaging navigation system include:

[0085] S100, extracting historical data and building a segmentation model based on the historical data;

[0086] S200, obtaining multiple medical images of the patient, and constructing a three-dimensional model of the patient based on the multiple medical images;

[0087] S300, constructing a visualized optimal path based on the three-dimensional model;

[0088] S400, acquiring optical tracking data, obtaining the needle tip position based on the optical tracking data, and correcting the puncture path based on the needle tip position;

[0089] It is understandable to those skilled in the art that historical data contains a large number of labeled medical images, which record the characteristics of lesion areas such as tumors in different patients. By building a segmentation model, the model can learn the differences in image characteristics between lesion areas and normal tissues, so that when processing new images of patients in the future, the lesion area can be accurately located and identified. A single medical image can only provide information at a certain level in the patient's body, and it is difficult to fully display the spatial relationship between the lesion area and the surrounding tissues and organs. By acquiring multiple medical images and building a three-dimensional model, the anatomical structure in the patient's body can be presented in a three-dimensional form, allowing doctors to more intuitively understand the location, morphology and relationship between the lesion area and the surrounding important structures. The optical tracking system can obtain the position information of the puncture needle in space in real time and obtain the precise position of the needle tip through calculation. During the puncture process, the position of the puncture needle may be offset due to factors such as the patient's respiratory movement and muscle contraction. By obtaining the needle tip position and timely correcting the puncture path, these offsets can be corrected so that the puncture needle always accurately points to the lesion area, thereby improving the accuracy of the puncture.

[0090] like Figure 2 As shown, extracting historical data and building a segmentation model based on the historical data includes the following steps:

[0091] S101, extracting multiple images from historical data and obtaining their corresponding labels;

[0092] S102, dividing the multiple images into a training set and a validation set;

[0093] S103, inputting the image in the training set into the segmentation model, and obtaining the prediction result through the forward propagation calculation of the segmentation model, wherein the segmentation model is based on U-Net;

[0094] S104. Calculate the loss function value based on the prediction result and the corresponding label;

[0095] S105, performing back propagation algorithm calculation on the loss function value to obtain the gradient, and updating the parameters of the segmentation model;

[0096] S106, repeat the above forward propagation, loss calculation and back propagation steps until the segmentation model converges or reaches a preset number of iterations;

[0097] S107, verifying the segmentation model after iteration based on the validation set;

[0098] S108. Based on the performance of the validation set, the parameters of the segmentation model are adjusted twice using regularization technology;

[0099] It will be understood by those skilled in the art that image labels are annotations of key information in images. For example, in tumor-related medical images, labels may clearly mark the location, boundaries and other information of the tumor. By obtaining these labeled images, the segmentation model can clearly know which areas need to be focused on and segmented during the learning process. The training set is used for the training process of the model. The model adjusts its own parameters by continuously learning the image features and label relationships in the training set to predict the target area as accurately as possible. The validation set is independent of the training set and is used to evaluate the performance of the model during the model training process. U-Net is a deep learning architecture specially designed for medical image segmentation tasks. It has an encoder-decoder structure. The encoder part gradually extracts high-level features of the image through convolutional layers and pooling layers, and can capture local and global information in the image; the decoder part gradually restores the extracted features to the original image size through upsampling and convolution operations, and generates segmentation results. The forward propagation process is to gradually transform the input image into the predicted segmentation result through a series of calculation operations of the model. The loss function is used to measure the degree of difference between the model prediction result and the true label. By calculating the loss function value, you can intuitively understand the degree of fit of the model to the training data under the current parameter settings. The backpropagation algorithm calculates the gradient of the loss function value to the model parameters to determine the direction and amplitude of the parameter adjustment. Repeating these steps can give the model enough opportunities to learn the data features in the training set multiple times. In each iteration, the model will predict the training data according to the current parameter settings, calculate the loss function value, and update the parameters. Through multiple iterations, the model can gradually capture the complex patterns and relationships in the data, improve the ability to fit the training data, and thus better The model learns the mapping relationship between image features and target areas. The data of the validation set is not seen by the model during the training process. By verifying the model on the validation set, the generalization ability of the model in practical applications can be objectively evaluated. If the model can also achieve good segmentation effect on the validation set, it means that the model not only performs well on the training data, but also can adapt well to new data and has good generalization performance. On the contrary, if the performance of the model on the validation set is poor, it may indicate that the model has problems such as overfitting and needs further adjustment and optimization. When the performance of the model on the validation set decreases, it may indicate that the model has overfitted to the training data. At this time, the regularization technology can be used to constrain the parameters of the model, limit the size of the parameters, and avoid the model being too complex.

[0100] like Figure 3As shown, obtaining multiple medical images of the patient and constructing a three-dimensional model of the patient based on the multiple medical images includes the following steps:

[0101] S201, preprocessing each medical image to obtain a preprocessed image;

[0102] S202, inputting the pre-processed image into the segmentation model to obtain the category of each region in the pre-processed image;

[0103] S203, performing SIFT feature extraction on the multiple pre-processed images to obtain key point sets and feature descriptors of the pre-processed images;

[0104] S204, obtaining the Euclidean distance between the feature descriptors in two adjacent pre-processed images, and obtaining the matching key point pairs in the two pre-processed images;

[0105] S205, obtaining two pre-processed image transformation parameters using a random sampling consensus algorithm based on the key point pairs;

[0106] S206, obtaining a registered paired image based on the pre-processed image and the corresponding transformation parameters;

[0107] S207, generating a three-dimensional model using surface rendering technology based on the paired images;

[0108] It is understood by those skilled in the art that medical images may be interfered with by various factors during the acquisition process, such as noise, artifacts, insufficient contrast, etc. Preprocessing operations (such as denoising, contrast enhancement, normalization, etc.) can effectively improve image quality, reduce noise interference, and make the tissue structure and lesion area in the image clearer and more identifiable. The segmentation model can divide the image into different regions based on the features in the preprocessed image and assign corresponding category labels to each region, such as tumor area, normal tissue area, blood vessel area, etc. The SIFT (Scale Invariant Feature Transform) algorithm can extract feature points (key points) that remain stable under conditions such as image rotation, scaling, translation, and brightness changes, and their corresponding feature descriptors. These key points are usually locations with significant features in the image, such as corner points and edge points, while feature descriptors are quantitative representations of the features of the local area around the key points, and have high discrimination. By calculating the Euclidean distance between the feature descriptors in two adjacent preprocessed images, the similarity of the key point features in different images can be quantified. When the Euclidean distance between two feature descriptors is small, it means that the local features they describe are relatively similar, and thus it can be considered that the two key points are similar. Key points are matched. During the feature matching process, some matched key point pairs may be incorrect due to noise, occlusion or other factors, i.e., abnormal matching points. The Random Sample Consensus Algorithm (RANSAC) can randomly select a portion of point pairs from all matched key point pairs to estimate the transformation parameters between the images, and then verify the other point pairs based on these parameters, and count the number of point pairs that conform to the transformation model (inliers). The registered paired images are the result of spatially aligning multiple pre-processed images according to the calculated transformation parameters. By aligning different images to the same coordinate system, these images are made consistent in spatial position, providing a unified spatial reference framework for subsequent 3D model construction. Comprehensive rendering technology can convert the registered paired images into an intuitive 3D model, displaying the anatomical structure and lesion area in the patient's body in a three-dimensional form. Doctors can observe the 3D model from different angles through operations such as rotation, scaling, and sectioning, and have a more comprehensive understanding of the location, morphology, size, and relationship of the lesion with surrounding tissues.

[0109] like Figure 4 As shown, preprocessing each medical image to obtain a preprocessed image includes the following steps:

[0110] S2011, performing Gaussian filtering on each medical image to obtain a filtered, denoised image;

[0111] S2012, performing sharpening processing on each noise reduction image to obtain a sharpened enhanced image;

[0112] S2013, normalizing the pixel values ​​of each pixel in the enhanced image to obtain multiple normalized images;

[0113] S2014, adjusting each normalized image to the same resolution to obtain an adjusted pre-processed image;

[0114] Those skilled in the art will understand that Gaussian filtering is a classic linear smoothing filtering method. It performs a weighted average of each pixel in the image and its neighboring pixels by performing a convolution operation with a Gaussian kernel. Since the Gaussian function has smoothing properties in the spatial domain, it can gradually reduce the influence of neighboring pixels away from the central pixel on the central pixel, thereby effectively suppressing high-frequency noise in the image and making the image clearer and smoother. In medical images, the boundaries between lesions and normal tissues are often subtle. Sharpening can enhance the contrast of these boundaries, allowing doctors to more accurately identify the location and extent of lesions. Normalization can eliminate dimensional differences in pixel values, making the values ​​between different pixels comparable. At the same time, the normalized pixel value distribution is more concentrated, which helps to accelerate the convergence of the model. In the process of medical image analysis and processing, multiple images usually need to be processed in batches. If the resolution of these images is inconsistent, problems will arise in subsequent steps such as image registration, segmentation, and feature extraction. Adjusting the normalized images to the same resolution can ensure that all images have the same size, allowing the algorithm to process them in a unified manner, improving processing efficiency and accuracy.

[0115] like Figure 5 As shown in the figure, the optimal path for building visualization based on the 3D model includes the following steps:

[0116] S301, rendering the three-dimensional model according to OpenGL, and obtaining the rendered visual model;

[0117] S302, extracting region categories and marking corresponding regions in the visual model, and determining puncture target points based on the markings;

[0118] S303, dividing the three-dimensional model into discrete voxels according to the Dijkstra algorithm, and constructing an adjacency relationship between each voxel, where one voxel represents a node;

[0119] S304, obtaining and constructing the adjacency relationship between each voxel, and constructing a graph structure based on the adjacency relationship and the voxels;

[0120] S305: Obtain the dangerous area in the visual model, and construct a cost function for each node based on the graph structure and the dangerous area;

[0121] S306: Obtain the puncture starting point, and obtain a path point sequence based on the puncture starting point, the puncture target point, the cost function, and the graph structure;

[0122] S307, optimizing the path point sequence according to the path optimization algorithm to obtain the optimized optimal path;

[0123] It is understood by those skilled in the art that OpenGL, as a powerful graphics rendering library, can transform abstract 3D model data into vivid and intuitive visual images. Doctors can observe the internal structure of the human body from different angles and perspectives through the rendered visual model, and clearly see the location, shape, size and relationship of the lesion with the surrounding tissues. By extracting the region category, doctors can accurately identify the lesion area, important organs, blood vessels and other key structures. By dividing the 3D model into discrete voxels, the continuous 3D space can be transformed into a discrete node set, thereby simplifying the complexity of the path planning problem. The adjacency relationship clarifies the connection between nodes, that is, which nodes can be directly reached. In 3D space, a voxel usually has an adjacency relationship with six or twenty-six adjacent voxels ( The specific details depend on the voxel division method and the definition of adjacency). By integrating the adjacency relationship and voxel information into the graph structure, the connectivity and relative position relationship between each part in the three-dimensional space can be clearly represented. By constructing a cost function for each node, the dangerous area information in the visual model can be quantified into the graph structure. According to the graph structure and cost function, a path search algorithm (such as the Dijkstra algorithm) can be used to start from the puncture starting point and search for a path to the puncture target point in the graph structure to obtain a path point sequence. The initial path point sequence may have certain errors or inaccuracies and cannot fully meet the clinical requirements for puncture accuracy. The path optimization algorithm can fine-tune the path point sequence and make the optimal path more accurate and stable by optimizing parameters such as the shape, length, and smoothness of the path.

[0124] like Figure 6 As shown, the path point sequence is optimized according to the path optimization algorithm, and obtaining the optimized optimal path includes the following steps:

[0125] S3071. Obtain the distance between each path point in the path point sequence and the dangerous area, and obtain the gradient based on the distance;

[0126] S3072. Obtaining a step length according to the specifications of the biopsy robot;

[0127] S3073, updating the path point position according to the step size and the gradient of each path point to obtain an updated new path point;

[0128] S3074. Replace the original path point in the path point sequence with the new path point to obtain a new path sequence;

[0129] S3075, obtaining an anatomical structure boundary according to the visual model, and determining a new path sequence according to the anatomical structure boundary;

[0130] S3076, looping steps S3071-S3077, and obtaining the path length and number of loops for each loop;

[0131] S3077, extracting a preset maximum number of cycles, and if the path length does not reach the minimum, or the number of cycles reaches the maximum number of cycles, outputting the path as the optimal path;

[0132] Those skilled in the art will appreciate that by calculating the distance between each path point in the path point sequence and the danger zone, the distance of each path point from the danger zone can be quantified, and a gradient can be obtained based on the distance between the path point and the danger zone. The gradient can be understood as the trend and rate at which the path point changes in space toward a safer direction. The difference in distance between different path points and the danger zone will result in different gradient values. This difference enables the path adjustment to be differentiated according to the degree of risk. The step length is obtained based on the specifications of the biopsy robot, which can ensure that the path point position update amount (i.e., step length) generated during the path optimization process is consistent with the actual operational capabilities of the robot. By combining the step length and the gradient of each path point to update the path point position, the algorithm can gradually adjust the position of the path point to move it away from the danger zone. Each position update is an optimization of the path safety. After multiple iterative updates, the optimal path will gradually become safer, reducing the puncture process. In order to reduce the risk of damage to dangerous areas during the process, the algorithm can record the results of each path optimization by continuously replacing the original path points with the updated new path points, and gradually construct a more optimized optimal path. The anatomical structure boundary is obtained through the visual model and used as the basis for judging the new path sequence, which can ensure that the optimized optimal path is reasonable in terms of anatomical structure. In each cycle, the algorithm will update the path point position according to information such as the distance between the path point and the dangerous area, the gradient, and the specifications of the biopsy robot, generate a new path sequence, and make judgments based on the anatomical structure boundary, extract the preset maximum number of cycles, and set the output path as the optimal path when the number of cycles reaches the maximum number of cycles. This can ensure that the algorithm converges within a limited time. Even if the path length has not reached the shortest at the maximum number of cycles, the algorithm will stop running and output the current path, avoiding the problem of the algorithm running too long due to the pursuit of the theoretical shortest path, thereby improving the efficiency of the algorithm.

[0133] like Figure 7 As shown, obtaining the anatomical structure boundary according to the visual model and determining the new path sequence according to the anatomical structure boundary include the following steps:

[0134] S30751. Obtaining anatomical structure boundaries based on a visual model;

[0135] S30752, and judging the new path sequence based on the anatomical structure boundary;

[0136] S30753: If the new path point in the new path sequence is within the boundary of the anatomical structure, the new path point is determined to be unqualified and the original path point does not need to be replaced;

[0137] S30754: If the new path point in the new path sequence does not involve an anatomical structure boundary, the new path point is determined to be qualified and replaces the original path point;

[0138] Those skilled in the art will appreciate that obtaining anatomical boundaries based on a visual model provides an objective and accurate basis for subsequent judgment of new path sequences. Compared to traditional empirical judgment or two-dimensional images, the anatomical boundaries provided by the visual model are more intuitive and comprehensive, and can avoid judgment errors caused by subjective factors or incomplete information. By judging the new path sequence based on anatomical boundaries, path points that may enter dangerous anatomical areas (such as important blood vessels, nerves, organ parenchyma, etc.) can be promptly identified. When a new path point involves an anatomical boundary, it means that the path point may cause the puncture needle to enter a critical anatomical structure, thereby posing a serious safety risk. Determining the new path point as unqualified and eliminating the need to replace the original path point can promptly prevent the use of such a dangerous path and avoid unnecessary harm to the patient during the actual puncture operation. When a new path point does not involve an anatomical boundary, it indicates that the path point is anatomically safe and is determined to be qualified and replace the original path point. By continuously replacing the original path point with qualified new path points, the optimal path gradually moves away from dangerous anatomical areas, and safety is gradually improved.

[0139] like Figure 8 As shown, obtaining optical tracking data, obtaining the needle tip position based on the optical tracking data, and correcting the optimal path based on the needle tip position include the following steps:

[0140] S401, obtaining optical tracking data, and obtaining the three-dimensional coordinates of the marking position according to the optical tracking data;

[0141] S402, obtaining the needle tip coordinates, and obtaining the needle tip position by the least square method based on the optical tracking data and the needle tip coordinates;

[0142] S403, obtaining a smooth trajectory of the needle tip according to the optimal path;

[0143] S404, correcting the puncture path according to the smooth trajectory;

[0144] It will be understood by those skilled in the art that by obtaining the coordinates of the needle tip (which may be preliminary coordinates obtained through other auxiliary measurement methods) and combining them with optical tracking data, and performing fitting calculations using the least squares method, the actual position of the needle tip can be determined more accurately. The optical tracking data may be affected by factors such as the position of the marker point and occlusion, resulting in inaccurate measurement of the overall position of the puncture tool; and the directly obtained needle tip coordinates may have certain measurement errors or be incomplete. By obtaining the smooth trajectory of the needle tip based on the optimal path, the movement of the needle tip can be planned and controlled so that it moves along a predetermined smooth path during the puncture process. During the puncture process, due to the influence of various factors (such as patient breathing, changes in tissue elasticity, etc.), the actual puncture path may deviate from the pre-planned path. By correcting the optimal path based on the smooth trajectory, the movement of the needle tip can be monitored in real time. When it is found that the needle tip trajectory deviates from the smooth trajectory, the puncture direction is adjusted in time to return the needle tip to the correct path.

[0145] like Figure 9 As shown, the correction of the optimal path according to the smooth trajectory includes the following steps:

[0146] S4041. Obtain the Euclidean distance between the smooth trajectory and the optimal path, and obtain the real-time error based on the Euclidean distance;

[0147] S4042. Extract the PID parameters of the biopsy robot, and obtain the control increment of the robotic arm based on the PID parameters and the real-time error;

[0148] S4043. Obtain the current state of the robotic arm, and obtain the joint angle increment by controlling the increment and the current state of the robotic arm according to inverse kinematics;

[0149] S4044, generating a robot arm adjustment instruction based on the joint angle increment;

[0150] S4045, correcting the puncture path according to the robot arm adjustment instruction;

[0151] It will be understood by those skilled in the art that, in a biopsy robot-assisted puncture operation, a smooth trajectory represents an ideal puncture path, while the actual puncture path may be offset due to various factors (such as the patient's physiological movement, robot system errors, etc.). By obtaining the Euclidean distance between the two, the degree of this offset can be quantified into a specific numerical value. In the biopsy robot system, extracting the PID parameters and obtaining the control increment of the robotic arm based on the PID parameters and the real-time error can achieve precise closed-loop control of the robotic arm movement. The state of the robotic arm includes its current position, posture, speed and other information. Obtaining the current state of the robotic arm is the basis for precise control. Inverse kinematics is a key technology for converting the position and posture of the end effector of the robotic arm (such as a puncture needle) to the angles of each joint. By combining the control increment with the current robotic arm state based on inverse kinematics to obtain the joint angle increment, the angle at which each joint of the robotic arm needs to rotate in order to achieve the desired movement (i.e., correct the puncture path) can be accurately calculated. Based on the joint angle increment, the robotic arm adjustment instructions are generated, and the abstract control strategy can be converted into instructions that the robotic arm can understand and execute. The puncture path is corrected according to the robotic arm adjustment instructions, and deviations in the puncture process can be corrected in time to prevent the puncture needle from accidentally injuring important blood vessels, nerves or organs around it. By monitoring and correcting the puncture path in real time, the risk of the puncture operation can be greatly reduced, ensuring the patient's life safety.

[0152] like Figure 10 As shown, a tumor puncture biopsy robot and its real-time imaging navigation system are proposed, which are used to implement the above-mentioned tumor puncture biopsy robot and its real-time imaging navigation method, including:

[0153] Control module: The control module is used to control the biopsy robot;

[0154] Image acquisition module: The image acquisition module is used to acquire medical images of patients;

[0155] Image processing module: The image acquisition module is used to pre-process medical images;

[0156] Image analysis module: The image analysis module is used to analyze medical images and build visual models;

[0157] Data processing module: The data processing module is used to generate a visual model and a path point sequence;

[0158] Data analysis module: The data analysis module is used to analyze the path point sequence, generate the optimal path, and correct the puncture path in real time.

[0159] In summary, the advantages of the present invention are: based on the three-dimensional model, the Dijkstra algorithm is used to divide it into discrete voxels, and the adjacency relationship and graph structure between voxels are constructed to clearly outline the precise boundaries of different tissues and organs, so that the complex anatomical structure areas can be bypassed when planning the puncture path, avoiding unnecessary detours and turns in the path; for dangerous areas, the system first determines the dangerous areas in the visual model, and then constructs the cost function of each node based on these dangerous areas. When planning the optimal path, the path point sequence is obtained according to the puncture starting point, target point, cost function and graph structure, and the path is continuously adjusted through the path optimization algorithm to keep the path away from important blood vessels, nerves and other key tissues, preventing the path from being too close to or directly passing through the dangerous area, thereby ensuring the rationality of the optimal path planning, enhancing the precise controllability of the puncture process, and correcting the puncture path in real time through the needle tip position to further improve the possibility of the solution.

[0160] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. Tumor puncture biopsy robot and real-time imaging navigation system, characterized by: include: S100, extracting historical data and building a segmentation model based on the historical data; S200, obtaining multiple medical images of the patient, and constructing a three-dimensional model of the patient based on the multiple medical images; S300, constructing a visualized optimal path based on the three-dimensional model; S400 , obtaining optical tracking data, obtaining the needle tip position based on the optical tracking data, and correcting the puncture path based on the needle tip position.

2. The tumor puncture biopsy robot and real-time imaging navigation system according to claim 1, characterized in that: The extraction of historical data and the construction of a segmentation model based on the historical data include the following steps: S101, extracting multiple images from historical data and obtaining their corresponding labels; S102, dividing the multiple images into a training set and a validation set; S103, inputting the image in the training set into the segmentation model, and obtaining the prediction result through the forward propagation calculation of the segmentation model, wherein the segmentation model is based on U-Net; S104. Calculate the loss function value based on the prediction result and the corresponding label; S105, performing a back propagation algorithm on the loss function value to obtain a gradient, and using the gradient to update the parameters of the segmentation model; S106, repeat the above forward propagation, loss calculation and back propagation steps until the segmentation model converges or reaches a preset number of iterations; S107, verifying the segmentation model after iteration based on the validation set; S108. Based on the performance of the validation set, regularization technology is used to perform secondary adjustments on the parameters of the segmentation model.

3. The tumor puncture biopsy robot and real-time imaging navigation system according to claim 1, characterized in that: Acquiring multiple medical images of the patient and constructing a three-dimensional model of the patient based on the multiple medical images includes the following steps: S201, preprocessing each medical image to obtain a preprocessed image; S202, inputting the pre-processed image into the segmentation model to obtain the category of each region in the pre-processed image; S203, performing SIFT feature extraction on the multiple pre-processed images to obtain key point sets and feature descriptors of the pre-processed images; S204, obtaining the Euclidean distance between the feature descriptors in two adjacent pre-processed images, and obtaining the matching key point pairs in the two pre-processed images; S205, obtaining two pre-processed image transformation parameters using a random sampling consensus algorithm based on the key point pairs; S206, obtaining a registered paired image based on the pre-processed image and the corresponding transformation parameters; S207: Generate a three-dimensional model based on the paired images using surface rendering technology.

4. The tumor puncture biopsy robot and real-time imaging navigation system according to claim 3, characterized in that: Preprocessing each medical image to obtain a preprocessed image includes the following steps: S2011, performing Gaussian filtering on each medical image to obtain a filtered, denoised image; S2012, performing sharpening processing on each noise reduction image to obtain a sharpened enhanced image; S2013, normalizing the pixel values ​​of each pixel in the enhanced image to obtain multiple normalized images; S2014. Adjust each normalized image to the same resolution to obtain an adjusted pre-processed image.

5. The tumor puncture biopsy robot and real-time imaging navigation system according to claim 1, characterized in that: The method of constructing a visualized optimal path based on the three-dimensional model includes the following steps: S301, rendering the three-dimensional model according to OpenGL, and obtaining the rendered visual model; S302, extracting region categories and marking corresponding regions in the visual model, and determining puncture target points based on the markings; S303, dividing the three-dimensional model into discrete voxels according to the Dijkstra algorithm, and constructing an adjacency relationship between each voxel, where one voxel represents a node; S304, obtaining and constructing the adjacency relationship between each voxel, and constructing a graph structure based on the adjacency relationship and the voxels; S305: Obtain the dangerous area in the visual model, and construct a cost function for each node based on the graph structure and the dangerous area; S306: Obtain the puncture starting point, and obtain a path point sequence based on the puncture starting point, the puncture target point, the cost function, and the graph structure; S307: Optimize the path point sequence according to the path optimization algorithm to obtain the optimized optimal path.

6. The tumor puncture biopsy robot and real-time imaging navigation system according to claim 5, characterized in that: Optimizing the path point sequence according to the path optimization algorithm to obtain the optimized optimal path includes the following steps: S3071. Obtain the distance between each path point in the path point sequence and the dangerous area, and obtain the gradient based on the distance; S3072. Obtaining a step length according to the specifications of the biopsy robot; S3073, updating the path point position according to the step size and the gradient of each path point to obtain an updated new path point; S3074. Replace the original path point in the path point sequence with the new path point to obtain a new path sequence; S3075, obtaining an anatomical structure boundary according to the visual model, and determining a new path sequence according to the anatomical structure boundary; S3076, looping steps S3071-S3077, and obtaining the path length and number of loops for each loop; S3077. Extract the preset maximum number of cycles. If the path length does not reach the shortest, or the number of cycles reaches the maximum number of cycles, output the path as the optimal path.

7. The tumor puncture biopsy robot and real-time imaging navigation system according to claim 6, characterized in that: The step of obtaining the anatomical structure boundary according to the visual model and determining the new path sequence according to the anatomical structure boundary includes the following steps: S30751. Obtaining anatomical structure boundaries based on a visual model; S30752, and judging the new path sequence based on the anatomical structure boundary; S30753: If the new path point in the new path sequence is within the boundary of the anatomical structure, the new path point is determined to be unqualified and the original path point does not need to be replaced; S30754: If the new path point in the new path sequence does not involve the anatomical structure boundary, the new path point is determined to be qualified and replaces the original path point.

8. The tumor puncture biopsy robot and real-time imaging navigation system according to claim 1, characterized in that: The obtaining of optical tracking data, obtaining the needle tip position according to the optical tracking data, and correcting the optimal path according to the needle tip position include the following steps: S401, obtaining optical tracking data, and obtaining the three-dimensional coordinates of the marking position according to the optical tracking data; S402, obtaining the needle tip coordinates, and obtaining the needle tip position by the least square method based on the optical tracking data and the needle tip coordinates; S403, obtaining a smooth trajectory of the needle tip according to the optimal path; S404: Correct the puncture path according to the smooth trajectory.

9. The tumor puncture biopsy robot and real-time imaging navigation system according to claim 8, characterized in that: The correction of the puncture path according to the smooth trajectory comprises the following steps: S4041. Obtain the Euclidean distance between the smooth trajectory and the optimal path, and obtain the real-time error based on the Euclidean distance; S4042. Extract the PID parameters of the biopsy robot, and obtain the control increment of the robotic arm based on the PID parameters and the real-time error; S4043. Obtain the current state of the robotic arm, and obtain the joint angle increment by controlling the increment and the current state of the robotic arm according to inverse kinematics; S4044, generating a robot arm adjustment instruction based on the joint angle increment; S4045. Correct the puncture path according to the robot arm adjustment instruction.

10. A tumor puncture biopsy robot and a real-time imaging navigation system thereof, for implementing the tumor puncture biopsy robot and a real-time imaging navigation method thereof as described in claims 1-9, characterized in that: include: Control module: The control module is used to control the biopsy robot; Image acquisition module: The image acquisition module is used to acquire medical images of patients; Image processing module: The image acquisition module is used to pre-process medical images; Image analysis module: The image analysis module is used to analyze medical images and build visual models; Data processing module: The data processing module is used to generate a visual model and a path point sequence; Data analysis module: The data analysis module is used to analyze the path point sequence, generate the optimal path, and correct the puncture path in real time.

Images