Motion track planning system and method for interventional puncture operation

Abstract

The invention relates to a motion trail planning system and method for interventional puncture surgery, and the system comprises a camera module which is used for collecting the motion trail of an instrument in an external environment and generating a first image; the image equipment is used for acquiring the movement track of the instrument in the internal environment and generating a second image; the preprocessing module is used for generating an external three-dimensional coordinate and an internal three-dimensional coordinate based on the first image and the second image, and forming a data pair of the movement track of the instrument based on the external three-dimensional coordinate and the internal three-dimensional coordinate; the control module comprises a machine learning unit and a trajectory planning unit; the machine learning unit is used for training a deep learning model based on the data of the instrument movement track and generating a prediction path based on the trained deep learning model; and the trajectory planning unit is used for optimizing and adjusting the predicted path. According to the method, the movement track of the interventional puncture operation of the specific target part can be accurately predicted and planned.

Description

Technical Field

[0001] The present invention relates to the technical field of computer-assisted medical treatment, and in particular to a motion trajectory planning system and method for interventional puncture surgery. Background Art

[0002] Interventional devices are an important type of medical device in the field of interventional medicine. They are widely used, covering a range of operations from biopsy to treatment. These devices can accurately reach the target tissue under the guidance of images, helping doctors to perform minimally invasive diagnosis and treatment, effectively improving the safety and effectiveness of treatment.

[0003] When performing percutaneous puncture, doctors need to consider a variety of factors, including the location of the lesion or organ, the patient's body position, and physiological conditions such as breathing. These factors prompt doctors to choose interventional instruments with different parameters during the operation. For example, some interventional instruments are rigidly designed. Such straight instruments can only operate along a straight path, so they are limited in the positioning and freedom of the target point. In addition, the interaction between the interventional instrument and the tissue organ, such as tissue deformation and slight bending of the needle under force, may cause the needle tip to deviate from the predetermined target position, thereby reducing the accuracy of the puncture.

[0004] In actual operation, if the doctor makes a deviation, he can only manually adjust or remove the interventional device, and then re-puncture to correct the deviation. However, such adjustments may cause secondary damage to the patient, increase his pain and prolong the postoperative recovery time. For example, during heart or lung cancer surgery, the dynamic changes in the patient's heartbeat or breathing make the organ position unstable; secondly, due to the large amount of gas in the lungs, the actual puncture position and angle of the interventional device when entering the lungs may be significantly different from the doctor's ideal preset.

[0005] Therefore, monitoring the motion trajectory of interventional instruments, as well as puncture modeling and path planning based on different target areas, have become important problems that need to be urgently solved in the field of minimally invasive surgery. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a system and method for motion trajectory planning for interventional puncture surgery, which can accurately predict and plan the motion trajectory of interventional puncture surgery at a specific target site to improve the safety and effectiveness of the interventional surgery.

[0007] The technical solution adopted by the present invention to solve the technical problem is: to provide a motion trajectory planning system for interventional puncture surgery, comprising:

[0008] A camera module, used for collecting a motion trajectory of the device in an external environment based on a first feature point of the device and generating a first image;

[0009] An imaging device, used to obtain a motion trajectory of the device in the internal environment based on a second feature point of the device and generate a second image;

[0010] a preprocessing module, configured to generate external three-dimensional coordinates and internal three-dimensional coordinates based on the first image and the second image, and to form a data pair of a movement trajectory of an instrument based on the external three-dimensional coordinates and the internal three-dimensional coordinates;

[0011] The control module includes a machine learning unit and a trajectory planning unit; the machine learning unit is used to train a deep learning model based on the data of the instrument's motion trajectory, and generate a predicted path based on the trained deep learning model; the trajectory planning unit is used to optimize and adjust the predicted path.

[0012] The camera module is a multi-eye camera, and the multi-eye camera adopts a non-parallel multi-eye vision system.

[0013] The multi-eye camera is fixed on an electrically driven arc guide rail, and the position and angle of the multi-eye camera on the arc guide rail are adjusted via the control module.

[0014] The preprocessing module comprises:

[0015] An image processing unit, used for performing edge detection, feature point extraction and image segmentation on the first image and the second image;

[0016] A coordinate conversion unit, configured to convert the coordinates of the processed first image into external three-dimensional coordinates based on the intrinsic parameters and extrinsic parameters of the camera module;

[0017] A coordinate reconstruction unit, configured to perform three-dimensional reconstruction based on the volume data provided by the imaging device and feature points extracted from the processed second image to generate internal three-dimensional coordinates;

[0018] The registration unit is used to register the external three-dimensional coordinates with the internal three-dimensional coordinates to form a data pair of the movement trajectory of the instrument.

[0019] The motion trajectory planning system for interventional puncture surgery also includes: a light source module, which is used to provide a light source, and the light source includes at least one of a ring light source, a point light source, and a surface light source; the light source module dynamically adjusts the angle, brightness and / or position of the provided light source according to the position of the camera module.

[0020] The machine learning unit comprises:

[0021] A collecting subunit, used to collect data pairs of information related to the interventional instrument, information related to the surgical object, and movement trajectory of the instrument;

[0022] A coding fusion subunit, used for uniformly coding the information related to the interventional instrument and the information related to the surgical object, and combining the data pairs of the instrument motion trajectory to form a training data set;

[0023] A training subunit, used to train the deep learning model using a training data set to obtain a trained deep learning model;

[0024] The generation subunit is used to input the relevant information of the interventional instrument to be punctured and the relevant information of the surgical object into the trained deep learning model to obtain a predicted path.

[0025] The trajectory planning unit comprises:

[0026] A graph search subunit, for finding a plan for the predicted path in three-dimensional space;

[0027] A path optimization subunit, used for optimizing the predicted path under the constraints of the operation constraints of the surgical instrument, the tissue structure of the target site and the safety of the surgery;

[0028] The real-time adjustment subunit is used to compare the actual motion trajectory of the surgical instrument with the optimized predicted path, and adjust the optimized predicted path when there is a deviation between the two.

[0029] The technical solution adopted by the present invention to solve the technical problem is: to provide a motion trajectory planning method for interventional puncture surgery, using the above-mentioned interventional puncture surgery motion trajectory planning system, specifically including the following steps:

[0030] A first image captured from the camera module and a second image generated from the imaging device;

[0031] Performing image registration based on the first image and the second image to generate three-dimensional coordinate information and form a data pair of the movement trajectory of the instrument;

[0032] The data pairs of the instrument motion trajectory, the relevant information of the interventional instrument and the relevant information of the surgical object are used as the input data of the deep learning model to train the deep learning model. The generated training results include the prediction of the motion trajectory of the instrument on different surgical objects, and the results are stored in the storage module;

[0033] The specific location information of the target part and the information of the instrument to be used are input, and based on the training results in the storage module, the motion trajectory path of the interventional puncture surgery for the specific target part is generated.

[0034] The technical solution adopted by the present invention to solve the technical problem is: providing a computer storage medium storing at least one program for enabling a computer to execute a method for planning a motion trajectory for interventional puncture surgery, the method comprising:

[0035] A first image captured from the camera module and a second image generated from the imaging device;

[0036] Performing image registration based on the first image and the second image to generate three-dimensional coordinate information and form a data pair of the movement trajectory of the instrument;

[0037] The data pairs of the instrument motion trajectory, the relevant information of the interventional instrument and the relevant information of the surgical object are used as the input data of the deep learning model to train the deep learning model. The generated training results include the prediction of the motion trajectory of the instrument on different surgical objects, and the results are stored in the storage module;

[0038] The specific location information of the target part and the information of the instrument to be used are input, and based on the training results in the storage module, the motion trajectory path of the interventional puncture surgery for the specific target part is generated.

[0039] Beneficial Effects

[0040] Due to the adoption of the above-mentioned technical scheme, the present invention has the following advantages and positive effects compared with the prior art: the present invention takes into account the dynamic changes of the environment, the motion characteristics of the interventional instrument and the real-time feedback of the patient's physiological state. This comprehensiveness enables the system to generate more accurate and personalized motion trajectory planning, thereby minimizing errors and risks during the puncture process. At the same time, the present invention uses machine learning technology to identify motion patterns in different situations, and combines the imaging information of the interventional puncture surgery and the patient's physiological indicators. It can quickly adapt to changes in different clinical scenarios, thereby realizing dynamic adjustment of the target position and optimizing path planning, which significantly improves the flexibility and adaptability of surgical operations, and improves the safety and effectiveness of interventional surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram showing the structure of a system for motion trajectory planning for interventional puncture surgery according to a first embodiment of the present invention;

[0042] Figure 2 A schematic diagram showing the structure of a control module in a first embodiment of the present invention is shown;

[0043] Figure 3 A schematic diagram showing the arrangement of a multi-camera and a light source in the first embodiment of the present invention is shown;

[0044] Figure 4 A schematic diagram showing the arrangement of a multi-camera and a light source in a first embodiment of the present invention is shown;

[0045] Figure 5 Another schematic diagram of the arrangement of a multi-camera and a light source in the first embodiment of the present invention is shown;

[0046] Figure 6 A schematic diagram showing the structure of a pre-processing module in a first embodiment of the present invention is shown;

[0047] Figure 7 A schematic flow chart showing a system planning method according to a second embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following detailed description is made with reference to the accompanying drawings, and the following detailed description is provided to assist in a comprehensive understanding of the various exemplary embodiments of the present disclosure. The following description includes various details to assist in understanding, but these details are considered to be examples only and not to limit the present disclosure, which is defined by the appended claims and their equivalents. The words and phrases used in the following description are only used to enable a clear and consistent understanding of the present disclosure. In addition, for the sake of clarity and brevity, descriptions of well-known structures, functions, and configurations may be omitted. Those of ordinary skill in the art will recognize that various changes and modifications may be made to the examples described herein without departing from the spirit and scope of the present disclosure.

[0049] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0050] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0051] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0052] The technical solution of the embodiment of the present invention is further described below based on the accompanying drawings.

[0053] The first embodiment of the present invention provides a system 1 for planning the motion trajectory of interventional puncture surgery, such as Figure 1 and Figure 2As shown. The system 1 is composed of a camera module 10, an imaging device 11, a pre-processing module 12 and a control module 13, and each module is integrated into a unified system architecture through data connection and control connection. Specifically, the camera module 10 and the imaging device 11 communicate with the pre-processing module 12 through a control interface, and transmit the captured first image and second image in real time. Data flows from the camera module 10 and the imaging device 11 to the pre-processing module 12, in which image processing and coordinate conversion are performed to generate data pairs for subsequent processing. Figure 2 Inside the control module 13, the machine learning unit 130 is responsible for receiving the data pairs transmitted from the preprocessing module 12 and is trained by applying advanced algorithms. The machine learning unit 130 is intended to predict the path. During the training process, the model parameters and the corresponding data pairs will be stored in the storage unit of the control module 13 for subsequent calls and optimizations. Based on the trained model and the parameter information pre-entered in the system, the trajectory planning unit 131 optimizes and adjusts the predicted path. Finally, the planning results are transmitted to the medical operator or other related systems through the output interface to achieve efficient surgical preparation and execution.

[0054] It should be noted that the "first image" and the "second image" generally refer to the image data and / or image information involved in the present system or method, which may be a digital image, an analog image, or an image acquired from a device. The image display type may be a color image, a black and white image, or a grayscale image, etc. The first image and the second image here may be from different devices or the same device but acquired at different times or parameters.

[0055] It should also be noted that the "imaging equipment" mentioned in the context of the present invention refers to imaging equipment in the field of interventional medicine, which refers to various technologies and instruments used to obtain, process and display the structure or function in the body. These devices visualize the physiological or pathological information inside the human body through different imaging principles to help doctors diagnose, monitor and treat. For example, they can be computed tomography (CT), magnetic resonance imaging (MRI), radionuclide imaging equipment, etc.

[0056] It is understandable that the machine learning model for fusion training of the acquired instrument information, target object information, specific part information, three-dimensional coordinate information of the instrument, etc., that is, the model adopted by the machine learning unit is not limited, for example, it can be a deep learning model, or a hybrid model, or a graph learning model. For example, when processing image data, a convolutional neural network (CNN model) can be used, when analyzing the physiological parameters of the target object, a recursive neural network model (RNN model) can be used, and when processing data sets or enhancing or rendering images, a generative adversarial network model (GAN model) can be used. In addition, the model used by the system may also have transfer learning capabilities, such as transferring knowledge from pre-trained models in related fields to improve the training results of medical data.

[0057] See also Figure 2 and Figure 6 As mentioned above, the control module 13 further includes a machine learning unit 130 and a trajectory planning unit 131, and the preprocessing module 12 further includes an image processing unit 120 and a coordinate conversion unit 121. The following example illustrates how the machine learning unit 130 uses instrument information, target object information, specific part information, and three-dimensional coordinate information of the instrument to perform fusion training and plan the surgical motion trajectory of the input specific target part.

[0058] The machine learning unit 130 in this embodiment includes: a collection subunit, which is used to collect data pairs of relevant information of the interventional instrument, relevant information of the surgical object, and the motion trajectory of the instrument; a coding fusion subunit, which is used to uniformly encode the relevant information of the interventional instrument and the relevant information of the surgical object, and combine the data pairs of the motion trajectory of the instrument to form a training data set; a training subunit, which is used to train the deep learning model using the training data set to obtain a trained deep learning model; and a generation subunit, which is used to input the relevant information of the interventional instrument to be punctured and the relevant information of the surgical object into the trained deep learning model to obtain a predicted path.

[0059] When the machine learning unit 130 is working, it first collects information such as geometric structures, functional parameters and operating characteristics of different types of interventional instruments, and stores this information in the instrument database of the storage module (not shown). Secondly, it also collects the intervention site of the target surgical object, the physiological parameters of the target surgical object, such as the heart rate, breathing and other physiological information during the operation, and the tissue characteristics of the intervention site, disease information, etc., which can be provided by the sensor module (not shown), the imaging device 11 and the camera module 10 of the system 1. In addition, there is also three-dimensional coordinate information of the instrument. The coordinate information of the interventional instrument in the three-dimensional space is obtained by the camera module 10, and the three-dimensional point cloud data of the interventional instrument is generated by the parallax principle, and converted into three-dimensional coordinate information in the standard coordinate system by the coordinate conversion unit 121. Subsequently, feature extraction is performed on the various types of information collected above: for example, a deep learning algorithm (such as a convolutional neural network CNN) is used to extract feature points, edges and texture information from image data; geometric features and operating characteristics are extracted from instrument information, and the extracted features from different sources are fused. Through multimodal data fusion technology, the instrument information, target object information and specific part information are uniformly encoded, and combined with the three-dimensional coordinate information of the instrument to form a high-dimensional feature vector, so that the collected fusion feature data can be used to construct a training data set. The training data set includes input features and corresponding surgical operation labels (such as surgical path, operation steps, etc.). After that, a suitable machine learning model (such as long short-term memory network LSTM, graph convolutional neural network GCN, etc.) is selected for training. During the training process, the supervised learning method is adopted to continuously optimize the model parameters through the back propagation algorithm to improve the prediction accuracy of the model. Finally, based on the prediction results of the machine learning unit 130, a preliminary surgical path is generated.

[0060] The trajectory planning unit 131 in this embodiment includes: a graph search subunit, used to find the planning of the predicted path in three-dimensional space; a path optimization subunit, used to optimize the predicted path under the constraints of the operation constraints of the surgical instrument, the tissue structure of the target site and the safety of the operation; a real-time adjustment subunit, used to compare the actual motion trajectory of the surgical instrument with the optimized predicted path, and adjust the optimized predicted path when there is a deviation between the two.

[0061] When the trajectory planning unit 131 is working, it can use a graph search algorithm (such as AAlgorithm algorithm, Dijkstra algorithm) to find a preliminary path planning in three-dimensional space, and can also use a path optimization algorithm (such as RRT, PRM, etc.) to optimize the initially generated path. Considering the operation constraints of the surgical instrument, the tissue structure of the target part and the safety of the operation, the path is continuously adjusted through an iterative optimization algorithm (such as the gradient descent method) to make it smoother and safer. During the operation, the real-time adjustment subunit continuously monitors the actual motion trajectory of the instrument and compares it with the predicted path in real time. If a deviation occurs, it is adjusted immediately. In this way, the path planning can be dynamically updated according to the real-time data during the operation (such as newly discovered lesions, tissue displacement, etc.). It can use an adaptive planning algorithm (such as MPC, RL, etc.) to dynamically adjust the path to ensure the efficiency and safety of the operation.

[0062] It can be understood that in the solution provided in the above embodiment, the prediction results of the machine learning unit 130 provide preliminary path planning for reference, and the combination of the graph search algorithm, path optimization algorithm and adaptive algorithm of the trajectory planning unit 131 makes the path more accurate, safe and effective.

[0063] Specifically, the machine learning unit 130 can predict the best path of the interventional instrument during surgery by training a large amount of sample data. These prediction results include the preliminary path from the starting point to the target surgical site, as well as the path that takes into account the characteristics of the instrument, the anatomical structure and the surgical requirements. In the trajectory planning unit 131, for example, the AAlgorithm algorithm is used to find the optimal path from the starting point to the target, because in interventional medicine, the heuristic function h(n) can be designed based on the physical characteristics of the surgical instrument and the complexity of the puncture target site, for example, the Manhattan distance or Euclidean distance from the target site extracted from the image data to the current instrument position. In the absence of an obvious heuristic, the Dijkstra algorithm can be used to gradually expand the nodes through the minimum cost priority strategy to ensure that the shortest path is found. For example, when it is necessary to find a path that can avoid important blood vessels or nerves from all possible paths from the starting point to the target site, the Dijkstra algorithm is a better choice. Of course, in interventional surgery, rapid random trees (RRT) can be used to generate paths that avoid critical tissues (such as large blood vessels or organs), while considering the operational limitations of surgical instruments. PRM may also be used in the preparation stage of interventional puncture surgery, through a large number of sampling and path calculations, to generate a global path map to help determine the optimal path. For another example, iterative optimization algorithms can be used to fine-tune the preliminary path through the gradient descent method to ensure that the path is smoother when passing through dense tissues or close to fragile structures, reducing surgical risks.

[0064] In addition, the trajectory planning unit 131 continuously monitors the actual motion trajectory of the interventional instrument based on the data information or image information of the camera module 10, the imaging device 11, and the sensor module, and compares it with the predicted path. If the actual trajectory deviates significantly from the predicted path, the system will immediately adjust it. During the operation, new image data or sensor data may show tissue displacement or new lesions. For example, when a new lesion location or tissue displacement is found during the operation, the system needs to update the path planning in real time to ensure safety. When the physiological environment around the target site may change, for example, the patient's lung state may change due to intraoperative breathing, an adaptive planning algorithm can be used at this time, which can dynamically optimize the control strategy according to real-time data to adapt to changes in the current environment. For example, during the operation, MPC can adjust the path of the interventional instrument in real time to cope with emergencies, such as tissue expansion or contraction. RL can learn the best path strategy by simulating a complex surgical environment during the interventional puncture surgery training phase, and adjust it according to real-time feedback during the operation. Through the close combination of the above algorithms, the interventional puncture surgery motion trajectory planning of the interventional instrument can be intelligent, dynamic, and precise, meeting the high requirements of surgery for accuracy and safety.

[0065] It should be noted that the "information or parameter information of the interventional instrument" mentioned in the context of the present invention includes but is not limited to the material of the interventional needle (for example, stainless steel commonly used for rigid needles, nickel-titanium alloy commonly used for flexible needles, etc.), the type of needle tip (for example, a bevel tip, a round tip, etc.), the diameter of the needle (for example, the outer diameter and the inner diameter, etc., usually expressed in G (Gauge), the larger the value, the smaller the diameter of the needle, such as 18G, 22G, etc.), the length of the needle, the structure of the needle (for example, a single-lumen needle, a multi-lumen needle, etc.), image-guided markers, reference marking points and other parameter information.

[0066] The "characteristic sites, first characteristic points, and second characteristic points of the interventional device" mentioned in the context of the present invention may also be referred to as "characteristic points" in certain scenarios, and generally refer to points with special significance in the structure and geometry of the needle. These characteristic sites play a key role in the puncture operation and / or image guidance, such as the main working point of the puncture, the needle tip (i.e., the front end of the needle), and the reference marking points on the needle body. It should be noted that these reference marking points may be inherent characteristic points of the interventional device itself, such as some specific geometric features (such as notches, grooves, protrusions, etc.) in the manufacturing process of the needle body or other interventional devices, or may be the use of the difference in radioactive density of different materials to embed high-contrast materials (such as metal markers) at specific positions in the needle body to form visual reference marking points. Of course, the reference marking points here may also be artificially added marking points (sticky markings or coating markings), for example, a high-contrast, easily identifiable marker (such as a red star-shaped label) is pasted on the surface of the needle body, and a high-contrast coating (such as a fluorescent, radioactive or magnetic coating) is applied on the surface of the needle body to form a clear reference marking point. Another example is the color or a certain position of the needle tail, or an image-guided marking point. In some scenarios, a marking point is added to a specific position of the needle that can be clearly displayed under imaging equipment such as CT and MRI. Another example is that an ultrasound reflection point is designed on the needle used under ultrasound guidance. Another example is the location of a sensor at a specific position of the needle tip or needle body, etc. Another example is the demarcation point of the coating, or the starting point of the inner cavity of the needle (i.e. the starting position of the cavity or channel inside the needle) or the intersection point of multiple cavities (i.e. the intersection point of different channels inside a multi-cavity needle), etc.

[0067] It is understandable that the device information may include at least one of the following information: device name, model, manufacturer, serial number, length, diameter, width, weight, shape, material, structure, image sample, and marking point.

[0068] It is understandable that the physiological information of the target object may include data related to its health status, vital signs and body functions, such as height, weight, body mass index, basal metabolic rate, body temperature, heart rate, blood pressure, respiratory rate, blood oxygen saturation, and may also be the results of blood or urine tests, and may also be data information related to physiological functions, such as electrocardiogram, pulmonary function test, electroencephalogram, electromyogram, etc.

[0069] It is understandable that the target site-related information generally refers to information related to the location of lesions in the site of interventional puncture surgery, such as the liver, lungs, kidneys, etc., such as the shape, boundaries and structural characteristics of the target site, such as whether the shape of the tumor is spherical, elliptical or irregular; the thickness or depth of the tissue, especially when it is necessary to puncture through multiple tissue layers; the specific location of the target site in the body and its relationship with surrounding organs and blood vessels; the size of the target site, including dimensions such as length, width, and height; the distribution of blood vessels in and around the target site, especially the location and direction of large and small blood vessels; the pathological type of the target site, such as benign tumors, malignant tumors, cysts, inflammatory lesions, etc.; the histological composition of the target site, such as cell type, tissue structure, cell proliferation, etc.; the functional state of the target organ or tissue, such as the metabolic function of the liver, the filtration function of the kidneys, etc.; the blood flow in and around the target site, especially the blood flow in the arteries and veins; the level of biomarkers related to lesions at the target site, such as tumor markers (such as CEA, AFP, etc.), such as C-reactive protein (CRP), white blood cell count, etc. In addition, it may also include information such as whether the target area has received treatment, such as surgery, radiotherapy, chemotherapy, etc., and changes after treatment.

[0070] Furthermore, in the preprocessing module 12, the image processing unit 120 is designed to perform processing operations such as edge detection, feature point extraction and image segmentation on the first image and the second image. The coordinate conversion unit 121 uses the internal and external parameters of the camera module 10 to convert the processed first image into external three-dimensional coordinates. In addition, the coordinate reconstruction unit 122 performs three-dimensional reconstruction on the feature points of the second image extracted by the image processing unit 120 based on the volume data provided by the imaging device 11, thereby generating internal three-dimensional coordinates. Finally, the registration unit 124 is responsible for accurately registering the external three-dimensional coordinates with the internal three-dimensional coordinates. By adopting advanced image processing technologies such as edge detection and feature point extraction, the system can effectively identify and extract key features, thereby enhancing the accuracy of capturing motion trajectories. In addition, the use of internal and external parameters for coordinate conversion and registration makes the relationship between the external and internal three-dimensional coordinates closer, ensuring the consistency and accuracy of the data.

[0071] Specifically, the image processing unit 120 processes interventional puncture surgery images (such as CT or MRI), intraoperative real-time images (such as ultrasound images or X-ray images), and interventional puncture surgery and images from the camera module 10 during the surgery. In order to perform accurate image segmentation, in one embodiment, the system uses the SAM2 model, which first accepts the input image and extracts low-level and high-level features of the image through a multi-layer convolutional neural network. The low-level features include edges and textures, while the high-level features include morphological features. Then, the SAM2 model uses the attention mechanism to enhance the features of key areas, such as the area where the interventional needle is located, while suppressing background noise, thereby generating a segmentation mask for the interventional needle and its surrounding area. This step ensures that the boundary of the interventional needle in the segmentation result is clear, providing accurate data for subsequent processing. After image segmentation, the image processing unit 120 continues to perform edge detection and feature point extraction. For edge detection, the system uses the Canny edge detection algorithm, which can extract clear edges in medical images with high noise through multi-level filtering and non-maximum suppression. Specifically, the input image is first processed by Gaussian blur to reduce noise, and then the gradient strength and direction of the image are calculated to detect edges. Through non-maximum suppression, only the local maximum value in the gradient direction is retained to form an edge line. Then, the double threshold method is applied to extract strong edges and weak edges, and finally the edge map of the interventional needle is generated. For feature point extraction, the image processing unit 120 uses the SIFT (Scale Invariant Feature Transform) algorithm. The algorithm detects extreme points in image spaces of different scales and preliminarily identifies the key feature points on the interventional needle. Then, by screening out stable key points, eliminating points with low contrast and strong edge response, and calculating the direction of each key point based on the local gradient direction, a key point descriptor is generated. Through these steps, the system can accurately extract the feature points of the interventional needle in the image and provide basic data for subsequent three-dimensional reconstruction and coordinate transformation. In addition, in order to optimize the visual background in the surgical environment and enhance the algorithm's ability to extract and segment the feature points of the puncture instrument, especially the complex and changeable surgical environment (such as multiple colors, textures and reflective objects) may interfere with the accuracy of the algorithm, so it is necessary to use a specific background (such as a surgical drape or surgical gown with a grid or a single color) to simplify the visual background and reduce interference. For example, in lung puncture surgery, the puncture needle is operated on a surgical drape with a grid. The computer vision algorithm can use the regularity of the grid to quickly identify and segment the outline of the puncture needle. For another example, using a single-color surgical drape, the visual algorithm can more easily distinguish the shape and position of the interventional instrument in this background.

[0072] After completing the image processing, the coordinate conversion unit 121 begins to use the internal and external parameters of the camera module 10 to convert the processed first image into external three-dimensional coordinates. The internal parameters of the camera include focal length, principal point coordinates and distortion coefficients, which are used for internal parameter matrix representation; the external parameters include the position and orientation of the camera, which are represented by rotation matrix and translation vector. Through these parameters, the system can convert the pixel coordinates in the image into three-dimensional coordinates in the camera coordinate system, and then further convert them into external three-dimensional coordinates in the world coordinate system. Next, the coordinate reconstruction unit 122 performs three-dimensional reconstruction on the second image feature points extracted by the image processing unit 120 based on the volume data provided by the imaging device 11. The volume data includes three-dimensional body data of medical images such as CT or MRI, which provides a detailed three-dimensional structure of the surgical environment. By matching and triangulating feature points in images of different perspectives, the system can reconstruct the three-dimensional coordinates of the interventional needle. Finally, the configuration unit 123 is responsible for accurately aligning the external three-dimensional coordinates with the internal three-dimensional coordinates. First, a preliminary alignment is performed through the ICP (iterative closest point) algorithm, and feature point pairs in the external and internal three-dimensional coordinates are selected, and the initial transformation matrix is ​​calculated to preliminarily align the two coordinate systems. Then, nonlinear optimization algorithms such as Levenberg-Marquardt are used for fine registration, further optimizing the transformation matrix, minimizing the registration error, and ensuring that all external three-dimensional coordinates are accurately registered to the internal three-dimensional coordinate system. Through the above steps, the system can use advanced image processing technology and coordinate transformation methods to capture and plan the motion trajectory of the interventional needle with high precision, ensuring the efficiency, safety and success of the operation.

[0073] Continue to see Figure 3 , Figure 3 The following is an example of an application scenario of the motion trajectory planning system for interventional puncture surgery mentioned in an embodiment of the present invention. Figure 3 As can be seen, the system 1 includes an imaging device 11 and a camera module 10. Further, the camera module 10 includes a multi-eye camera 100 and an arc guide rail 101. In order to improve the quality of the image captured by the camera module 10, a light source module 14 is also provided. The operator performs a puncture intervention operation on the target object 2 through the interventional instrument 3.

[0074] It should be noted that the multi-eye camera 100 in the embodiment of the present invention preferably adopts a non-parallel multi-eye vision system for imaging. Through the non-parallel multi-eye vision system, such cameras can capture the same scene from multiple angles, provide more comprehensive and accurate external three-dimensional coordinates, and reduce the errors caused by viewing angle limitations. Specifically, cameras with different numbers of eyes will have different installation and arrangement methods, as well as corresponding light source module configurations and control methods to meet different surgical needs and environments.

[0075] See also Figure 3 , Figure 3A six-eye camera is provided in the system. In the six-eye camera system, six cameras are mounted on an electrically driven arc guide rail 101 to form an irregular but evenly covered three-dimensional observation network. These cameras are dynamically adjusted in position and angle through the control module to ensure that they are always in the best observation position during the operation. For example, in interventional needle puncture, the six camera lenses can adjust the position and angle in real time according to the changes in the target site to capture images from different perspectives and perform three-dimensional reconstruction. The light source module 14 usually adopts a combination of annular light source and point light source in this configuration. The annular light source provides uniform overall illumination, while the point light source is used for focused illumination of specific areas to ensure that the details of the puncture site are clearly visible. The control module dynamically adjusts the brightness and angle of the light source according to the position and angle of the camera to optimize the lighting conditions and ensure high-quality image data acquisition.

[0076] It is understood that the multi-eye camera includes at least two camera lenses. Figure 4 , showing an exemplary arrangement diagram of ten-eye cameras; see Figure 5, showing an exemplary arrangement diagram of a four-eye camera. For a four-eye camera, it can be evenly distributed at four key positions of the arc guide rail, such as 0 degrees, 90 degrees, 180 degrees and 270 degrees. The light source module 14 is usually preferably a combination of an annular light source and a point light source. The annular light source provides basic lighting, while the point light source is used for accent lighting of a specific area. The light source position can be fixed at the central position of the arc guide rail to ensure uniform basic lighting. Under the control of the control module, the four-eye camera system dynamically adjusts the angle and position of the camera through the electrically driven arc guide rail. For example, in a specific surgical step, the control module can instruct one or more cameras to adjust the angle to better capture the image of the key parts. At the same time, the brightness and angle of the light source module will also be adjusted in time according to the current position of the camera and the surgical environment to ensure the best lighting conditions. For a six-eye camera, an interval distribution method is adopted, which can ensure that the camera captures the surgical area from multiple different angles. Six cameras are mounted on the arc guide rail to ensure that the spacing between each camera is as uniform as possible. For example, if the curved rail is semicircular, the cameras can be installed at 30 degrees, 60 degrees, 90 degrees, 120 degrees, 150 degrees and 180 degrees respectively. The light source module preferably adopts a combination of annular light sources and point light sources. The annular light source is installed in the central position of the curved rail to provide overall uniform illumination, while the point light source can be flexibly installed at several key points of the curved rail for focused illumination of specific areas. In addition, for the ten-eye camera, a dense and asymmetric distribution can be adopted to cover a wider and more detailed observation area. The cameras can be distributed on the curved rail at intervals of 18 degrees, but the specific positions will be fine-tuned according to surgical needs. For example, the cameras can be installed at 0 degrees, 18 degrees, 36 degrees, 54 degrees, 72 degrees, 90 degrees, 108 degrees, 126 degrees, 144 degrees and 162 degrees. The light source module uses a comprehensive configuration of annular light sources, point light sources and surface light sources. The annular light source provides global illumination, the surface light source is used for uniform background illumination, and the point light source is used for fine illumination of key areas.

[0077] It should be noted that the arrangement of the camera module 10 and the light source module 14 of the system 1 in the embodiment of the present invention can be arbitrarily combined and arranged as needed, and is not limited to the several possible arrangements listed. For example, two camera lenses may form a group of lenses for image acquisition, etc.

[0078] The "ring light source" mentioned in the context of the embodiments of the present invention is a light source in the shape of a ring, usually composed of multiple small light sources (such as LEDs) arranged in a ring structure. A "point light source" is a very small size, which can be considered as a point relative to the irradiation distance, and the light emitted is evenly distributed in all directions. A small-sized light source such as a single LED lamp can be approximated as a point light source, and a light source such as a laser pointer can be regarded as a point light source. A "surface light source" is a light source that emits light evenly from a certain area, and the light emitted in different directions has a consistent brightness. When using a multi-eye camera system for three-dimensional reconstruction, a ring light source can be installed around the camera to provide uniform lighting to ensure that there are no shadows on the instrument and tissue surface, and to improve the clarity and details of the imaging. When precise illumination of a specific area (such as a blood vessel or puncture point) is required, a point light source can be installed at the end of the camera or surgical instrument to provide a concentrated beam to highlight important details. In scenes where large-area uniform illumination is required (such as overall observation of the surgical field), a surface light source can be installed on the ceiling of the operating room or around the operating table to provide soft lighting and reduce visual fatigue and interference caused by uneven brightness.

[0079] The "non-parallel multi-eye vision system" mentioned in the context of the embodiments of the present invention is generally relative to the parallel multi-eye vision system. Unlike the parallel multi-eye vision system, the cameras of the non-parallel multi-eye vision system are not strictly arranged in parallel, but are arranged at different angles and positions. The cameras can be installed at different heights, angles and positions to ensure that the images captured by each camera have different viewing angles. Multiple cameras take images of the same scene at the same time to ensure that the captured images are temporally consistent. In interventional surgery, the non-parallel multi-eye vision system can provide real-time three-dimensional imaging to help doctors operate more accurately. For example, in complex vascular surgery, multiple cameras are used to capture images of the interventional needle from different angles to generate an accurate three-dimensional model, thereby assisting doctors in performing delicate operations.

[0080] Further, the system 1 also includes a storage module (not shown), which further includes an instrument database, which stores conventional interventional instrument information, and the instrument database also allows the operator to manually enter instrument information. The instrument database pre-stores a large amount of information about conventional interventional instruments. This information includes but is not limited to the name, model, specification, manufacturer, purpose and method of use of the instrument. By pre-storing this information, the system can quickly identify and call instrument information to assist doctors in performing surgical operations. For example, a doctor or operator first queries the information of the required instrument in the instrument database. If the information of the instrument is found, the system will display the relevant data for reference and use. If the query fails, the doctor can manually enter the instrument information, including the instrument name, model, specification, manufacturer, purpose and any other relevant information. This information will be saved in the database for subsequent use. Scenarios in which doctors need to manually enter instrument information include the introduction of newly developed interventional instruments that do not yet exist in the database, or some special cases may require customized instruments, new or upgraded versions of existing instruments are launched, and the database has not yet been updated.

[0081] Furthermore, the system 1 further comprises a display module (not shown), the display module comprises a display, and the display displays the three-dimensional model of the instrument, the navigation path, and the target part information in real time. The display module is configured to superimpose the real-time position information of the instrument on the image acquired by the CT scanning device, and / or render the three-dimensional model and navigation path of the instrument in real time on the augmented reality device.

[0082] Specifically, the display module may include a display, a display processing unit, and an augmented reality device (AR). The display may be a high-definition display or a touch screen. The high-definition display is used to display high-resolution images and three-dimensional models to ensure that the doctor can clearly view the real-time data during the operation. The touch screen allows the doctor to zoom in, zoom out, rotate, and other operations through the touch operation interface to facilitate viewing of images at different angles. The display processing unit is used to process and render complex three-dimensional models and image data to ensure the smoothness and accuracy of real-time display, or to process images and data from multiple sources, superimpose and fuse the real-time position information of the instrument with the CT scan image, or to superimpose virtual information in the real environment for a more immersive augmented reality experience, and to display the three-dimensional model and navigation path of the instrument in real time.

[0083] For example, the three-dimensional model of conventional instruments is pre-stored in the system. When the instrument enters the use state, the display processing unit renders its three-dimensional model in real time and displays it on the display. The doctor can rotate, scale and move the three-dimensional model through the touch screen or mouse and keyboard to view different angles and details. The system calculates the best path according to the target part and the current position of the instrument, renders the navigation path in real time, and superimposes it on the three-dimensional model and CT image to guide the doctor's operation. The image data obtained by the CT scanning device is transmitted to the display module system in real time to automatically or manually mark the target part (such as the lesion area, the entrance of the blood vessel, etc.) and display it on the image. In addition, the display processing unit can also superimpose the processed instrument position information on the CT image and display it in real time on the display. The doctor can clearly see the specific position and movement trajectory of the instrument in the body. For another example, the doctor can interact with the AR device through gestures or touch pads to adjust the viewing angle, zoom in / out the model, etc. The AR device updates the display information in real time according to the doctor's operation and the movement of the instrument to ensure the dynamic accuracy of the information. The navigation path and target part information are displayed in the real environment in the form of virtual markings to help the doctor perform intuitive operations.

[0084] In an embodiment of the present invention, the system further comprises a sensing module, which is configured to capture environmental data and / or physiological information of the target object. It is understandable that in some scenarios, the sensing module may not be directly integrated into the system, but as an independent unit, it only communicates data with the system. For example, during lung puncture, real-time monitoring of the patient's respiratory rate is extremely important for planning the motion trajectory. One or more respiratory sensors (such as piezoelectric sensors, strain gauge sensors or photoelectric volume pulse wave sensors) can be placed on the patient's chest or abdomen. The sensor monitors the expansion and contraction of the chest or abdomen in real time to obtain the respiratory rate. Since the lungs move continuously with breathing, it is necessary to monitor and predict this movement in real time during the operation to ensure that the path of the needle reaches the target site accurately. Since the chest cavity expands and contracts during breathing, the lungs move up and down. This movement will change the position of various anatomical structures in the lungs, including the target lesion site. If the displacement caused by breathing is not considered, the needle may deviate from the target, increasing the risk of mispuncture or damage to other tissues.

[0085] Specifically, taking lung puncture surgery as an example, the system uses respiratory data to train and plan the needle's motion trajectory through the following steps to ensure that the needle reaches the target site accurately within the respiratory cycle. The system trains the model through machine learning algorithms (such as neural networks, support vector machines, etc.) to learn the relationship between respiratory data and actual lung movement. The system uses the trained respiratory model to predict the lung motion trajectory within the respiratory cycle. According to the predicted lung movement, the system plans the dynamic trajectory of the needle to ensure that the needle can accurately reach the target site at different stages of the respiratory cycle. The system superimposes the predicted trajectory on real-time images (such as CT, MRI or X-rays) to provide intuitive navigation information. The doctor views the needle's path and target position through a real-time display or augmented reality device and performs operations. The system monitors the respiratory data in real time, dynamically adjusts the needle's path and speed, and ensures that the needle always remains on the optimal trajectory. Therefore, in lung puncture surgery, the system collects and processes the patient's respiratory data in real time, establishes an individualized respiratory model, uses machine learning algorithms to train and predict the lung motion trajectory caused by breathing, and dynamically plans the needle's path and speed to ensure that the needle reaches the target site accurately within the respiratory cycle.

[0086] It is understandable that the motion trajectory planning system for interventional puncture surgery provided in the embodiment of the present invention can be used in the process of interventional puncture surgery, and the system can adjust and optimize the motion trajectory in real time according to the doctor's operation. Here, the "interventional puncture surgery process" not only refers to the preoperative preparation and the operation stage of the interventional instrument entering the body, but also covers the real-time navigation and path adjustment during the operation. The system can be trained based on the sample data collected and analyzed in advance. In this way, when the doctor determines the lesion information and is ready to perform the puncture surgery, the system can generate a preliminary motion trajectory based on the information of the specific target object and target part. This preliminary trajectory is for the doctor to select, adjust and refer to, so as to formulate a more accurate surgical plan. In addition, during the operation, when the doctor operates according to the preliminary motion trajectory, if there is a sudden change in the operation (such as a change in the location of the lesion or a new abnormality is found or the patient's physiological parameters change dynamically), the system can respond quickly and optimize the motion trajectory in time. These optimized trajectory information will be displayed to the doctor in real time to ensure that the doctor can adjust the operation immediately and improve the success rate and safety of the operation.

[0087] It can also be understood that the interventional puncture surgery motion trajectory planning system and method of the embodiment of the present invention are mainly used to assist doctors in performing interventional puncture surgery. The system generates a preliminary motion trajectory based on the information of a specific target object and target part by adjusting and optimizing the motion trajectory in real time, and performs dynamic optimization when there is a sudden change during the operation to ensure the accuracy and safety of the operation. However, the present invention is not limited to use under the guidance of experienced doctors. In particular, in areas where medical experience and medical conditions are relatively weak, the system also has potential automated application possibilities. For example, by introducing a robotic arm and performing autonomous operation based on training data, puncture surgery can be performed automatically. For example, by introducing a high-precision robotic arm, combined with the motion trajectory planning function of the system, interventional puncture surgery can be performed automatically. The robotic arm can operate according to a predetermined trajectory and real-time optimization instructions under the guidance of the system. The system uses a large amount of sample data previously collected for deep learning and training, and accumulates rich lesion characteristics and surgical trajectory information. Based on these data, the system can autonomously complete lesion positioning, trajectory planning, and puncture operations in automatic mode. During the automated operation, a remote monitoring system can be used to allow experienced doctors to view the progress of the operation in real time and intervene and guide when necessary. In this way, the high quality and safety of surgery can be ensured even in areas with poor medical conditions.

[0088] The second embodiment of the present invention provides a method for planning the motion trajectory of an interventional puncture surgery, which adopts the planning system of the first embodiment, see Figure 7 , including the following steps: acquiring image data from multiple data sources, including a first image captured by a camera module and a second image generated by a CT scanning device; performing image registration based on the first image and the second image, generating three-dimensional coordinate information and forming a data pair of the instrument motion trajectory; using the data pair of the instrument motion trajectory, the instrument information and the target object information as input data, performing deep learning model training, and generating training results including the prediction of the instrument motion trajectory on different target objects, and storing the results in a storage module; entering the specific location information of the target part and the instrument information to be used, and generating an interventional puncture surgery motion trajectory path for a specific target part based on the stored training results.

[0089] In addition, a third embodiment of the present invention provides a computer storage medium, which stores computer executable instructions, and the computer executable instructions are used to execute the above-mentioned method for planning the motion trajectory of interventional puncture surgery.

[0090] It can be seen that the contents of the above-mentioned embodiments of the method for planning the motion trajectory of interventional puncture surgery are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above-mentioned embodiments of the method for planning the motion trajectory of interventional puncture surgery, and the beneficial effects achieved are also the same as those achieved by the above-mentioned embodiments of the method for planning the motion trajectory of interventional puncture surgery.

[0091] It is understood that all or some steps and systems in the disclosed method above can be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components can be implemented as processors, such as software executed by a central processing unit, a digital information processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media generally contain computer readable instructions, data structures, program modules, or other data in modulated data information such as carrier waves or other transport mechanisms, and may include any information delivery media.

[0092] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the technical field without departing from the purpose of the present invention.

Claims

1. A motion trajectory planning system for interventional puncture surgery, characterized in that: include: A camera module, used for collecting a motion trajectory of the device in an external environment based on a first feature point of the device and generating a first image; An imaging device, used to obtain a motion trajectory of the device in the internal environment based on a second feature point of the device and generate a second image; a preprocessing module, configured to generate external three-dimensional coordinates and internal three-dimensional coordinates based on the first image and the second image, and to form a data pair of a movement trajectory of an instrument based on the external three-dimensional coordinates and the internal three-dimensional coordinates; The control module includes a machine learning unit and a trajectory planning unit; the machine learning unit is used to train a deep learning model based on the data of the instrument's motion trajectory, and generate a predicted path based on the trained deep learning model; the trajectory planning unit is used to optimize and adjust the predicted path.

2. The motion trajectory planning system for interventional puncture surgery according to claim 1, characterized in that: The camera module is a multi-eye camera, and the multi-eye camera adopts a non-parallel multi-eye vision system.

3. The motion trajectory planning system for interventional puncture surgery according to claim 2, characterized in that: The multi-eye camera is fixed on an electrically driven arc guide rail, and the position and angle of the multi-eye camera on the arc guide rail are adjusted via the control module.

4. The motion trajectory planning system for interventional puncture surgery according to claim 1, characterized in that: The preprocessing module comprises: An image processing unit, used for performing edge detection, feature point extraction and image segmentation on the first image and the second image; A coordinate conversion unit, configured to convert the coordinates of the processed first image into external three-dimensional coordinates based on the intrinsic parameters and extrinsic parameters of the camera module; A coordinate reconstruction unit, configured to perform three-dimensional reconstruction based on the volume data provided by the imaging device and feature points extracted from the processed second image to generate internal three-dimensional coordinates; The registration unit is used to register the external three-dimensional coordinates with the internal three-dimensional coordinates to form a data pair of the movement trajectory of the instrument.

5. The motion trajectory planning system for interventional puncture surgery according to claim 1, characterized in that: Also includes: A light source module, wherein the light source module is used to provide a light source, wherein the light source includes at least one of a ring light source, a point light source, and a surface light source; the light source module dynamically adjusts the angle, brightness, and / or position of the provided light source according to the position of the camera module.

6. The motion trajectory planning system for interventional puncture surgery according to claim 1, characterized in that: The machine learning unit comprises: A collecting subunit, used to collect data pairs of information related to the interventional instrument, information related to the surgical object, and movement trajectory of the instrument; The coding fusion subunit is used to uniformly encode the relevant information of the interventional instrument and the relevant information of the surgical object. and combining the data pairs of the movement trajectories of the apparatus to form a training data set; A training subunit, used to train the deep learning model using a training data set to obtain a trained deep learning model; The generation subunit is used to input the relevant information of the interventional instrument to be punctured and the relevant information of the surgical object into the trained deep learning model to obtain a predicted path.

7. The motion trajectory planning system for interventional puncture surgery according to claim 1, characterized in that: The trajectory planning unit comprises: A graph search subunit, for finding a plan for the predicted path in three-dimensional space; A path optimization subunit, used for optimizing the predicted path under the constraints of the operation constraints of the surgical instrument, the tissue structure of the target site and the safety of the surgery; The real-time adjustment subunit is used to compare the actual motion trajectory of the surgical instrument with the optimized predicted path, and adjust the optimized predicted path when there is a deviation between the two.

8. A motion trajectory planning method for interventional puncture surgery, characterized in that: The interventional puncture surgery motion trajectory planning system as claimed in any one of claims 1 to 7 specifically comprises the following steps: A first image captured from the camera module and a second image generated from the imaging device; Performing image registration based on the first image and the second image to generate three-dimensional coordinate information and form a data pair of the movement trajectory of the instrument; The data pairs of the instrument motion trajectory, the relevant information of the interventional instrument and the relevant information of the surgical object are used as the input data of the deep learning model to train the deep learning model. The generated training results include the prediction of the motion trajectory of the instrument on different surgical objects, and the results are stored in the storage module; The specific location information of the target part and the information of the instrument to be used are input, and based on the training results in the storage module, the motion trajectory path of the interventional puncture surgery for the specific target part is generated.

9. A computer storage medium storing at least one program for causing a computer to execute a method for planning a motion trajectory for an interventional puncture surgery, the method comprising: A first image captured from the camera module and a second image generated from the imaging device; Performing image registration based on the first image and the second image to generate three-dimensional coordinate information and form a data pair of the movement trajectory of the instrument; The data pairs of the instrument motion trajectory, the relevant information of the interventional instrument and the relevant information of the surgical object are used as the input data of the deep learning model to train the deep learning model. The generated training results include the prediction of the motion trajectory of the instrument on different surgical objects, and the results are stored in the storage module; The specific location information of the target part and the information of the instrument to be used are input, and based on the training results in the storage module, the motion trajectory path of the interventional puncture surgery for the specific target part is generated.

Images