A flexible organ surgery navigation system and a flexible organ surgery robot

Abstract

The present invention discloses a flexible organ surgery navigation system and a flexible organ surgery robot, which include: an image processing module, which is used to perform rigid registration based on preoperative image data and mark the feature points of the flexible organ; a triangular model is constructed based on the feature points, and the key parameters of the surgical path are determined based on the triangular model; a navigation module is used to monitor the position change of the flexible organ based on real-time image data, perform non-rigid registration, and update the triangular model; according to the updated triangular model, the key parameters of the surgical path are dynamically adjusted to generate an optimal operation path; and navigation data is output according to the optimal operation path. The present invention can realize the fully automated operation of the surgical robot in flexible organ surgery.

Description

Technical Field

[0001] The present invention relates to the technical field of medical robots, and in particular to a flexible organ surgery navigation system and a flexible organ surgery robot. Background Art

[0002] In the field of modern surgery, fully automated surgical robots have always been an important direction of technological development. With the widespread application of artificial intelligence deep learning technology and the improvement of chip computing power, surgical robots, especially robotic technology for flexible organ surgery, are developing rapidly.

[0003] Current surgical robots can be divided into two categories: one is for hard organs, such as bones and bronchi; the other is for flexible organs, such as the liver, pancreas, gastrointestinal tract, and kidneys. Surgery on hard organs is relatively easy because these organs do not deform during surgery, and intraoperative images can maintain a high degree of consistency with preoperative images. In contrast, flexible organs deform and move during surgery, resulting in inconsistencies between intraoperative images and preoperative images, which places higher demands on the accuracy of surgical robots.

[0004] Although the traditional da Vinci surgical robot can assist surgeons in delicate operations, it still requires human control and is difficult to achieve full automation. To realize a truly fully automatic flexible organ surgical robot, it is necessary to overcome the challenges brought about by organ deformation during surgery, especially how to calibrate images in real time during surgery to ensure the accuracy and safety of the surgery. Summary of the invention

[0005] According to one aspect of the present invention, a flexible organ surgery navigation system and a flexible organ surgery robot are provided to achieve fully automated operation of the surgical robot in flexible organ surgery.

[0006] In order to solve the above technical problems, the first aspect of the present invention discloses a flexible organ surgery navigation system, comprising:

[0007] An image processing module is used to perform rigid registration based on preoperative image data and mark the feature points of flexible organs; construct a triangular model based on the feature points and determine key parameters of the surgical path based on the triangular model;

[0008] The navigation module is used to monitor the position change of the flexible organ based on real-time image data, perform non-rigid registration, and update the triangular model; dynamically adjust the key parameters of the surgical path according to the updated triangular model to generate an optimal operation path; and output navigation data according to the optimal operation path.

[0009] In some embodiments, the image processing module includes:

[0010] The rigid registration unit performs a registration based on the preoperative image data; in the in-situ state, the characteristic points of the channel surface of the flexible organ are determined, and the characteristic points include the target point and the marking point and the entry point of the surface of the flexible organ;

[0011] The triangular model building unit builds a triangular model according to the entry point and the control parameters, the marking point and the target point, and associates the triangular model with the marking point and the entry point.

[0012] In some implementations, the navigation module further includes:

[0013] The non-rigid registration unit is used to perform secondary registration based on real-time image data and update the triangulated model, including:

[0014] Acquire first real-time image data to form a continuous triangle set in the same channel; the continuous triangle set is centered at the target point, and the angles of the triangles in the continuous triangle set are the same and the side lengths are within a preset range; the first real-time image data is real-time image data after the organ is freed;

[0015] Acquire second real-time image data, and extract a curved surface with the target point as the center and the distance between the target point and the entry point as the radius from the second real-time image; the second real-time image data is the real-time image data for performing a preset operation;

[0016] Mark the triangles where the blood vessels exist on each surface and update the triangular model;

[0017] A path adjustment unit, used to dynamically adjust key parameters of the surgical path according to the updated triangular model;

[0018] A path conversion unit is used to generate an optimal operation path according to the key parameters; the key parameters include an entry point and control parameters of the entry point, and the control parameters include an entry angle, a first side length, a second side length and / or a third side length of a triangular model.

[0019] In some embodiments, the non-rigid registration unit further comprises:

[0020] If the target point is located in an important pipeline, the key parameters of the surgical path are determined based on the preoperative imaging data, and real-time positioning measurement is performed through laser ranging and magnetic ultrasonic knife to determine the triangular position of the target point.

[0021] In some embodiments, the path adjustment unit dynamically adjusts key parameters of the surgical path according to the updated triangular model, and generates the optimal operation path including:

[0022] If the operation area is a regular area, the entry point is updated by the intersection line of the preset section and the curved surface, the section depth is determined by the real-time distance between the entry point and the target point, and the optimal operation path is adjusted.

[0023] In some embodiments, the path adjustment unit dynamically adjusts key parameters of the surgical path according to the updated triangular model, and generating the optimal operation path further comprises:

[0024] If the operation area is an irregular area, the entry points are determined in the transverse, sagittal and coronal planes of the operation area respectively, and non-rigid registration is performed according to the entry points and control parameters so that the real-time entry angle of each entry point is the same as the planned entry angle;

[0025] The path conversion unit generates an optimal operation path according to the entry point and the second side length of the triangular model.

[0026] In some embodiments, the rigid registration unit performs rigid registration of preoperative image data and real-time image data through an automatic rigid registration network; the automatic rigid registration network uses U-Net convolutional neural network to perform supervised deep learning to construct an AI registration model.

[0027] In some embodiments, rigid registration is performed based on preoperative image data, including:

[0028] Calibration is performed based on the calibration parameters of the entry point in the preoperative image data and the calibration parameters of the entry point in the electromagnetic coordinate system in the real-time image data.

[0029] According to a second aspect of the present invention, a flexible organ surgery robot is disclosed, comprising a flexible organ surgery navigation system and a robotic arm operation module as described above, wherein the robotic arm operation module comprises a robotic arm and an end effector tool; and is used to generate robotic arm control instructions according to the navigation data, and control the robotic arm and the end effector tool to perform predetermined operations.

[0030] In some embodiments, the surgical robot arm includes a scanning detection device, which is used to detect the working range of the robot arm operation module, determine the operation area, and monitor the operation process of the end-effector tool in real time; the end-effector tool includes a switching component and one or more of the following instruments: surgical energy instruments, forceps, tweezers, scissors, and closed pipe devices; the end-effector tool has a built-in sensor in the instrument.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention provides a flexible organ surgery navigation system and a flexible organ surgery robot. By constructing a triangular model and combining rigid and non-rigid registration, the surgical path is adjusted in real time to ensure that during flexible organ surgery, even if the organ is displaced and deformed, the surgical robot can accurately operate along the optimal path, avoid uncontrollable factors randomly generated during the operation, significantly improve the accuracy and safety of the operation, reduce damage to healthy tissues, reduce the risk of intraoperative complications, improve surgical efficiency, and improve the patient's postoperative recovery effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a schematic diagram of the flexible organ surgery navigation system provided by the present invention;

[0034] Figure 2 A schematic diagram of the automatic rigid registration network provided by the present invention;

[0035] Figure 3 A schematic diagram of constructing a triangular model provided by the present invention;

[0036] Figure 4 A schematic diagram of real-time updating of the triangular model provided by the present invention;

[0037] Figure 5 The non-rigid registration provided by the present invention is shown in FIG. Figure 1 ;

[0038] Figure 6 The non-rigid registration provided by the present invention is shown in FIG. Figure 2 ;

[0039] Figure 7 The non-rigid registration provided by the present invention is shown in FIG. Figure 3 ;

[0040] Figure 8 This is a schematic diagram of the flexible organ surgery robot provided by the present invention;

[0041] In the figure: 1. Robotic arm; 2. End effector. DETAILED DESCRIPTION

[0042] For better understanding and implementation, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products or apparatus.

[0044] The embodiment of the present invention discloses a flexible organ surgery navigation system, namely a flexible organ surgery robot, which realizes the fully automated operation of the surgery robot in flexible organ surgery.

[0045] like Figure 1 As shown, a flexible organ surgery navigation system is applied to a flexible organ surgery robot, including an image processing module and a navigation module. The image processing module is used to perform rigid registration based on preoperative image data to mark the feature points of the flexible organ; a triangular model is constructed based on the feature points, and key parameters of the surgical path are determined based on the triangular model; the navigation module is used to monitor the position change data of the flexible organ based on real-time image data, update the triangular model, dynamically adjust the key parameters of the surgical path based on the updated triangular model, and generate an optimal operation path; and the navigation data is output based on the optimal operation path.

[0046] Specifically, the image processing module is used to process the calibrated medical image data and mark the feature points for planning the surgical path. The medical image data may include preoperative three-dimensional image data obtained by CT (computed tomography) or MRI (magnetic resonance imaging) equipment.

[0047] The image processing module includes a rigid registration unit, which performs a registration based on the preoperative image data; in the in-situ state, the characteristic points of the channel surface of the flexible organ are determined, and the characteristic points include the target point and the marking point and the entry point of the flexible organ surface. Rigid registration is used to align the patient's three-dimensional image data, that is, the preoperative image data with the real-time image data during the operation, before the operation to establish the initial coordinate system of the operation.

[0048] In some embodiments, Figure 2As shown, the rigid registration unit is calibrated according to the calibration parameters of the entry point in the preoperative image data and the calibration parameters of the entry point in the electromagnetic coordinate system in the real-time image data. Specifically, the rigid registration of the preoperative image data and the real-time image data is performed by an automatic rigid registration network; the automatic rigid registration network uses U-Net convolutional neural network for supervised deep learning to construct an AI registration model. During the training process, 3,000 CT image data of pig livers are collected, and the coordinates of the pig liver in the electromagnetic coordinate system are used for data annotation to establish a standard data set. The U-Net convolutional neural network is used for supervised deep learning to construct an AI registration model, and the coordinates of 200 pig liver CTs are automatically marked based on the model to verify the degree of registration with the electromagnetic coordinate system. The constructed AI registration model is used to determine the specific coordinates of the pre-entry point b on the preoperative CT image in the intraoperative electromagnetic coordinate system, thereby completing the rigid registration and achieving the goal of corresponding the point b on the preoperative MR or CT image of the pig liver to the actual point b position of the pig liver in the intraoperative electromagnetic coordinate system.

[0049] The image processing module also includes a triangular model construction unit, which constructs a triangular model according to the entry point and control parameters, marking points and target points, and associates the triangular model with the marking points and entry points; the control parameters include the entry angle, the first side length, the second side length and the third side length of the triangular model. The path defined by the triangular model can be understood as a pipeline, such as a bronchus, bile duct, etc., which can be imaged in the pipeline to form a synthetic image set of several tunnels leading to the same target point. Different channels can be non-imaging areas, and different channels can be connected after the main channel is resected. In other words, a section may be composed of multiple curved surfaces, each of which can be replaced by a triangular model, which is equivalent to a channel.

[0050] After rigid registration, in the flexible organ in situ state, such as Figure 3 As shown, the feature points of each channel surface, such as the entry point b, the marking point a, and the target point c, are determined by the preoperative image data, and a triangular model is determined by the above feature points, and the triangle is associated with the marking point a and the entry point b marked on the surface of the flexible organ. The association can be achieved by calibrating the real-time image data with the preoperative image data and then performing feature point correspondence.

[0051] For example, the triangular model is constructed on the three-dimensional reconstructed image constructed by preoperative CT / MRI. The projection point of the largest cross-section of the liver tumor on the liver surface is selected and marked as the marker point a; then the deepest part of the tumor in the largest cross-section of the liver tumor is selected and marked as the target point c, which is usually the oncological benefit margin. ac The third side length can be measured by CT or MRI images before surgery. After determining the preset angle α, substitute the inverse sine function formula and the Pythagorean theorem to calculate the first side length Lab and the second side length L bc Among them, L ab Represents the distance from the liver surface entry point b to the marked point a, and the second side length L bc Represents the distance from the incision point b to the deepest target point c on the liver section. Angle α is the angle between the liver section and the liver surface at point b. The right triangle △bac is a fictitious right triangle with the largest section of the tumor as the center, ∠bac=90°∠abc=α. A circle of points b is formed on the liver surface, and a group of triangles formed at the same time form a "bowl" section covering the tumor.

[0052] After association, the organs are freed. After the organs are freed and displaced to the intended surgical position, since the position of the flexible organ has changed at this time, the position information of the feature points in the corresponding triangular model changes accordingly. In the present application, an ultrasonic knife with a built-in sensor is used to locate key points, and a laser rangefinder is used to measure the key point parameters of the triangular model in real time. The liver surgery navigation process is divided into two parts: the picture and the data. A laser rangefinder-assisted real-time electromagnetic liver surgery navigation system is constructed to output real-time measurement data, estimate the relative position information of the target anatomical point and the energy device used by the doctor on the triangular model construction plane, and assist the doctor in judging the remaining distance from the target point, thereby realizing data navigation. ICG fluorescence staining and augmented reality technology can provide picture navigation, in order to achieve synchronization of picture and data navigation.

[0053] Therefore, non-rigid registration is performed through the non-rigid registration unit of the navigation module. The non-rigid registration unit is used to perform secondary registration based on real-time image data and update the triangulated model, including:

[0054] Acquire first real-time image data to form a continuous triangle set in the same channel; the continuous triangle set is centered at the target point, and the angles of the triangles in the continuous triangle set are the same and the side lengths are within a preset range; the first real-time image data is real-time image data after the organ is freed;

[0055] Acquire second real-time image data, and extract a curved surface with the target point as the center and the distance between the target point and the entry point as the radius from the second real-time image; the second real-time image data is the real-time image data for performing a preset operation;

[0056] The triangles where the blood vessels exist on each surface are marked and the triangular model is updated.

[0057] In the secondary registration process, the triangular model constructed by the laser ranging assisted electromagnetic liver surgery navigation system is the underlying logic, providing accurate positioning for the secondary registration, that is, non-rigid registration. By measuring the key parameters of the triangular model, the problem of three-dimensional space is reduced to two-dimensional space, so as to realize the data navigation of real-time resection distance on the two-dimensional section. The laser ranging device used can be fixed on the surface of the flexible organ and can move with the liver, but always keep consistent with the section, avoid mutual interference in the positioning and data measurement of multiple key points at the same time, minimize the occupation of minimally invasive surgery poking hole resources, reduce the influence of heat source on magnetic positioning and magnetic field on surgical instruments, try to use wireless source targets, and merge wired source elements with instrument line sources. At the same time, it is convenient for real-time measurement, and is not affected by the deformation of three-dimensional space. At the same time, it is convenient to integrate with the resection surface determined by fluorescence, or assist augmented reality technology to determine the target pipeline that needs to be cut off on the section.

[0058] The electromagnetic surgical navigation system and laser ranging equipment integrate optics, electromagnetics, computers, stereotactic orientation, image processing, artificial intelligence and other technologies. The electromagnetic frame and precision measuring device used in surgery should cooperate with minimally invasive imaging, surgical instruments, display device layout, personnel division of labor and cooperation, data collection and analysis, and feedback, etc., to rationally layout related auxiliary equipment such as power supply, energy supply, data transmission and display of the navigation system, and ensure that the overall operation meets the standards and specifications of the operating room working environment.

[0059] In the present application, the laser distance measuring device includes a laser displacement sensor, a cylindrical base and a joint, a transmission rod, and a data screen. The laser displacement sensor emits two coaxial light beams L1 and L2 respectively, and by rotating the shaft, L1 and L2 form a certain angle α, and the laser displacement sensor is in an open state to accurately measure the angle.

[0060] The barrel base is 2 to 4 cm long and is in the shape of a hollow barrel. The roots of L1 and L2 are fixed in the base. When the bases L1 and L2 are combined and the angle α is 0°, they can be stored in the hollow barrel base, which is convenient for passing through the poking hole. There are two fixing holes on both sides of the barrel base, which are used to fix the base on the surface of the solid organ. According to the toughness of the organ substance, it can be fixed by suturing or clamping. The roots of L1 and L2 are connected to the transmission rod. Various external circuits of the laser displacement sensor are inside the rod, which are connected to the data screen outside the body through the transmission rod. The data screen is equipped with a rotating disk to adjust the angle between L1 and L2. The data screen displays the beam distance measurement of L1 and L2 and the real-time value of the angle α. The connecting rod is connected to the barrel base through the joint in the body. When the barrel base is fixed on the surface of the organ, due to the existence of the joint, the transmission rod can move freely with the displacement and flipping of the organ. The transmission rod is equipped with a locking button outside the body to lock and release the joint. The in vivo part of the device is finely crafted and can be inserted into the abdominal or thoracic cavity through a 12cm diameter poking card.

[0061] The electromagnetic tracking is used to assist the positioning of the triangular model for real-time secondary registration and update the triangular model. An external electromagnetic field is constructed in the operation area so that the operation area forms a 3D coordinate system, and non-rigid registration is performed through the electromagnetic field.

[0062] like Figure 4 , 5 , 6, 7, in the electromagnetic field, determine the real-time position information of the preset first target, the real-time position information of the second target and the real-time position information of the third target; calculate and generate the best operation path of triangulation positioning through the real-time position information of the first target, the real-time position information of the second target and the real-time position information of the third target; wherein, the first target, that is, the base point target, is used to determine the positioning information of the fixed point in the surgical area to be navigated, that is, point a; the second target, that is, the operation target, is installed in the working part of the operating instrument such as ultrasonic knife, CUSA, etc., and is used to determine the real-time moving position information of the operating instrument in the surgical area to be navigated; the third target, that is, the detection target, is used for the position of the entry point b, so as to determine the real-time angle information of triangulation positioning in the surgical area to be navigated, that is, the entry angle. Thereby, deep tumors and large blood vessels can be avoided in real time and accurately, the correct surgical section and sufficient tumor cutting margin can be ensured, and the efficiency and accuracy of the operation can be improved.

[0063] Exemplarily, a sensor is arranged in the blade head of the ultrasonic scalpel, and the sensor is movably arranged in the splint of the blade head. The sensor is located on the rotatable splint and can cater to the positioning area. At the same time, the surface of the first splint is covered with an insulating layer. It is used to protect the sensor from heat and current interference. An in vitro adjustment device is arranged at the handle of the ultrasonic scalpel. The doctor can control the adjustment device in vitro to adjust the position of the sensor on the blade head. The power cord of the sensor and the linkage line for adjusting the position of the sensor are arranged in the blade body and connected to the handle and the host. The sensor is connected to the sensor receiving unit, which transmits the signal to the ultrasonic scalpel control unit and then to the host. During the operation of the scalpel head in the electromagnetic field, the sensor sends a signal to capture the position information of the scalpel head in the magnetic field generated by the magnetic field generator, thereby presenting the position of the scalpel head visually.

[0064] During the operation, L1 and L2 of the laser distance measuring device represent the distance between the marked point and the entry point in the triangulated model and the distance between the entry point and the target point, respectively. c' is defined as the deepest point where the tool is cut into the end of the operation area, that is, any point from the entry point b to the point c. The measured value of L1 is the length of the first side L ab , the measurement value of L2 is the second side length L bc’ The base point positioning target is fixed at the entry point b, and the operation tracking target is used to track the movement of the liver cutting instrument such as the ultrasonic knife in the L2 direction, so as to determine the real-time entry depth distance L bc’, represents the real-time section depth. In this process, a continuous triangle set is formed in the same channel, with the target point c as the center. The cutting angles in the triangle set are all the planned cutting angles α, and the second and third side lengths are within the preset lengths.

[0065] After the rigid registration unit completes the rigid registration in the electromagnetic field, it determines the preset section of the flexible organ through ultrasound or other preoperative imaging data, that is, the coordinates of the projection point a of the maximum section of the tumor on the liver surface in the transverse, sagittal and coronal planes, to determine the marking point a. The key parameters of the entry point b corresponding to the preset section are calculated based on the preoperative imaging data, namely the first side length, the second side length and the third side length as well as the entry angle α. The non-rigid registration unit determines the first side length L of the triangular model through the first target, the second target and the third target respectively from the entry point b in the transverse, sagittal and coronal planes. ab , the second side length L bc’ and the length of the third side L ac’ , where the second side length is L bc’ With the third side length L ac’ is the real-time depth determined by real-time image data. On this basis, the incision angle is calculated by the arc cosine function:

[0066] cosα=(L ab 2 +L bc’ 2 -L ac’ 2 ) / 2L ab L bc’

[0067] The real-time incision angle is compared with the planned incision angle of the preoperative plan, and the real-time incision angle is adjusted to make the real-time incision angle the same as the planned incision angle α. All incision points on the transverse, sagittal and coronal planes form the incision plane planned before the operation, and the second side length L is determined on the incision plane by using the ultrasonic knife and laser ranging equipment. bc’ In actual operation, no matter how the flexible organ deforms, the relative measurement value of the distance between the entry point b on the preset section and the deepest point c' on the section is limited by the displacement deformation, and the measurement process is controllable and has little impact on the surgical process. bc’ It can be used as a relatively constant determining factor in the surgical navigation system to calculate the second side length L in real time. bc’ and other key parameters.

[0068] Furthermore, non-rigid registration can be assisted by a laser rangefinder. A laser rangefinder is fixed at the entry point b. The L1 and L2 laser displacement sensors of the device point from point b to ba and bc respectively, and their included angles are set to the angle α corresponding to point b before the operation. When the point b is resected to angle α, the laser emitted from this point can be presented on the cut surface. bc’ The laser beam emitted by laser L2 can be projected to the cutting point where the ultrasonic knife is located, and real-time measurement can be performed until L bc’ =L bc .

[0069] Non-rigid registration by electromagnetic field or laser ranging equipment complements and verifies each other. When the section deformation is large, the beam of the laser ranging equipment may be affected to a certain extent, resulting in deviations in measurement accuracy, which requires verification by electromagnetic coordinate measurement. When the patient's position needs to change frequently, the electromagnetic coordinates may be displaced, and laser ranging can correct the deviation. Therefore, the combination of the two methods can most accurately measure the position information data of the key points of the triangulated model, and implement precise navigation according to the preoperative plan.

[0070] The non-rigid registration unit also includes: if the target point is located in an important pipeline, the key parameters of the surgical path are determined according to the preoperative imaging data, and the triangle position where the target point is located is determined by real-time positioning measurement through laser ranging and magnetic ultrasonic knife. There are areas such as blood vessels in flexible organs that require special attention, which need to be avoided or other operations need to be performed during the operation. Therefore, when the target point is located in an important channel, the triangle where the large blood vessels that may be encountered on each curved surface are marked accordingly. When cutting into the channel, it is necessary to determine the triangle where the blood vessel is located in advance, so as to accurately locate the blood vessel.

[0071] The navigation module is also used to dynamically adjust the key parameters of the surgical path according to the triangular model obtained by the image processing module to generate an optimal operation path; output navigation data according to the optimal operation path, including a path adjustment unit and a path conversion unit, the path adjustment unit is used to dynamically adjust the key parameters of the surgical path according to the updated triangular model to generate an optimal operation path; the path conversion unit is used to generate an optimal operation path according to the key parameters; the key parameters include the entry point and the control parameters of the entry point, and the control parameters include the entry angle, the first side length, the second side length and / or the third side length of the triangular model.

[0072] The key parameters of each triangle model include 4 parameters, forming a data channel, which forms an imaging point in the preoperative imaging data, and then selects N triangles to form a three-dimensional data channel, simulating N virtual data imaging channels. During the operation, the imaging data formed by each channel is converted into navigation data to guide the ultrasonic knife with real-time positioning function into each imaging channel to complete the navigation process.

[0073] In the actual operation process, if the operation area is a regular area, the entry point is updated by the intersection line of the preset section and the curved surface, and the section depth is determined by the real-time distance between the entry point and the target point, and the optimal operation position is adjusted. Through the preoperative imaging data, the point b determined by the intersection of the section determined by the liver watershed analysis (such as fluorescent staining or ischemic area) and the section determined by the triangular model can be pre-resected as the entry point. On the section determined by fluorescence, if the target point c is determined as the target blood vessel, the navigation module path adjustment unit can calculate the depth of incision required to reach the target cut-off blood vessel on the fluorescent section. That is, on this plane, L bc’ Represents the real-time distance required to be resected when encountering the target blood vessel. This distance represents the relative distance between two points on the section and is less affected by the overall displacement and deformation of the liver.

[0074] If the operation area is an irregular area, the entry points are determined in the cross section, sagittal plane and coronal plane of the operation area respectively, and non-rigid registration is performed according to the entry points and control parameters so that the real-time entry angle of each entry point is the same as the planned entry angle; the optimal operation path is generated according to the entry point and the second side length of the triangular model.

[0075] After the rigid registration unit completes the rigid registration, the entry operation is performed according to the control parameters of the entry point, and the non-rigid registration unit performs non-rigid registration until the actual entry angle is the same as the real-time entry angle and the planned entry angle α. The second side length L is cut according to the entry point and control parameters determined by the rigid registration unit. bc During the cut, the second side length L bc’ It is real-time navigation data. Cut in at the entry point of the preset section to form a second side length L with the target point as the center. bc On the curved surface, regardless of the random displacement and deformation of the flexible organ, the real-time navigation data is approximately the penetration depth.

[0076] The path conversion unit generates an optimal operation path according to the entry point, the second side length of the triangular model and the real-time navigation data, and outputs navigation data according to the optimal operation path to control the robot arm operation module to generate robot arm control instructions according to the navigation data, and control the robot arm and the end execution tool to perform predetermined operations.

[0077] Based on the same inventive idea, such as Figure 8As shown, the present application also provides a flexible organ surgery robot equipped with the above-mentioned operating system and a robotic arm operation module. The robotic arm operation module includes a robotic arm 1, an end effector 2, and a scanning detection tool. The robotic arm uses joints with a certain degree of freedom to form the main body of the robot, and controls the joints and the end effector to complete the specified operation according to the robotic arm control instructions. The scanning detection tool is used to detect the working range of the robotic arm operation module, determine the operation area, and monitor the entire process of the end effector operation in real time to perform quality control of the entire process. When it is found that there is an error between the operation area and the operation area of ​​the end effector, or when the end effector is operated improperly, timely feedback is provided.

[0078] When the flexible organ surgery robot is operating, any external factors, including patient twitching and similar activities caused by factors such as patient breathing and the depth of anesthesia, which cause the patient's body position to vibrate to a certain extent, as well as deviations of surgical instruments in the magnetic field and magnetic field interference in the external environment, will cause certain interference. The scanning detection tool also monitors the external environment and conducts real-time verification when external interference is found, so as to detect deviations and calibrate them in time. Real-time scanning can detect these abnormalities in time and provide timely feedback. The triangulation model can be updated in time by manual intervention or the image processing module, and the navigation module can dynamically adjust the navigation data.

[0079] The end effector 2 can be fixed to the end of the robot arm 1, including a conversion component and surgical energy instruments, forceps, tweezers, scissors, closed pipe devices, etc., which are required to complete incision, hemostasis, clamping, suturing and other operations. The switching of the instrument is completed by the switching component. The instrument of the end effector has a built-in sensor. After the switching component completes the instrument switching, the instrument is located at the specified working position, the built-in sensor is turned on, and the image processing module and the navigation module can receive the signal from the built-in sensor of the instrument in real time to determine the location information. In addition, the end effector can also be provided with a trigger feedback component, which builds a feedback mechanism through visual, tactile, pressure, temperature and other measurement sensors, and judges or triggers the simulation of the surgical tool through the AI ​​model.

[0080] The present invention provides a flexible organ surgery navigation system and a flexible organ surgery robot. By constructing a triangular model and combining rigid and non-rigid registration, the surgical path is adjusted in real time to ensure that during flexible organ surgery, even if the organ is displaced and deformed, the surgical robot can accurately operate along the optimal path, avoid uncontrollable factors randomly generated during the operation, significantly improve the accuracy and safety of the operation, reduce damage to healthy tissues, reduce the risk of intraoperative complications, improve surgical efficiency, and improve the patient's postoperative recovery effect.

[0081] The embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without paying creative labor.

[0082] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0083] Finally, it should be noted that the embodiments disclosed in the present invention are only preferred embodiments of the present invention, which are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A flexible organ surgery navigation system, characterized in that: include: An image processing module is used to perform rigid registration based on preoperative image data and mark feature points of flexible organs; constructing a triangular model according to the characteristic points, and determining key parameters of the surgical path based on the triangular model; The navigation module is used to monitor the position change of the flexible organ based on real-time image data, perform non-rigid registration, and update the triangular model; dynamically adjust the key parameters of the surgical path according to the updated triangular model to generate an optimal operation path; and output navigation data according to the optimal operation path.

2. A flexible organ surgery navigation system according to claim 1, characterized in that: The image processing module comprises: The rigid registration unit performs a registration based on the preoperative image data; in the in-situ state, the characteristic points of the channel surface of the flexible organ are determined, and the characteristic points include the target point and the marking point and the entry point of the surface of the flexible organ; The triangular model building unit builds a triangular model according to the entry point and the control parameters, the marking point and the target point, and associates the triangular model with the marking point and the entry point.

3. A flexible organ surgery navigation system according to claim 2, characterized in that: The navigation module also includes: The non-rigid registration unit is used to perform secondary registration based on real-time image data and update the triangulated model, including: Acquire first real-time image data to form a continuous triangle set in the same channel; the continuous triangle set is centered at the target point, and the angles of the triangles in the continuous triangle set are the same and the side lengths are within a preset range; the first real-time image data is real-time image data after the organ is freed; Acquire second real-time image data, and extract a curved surface with the target point as the center and the distance between the target point and the entry point as the radius from the second real-time image; the second real-time image data is the real-time image data for performing a preset operation; Mark the triangles where the blood vessels exist on each surface and update the triangular model; A path adjustment unit, used to dynamically adjust key parameters of the surgical path according to the updated triangular model; A path conversion unit is used to generate an optimal operation path according to the key parameters; the key parameters include an entry point and control parameters of the entry point, and the control parameters include an entry angle, a first side length, a second side length and / or a third side length of a triangular model.

4. The flexible organ surgery navigation system according to claim 3, characterized in that: The non-rigid registration unit also includes: If the target point is located in an important pipeline, the key parameters of the surgical path are determined based on the preoperative imaging data, and real-time positioning measurement is performed through laser ranging and magnetic ultrasonic knife to determine the triangular position of the target point.

5. The flexible organ surgery navigation system according to claim 3, characterized in that: The path adjustment unit dynamically adjusts key parameters of the surgical path according to the updated triangular model to generate the optimal operation path, including: If the operation area is a regular area, the entry point is updated by the intersection line of the preset section and the curved surface, the section depth is determined by the real-time distance between the entry point and the target point, and the optimal operation path is adjusted.

6. The flexible organ surgery navigation system according to claim 5, characterized in that: The path adjustment unit dynamically adjusts key parameters of the surgical path according to the updated triangular model, and generates the optimal operation path further comprising: If the operation area is an irregular area, the entry points are determined in the transverse, sagittal and coronal planes of the operation area respectively, and non-rigid registration is performed according to the entry points and control parameters so that the real-time entry angle of each entry point is the same as the planned entry angle; The path conversion unit generates an optimal operation path according to the entry point and the second side length of the triangular model.

7. The flexible organ surgery navigation system according to claim 2, characterized in that: The rigid registration unit performs rigid registration of preoperative image data and real-time image data through an automatic rigid registration network; the automatic rigid registration network uses U-Net convolutional neural network to perform supervised deep learning to build an AI registration model.

8. The flexible organ surgery navigation system according to claim 2, characterized in that: Rigid registration is performed based on preoperative imaging data, including: Calibration is performed based on the calibration parameters of the entry point in the preoperative image data and the calibration parameters of the entry point in the electromagnetic coordinate system in the real-time image data.

9. A flexible organ surgery robot, characterized in that: It comprises a flexible organ surgery navigation system and a robotic arm operation module as described in any one of claims 1 to 8, wherein the robotic arm operation module comprises a robotic arm and an end effector tool; and is used to generate robotic arm control instructions according to the navigation data to control the robotic arm and the end effector tool to perform predetermined operations.

10. The flexible organ surgery robot according to claim 9, characterized in that: The robotic arm includes a scanning detection device, which is used to detect the working range of the robotic arm operation module, determine the operation area, and monitor the operation process of the end-effector tool in real time; the end-effector tool includes a switching component and one or more of the following instruments: surgical energy instruments, forceps, tweezers, scissors, and closed pipe devices; the end-effector tool has a built-in sensor in the instrument.

Images