Robot surgical navigation system and method based on soft tissue deformation model real-time registration

Abstract

The invention provides a robot surgical navigation system and method based on soft tissue deformation model real-time registration, and relates to the technical field of medical robots, the system comprises an imaging device, a display device and a control device; the control device comprises a model management module used for acquiring a pre-constructed three-dimensional soft tissue deformation model associated with a target tissue; the feature extraction module is used for processing the real-time image and extracting surface contour features and deformation mechanical correlation features of the target tissue; the real-time registration module is used for performing real-time registration on the target tissue and the three-dimensional soft tissue deformation model; and the navigation generation module is used for fusing the registered three-dimensional soft tissue deformation model with a real-time image. According to the method, real deformation of the tissue in the operation can be adapted in real time through deformation driving of the three-dimensional soft tissue deformation model, high-precision registration of the three-dimensional soft tissue deformation model and the target tissue is achieved, and the registration relation can be kept stable and continuous when the tissue state or the imaging view changes.

Description

Technical Field

[0002] This invention relates to the technical field of medical robots, and in particular to a robotic surgical navigation system and method based on real-time registration of soft tissue deformation models. Background Technology

[0004] In computer-assisted surgery, augmented reality (AR) technology has been gradually introduced into clinical applications to overcome the disconnect between traditional imaging examinations and real-time intraoperative vision. This technology aims to fuse and overlay preoperative or intraoperative 3D virtual models with real-time surgical field video, thereby helping doctors "see through" the tissue surface and intuitively observe key anatomical structures underneath, such as blood vessels, nerves, or tumors, to improve the accuracy and safety of the surgery.

[0005] However, existing AR-assisted surgical systems still have some problems in practical applications. On the one hand, soft tissues are easily deformed by external forces such as traction and compression during surgery, while traditional registration methods are mostly based on rigid or semi-rigid models, which are difficult to truly reflect and dynamically adapt to the non-rigid deformation characteristics of soft tissues. This results in the inability to achieve accurate and adaptive registration between the virtual model and the actual anatomical structure during surgery. On the other hand, during surgical operations, the position of the imaging device often changes dynamically due to factors such as the doctor's operation and changes in the patient's position. The established registration relationship will become invalid, and traditional methods usually require re-registration. It is difficult to achieve continuous, stable real-time registration and accurate superposition in dynamic surgical scenarios, which seriously affects the continuity and reliability of surgical guidance. Summary of the Invention

[0007] The purpose of this application is to provide a robotic surgical navigation system and method based on real-time registration of a soft tissue deformation model. It can adapt to the actual deformation of the tissue during surgery in real time through the deformation drive of the three-dimensional soft tissue deformation model, and achieve high-precision registration between the three-dimensional soft tissue deformation model and the target tissue. It can maintain the stability and continuity of the registration relationship when the tissue state or imaging field of view changes, thereby improving the stability, accuracy and practicality of intraoperative guidance.

[0008] In a first aspect, embodiments of this application provide a robotic surgical navigation system based on real-time registration of a soft tissue deformation model, comprising: Imaging device for acquiring real-time images containing target tissue; A display device is used to present a navigation screen to the operator; The control device is communicatively connected to both the imaging device and the display device. The control device includes: The model management module is used to acquire a pre-built three-dimensional soft tissue deformation model associated with the target tissue, the three-dimensional soft tissue deformation model being able to deform in response to the action of a virtual driving force; The feature extraction module is used to process the real-time image and extract the surface contour features and deformation mechanical correlation features of the target tissue; The real-time registration module is used to calculate the deformation difference between the three-dimensional soft tissue deformation model and the target tissue based on the surface contour features and deformation mechanical correlation features, map the deformation difference into a virtual driving force applied to the three-dimensional soft tissue deformation model, and control the three-dimensional soft tissue deformation model to deform under the action of the virtual driving force, so that the target tissue and the three-dimensional soft tissue deformation model can be registered in real time. The navigation generation module is used to fuse the registered three-dimensional soft tissue deformation model with the real-time image to generate a navigation screen and output it to the display device.

[0009] Furthermore, the navigation system also includes a preoperative planning module for constructing the three-dimensional soft tissue deformation model based on preoperative medical imaging data; The preoperative planning module includes: A geometric modeling unit is used to reconstruct a three-dimensional geometric model of the target tissue based on the preoperative medical imaging data. The mechanical property assignment unit is used to assign biomechanical properties that characterize tissue elasticity to the three-dimensional geometric model in order to obtain the three-dimensional soft tissue deformation model.

[0010] Furthermore, the geometric modeling unit includes: The image acquisition subunit is used to acquire preoperative CT and MRI image data of the target object. The image segmentation and fusion subunit is used to segment the preoperative CT image data and the preoperative MRI image data to obtain their respective tissue contour information, and to fuse the tissue contour information of the two to generate fused image data containing details of the target tissue anatomical structure. The three-dimensional reconstruction subunit is used to reconstruct the three-dimensional geometric model of the target tissue based on the fused image data.

[0011] Furthermore, the mechanical property endowing unit is configured to endow the three-dimensional geometric model with biomechanical properties by constructing a particle-fiber bundle network model; The mechanical property-assigning unit includes: The particle system generation subunit is used to determine the soft tissue region and the hard tissue region based on the anatomical structural features of the three-dimensional geometric model, and to distribute particles in each region to generate a particle system; wherein, the particle distribution density in the hard tissue region is greater than that in the soft tissue region, and the mass of each particle is set according to the particle distribution density of its region. A fiber bundle network construction subunit is used to establish elastic connections between adjacent particles in the particle system to form a fiber bundle network; wherein the mechanical parameters of the elastic connection are set according to the biomechanical properties of the regions where the particles at both ends are located.

[0012] Furthermore, the mechanical property imparting unit also includes: An image processing subunit is used to process the fused image data to extract the anatomical structural features of the target tissue; The processing of the fused image data includes noise reduction and contrast enhancement.

[0013] Furthermore, the fiber bundle network construction subunit is also configured as follows: Based on the anatomical structure characteristics of the target tissue, fiber bundles are arranged in the fiber bundle network along the anatomical direction of muscle fibers or collagen fibers to simulate the anisotropic mechanical properties of the target tissue.

[0014] Furthermore, the feature extraction module includes: The surface contour extraction unit is used to extract the surface contour information of the target tissue from the real-time image using a dual-gradient contour extraction algorithm under anatomical constraints.

[0015] Furthermore, the feature extraction module also includes: The gray-level gradient analysis unit is used to perform semantic segmentation on the real-time image, obtain a mask image of the target tissue, and calculate the gray-level gradient of the surface contour region of the target tissue based on the mask image. The mechanical feature extraction unit is used to generate the deformation mechanical correlation features that characterize the local mechanical response of the target tissue based on the gray-level gradient of the gray value and the edge detection algorithm that integrates anatomical constraints.

[0016] Secondly, embodiments of this application provide a method for a robotic surgical navigation system based on real-time registration of a soft tissue deformation model, comprising: Obtain a pre-constructed three-dimensional soft tissue deformation model associated with the target tissue, the three-dimensional soft tissue deformation model being able to deform in response to the action of a virtual driving force; The real-time image is processed to extract the surface contour features and deformation mechanical correlation features of the target tissue; Based on the surface contour features and deformation mechanics correlation features, the deformation difference between the three-dimensional soft tissue deformation model and the target tissue is calculated, the deformation difference is mapped to a virtual driving force applied to the three-dimensional soft tissue deformation model, and the three-dimensional soft tissue deformation model is controlled to deform under the action of the virtual driving force so that the target tissue and the three-dimensional soft tissue deformation model can be registered in real time. The registered three-dimensional soft tissue deformation model is fused with the real-time image to generate a navigation screen, which is then output to the display device.

[0017] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the second aspect above.

[0018] The robotic surgical navigation system and method based on real-time registration of soft tissue deformation models provided in this application have at least the following beneficial effects: Compared with existing technologies, this invention is based on a three-dimensional soft tissue deformation model that can deform in response to virtual driving forces. The feature extraction module extracts surface contour features and deformation mechanics correlation features from real-time images, and the real-time registration module calculates deformation differences and maps virtual driving forces to drive model deformation based on these features. This achieves high-precision dynamic adaptation of real deformation of soft tissue under external forces (traction, compression, etc.) during surgery, solves the registration error problem caused by the inability of traditional rigid models to match dynamic tissue deformation, and significantly improves registration accuracy.

[0019] Furthermore, by continuously performing deformation difference calculation, driving force mapping, and model deformation control through the real-time registration module, it is possible to continuously complete the real-time registration between the three-dimensional soft tissue deformation model and the target tissue when the soft tissue state or imaging field of view changes, and update the registration status in a timely manner. This effectively avoids the registration failure caused by dynamic changes in the scene in traditional methods, thereby ensuring the continuity and stability of surgical navigation.

[0020] Furthermore, by fusing the registered 3D soft tissue deformation model with real-time images through the navigation generation module, a navigation screen can be generated, which can intuitively present an augmented reality view with a "perspective" effect on the display device, clearly revealing the key anatomical structures under the surface tissue, providing doctors with spatial information that cannot be directly obtained by traditional vision, thereby significantly improving the accuracy of intraoperative positioning and decision-making and the safety of surgical procedures. Attached Figure Description

[0022] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 A schematic diagram of the structure of a robotic surgical navigation system based on real-time registration of a soft tissue deformation model, provided in an embodiment of this application; Figure 2 This is a block diagram of the control device. Figure 3 A flowchart of a robotic surgical navigation method based on real-time registration of a soft tissue deformation model, provided in an embodiment of this application.

[0024] icon: 100 - Display device; 200 - Imaging device; 300 - Control device; 310 - Model management module; 320 - Feature extraction module; 330 - Real-time registration module; 340 - Navigation generation module. Detailed Implementation

[0026] The technical solutions of this application will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] This embodiment provides a robotic surgical navigation system based on real-time registration of a soft tissue deformation model, such as... Figure 1 and Figure 2 As shown, the system includes: Imaging device 200 is used to acquire real-time images containing target tissue; Display device 100 is used to present a navigation screen to the operator; The control device 300 is communicatively connected to the imaging device 200 and the display device 100, respectively. The control device 300 includes: The model management module 310 is used to acquire a pre-built three-dimensional soft tissue deformation model associated with the target tissue. The three-dimensional soft tissue deformation model can deform in response to the action of virtual driving force. The feature extraction module 320 is used to process real-time images and extract surface contour features and deformation mechanical correlation features of the target tissue; The real-time registration module 330 is used to calculate the deformation difference between the three-dimensional soft tissue deformation model and the target tissue based on the surface contour features and deformation mechanics correlation features, map the deformation difference into a virtual driving force applied to the three-dimensional soft tissue deformation model, and control the three-dimensional soft tissue deformation model to deform under the action of the virtual driving force, so as to enable the target tissue and the three-dimensional soft tissue deformation model to be registered in real time. The navigation generation module 340 is used to fuse the registered three-dimensional soft tissue deformation model with real-time images to generate a navigation screen and output it to the display device 100.

[0028] Compared with existing technologies, this invention is based on a three-dimensional soft tissue deformation model that can deform in response to virtual driving forces. The feature extraction module 320 extracts surface contour features and deformation mechanics correlation features from real-time images, and the real-time registration module 330 calculates deformation differences and maps virtual driving forces to drive model deformation based on these features. This achieves high-precision dynamic adaptation of the real deformation of soft tissue under external forces (traction, compression, etc.) during surgery, solves the registration error problem caused by the inability of traditional rigid models to match dynamic tissue deformation, and significantly improves registration accuracy.

[0029] Furthermore, by continuously performing deformation difference calculation, driving force mapping and model deformation control through the real-time registration module 330, it is possible to continuously complete the real-time registration between the three-dimensional soft tissue deformation model and the target tissue when the soft tissue state or imaging field of view changes, and update the registration status in a timely manner. This effectively avoids the registration failure caused by dynamic changes in the scene in traditional methods, thereby ensuring the continuity and stability of surgical navigation.

[0030] Furthermore, the navigation generation module 340 fuses the registered 3D soft tissue deformation model with real-time images to generate a navigation screen. This screen can be intuitively presented on the display device 100 with an augmented reality view that provides a "perspective" effect, clearly revealing key anatomical structures beneath the surface tissue. This provides surgeons with spatial information that cannot be directly obtained through traditional vision, thereby significantly improving the accuracy of intraoperative positioning and decision-making, as well as the safety of surgical procedures. In addition, the 3D soft tissue deformation model used in this invention is constructed based on physical principles, and its deformation response mechanism is related to the mechanical properties of the tissue. This gives the system framework good versatility and scalability, making it easy to adapt to the navigation needs of soft tissue surgeries in different locations.

[0031] For ease of explanation, the following will be used as follows: Figure 1 The robotic surgical navigation system based on real-time registration of a soft tissue deformation model is used as an example for illustration.

[0032] Figure 1This is a schematic diagram of the structure of a robotic surgical navigation system based on real-time registration of a soft tissue deformation model, as provided in an embodiment of this application. The robotic surgical navigation system includes a doctor's console, a patient's surgical platform, and a control device 300 that enables communication between the two.

[0033] The doctor's console serves as the main control unit, providing users with an immersive operating environment. The robotic master hand, acting as an interactive device, is a multi-degree-of-freedom control handle used to manipulate surgical instruments on the patient's surgical platform. The doctor's console also integrates a display device 100 to present navigation information to the operator, assisting the doctor in observation and decision-making.

[0034] The patient surgical platform, serving as the system's execution end, is deployed in the surgical area to directly perform procedures on the patient. This platform primarily comprises the following components: A robotic arm system, comprising one or more high-precision robotic arms, with specialized surgical instruments at their ends, is capable of performing delicate surgical operations such as grasping, cutting, and suturing.

[0035] The imaging device 200 is typically mounted on a separate robotic arm, whose end effector can be flexibly positioned and inserted into the patient's body to continuously acquire high-definition real-time images or video streams containing target tissues (such as lesion areas, blood vessels, or nerves). The image data acquired by the imaging device 200 is transmitted in real time to the control device 300 for processing and is ultimately presented on the display device 100 in the doctor's console, providing the doctor with a real-time intraoperative view.

[0036] The control device 300 maintains a real-time connection with the various components of the doctor's console and the patient's surgical platform via wired communication, and is responsible for performing data processing, logical operations and collaborative control tasks of the system.

[0037] The aforementioned functions of the control device 300 are achieved through a combination of internally cooperating software and hardware modules. The specific functions of each module are as follows: The model management module 310 is used to acquire a pre-built three-dimensional soft tissue deformation model associated with the target tissue, which is capable of deforming in response to virtual driving forces.

[0038] Specifically, the model management module 310 is responsible for loading and dynamically calling three-dimensional soft tissue deformation models. In actual implementation, this module can be configured to call the corresponding three-dimensional soft tissue deformation model from the pre-stored model library based on the target tissue type of the current surgery (such as liver, kidney, brain tissue or gastrointestinal tract).

[0039] Through the above settings, the model management module 310 provides a computational foundation for subsequent real-time registration by calling a pre-constructed three-dimensional soft tissue deformation model that matches the target tissue type and possesses biomechanical response characteristics. This model accurately simulates the dynamic deformation of tissues during surgery. The dynamic loading and calling mechanism implemented by this module ensures that an accurate geometric and biomechanical correspondence can be quickly established between the virtual model and the real tissue during surgery, thereby fundamentally solving the systematic registration error caused by the inability of traditional rigid registration methods to adapt to the dynamic deformation of soft tissues. Based on this key model foundation, the entire navigation system can achieve high-precision real-time registration and continuous stable tracking during surgery.

[0040] In practical applications, three-dimensional soft tissue deformation models can be pre-constructed using the system's built-in preoperative planning module. Based on the patient's preoperative medical imaging data (such as CT, MRI, etc.), the preoperative planning module first reconstructs a three-dimensional geometric model of the target tissue using a geometric modeling unit, and then assigns biomechanical properties that characterize tissue elasticity to this geometric model using a mechanical property assignment unit, thereby forming a three-dimensional soft tissue deformation model that can respond to virtual driving forces.

[0041] By integrating a preoperative planning module, the system can complete high-precision geometric reconstruction and physical property modeling of the target tissue in the preoperative stage, thereby providing a deformation model that highly matches the patient's individual anatomical structure and tissue mechanical properties for real-time intraoperative registration. This ensures the realism and accuracy of the model in deformation simulation, providing a physical basis for overcoming the errors of traditional rigid registration; on the other hand, it also realizes end-to-end data integration from preoperative planning and model construction to intraoperative registration and navigation, improving the system's overall integrity and clinical applicability.

[0042] More specifically, a preferred embodiment of the geometric modeling unit includes the following cooperating sub-units: The image acquisition subunit is used to acquire preoperative CT and MRI image data of the target object; the image segmentation and fusion subunit is used to segment and process the CT and MRI image data respectively, extract their respective tissue contour information, and then register and fuse the contour information from different modalities to generate a fused image data that integrates the high spatial resolution of CT images and the excellent soft tissue contrast of MRI images; the 3D reconstruction subunit is used to reconstruct a 3D geometric model of the target tissue with fine anatomical details based on the fused image data through surface rendering or volume rendering algorithms.

[0043] By integrating the complementary advantages of CT and MRI images, the generated three-dimensional geometric model can significantly improve the identification accuracy of different soft tissue interfaces (such as organ boundaries and blood vessel walls) while maintaining accurate anatomical structures, thus providing a highly realistic geometric model foundation for the accurate assignment of subsequent biomechanical properties. Secondly, the aforementioned modeling process reduces the dependence on manual contour drawing, improves the efficiency of model construction and the consistency of results, and thus ensures the standardization and reliability of data transfer from preoperative images to intraoperative navigation models.

[0044] Furthermore, the mechanical property endowing unit is configured to endow the three-dimensional geometric model with biomechanical properties by constructing a particle-fiber bundle network model. The mechanical property endowing unit includes: a particle system generation subunit, used to determine soft tissue regions and hard tissue regions according to the anatomical structural features of the three-dimensional geometric model, and to distribute particles in each region to generate a particle system; wherein the particle distribution density in the hard tissue region is greater than that in the soft tissue region, and the mass of each particle is set according to the particle distribution density of its region; and a fiber bundle network construction subunit, used to establish elastic connections between adjacent particles in the particle system to form a fiber bundle network; wherein the mechanical parameters of the elastic connection are set according to the biomechanical properties of the regions where the particles at both ends are located.

[0045] Furthermore, based on the anatomical features of the target tissue, fiber bundles can be placed in the fiber bundle network along the anatomical direction of muscle fibers or collagen fibers. By directionally placing fiber bundles along the anatomical direction, the model can effectively simulate the anisotropic mechanical properties of soft tissue, making the deformation behavior of the three-dimensional soft tissue deformation model highly consistent with the mechanical performance of real tissue when responding to complex surgical operations (such as traction and shearing), thus providing a highly realistic mechanical simulation basis for real-time intraoperative registration.

[0046] By distinguishing between soft and hard tissue regions and differentiating particle density and mass, the model can more realistically reflect the differences in the response of different tissues under stress, thus providing a biophysical foundation for overall deformation simulation at the micromechanical level. Furthermore, by constructing a fiber bundle network through elastic connections and setting corresponding parameters based on the biomechanical characteristics of the connected regions, a highly efficient discretized simulation of the mechanical behavior of tissue continuum is achieved. This significantly improves the physical realism and numerical stability of the deformation simulation while ensuring computational efficiency.

[0047] Furthermore, the mechanical property imparting unit also includes an image processing subunit, used to process the fused image data to extract the anatomical structural features of the target tissue; wherein, the processing of the fused image data includes noise reduction processing and contrast enhancement processing.

[0048] By preprocessing the fused image data with noise reduction and contrast enhancement, the system can extract the anatomical details of the target tissue (such as tissue layers, internal texture, and microvascular orientation) more clearly and stably. On the one hand, it provides a more reliable image basis for the regional division of the particle system (soft tissue region and hard tissue region) and the directional construction of fiber bundle networks, thereby improving the accuracy of the anatomical correspondence given by mechanical properties. On the other hand, this preprocessing enhances the system's adaptability to fluctuations in the quality of the original image, ensuring the consistency and reliability of the mechanical model construction under different imaging conditions.

[0049] In this embodiment, the feature extraction module 320 includes a surface contour extraction unit, which is used to extract the surface contour information of the target tissue from the real-time image using a dual-gradient contour extraction algorithm under anatomical constraints.

[0050] By employing a dual-gradient contour extraction algorithm that integrates prior anatomical constraints, the surface contour extraction unit can effectively distinguish the true anatomical boundaries of the target tissue from imaging artifacts and noise interference caused by uneven illumination, tissue fluid reflection, or temporary tissue adhesion under complex intraoperative imaging conditions. This algorithm combines the spatial grayscale gradient information of the image itself with structural constraints derived from prior knowledge of the target tissue's anatomical structure (such as surface continuity and curvature features). During contour extraction, it achieves dual verification of the physical rationality and anatomical consistency of edge responses, thereby significantly improving the accuracy and stability of surface contour information extraction and providing a more reliable data foundation for subsequent contour-based real-time deformation registration.

[0051] Furthermore, the feature extraction module 320 also includes a gray-level gradient analysis unit and a mechanical feature extraction unit. The gray-level gradient analysis unit is used to perform semantic segmentation on the real-time image to obtain a mask image of the target tissue, and calculate the gray-level gradient of the surface contour region of the target tissue based on the mask image. The mechanical feature extraction unit is used to generate deformation mechanical correlation features that characterize the local mechanical response of the target tissue based on the gray-level gradient and the edge detection algorithm that integrates anatomical constraints.

[0052] Through the collaboration of the gray-level gradient analysis unit and the mechanical feature extraction unit, the system can extract deep features from real-time images that not only reflect geometric contours but also characterize the local physical state of tissues. Semantic segmentation and masking ensure the accuracy of the analyzed regions and avoid background interference. Edge detection with anatomical constraints transforms image gray-level changes into feature descriptions with clear biomechanical significance. On the one hand, this allows the subsequent real-time registration process to be built on a more realistic physical response model, significantly improving the physical consistency between deformation simulation and registration. On the other hand, by directly associating image features with mechanical behavior, the system enhances its ability to perceive and analyze the subtle and complex deformation patterns of soft tissue under intraoperative stress, providing crucial underlying feature support for high-precision, adaptive dynamic registration, thereby improving the overall accuracy of the navigation system in real surgical scenarios.

[0053] The real-time registration module 330 first matches the extracted surface contour features with the current surface of the 3D soft tissue deformation model, calculating the geometric position difference. Simultaneously, combining the local physical state of the target tissue reflected by the deformation mechanics correlation features, it mechanically corrects and weights this geometric difference, transforming it into a set of physical quantities describing the stretching, compression, or bending deformations required by the model. Subsequently, the module uses a preset mapping function to convert these physical quantities into virtual driving forces (such as force vectors or displacement boundary conditions) applied to the corresponding nodes or regions of the 3D soft tissue deformation model. Finally, by solving the mechanical equilibrium state of the physical deformation model under the action of the virtual driving forces, it drives the update of the model node positions, causing them to produce deformations matching the current state of the target tissue, thus completing one registration iteration. This process continues, achieving real-time tracking and adaptive registration of the target tissue's dynamic deformation.

[0054] Through the aforementioned closed-loop registration mechanism, the system incorporates the actual intraoperative tissue mechanical response into the registration process, ensuring that the deformation behavior of the 3D soft tissue deformation model aligns with biophysical laws. This effectively solves the registration distortion and lag problems caused by traditional rigid or purely geometric registration methods that neglect the physical properties of soft tissue, thus improving registration accuracy. Simultaneously, the closed-loop feedback mechanism enables the system to adapt to continuous tissue deformation and dynamic changes in the surgical scene, maintaining the stability of the registration relationship and providing real-time, reliable spatial mapping support for augmented reality surgical navigation.

[0055] Based on the above, the navigation generation module 340 renders the 3D soft tissue deformation model according to the spatial relationship matching the real-time image, and then overlays and fuses it with the real-time image. During the fusion process, based on the depth information of the corresponding area of ​​the 3D soft tissue deformation model in the real-time image and the preset display rules, the module handles the occlusion and transparency relationship between the virtual model and the real image, so that the internal and deep structure of the 3D model is presented on the real-time image in a preset visualization form. Finally, the module converts the fused image data into a signal format supported by the display device 100, and generates and outputs the navigation screen in real time.

[0056] like Figure 3 As shown, this embodiment also provides a navigation method for a robotic surgical navigation system based on real-time registration of a soft tissue deformation model, as described in the foregoing embodiments, comprising: S100, Obtain a pre-constructed three-dimensional soft tissue deformation model associated with the target tissue, the three-dimensional soft tissue deformation model being able to deform in response to the action of virtual driving force.

[0057] This three-dimensional soft tissue deformation model is reconstructed based on preoperative medical imaging data and endowed with biomechanical properties that can characterize tissue elasticity, enabling it to simulate real physical deformation behavior under the action of virtual driving force.

[0058] S200 processes real-time images to extract surface contour features and deformation mechanical correlation features of the target tissue.

[0059] Semantic segmentation is performed on images acquired in real time by the imaging device 200 to obtain the mask region of the target tissue. Based on this, a dual-gradient contour extraction algorithm under anatomical constraints is used to extract the surface contour features of the target tissue. Simultaneously, by analyzing the gray-level gradient distribution within the target tissue region and fusing anatomical structural constraints, deformation mechanical correlation features describing the local tissue stiffness and deformation tendency are generated.

[0060] S300 calculates the deformation difference between the three-dimensional soft tissue deformation model and the target tissue based on surface contour features and deformation mechanics correlation features. It maps the deformation difference into a virtual driving force applied to the three-dimensional soft tissue deformation model and controls the three-dimensional soft tissue deformation model to deform under the action of the virtual driving force, so as to enable the target tissue and the three-dimensional soft tissue deformation model to be registered in real time.

[0061] Based on the extracted surface contour features, the geometric difference between the current 3D model and the target tissue is calculated, and the difference is physically corrected and weighted by combining deformation mechanics correlation features. Subsequently, the corrected difference is transformed into a virtual driving force acting on the 3D soft tissue deformation model through a preset mapping function, controlling the update of the model's node positions in the physical simulation, so that the model deformation is dynamically aligned with the real state of the target tissue, achieving continuous closed-loop real-time registration.

[0062] S400 fuses the registered 3D soft tissue deformation model with the real-time image to generate a navigation screen and output it to the display device 100.

[0063] Based on the real-time registration results, the three-dimensional soft tissue deformation model is rendered according to the spatial perspective relationship consistent with the real-time image. Based on the model depth information processing and the occlusion relationship with the real image, the rendered virtual model structure is superimposed on the real-time image through the fusion algorithm to generate an augmented reality navigation screen with a "perspective" effect. Finally, it is output to the display device 100 to provide doctors with intuitive and accurate intraoperative guidance.

[0064] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the robotic surgical navigation method based on real-time registration of a soft tissue deformation model as described above.

[0065] Specifically, computer-readable storage media include, but are not limited to, at least one of USB flash drives, portable hard drives, solid-state drives, read-only memory, and random access memory.

[0066] It should be understood that the embodiments of the present invention are not limited to the specific structures and control flows listed in the above embodiments. Those skilled in the art can make appropriate adjustments or substitutions to the implementation methods of each module, the specific parameters of the coordinate mapping relationship, the mechanical structure form, etc., without departing from the principles of the present invention, and such modifications and variations should also be considered to fall within the protection scope defined by the claims of the present invention.

[0067] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used solely for the convenience of describing this application and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0068] Finally, it should be noted that the above embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A robotic surgical navigation system based on real-time registration of a soft tissue deformation model, characterized in that, include: Imaging device for acquiring real-time images containing target tissue; A display device is used to present a navigation screen to the operator; The control device is communicatively connected to both the imaging device and the display device. The control device includes: The model management module is used to acquire a pre-built three-dimensional soft tissue deformation model associated with the target tissue, the three-dimensional soft tissue deformation model being able to deform in response to the action of a virtual driving force; The feature extraction module is used to process the real-time image and extract the surface contour features and deformation mechanical correlation features of the target tissue; The real-time registration module is used to calculate the deformation difference between the three-dimensional soft tissue deformation model and the target tissue based on the surface contour features and deformation mechanical correlation features, map the deformation difference into a virtual driving force applied to the three-dimensional soft tissue deformation model, and control the three-dimensional soft tissue deformation model to deform under the action of the virtual driving force, so that the target tissue and the three-dimensional soft tissue deformation model can be registered in real time. The navigation generation module is used to fuse the registered three-dimensional soft tissue deformation model with the real-time image to generate a navigation screen and output it to the display device.

2. The robotic surgical navigation system based on real-time registration of a soft tissue deformation model according to claim 1, characterized in that, The navigation system also includes a preoperative planning module, which is used to construct the three-dimensional soft tissue deformation model based on preoperative medical imaging data; The preoperative planning module includes: A geometric modeling unit is used to reconstruct a three-dimensional geometric model of the target tissue based on the preoperative medical imaging data. The mechanical property assignment unit is used to assign biomechanical properties that characterize tissue elasticity to the three-dimensional geometric model in order to obtain the three-dimensional soft tissue deformation model.

3. The robotic surgical navigation system based on real-time registration of a soft tissue deformation model according to claim 2, characterized in that, The geometric modeling unit includes: The image acquisition subunit is used to acquire preoperative CT and MRI image data of the target object. The image segmentation and fusion subunit is used to segment the preoperative CT image data and the preoperative MRI image data to obtain their respective tissue contour information, and to fuse the tissue contour information of the two to generate fused image data containing details of the target tissue anatomical structure. The three-dimensional reconstruction subunit is used to reconstruct the three-dimensional geometric model of the target tissue based on the fused image data.

4. The robotic surgical navigation system based on real-time registration of a soft tissue deformation model according to claim 3, characterized in that, The mechanical property imparting unit is configured to impart biomechanical properties to the three-dimensional geometric model by constructing a particle-fiber bundle network model; The mechanical property imparting unit includes: The particle system generation subunit is used to determine the soft tissue region and the hard tissue region based on the anatomical structural features of the three-dimensional geometric model, and to distribute particles in each region to generate a particle system; wherein, the particle distribution density in the hard tissue region is greater than that in the soft tissue region, and the mass of each particle is set according to the particle distribution density of its region. A fiber bundle network construction subunit is used to establish elastic connections between adjacent particles in the particle system to form a fiber bundle network; wherein the mechanical parameters of the elastic connection are set according to the biomechanical properties of the regions where the particles at both ends are located.

5. The robotic surgical navigation system based on real-time registration of a soft tissue deformation model according to claim 4, characterized in that, The mechanical property imparting unit also includes: An image processing subunit is used to process the fused image data to extract the anatomical structural features of the target tissue; The processing of the fused image data includes noise reduction and contrast enhancement.

6. The robotic surgical navigation system based on real-time registration of a soft tissue deformation model according to claim 5, characterized in that, The fiber bundle network construction subunit is also configured to: Based on the anatomical structure characteristics of the target tissue, fiber bundles are arranged in the fiber bundle network along the anatomical direction of muscle fibers or collagen fibers to simulate the anisotropic mechanical properties of the target tissue.

7. The robotic surgical navigation system based on real-time registration of a soft tissue deformation model according to claim 1, characterized in that, The feature extraction module includes: The surface contour extraction unit is used to extract the surface contour information of the target tissue from the real-time image using a dual-gradient contour extraction algorithm under anatomical constraints.

8. The robotic surgical navigation system based on real-time registration of a soft tissue deformation model according to claim 1, characterized in that, The feature extraction module further includes: The gray-level gradient analysis unit is used to perform semantic segmentation on the real-time image, obtain a mask image of the target tissue, and calculate the gray-level gradient of the surface contour region of the target tissue based on the mask image. The mechanical feature extraction unit is used to generate the deformation mechanical correlation features that characterize the local mechanical response of the target tissue based on the gray-level gradient of the gray value and the edge detection algorithm that integrates anatomical constraints.

9. A navigation method for a robotic surgical navigation system based on real-time registration of a soft tissue deformation model as described in any one of claims 1-8, characterized in that, include: Obtain a pre-constructed three-dimensional soft tissue deformation model associated with the target tissue, the three-dimensional soft tissue deformation model being able to deform in response to the action of a virtual driving force; The real-time image is processed to extract the surface contour features and deformation mechanical correlation features of the target tissue; Based on the surface contour features and deformation mechanics correlation features, the deformation difference between the three-dimensional soft tissue deformation model and the target tissue is calculated, the deformation difference is mapped to a virtual driving force applied to the three-dimensional soft tissue deformation model, and the three-dimensional soft tissue deformation model is controlled to deform under the action of the virtual driving force so that the target tissue and the three-dimensional soft tissue deformation model can be registered in real time. The registered three-dimensional soft tissue deformation model is fused with the real-time image to generate a navigation screen, which is then output to the display device.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the method as described in claim 9 above.

Images