PCNL fusion image puncture and lithotripsy navigation method and equipment based on optical positioning

Abstract

The invention relates to the technical field of medical image processing, and discloses a PCNL fusion image puncture and lithotripsy navigation method and equipment based on optical positioning. The method comprises the following steps: acquiring a real-time ultrasonic image of a target object, and fusing the real-time ultrasonic image with a pre-acquired three-dimensional CT image of the target object to obtain a fused image; establishing a simulated skin-kidney channel according to the fused image; wherein the simulated skin-kidney channel is used for assisting in constructing a lithotripsy path for a target object; acquiring pose information of a navigation marker of the endoscope relative to the target object in real time, and displaying the pose information in the three-dimensional CT image in real time to obtain a target three-dimensional CT image displaying the pose information of the navigation marker of the endoscope in real time; wherein the target three-dimensional CT image is used for assisting in navigating the lithotripsy position of the target object.

Description

Technical Field

[0001] This application belongs to the field of medical image processing technology, specifically relating to a PCNL fusion image puncture and lithotripsy navigation method based on optical positioning, a PCNL fusion image puncture and lithotripsy navigation device based on optical positioning, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Percutaneous nephrolithotomy (PCNL) effectively improves surgical efficiency and the rate of stone removal in one stage for treating kidney stones larger than 2 cm and complex upper urinary tract stones, and is characterized by minimal trauma and rapid postoperative recovery. Currently, the two most widely used puncture localization methods are X-ray (such as CT imaging) and ultrasound imaging. While X-ray guidance can clearly display the renal collecting system structure and ensure puncture accuracy, it has significant drawbacks such as repeated machine movement during the procedure, radiation exposure, and the inability to identify surrounding organs. Ultrasound imaging localization has the advantage of requiring simple equipment and being quick to operate. Under ultrasound monitoring, the structure of surrounding organs and the kidney itself can be clearly identified, preventing collateral damage and avoiding radiation exposure. However, ultrasound imaging is insufficient for identifying anatomical structures, resulting in unclear anatomical visualization. Furthermore, in the application of PCNL navigation technology based on 3D images, although 3D printing and AR / MR technologies are relatively advanced visualization image reconstruction technologies, most current research is limited to preoperative planning and design, simulated puncture and virtual surgical operations, and it is still impossible to achieve real-time planning, guidance and navigation of lithotripsy path based on 3D images during surgery. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a PCNL fusion image puncture and lithotripsy navigation method and device based on optical positioning.

[0004] In a first aspect, embodiments of this application provide a PCNL fusion image puncture and lithotripsy navigation method based on optical positioning, including:

[0005] Acquire real-time ultrasound images of the target object, and fuse the real-time ultrasound images with the pre-acquired 3D CT images of the target object to obtain a fused image;

[0006] A simulated percutaneous renal access channel is established based on the fused images; the simulated percutaneous renal access channel is used to assist in constructing a lithotripsy path for the target object;

[0007] The pose information of the navigation markers of the endoscope relative to the target object is acquired in real time and displayed in the 3D CT image to obtain the target 3D CT image with the pose information of the navigation markers of the endoscope displayed in real time; the target 3D CT image is used to assist in navigating the location of the stone fragments of the target object.

[0008] In some embodiments, the method further includes:

[0009] Multiple positioning sensors fixed on the surface of the target object are used to acquire pose information of different positions on the surface of the target object throughout the entire respiratory cycle.

[0010] CT image sequences of the target object are obtained through computer-aided plain scan and computed tomography.

[0011] Based on pose information and CT image sequences, three-dimensional CT images are obtained through three-dimensional reconstruction.

[0012] In some embodiments, a three-dimensional CT image is obtained through three-dimensional reconstruction based on pose information and a CT image sequence, including:

[0013] For each CT image in the CT image sequence, the target region is segmented from the CT image; wherein, the target markers include at least one of the target object's kidney, stones, and blood vessels;

[0014] Obtain data on the location, shape, characteristic points, and amplitude of respiratory movements of target markers within the target area;

[0015] Based on the arterial phase of CT images, the venous phase and excretory phase data of CT images are fused to obtain an initial CT image model.

[0016] The trajectory of the target marker during respiration in the initial CT image model is quantified based on the location, shape, feature points, and amplitude data of the target marker as it moves with respiration, thus obtaining a three-dimensional CT image.

[0017] In some embodiments, the pose information of the endoscope's navigation markers relative to the target object is acquired in real time, and the pose information is displayed in a three-dimensional CT image in real time, including:

[0018] Using the target object as the first coordinate system, obtain the first pose data of the endoscope's navigation marker in the first coordinate system;

[0019] Using 3D CT images as the second coordinate system, determine the transformation matrix between the first and second coordinate systems;

[0020] Based on the transformation matrix, the first pose data is converted into the second pose data in the second coordinate system, and the second pose data is displayed in the 3D CT image in real time.

[0021] In some embodiments, fusing real-time ultrasound images with pre-acquired three-dimensional CT images of the target object includes:

[0022] Determine whether the target landmarks and their cross-sectional information mapped from real-time ultrasound images are consistent with those mapped from 3D CT images;

[0023] In response to the consistency between the target markers and their cross-sectional information mapped from real-time ultrasound images and those mapped from 3D CT images, it is determined that the real-time ultrasound images and 3D CT images will be fused.

[0024] In some embodiments, the method further includes:

[0025] The real-time position information of the navigation markers of the endoscope is continuously acquired through MEMS sensors.

[0026] Based on the mechanical arrangement algorithm, the real-time coordinates of the endoscope on the target 3D CT image are calculated according to the real-time position information;

[0027] The relative position of the endoscope and the target stone is determined based on real-time coordinates.

[0028] In some embodiments, the method further includes:

[0029] The real-time position information acquired by the MEMS sensor is constrained by the linear motion of the endoscope during the operation, thereby correcting the error that accumulates over time during the acquisition of real-time position information by the MEMS sensor.

[0030] Secondly, embodiments of this application provide a PCNL fusion image puncture and lithotripsy navigation device based on optical positioning, comprising:

[0031] The acquisition module is configured to acquire real-time ultrasound images of the target object and fuse the real-time ultrasound images with the pre-acquired three-dimensional CT images of the target object to obtain a fused image.

[0032] The channel creation module is configured to create a simulated percutaneous renal channel based on the fused image; wherein, the simulated percutaneous renal channel is used to assist in constructing a lithotripsy path for the target object;

[0033] The lithotripsy navigation module is configured to acquire the pose information of the endoscope's navigation markers relative to the target object in real time, and display the pose information in the 3D CT image in real time to obtain the target 3D CT image with the pose information of the endoscope's navigation markers displayed in real time; wherein, the target 3D CT image is used to assist in navigating the lithotripsy position of the target object.

[0034] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, they implement the steps of the PCNL fusion image puncture and lithotripsy navigation method based on optical positioning as described in the first aspect.

[0035] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the PCNL fusion image puncture and lithotripsy navigation method based on optical positioning as described in the first aspect.

[0036] The technical solution provided in this application accurately matches and fuses real-time intraoperative ultrasound images with preoperative CT images, enabling the surgeon to quickly develop spatial awareness during ultrasound-guided puncture. This avoids the problem of poor positioning caused by the surgeon's unfamiliarity with ultrasound images. It leverages the convenience and radiation-free advantages of ultrasound-guided technology while fully utilizing the clear and accurate three-dimensional reconstructed images from preoperative CT images. This reduces the difficulty of PCNL surgery, improves surgical safety, shortens the PCNL learning curve, effectively improves surgical precision, optimizes the lithotripsy route, and increases surgical efficiency and stone clearance rate. Furthermore, based on the preoperative three-dimensional CT images for spatial navigation and overall surgical lithotripsy layout, navigation markers transmit and display the pose information of the movement position in real time on the three-dimensional CT image. The real-time position information of the endoscope is then superimposed on the three-dimensional CT image, using three-dimensional images to guide the intracavitary image, thereby providing precise navigation for the entire lithotripsy procedure.

[0037] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0038] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0039] Figure 1 This is a flowchart of a PCNL fusion image puncture and lithotripsy navigation method based on optical positioning according to some embodiments of the present invention;

[0040] Figure 2 A flowchart of a method for obtaining three-dimensional CT images according to some embodiments of the present invention;

[0041] Figure 3 A flowchart illustrating a method for performing a puncture procedure according to some embodiments of the present invention;

[0042] Figure 4A flowchart illustrating the process of obtaining the relative position of an endoscope and a stone within a target body according to some embodiments of the present invention;

[0043] Figure 5 This is a schematic diagram of a lithotripsy guidance device based on CT and ultrasound image fusion according to some embodiments of the present invention;

[0044] Figure 6 This is a block diagram of an electronic device according to some embodiments of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0046] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this invention should have the ordinary meaning understood by those skilled in the art. The terms "first," "second," and similar terms used in the embodiments of this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0047] As described in the background section, the incidence of urinary tract stones is gradually increasing worldwide. The incidence of kidney stones varies significantly due to numerous factors, including geography, climate, diet, fluid intake, genetics, gender, occupation, and age. Current treatments for kidney stones primarily include minimally invasive surgery, adjuvant medication, and dietary adjustments. However, due to the high incidence and recurrence rate of new stones, kidney stones cannot be completely cured. Furthermore, the treatment and management of kidney stones are extremely expensive, resulting in substantial medical costs.

[0048] PCNL effectively improves surgical efficiency and one-stage stone removal rate in the treatment of kidney stones larger than 2cm and complex upper urinary tract stones, and features minimal trauma and rapid postoperative recovery. It is particularly important for staghorn calculi with large stone volume and burden, as well as for patients with stones complicated by infection or abnormalities in the collecting system structure such as the renal pelvis and calyces. PCNL involves percutaneous puncture into the renal collecting system under image guidance, creating a 6-8mm diameter percutaneous renal channel, and then using a nephroscope combined with lithotripsy tools to fragment and remove the stones. Currently, the two most widely used puncture guidance methods are X-ray (such as CT imaging) and ultrasound imaging. While X-ray guidance can clearly display the renal collecting system structure and ensure puncture accuracy, it has significant drawbacks such as repeated intraoperative machine movement, radiation exposure, and inability to identify surrounding organs. Ultrasound imaging guidance offers the advantages of simple equipment and quick operation. Under ultrasound monitoring, surrounding organs and the structure of the kidney itself can be clearly identified, preventing collateral damage and avoiding radiation exposure.

[0049] The successful completion of PCNL depends heavily on the successful establishment of the surgical access. Appropriate puncture point selection and precise puncture localization are crucial factors in effectively reducing postoperative bleeding complications and improving safety. Achieving precise puncture relies on the assistance of ultrasound localization. However, due to the highly specialized nature of ultrasound imaging, mastering ultrasound diagnostic and exploration techniques requires extensive clinical experience. This poses a challenge for urologists who are more familiar with CT imaging, requiring not only familiarity with ultrasound techniques but also an abstract, three-dimensional understanding of the renal collecting system during the procedure. Therefore, the learning curve for ultrasound-guided PCNL is relatively long.

[0050] After establishing the percutaneous renal access, the surgeon uses a nephroscope combined with EMS (Electro Medical Systems) for lithotripsy. For multiple kidney stones, staghorn calculi, etc., the stone load is large and dispersed, and many stones are composed of calcium oxalate, making them quite hard, resulting in prolonged surgery time and a difficult lithotripsy process. Furthermore, the anatomical structure of the renal pelvis and calyces collecting system for this type of stone is complex, making it difficult to distinguish the positional relationship of the anterior and posterior calyces during lithotripsy and leading to missed stones. In addition, most staghorn calculi require multi-channel lithotripsy, and the above factors can also cause deviations or errors in the location or sequence of channel establishment, thus affecting the overall stone removal effect. Repeatedly searching for blind spots in the endoscopic view of the calyces and stones can also significantly prolong the operation time. Excessive residual stones after surgery may require secondary or even multiple surgeries. Therefore, a clear and reasonable lithotripsy path during the operation is significantly helpful in effectively improving surgical efficiency and reducing the probability of residual stones and secondary surgeries.

[0051] Image fusion technology is increasingly used in urology. This technology combines two relatively mature imaging techniques, such as the fusion of ultrasound and CT (Computed Tomography) or ultrasound and MR (Magnetic Resonance Imaging). Guiding percutaneous nephrolithotomy with fused ultrasound and CT images can effectively improve puncture accuracy and reduce the learning curve. This technique imports preoperative or intraoperative CT files into the fused image ultrasound machine and uses computer image matching and recognition technology to display real-time ultrasound and CT images on the same plane, overcoming the shortcomings of both techniques and ultimately achieving precise puncture. CT images can clearly display anatomical structures, while ultrasound images can monitor the kidney and surrounding tissues in real time. Fusion imaging combines the advantages of both, compensating for the insufficient recognition of anatomical structures by ultrasound images and overcoming the limitation of CT images in real-time monitoring of surrounding tissue structures.

[0052] In the process of developing this invention, the applicant discovered that current ultrasound-CT image fusion technology has limitations. Furthermore, real-time ultrasound features include respiratory motion, increasing the difficulty of fusing real-time ultrasound features with static CT images. Further optimization of the depth perception effect in the fused display is needed to achieve enhanced fusion of ultrasound images and CT reconstructed images, while eliminating respiratory motion interference. Regarding the application of PCNL navigation technology for three-dimensional images, although 3D printing and AR (Augmented Reality) / MR technologies belong to visual image reconstruction technologies, most current research is limited to preoperative planning, simulated puncture, and virtual surgical operations, and cannot yet achieve real-time planning, guidance, and navigation of lithotripsy paths using three-dimensional images during surgery.

[0053] Multimodal medical image fusion involves rigidly and flexibly registering images from multiple imaging sources and displaying them in two or three dimensions within the same coordinate system. This technology overcomes the limitations of single-modal medical images in information presentation, providing clinicians with more intuitive and necessary information, thereby achieving precision in diagnosis, surgery, or other treatments. PCNL, as a first-line surgical option for treating complex kidney stones, is irreplaceable, but it also presents significant surgical challenges and a high rate of perioperative complications. Simultaneously applying fusion imaging technology and 3D image-guided lithotripsy technology in PCNL, through rapid ultrasound image recognition and precise analysis of the lithotripsy path, fully covers the two key processes of puncture and lithotripsy, making the overall surgery precise, minimally invasive, and controllable. This effectively reduces the learning curve, lowers postoperative complications, improves overall surgical safety, increases efficiency, and reduces treatment costs and patient burden.

[0054] Therefore, this invention accurately matches and fuses real-time intraoperative ultrasound images with preoperative CT images, enabling the surgeon to quickly develop spatial awareness during ultrasound-guided puncture. This avoids the problem of poor positioning caused by the surgeon's unfamiliarity with ultrasound images. It can take advantage of the convenience and radiation-free nature of ultrasound-guided technology, and make full use of the clear and accurate three-dimensional reconstruction images from preoperative CT images. This reduces the difficulty of PCNL surgery, improves surgical safety, shortens the PCNL learning curve, effectively improves surgical precision, optimizes the lithotripsy route, and improves surgical efficiency and stone clearance rate.

[0055] The following description, with reference to the accompanying drawings, describes the PCNL fusion image puncture and lithotripsy navigation method and device based on optical positioning proposed in the embodiments of the present invention.

[0056] In some embodiments, the optical positioning-based PCNL fusion image puncture and lithotripsy navigation system of the present invention includes a first navigation marker, a second navigation marker, an optical tracker, a control processor, and a display.

[0057] Specifically, the first navigation marker can be placed on the surgical target body, i.e., on the target patient, to rise and fall with the patient's breathing; the second navigation marker can be placed on the ultrasound detection handle to move with the movement of the ultrasound detection handle; an optical tracker is correspondingly set with the first and second navigation markers and can receive the marking signals fed back by the first and second navigation markers; a control processor is electrically connected to the optical tracker and the ultrasound detection handle; wherein, the control processor includes a first processor, a second processor, and a first memory, the first processor can obtain the position of the first navigation marker to determine the respiratory state of the surgical target body; the second processor can obtain the position of the second navigation marker to determine the position of the ultrasound detection handle; the first memory can store a set of CT images of the surgical target body throughout the entire respiratory cycle, the first memory is electrically connected to the first and second processors to retrieve the CT images corresponding to the respiratory state determined by the first navigation marker and the position of the ultrasound detection handle determined by the second navigation marker; a display is electrically connected to the control processor to display the CT images retrieved from the memory; the display can also be electrically connected to the ultrasound detection handle to display the ultrasound images detected by the ultrasound detection handle.

[0058] Therefore, the PCNL fusion image puncture and lithotripsy navigation system based on optical positioning provided by this invention can combine intraoperative ultrasound images with preoperative CT images through optical positioning to complete precise puncture under fusion image guidance and establish a percutaneous renal access. Secondly, using the preoperative three-dimensional CT image of the kidney as a whole spatial holographic map, an optical tracker is used to capture navigation markers at the endoscope operating end, and the coordinate information of the endoscope position is presented in real time on the three-dimensional image display terminal, thereby providing precise navigation for the overall lithotripsy surgery.

[0059] refer to Figure 1 This is a flowchart of a PCNL fusion image puncture and lithotripsy navigation method based on optical positioning according to some embodiments of the present invention.

[0060] like Figure 1 As shown, the PCNL fusion image puncture and lithotripsy navigation method based on optical positioning in this embodiment of the invention may include the following steps:

[0061] S101, acquire real-time ultrasound images of the target object, and fuse the real-time ultrasound images with the pre-acquired three-dimensional CT images of the target object to obtain a fused image.

[0062] In this step, real-time ultrasound images of the target patient (i.e., the patient requiring PCNL surgery) are first acquired. This process uses ultrasound equipment to scan the patient's kidneys and related areas in real time, acquiring ultrasound images reflecting the current tissue structure and organ status. Subsequently, the real-time ultrasound images are fused with pre-acquired 3D CT images of the target patient. 3D CT images are typically generated by computer processing after a comprehensive CT scan of the patient before surgery, providing images with three-dimensional structural information that clearly presents key information such as the kidney's anatomical structure and stone location. Using specific image fusion algorithms and techniques, the real-time ultrasound images and 3D CT images are matched and fused in the same coordinate system to obtain a fused image. The fused image combines the real-time nature of ultrasound images with the high resolution and three-dimensional structural information of CT images, allowing the surgeon to simultaneously observe real-time dynamic tissue changes and precise anatomical structures within the patient's body, providing a more comprehensive and accurate visual basis for subsequent surgical procedures. For example, while observing renal blood flow, a clear understanding of the positional relationship between the stones and surrounding blood vessels, renal pelvis, and calyces helps in more accurately planning the surgical path.

[0063] S102, establish a simulated percutaneous renal channel based on the fused image; wherein, the simulated percutaneous renal channel is used to assist in constructing a lithotripsy path for the target object.

[0064] In this step, based on the fused image obtained in step S101, a simulated percutaneous renal access route is established using computer-aided design software or a specialized surgical planning system, taking into account factors such as the location, size, and shape of the stone, as well as the anatomical structure of the kidney. The simulated percutaneous renal access route is a virtual model of a channel from the body surface through the renal cortex to the location of the stone within the renal collecting system. Its establishment process requires consideration of parameters such as the length, angle, and diameter of the channel to ensure safe and effective access to the stone site during actual surgery. The establishment of the simulated percutaneous renal access route provides important assistance in constructing a lithotripsy path tailored to the target patient. Through the simulated channel, the surgeon can pre-plan and evaluate the surgical path before surgery, identifying potential risks and difficulties in advance, such as avoiding important blood vessels and nerve tissues, and selecting the optimal puncture point and angle. This helps improve the precision and safety of the surgery, reduces the occurrence of complications such as intraoperative bleeding and damage to surrounding organs, and also helps shorten the operation time and improve surgical efficiency.

[0065] S103, acquire the pose information of the navigation marker of the endoscope relative to the target object in real time, and display the pose information in the three-dimensional CT image in real time to obtain the target three-dimensional CT image with the pose information of the navigation marker of the endoscope displayed in real time; wherein, the target three-dimensional CT image is used to assist in navigating the location of the stone fragments of the target object.

[0066] In this step, during the surgery, the pose information of the endoscopic navigation markers relative to the target object is acquired in real time. Navigation markers are typically specific optical markers or sensors fixed to the endoscope, and their position and orientation changes can be tracked in real time using an optical positioning system (such as an infrared optical locator). The acquired pose information is transmitted to a computer system in real time and matched and calculated with pre-acquired 3D CT images. The pose information is then displayed in the 3D CT images in real time, resulting in a target 3D CT image displaying the pose information of the endoscopic navigation markers. This target 3D CT image assists in navigating the target object to the location of the lithotripsy stones. During the surgery, the surgeon can observe the target 3D CT image to understand the position and orientation of the endoscope within the patient's body, as well as the relative position of the endoscope tip to the stone. This helps the surgeon to more precisely control the operation of the endoscope, accurately guide the lithotripsy tool to the stone site, avoid blind operation and repeated adjustments, improve the accuracy and efficiency of lithotripsy, reduce damage to surrounding normal tissues, and further improve the safety and success rate of the surgery.

[0067] The system utilizes integrated inertial sensors (accelerometer + gyroscope) mounted on the endoscope to detect its real-time position as it enters the target body from the puncture point. This real-time position information is then transmitted to a wireless transmission module, which transmits the endoscope's position back in real time to determine its location within the 3D CT image model. This allows for the fusion of image-guided puncture and 3D lithotripsy navigation in ultrasound-guided PCNL. Intraoperative ultrasound images are combined with preoperative CT images using optical positioning to achieve precise puncture and establish a percutaneous renal access under fused image guidance. Furthermore, using the preoperative 3D renal CT image as a holographic map, an optical tracker captures navigation markers at the endoscope's operating end, displaying the endoscope's coordinates in real time on a 3D image display terminal, thus providing precise navigation for the entire lithotripsy procedure.

[0068] The wireless transmission module can be a low-power Bluetooth processor. The low-power Bluetooth processor collects the motion information of the broken stones (including the stability and real-time update of position and attitude data, the accuracy and stability of angular velocity and acceleration, etc.), and obtains the motion state of the wireless transmission module through mechanical arrangement and other algorithms. The position of the endoscope in the three-dimensional CT image model is sent to the image fusion display terminal (the time display delay error does not exceed 0.5s, and the relative position error does not exceed 1cm).

[0069] As an optional embodiment, the method further includes: acquiring pose information of different positions on the body surface of the target object during the entire respiratory cycle through multiple positioning sensors fixed on the body surface of the target object; obtaining CT image sequences of the target object through computer plain scan and tomographic scan; and obtaining three-dimensional CT images through three-dimensional reconstruction based on the pose information and CT image sequences.

[0070] In this step, multiple positioning sensors are fixed to the surface of the target object. These sensors can accurately detect movement and positional changes at different locations on the body surface. The pose information from these sensors is continuously acquired throughout the target object's entire respiratory cycle. Respiratory movements cause changes in the position of internal organs; by collecting pose information throughout the entire respiratory cycle, a comprehensive understanding of the correlation between surface movement and changes in organ position can be obtained. This provides accurate surface movement data for subsequent 3D reconstruction, enabling the reconstructed 3D CT images to more realistically reflect the target object's anatomical structure under actual physiological conditions, reducing image errors caused by respiratory movements, and improving the accuracy and reliability of the images.

[0071] For example, preoperatively, six optical positioning sensors are attached to the target patient's body surface (chest and abdomen), covering the projected areas of the liver and kidneys. The target patient's surface movement trajectory during free breathing is recorded (sampling frequency can be 100Hz), continuously acquiring data for five respiratory cycles to obtain pose information at different locations on the body surface throughout the entire respiratory cycle. The target patient undergoes a computed tomography (CT) scan (three-phase contrast-enhanced scan) to obtain CT image sequences of the kidney region containing stones. Based on the pose information and the CT image sequence, the kidney, stone, and blood vessel (renal artery / vein) regions are extracted from the CT images using threshold segmentation and morphological algorithms, and the coordinates of the stone's center point are calculated. Based on the surface sensor data, the displacement range of the kidney with respiration is quantified (±8mm vertically, ±5mm anteroposteriorly), generating a respiratory motion trajectory curve. The three-phase CT data are registered to the arterial phase reference coordinate system, and a non-rigid registration algorithm is used to eliminate image misalignment caused by respiratory motion, generating an initial CT image model. The geometric data of the target organs (kidney, stone) are combined with the respiratory motion trajectory to generate a three-dimensional CT image model with respiratory compensation.

[0072] As an optional embodiment, a three-dimensional CT image is obtained through three-dimensional reconstruction based on pose information and CT image sequence, including: segmenting a target region from each CT image in the CT image sequence; wherein the target marker includes at least one of the target object's kidney, stones, and blood vessels; acquiring the position, shape, feature points, and amplitude data of the target marker with respiratory motion from the target region; fusing the venous phase and excretory phase data of the CT image with the arterial phase as a reference to obtain an initial CT image model; quantifying the trajectory of the target marker with respiratory motion in the initial CT image model based on the position, shape, feature points, and amplitude data of the target marker with respiratory motion to obtain a three-dimensional CT image.

[0073] Specifically, multi-slice spiral CT was used to acquire imaging data of the target subject. The target subject was placed in a supine-prone position (consistent with the intraoperative position) and plain CT images were acquired first. This stage was mainly used to acquire basic images, assess the anatomical structure and lesions of the target subject, and provide a reference for subsequent tomographic scans. Subsequently, a non-ionic contrast agent was injected for enhanced scanning, and continuous scanning began immediately after injection. The enhanced scan consisted of three phases: ① Arterial phase: rapid scanning after contrast agent injection to capture the arterial filling state; ② Venous phase: scanning was performed after the arterial phase; ③ Excretion phase: this phase mainly observed the excretion of the contrast agent in the renal collecting system to help assess renal function and related lesions. The plain and enhanced images from all three phases were uploaded to the workstation for preliminary multiplanar reconstruction and volumetric rendering to observe relevant lesions. Before starting 3D reconstruction, the original Digital Imaging and Communications in Medicine (DICOM) format data needed to be imported into the 3D image editing software.

[0074] Image segmentation is a crucial step in extracting target regions containing target landmarks from CT images. Specific methods include: ① Threshold-based segmentation: By setting one or more thresholds, regions with different grayscale values ​​in the image are segmented. This method is often used when there is a significant density difference between the target landmark (such as blood vessels, kidneys, or stones) and surrounding tissues. The threshold range is adjusted to select the target region containing the target landmark. Software can be used for automatic or semi-automatic segmentation to improve efficiency. ② Region segmentation: Based on connectivity or region growing algorithms, target landmarks (such as the vascular system) are automatically identified and segmented. Starting with a seed point (a point determined by the observer), the software automatically expands the segmentation area to ultimately obtain the target region. Through these segmentation techniques, the location, shape, feature points, and amplitude data of the target landmark during respiratory motion are effectively extracted, laying the foundation for subsequent reconstruction and analysis.

[0075] Three-dimensional reconstruction is the core process of post-processing, aiming to create a dynamic three-dimensional model. The specific process includes: (1) Model setting and registration: ① Standard model setting: The arterial phase of CT images is usually selected as the reconstruction benchmark because the blood vessels are most clearly visualized at this time. ② Model merging: The venous phase and excretory phase data of CT images are fused, and the images are registered through characteristic landmarks (such as bifurcation points, the starting position of blood vessels, etc.), that is, the image data at different time points are aligned and harmonized. (2) Manual adjustment: The interactive tools of the software are used, such as manual rotation, movement, scaling and cropping, to ensure that the models of different phases are accurately registered in space. This step ensures the accuracy of fusion and visualization effect. (3) Finally, the initial CT image model is obtained, which can realistically reflect the anatomical structure of the target object's tissues and organs.

[0076] After the three-dimensional reconstruction is completed, the initial CT image model needs to be further adjusted and supplemented to improve image quality and analysis effect: (1) Smoothing and denoising: ① Smoothing: Reduce the irregularity of the surface or boundary of the initial CT image model through algorithms to make the image look more natural and smooth. ② Denoising technology: Apply image processing algorithms to eliminate noise in the initial CT image model to ensure a high signal-to-noise ratio during observation and analysis. (2) Filling the model: Fill the irregular boundary parts to make the reconstructed three-dimensional model closed and avoid holes or gaps in the model. (3) Structural value rendering and transparency adjustment: ① Structural value rendering: Color according to the density of different structures (such as blood vessels, bones and fat, etc.) to facilitate the differentiation of different tissues or lesions. ② Transparency adjustment: By adjusting the transparency of the object, the internal structure can be better observed in the three-dimensional view, helping doctors understand the relationship between the lesion and the surrounding tissue. Finally, after all processing steps, the trajectory of the target marker in the initial CT image model with respiratory movement is quantified according to the position, shape, feature point and amplitude data of the target marker, and output as a three-dimensional CT image model.

[0077] As a specific example, such as Figure 2 As shown, the flowchart of the method for obtaining three-dimensional CT images according to the present invention may include the following steps:

[0078] S201: Multiple positioning sensors fixed on the surface of the target object are used to acquire the pose information of different positions on the surface of the target object during the entire respiratory cycle.

[0079] S202, obtains CT image sequences of the target object through computer plain scan and tomographic scan.

[0080] S203, for each CT image in the CT image sequence, segment the target region containing the target marker from the CT image.

[0081] S204 Extract target markers from the target area and obtain data on the location, shape, feature points, and amplitude of respiratory movements of the target markers.

[0082] S205 uses the arterial phase of CT images as a benchmark, and integrates the venous phase and excretory phase data of CT images to obtain an initial CT image model.

[0083] S206. Based on the location, shape, feature points, and amplitude data of the target marker in the initial CT image model, the trajectory of the target marker with respiratory motion is quantified to obtain a three-dimensional CT image model.

[0084] As an optional embodiment, the real-time acquisition of the pose information of the endoscope's navigation marker relative to the target object and the real-time display of the pose information in the three-dimensional CT image includes: using the target object as a first coordinate system, acquiring the first pose data of the endoscope's navigation marker in the first coordinate system; using the three-dimensional CT image as a second coordinate system, determining the transformation matrix between the first and second coordinate systems; converting the first pose data into second pose data in the second coordinate system according to the transformation matrix, and displaying the second pose data in the three-dimensional CT image in real time.

[0085] Specifically, positioning sensors mounted on the ultrasound probe can detect the probe's first pose data in a first coordinate system, where the first coordinate system can be the target object. These positioning sensors detect the target object's pose data at different positions on its body surface throughout the entire respiratory cycle, enabling real-time monitoring of the target object's surface displacement. These sensors typically employ optical measurement technology, allowing for rapid and high-precision recording of motion data. During both static and dynamic breathing, the positioning sensors continuously record the ultrasound probe's first pose data in the first coordinate system, including displacement data along the X, Y, and Z axes, to monitor and record the target object's surface motion in real time.

[0086] Using the coordinate system of the 3D CT image model as the second coordinate system, the ultrasound-CT fusion imaging puncture technique based on optical positioning technology first requires understanding the coordinate parameters of the CT image, including the origin, coordinate axis direction, and scale. By selecting four feature points on the CT image—for example, certain anatomical landmarks; for kidney stones, this could be the upper and lower poles of the kidney and two points at different dimensions of the stone—these feature points in the second coordinate system are determined. Similarly, four corresponding feature points are found on the ultrasound image with a positioning sensor. Then, the positioning sensor measures the initial spatial coordinates and orientation data of the ultrasound plane to achieve registration between the 2D ultrasound and the 3D stone model, obtaining the transformation relationship between the two coordinate systems. This allows for the calculation of the transformation matrix between the first and second coordinate systems. Based on the transformation matrix, the first pose data is transformed to obtain the pose data in the 3D CT image model. This pose data is used as the second pose data to ensure accurate spatial correspondence between the two images. Based on this coordinate transformation relationship, the matching of the actual coordinate system and the virtual coordinate system in the surgical navigation environment can be achieved. The coordinate transformation relationship between the puncture needle and the tracker, and the coordinate transformation relationship between the ultrasound probe and the magnetic field transmitter can be obtained through the magnetic positioning system.

[0087] As an optional embodiment, fusing real-time ultrasound images with pre-acquired three-dimensional CT images of the target object includes: determining whether the target markers and their cross-sectional information mapped by the real-time ultrasound images are consistent with the target markers and their cross-sectional information mapped by the three-dimensional CT images; and determining to fuse the real-time ultrasound images with the three-dimensional CT images in response to the consistency between the target markers and their cross-sectional information mapped by the real-time ultrasound images and the target markers and their cross-sectional information mapped by the three-dimensional CT images.

[0088] Specifically, by comparing the correspondence between the target markers and their cross-sectional information mapped from the target ultrasound image and the target markers and their cross-sectional information mapped from the target CT image, physicians can determine the accuracy of registration, ensuring the quality and reliability of image fusion. When the target markers and their cross-sectional information mapped from the target ultrasound image match those mapped from the target CT image, the puncture procedure can be performed. At this point, the correspondence between the target CT image cross-section and the target ultrasound cross-section corresponding to the first pose data is confirmed. If the puncture site and path are clearly visible on ultrasound, ultrasound-guided puncture is the primary method, with CT images serving as an auxiliary and reference aiding in ultrasound image correction and optimization. If the ultrasound image cannot provide a sufficiently clear puncture path and target renal calyx, puncture can be performed entirely based on CT images.

[0089] As a specific example, such as Figure 3 As shown in the flowchart of the method for completing a puncture procedure according to the present invention, it may include the following steps:

[0090] S301, the first pose data of the ultrasonic probe in the first coordinate system is obtained by the positioning sensor set on the ultrasonic probe.

[0091] S302, using the coordinate system of the 3D CT image model as the second coordinate system, determine the transformation matrix between the first and second coordinate systems.

[0092] S303, based on the transformation matrix, convert the first pose data into the second pose data.

[0093] S304, determine whether the target landmarks and their section information mapped by the target ultrasound section are consistent with the target landmarks and their section information mapped by the target CT image section. If yes, proceed to step S305.

[0094] S305, determine the correspondence between the target CT image section and the target ultrasound section corresponding to the first pose data.

[0095] S306, Determine the puncture point guided by lithotripsy and complete the puncture procedure.

[0096] As an optional embodiment, the method further includes: continuously acquiring real-time position information of the endoscope's navigation markers via a MEMS sensor; calculating the real-time coordinates of the endoscope on the target 3D CT image based on the real-time position information using a mechanical arrangement algorithm; and determining the relative position of the endoscope and the target stone based on the real-time coordinates.

[0097] Specifically, MEMS sensors mounted on the endoscope continuously monitor its real-time position. This real-time position information includes triaxial angular velocity and triaxial acceleration data. These data are then used in a mechanical arrangement algorithm to obtain the endoscope's real-time coordinates within the 3D CT imaging model, representing its real-time triaxial displacement and rotation. After obtaining these real-time coordinates, the relative position of the endoscope to the stones within the target patient's body can be determined based on the endoscope's real-time coordinates within the 3D CT imaging model and the preoperative location of the stones.

[0098] In some embodiments, a Bluetooth sensor acquisition and processing module can be installed at the endoscope. This module acquires real-time position information of the endoscope continuously detected by the MEMS sensor via an SPI (Serial Peripheral Interface) or IIC (Inter-Integrated Circuit) interface. The Bluetooth sensor acquisition and processing module can enter a sleep mode when not in operation, featuring low power consumption and reduced battery power requirements.

[0099] Among them, the mechanical orchestration algorithm is a core algorithm in the field of inertial navigation, used to convert raw data from MEMS sensors into precise position and attitude information. In medical navigation systems, this algorithm is crucial for achieving high-precision positioning of endoscopes and other devices.

[0100] As an optional embodiment, the method further includes: constraining the real-time position information acquired by the MEMS sensor based on the linear motion of the endoscope during the operation, and correcting the error accumulated by the MEMS sensor over time during the acquisition of real-time position information.

[0101] For example, when the endoscope is detected to have moved more than 10 mm continuously, the system automatically fits the five most recent position points as an ideal straight line and projects subsequent coordinates onto this line. A physiological deviation of ±1.5 mm in the vertical direction is allowed. If this deviation exceeds the limit, an audible and visual alarm is triggered, and the navigation display is frozen. This constrains the real-time position information acquired by the MEMS sensor based on the linear movement of the endoscope during the procedure. A sliding window least squares method is used, updating the optimal fitted straight line every 100 ms. A Kalman filter is applied to the raw coordinates output by the MEMS sensor, with the gain coefficient dynamically adjusted according to the movement speed. When the optical system is visible, the MEMS sensor data is directly overwritten; when the optical system is obstructed, virtual correction based on the geometric constraints of the puncture path is enabled to correct errors accumulated over time by the inertial sensor.

[0102] As a specific example, such as Figure 4 As shown in the flowchart, the method for determining the relative position of the endoscope and the stones within the target body according to the present invention may include the following steps:

[0103] S401 continuously acquires the real-time position information of the endoscope through MEMS sensors.

[0104] S402, based on a mechanical arrangement algorithm, calculates the real-time coordinates of the endoscope in the three-dimensional CT image model according to the real-time position information.

[0105] S403 determines the relative position of the endoscope and the stones inside the target body based on real-time coordinates.

[0106] S404 constrains the real-time position information acquired by the MEMS sensor based on the linear motion of the endoscope during the operation.

[0107] S405 corrects errors that accumulate over time during the acquisition of real-time location information by MEMS sensors.

[0108] In some embodiments, 3D image-guided lithotripsy technology uses preoperative CT 3D reconstructed images for spatial navigation, providing an overall surgical lithotripsy layout. Bluetooth sensors at the lithotripsy end transmit simulated signals of surgical movement position information in real time and display them on a 3D image display terminal. The positional information of the endoscope (the location of the lithotripsy point in the kidney and the location of adjacent stones) is superimposed on the panoramic 3D image, using 3D imaging to guide the intracavitary image, thus providing precise navigation for the entire lithotripsy procedure and helping doctors to more accurately fragment and remove stones. Through 3D image-guided navigation, doctors have a holistic view of the surgery and a clear lithotripsy path, thereby maximizing surgical efficiency and reducing postoperative residual stone rate.

[0109] Therefore, this invention primarily employs elastic registration fusion technology under automatic respiratory compensation calibration in ultrasound-CT fusion imaging to achieve dynamic real-time image fusion and guide puncture, thereby achieving a rapid and precise channel establishment process. Based on optical positioning, multimodal ultrasound-CT fusion image-guided percutaneous nephrolithotomy (PCNL) is combined with 3D image-sensing navigation lithotripsy technology, simultaneously applying it to the minimally invasive treatment of PCNL. This addresses two major challenges simultaneously: the low accuracy and complex anatomical structures caused by single ultrasound-guided two-dimensional imaging puncture, and the directional deviation of intraoperative stone fragmentation due to stone load, thus improving the precision and controllability of the surgery.

[0110] Based on a computer-aided navigation system and preoperative CT 3D reconstruction images as the overall spatial navigation layout, the distal end of the lithotripsy endoscope is set as the surgical starting point. Clicking the "start positioning" command sends the endoscope's movement position information to a 3D image display terminal in real time for spatial overlay, obtaining the specific spatial location of the lithotripsy point within the kidney, as well as the location, shape, and size of the stones. This provides precise navigation for the entire lithotripsy procedure. The 3D images are registered with the actual surgical area using surgical instruments and sensors. The surgeon observes the 3D image navigation in real time through a visual interface, intuitively understanding the location of the lithotripsy stones within the kidney, the distribution of stones, and surrounding anatomical structures to guide the surgical procedure.

[0111] In summary, the PCNL fusion image puncture and lithotripsy navigation method based on optical positioning according to embodiments of the present invention first acquires real-time ultrasound images of the target object, and then fuses the real-time ultrasound images with pre-acquired three-dimensional CT images of the target object to obtain a fused image; further, a simulated percutaneous renal access channel is established based on the fused image; wherein, the simulated percutaneous renal access channel is used to assist in constructing a lithotripsy path for the target object; finally, the pose information of the endoscope navigation markers relative to the target object is acquired in real time, and the pose information is displayed in the three-dimensional CT image in real time to obtain a target three-dimensional CT image displaying the pose information of the endoscope navigation markers in real time; wherein, the target three-dimensional CT image is used to assist in navigating the lithotripsy position of the target object. This technique accurately matches and fuses real-time intraoperative ultrasound images with preoperative CT images, enabling surgeons to quickly develop spatial awareness during ultrasound-guided puncture. This avoids positioning errors caused by surgeons' unfamiliarity with ultrasound images. It leverages the convenience and radiation-free nature of ultrasound guidance while fully utilizing the clear and accurate 3D reconstructed images from preoperative CT, reducing the difficulty of PCNL surgery, improving surgical safety, shortening the PCNL learning curve, effectively increasing surgical precision, optimizing the lithotripsy route, and improving surgical efficiency and stone clearance rate. Furthermore, based on preoperative 3D CT images for spatial navigation and overall lithotripsy layout, navigation markers transmit and display the pose information of their movement positions in real time on the 3D CT images. The real-time position information of the endoscope is then superimposed on the 3D CT images, using 3D images to guide the intracavitary image, thus providing precise navigation for the entire lithotripsy procedure.

[0112] It should be noted that the method of this embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this embodiment, and the multiple devices will interact with each other to complete the above method.

[0113] It should be noted that the above description describes some embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] Corresponding to the above embodiments, the present invention also proposes a PCNL fusion image puncture and lithotripsy navigation device based on optical positioning.

[0115] like Figure 5 As shown, the PCNL fusion image puncture and lithotripsy navigation device based on optical positioning according to an embodiment of the present invention includes: an acquisition module 510, a channel establishment module 520, and a lithotripsy navigation module 530.

[0116] The acquisition module 510 is configured to acquire real-time ultrasound images of the target object and fuse the real-time ultrasound images with the pre-acquired three-dimensional CT images of the target object to obtain a fused image.

[0117] The channel establishment module 520 is configured to establish a simulated percutaneous renal channel based on the fused image; wherein, the simulated percutaneous renal channel is used to assist in constructing a lithotripsy path for the target object;

[0118] The lithotripsy navigation module 530 is configured to acquire the pose information of the navigation markers of the endoscope relative to the target object in real time, and display the pose information in the three-dimensional CT image in real time to obtain the target three-dimensional CT image with the pose information of the navigation markers of the endoscope displayed in real time; wherein, the target three-dimensional CT image is used to assist in navigating the lithotripsy position of the target object.

[0119] Optionally, the acquisition module 510 is further configured to:

[0120] Multiple positioning sensors fixed on the surface of the target object are used to acquire pose information of different positions on the surface of the target object throughout the entire respiratory cycle.

[0121] CT image sequences of the target object are obtained through computer-aided plain scan and computed tomography.

[0122] Based on pose information and CT image sequences, three-dimensional CT images are obtained through three-dimensional reconstruction.

[0123] Optionally, the acquisition module 510 is further configured to:

[0124] For each CT image in the CT image sequence, the target region is segmented from the CT image; wherein, the target markers include at least one of the target object's kidney, stones, and blood vessels;

[0125] Obtain data on the location, shape, characteristic points, and amplitude of respiratory movements of target markers within the target area;

[0126] Based on the arterial phase of CT images, the venous phase and excretory phase data of CT images are fused to obtain an initial CT image model.

[0127] The trajectory of the target marker during respiration in the initial CT image model is quantified based on the location, shape, feature points, and amplitude data of the target marker as it moves with respiration, thus obtaining a three-dimensional CT image.

[0128] Optionally, the gravel navigation module 530 is further configured to:

[0129] Using the target object as the first coordinate system, obtain the first pose data of the endoscope's navigation marker in the first coordinate system;

[0130] Using 3D CT images as the second coordinate system, determine the transformation matrix between the first and second coordinate systems;

[0131] Based on the transformation matrix, the first pose data is converted into the second pose data in the second coordinate system, and the second pose data is displayed in the 3D CT image in real time.

[0132] Optionally, the gravel navigation module 530 is further configured to:

[0133] Determine whether the target landmarks and their cross-sectional information mapped from real-time ultrasound images are consistent with those mapped from 3D CT images;

[0134] In response to the consistency between the target markers and their cross-sectional information mapped from real-time ultrasound images and those mapped from 3D CT images, it is determined that the real-time ultrasound images and 3D CT images will be fused.

[0135] Optionally, the gravel navigation module 530 is further configured to:

[0136] The real-time position information of the navigation markers of the endoscope is continuously acquired through MEMS sensors.

[0137] Based on the mechanical arrangement algorithm, the real-time coordinates of the endoscope on the target 3D CT image are calculated according to the real-time position information;

[0138] The relative position of the endoscope and the target stone is determined based on real-time coordinates.

[0139] Optionally, the gravel navigation module 530 is further configured to:

[0140] The real-time position information acquired by the MEMS sensor is constrained by the linear motion of the endoscope during the operation, thereby correcting the error that accumulates over time during the acquisition of real-time position information by the MEMS sensor.

[0141] In summary, the PCNL fusion image puncture and lithotripsy navigation method based on optical positioning according to embodiments of the present invention first acquires real-time ultrasound images of the target object, and then fuses the real-time ultrasound images with pre-acquired three-dimensional CT images of the target object to obtain a fused image; further, a simulated percutaneous renal access channel is established based on the fused image; wherein, the simulated percutaneous renal access channel is used to assist in constructing a lithotripsy path for the target object; finally, the pose information of the endoscope navigation markers relative to the target object is acquired in real time, and the pose information is displayed in the three-dimensional CT image in real time to obtain a target three-dimensional CT image displaying the pose information of the endoscope navigation markers in real time; wherein, the target three-dimensional CT image is used to assist in navigating the lithotripsy position of the target object. This technique accurately matches and fuses real-time intraoperative ultrasound images with preoperative CT images, enabling surgeons to quickly develop spatial awareness during ultrasound-guided puncture. This avoids positioning errors caused by surgeons' unfamiliarity with ultrasound images. It leverages the convenience and radiation-free nature of ultrasound guidance while fully utilizing the clear and accurate 3D reconstructed images from preoperative CT, reducing the difficulty of PCNL surgery, improving surgical safety, shortening the PCNL learning curve, effectively increasing surgical precision, optimizing the lithotripsy route, and improving surgical efficiency and stone clearance rate. Furthermore, based on preoperative 3D CT images for spatial navigation and overall lithotripsy layout, navigation markers transmit and display the pose information of their movement positions in real time on the 3D CT images. The real-time position information of the endoscope is then superimposed on the 3D CT images, using 3D images to guide the intracavitary image, thus providing precise navigation for the entire lithotripsy procedure.

[0142] For ease of description, the above system is described by dividing it into various modules based on their functions. Of course, in implementing this invention, the functions of each module can be implemented in one or more software and / or hardware components.

[0143] The system described in the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0144] Corresponding to the above embodiments, the present invention also proposes an electronic device.

[0145] refer to Figure 6 The diagram below is a block diagram of an electronic device according to some embodiments of the present invention. It shows a more specific hardware structure diagram of an electronic device provided in this embodiment. The device may include: a display screen 630; one or more processors 610; one or more memory 620; the display screen 630 is used to display a graphical user interface; the one or more memory 620 is used to store one or more computer programs and data information; wherein the one or more computer programs include instructions; when the instructions are executed by one or more processors 610, the electronic device performs the steps of the above-described screen projection method.

[0146] According to the electronic device of the present invention, by executing the above-described screen projection method, applications in the smart cockpit system that require screen projection functionality do not need to create their own screen projection functions individually. This allows all applications in the smart cockpit system to share display between the main screen and the secondary screen or the rear screen, thereby improving the user experience.

[0147] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0148] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, the present invention also provides a computer-readable storage medium storing computer instructions for causing a computer to perform the methods of any of the above embodiments.

[0149] The aforementioned computer-readable storage medium can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0150] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the methods of any of the above exemplary method sections, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0151] Furthermore, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowchart may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0152] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0153] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this invention should have the ordinary meaning understood by those skilled in the art. The terms "first," "second," and similar terms used in the embodiments of this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0154] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the appended claims is to be interpreted in the broadest sense, thereby encompassing all such modifications and equivalent structures and functions.

Claims

1. A PCNL fusion image puncture and lithotripsy navigation method based on optical positioning, characterized in that, include: Acquire real-time ultrasound images of the target object, and fuse the real-time ultrasound images with pre-acquired three-dimensional CT images of the target object to obtain a fused image; A simulated percutaneous renal channel is established based on the fused image; wherein, the simulated percutaneous renal channel is used to assist in constructing a lithotripsy path for the target object; The pose information of the endoscope's navigation markers relative to the target object is acquired in real time, and the pose information is displayed in the three-dimensional CT image in real time to obtain a target three-dimensional CT image that displays the pose information of the endoscope's navigation markers in real time; wherein, the target three-dimensional CT image is used to assist in navigating the location of the stone fragments of the target object.

2. The PCNL fusion image puncture and lithotripsy navigation method based on optical positioning according to claim 1, characterized in that, The method further includes: Multiple positioning sensors fixed to the surface of the target object are used to acquire the pose information of different positions on the surface of the target object during the entire respiratory cycle. The CT image sequence of the target object was obtained by computer plain scan and tomographic scan. The three-dimensional CT image is obtained by three-dimensional reconstruction based on the pose information and the CT image sequence.

3. The PCNL fusion image puncture and lithotripsy navigation method based on optical positioning according to claim 2, characterized in that, The step of obtaining the three-dimensional CT image through three-dimensional reconstruction based on the pose information and the CT image sequence includes: For each CT image in the CT image sequence, a target region is segmented from the CT image; wherein the target markers include at least one of the target object's kidney, stones, and blood vessels; The location, shape, feature points, and amplitude data of the target markers during breathing are obtained from the target area. Based on the arterial phase of the CT images, the venous phase and excretory phase data of the CT images are fused to obtain an initial CT image model. The trajectory of the target marker in the initial CT image model is quantified based on the position, shape, feature points, and amplitude data of the target marker's movement with respiration, thereby obtaining the three-dimensional CT image.

4. The PCNL fusion image puncture and lithotripsy navigation method based on optical positioning according to claim 3, characterized in that, The real-time acquisition of the pose information of the navigation markers of the endoscope relative to the target object, and the real-time display of the pose information in the three-dimensional CT image, includes: Using the target object as the first coordinate system, the first pose data of the navigation marker of the endoscope in the first coordinate system is obtained; Using the three-dimensional CT image as the second coordinate system, determine the transformation matrix between the first coordinate system and the second coordinate system; According to the transformation matrix, the first pose data is converted into second pose data in the second coordinate system, and the second pose data is displayed in the three-dimensional CT image in real time.

5. The PCNL fusion image puncture and lithotripsy navigation method based on optical positioning according to claim 4, characterized in that, The step of fusing the real-time ultrasound image with the pre-acquired three-dimensional CT image of the target object includes: Determine whether the target markers and their cross-sectional information mapped by the real-time ultrasound image are consistent with the target markers and their cross-sectional information mapped by the three-dimensional CT image; In response to the consistency between the target marker and its cross-sectional information mapped by the real-time ultrasound image and the target marker and its cross-sectional information mapped by the three-dimensional CT image, it is determined to fuse the real-time ultrasound image and the three-dimensional CT image.

6. The PCNL fusion image puncture and lithotripsy navigation method based on optical positioning according to claim 5, characterized in that, The method further includes: The real-time position information of the navigation markers of the endoscope is continuously acquired through MEMS sensors. Based on the mechanical arrangement algorithm, the real-time coordinates of the endoscope in the target three-dimensional CT image are calculated according to the real-time position information; The relative position of the endoscope and the target stone is determined based on the real-time coordinates.

7. The PCNL fusion image puncture and lithotripsy navigation method based on optical positioning according to claim 6, characterized in that, The method further includes: The real-time position information acquired by the MEMS sensor is constrained by the linear motion of the endoscope during the operation, thereby correcting the error that accumulates over time during the acquisition of the real-time position information by the MEMS sensor.

8. A PCNL fusion image puncture and lithotripsy navigation device based on optical positioning, characterized in that, include: The acquisition module is configured to acquire real-time ultrasound images of the target object and fuse the real-time ultrasound images with pre-acquired three-dimensional CT images of the target object to obtain a fused image. The channel establishment module is configured to establish a simulated percutaneous renal channel based on the fused image; wherein the simulated percutaneous renal channel is used to assist in constructing a lithotripsy path for the target object; The lithotripsy navigation module is configured to acquire the pose information of the endoscope's navigation markers relative to the target object in real time, and display the pose information in the three-dimensional CT image in real time to obtain a target three-dimensional CT image that displays the pose information of the endoscope's navigation markers in real time; wherein, the target three-dimensional CT image is used to assist in navigating the lithotripsy position of the target object.

9. An electronic device, characterized in that, include: A processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions being executed by the processor to implement the steps of the PCNL fusion image puncture and lithotripsy navigation method based on optical positioning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the PCNL fusion image puncture and lithotripsy navigation method based on optical positioning as described in any one of claims 1 to 7.

Images