Catheter control device, method and storage medium

Abstract

The application relates to a catheter control device, method and storage medium. The device comprises an acquisition module for acquiring a catheter radius, a real-time position of the catheter, a real-time respiratory amplitude of a target object and a first three-dimensional model; a determination module for determining a target object radius according to the real-time position; a first control module for controlling the catheter to run in a first branch of the target object according to a first registration matrix and the first three-dimensional model when the difference between the target object radius and the catheter radius is greater than a preset difference value; and a second control module for determining a second three-dimensional model according to the real-time respiratory amplitude, a preset respiratory amplitude, the real-time position, the first registration matrix and the first three-dimensional model, and controlling the catheter to run in a second branch of the target object when the difference between the target object radius and the catheter radius is not greater than the preset difference value. The method can improve the catheter control accuracy.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a conduit control device, method and storage medium. Background Technology

[0002] In bronchoscopy, the advancement or bending of the endotracheal tube is often controlled manually using a handle or roller to navigate within the narrow bronchial lumen until it reaches the vicinity of the lesion. However, the shape of the bronchus is greatly affected by respiration, and manual control of the endotracheal tube suffers from low accuracy.

[0003] Meanwhile, electromagnetic sensors are commonly used for real-time navigation and positioning during catheter control. However, these sensors can only reflect the spatial positions of a few magnetic patch points under the magnetic field generator, lacking attitude information, and the accuracy of this spatial position information is low, exhibiting drift. When X-ray or fluorescence imaging is required for puncture, the magnetic field generator of the electromagnetic sensors needs to be removed. Any change in the generator's position will lead to inaccurate registration matrices. Therefore, real-time navigation and positioning based on electromagnetic sensors during catheter control suffers from low accuracy. Consequently, there is an urgent need for a catheter control device that can improve the accuracy of catheter control. Summary of the Invention

[0004] Therefore, it is necessary to provide a catheter control device, method, and computer-readable storage medium that can improve the accuracy of catheter control in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application also provides a catheter control device. The device includes:

[0006] The acquisition module is used to acquire the catheter radius, the real-time position of each catheter position point in the catheter when the catheter is running in the target object, the real-time respiratory amplitude of the target object, and the first three-dimensional model of the target object under the preset respiratory amplitude; the target object includes a first branch and a second branch;

[0007] The determination module is used to determine the radius of the target object at each target location point in the target object where the conduit has traveled, based on the real-time location.

[0008] The first control module is used to determine a first registration matrix based on the real-time position in the first branch and the first three-dimensional model when the difference between the radius of the target object and the radius of the catheter is greater than a preset difference; and to control the catheter to run in the first branch of the target object based on the first registration matrix and the first three-dimensional model.

[0009] The second control module is used to determine a second three-dimensional model based on real-time respiratory amplitude, preset respiratory amplitude, real-time position, first registration matrix, and first three-dimensional model, provided that the difference between the radius of the target object and the radius of the catheter is not greater than a preset difference; to perform registration based on the first three-dimensional model and the second three-dimensional model to obtain a second registration matrix and a registered three-dimensional model; and to control the catheter to run in the second branch of the target object based on the first registration matrix, the second registration matrix, the first three-dimensional model, and the registered three-dimensional model.

[0010] In one embodiment, a first registration matrix is ​​determined based on the real-time position in the first branch and the first 3D model, and the first control module is further configured to:

[0011] Based on the real-time location, a third 3D model of the target object is obtained through 3D reconstruction;

[0012] Based on the third 3D model corresponding to the first branch and the first 3D model corresponding to the first branch, the first registration matrix is ​​obtained through rigid body registration.

[0013] In one embodiment, based on a first registration matrix and a first three-dimensional model, the first control module controls the conduit to run in a first branch of the target object, and the first control module is further configured to:

[0014] Based on the first 3D model, determine the initial navigation path;

[0015] Based on the first registration matrix, the initial navigation path is transformed to obtain the transformed navigation path;

[0016] The control conduit runs along the transformed navigation path in the first branch of the target object.

[0017] In one embodiment, a second three-dimensional model is determined based on real-time respiratory amplitude, preset respiratory amplitude, real-time position, a first registration matrix, and a first three-dimensional model. The second control module is further configured to:

[0018] When the real-time respiratory amplitude is equal to the preset respiratory amplitude, determine the first real-time position of each catheter location point;

[0019] When the real-time respiratory amplitude is not equal to the preset respiratory amplitude, the second real-time position of each catheter position point at each respiratory moment is determined; based on the second real-time position of each catheter position point at each respiratory moment, the first real-time position of each catheter position point, the first registration matrix and the first three-dimensional model, the second three-dimensional model is determined.

[0020] In one embodiment, the second three-dimensional model is determined based on the second real-time position of each catheter position point at each breathing time, the first real-time position of each catheter position point, the first registration matrix, and the first three-dimensional model. The second control module is further configured to:

[0021] For the current breathing moment, the position offset vector of each catheter position point is determined based on the second real-time position and the first real-time position of each catheter position point at the current breathing moment;

[0022] The position offset vector is transformed into coordinates based on the first registration matrix to obtain the transformed offset vector; the second three-dimensional model is determined based on the transformed offset vectors at each catheter position point at the current breathing time and the first three-dimensional model.

[0023] In one embodiment, a second three-dimensional model is determined based on the converted offset vectors at each catheter position point at the current breathing moment and the first three-dimensional model. The second control module is further configured to:

[0024] Determine the cross-section and centerline corresponding to each catheter location point in the first three-dimensional model;

[0025] The corresponding cross section and the corresponding center line are updated based on the converted offset vector at each catheter position point at the current breathing time, resulting in the updated cross section and the updated center line.

[0026] The second three-dimensional model is determined based on the updated cross-sections and updated centerlines at each catheter location point at each breathing time.

[0027] In one embodiment, based on a first registration matrix, a second registration matrix, a first 3D model, and a registered 3D model, the conduit is controlled to run in a second branch of the target object. The second control module is further configured to:

[0028] Determine the calibration cycle; each calibration cycle includes multiple respiratory cycles.

[0029] For the first respiratory cycle of each calibration cycle, the dynamic navigation path is determined based on the position offset vector of each catheter position point at each respiratory moment in the first respiratory cycle and the initial navigation path; the initial navigation path is obtained based on the first three-dimensional model.

[0030] For each calibration cycle, the dynamic navigation path is determined at each respiratory moment in the non-first respiratory cycle based on the second registration matrix and the initial navigation path.

[0031] Based on the first registration matrix, coordinate transformation is performed on the dynamic navigation path at each breathing time in each calibration cycle to obtain the transformed dynamic navigation path.

[0032] Based on the registered 3D model, the control conduit runs in the second branch of the target object according to the transformed dynamic navigation path.

[0033] In one embodiment, based on the registered 3D model, the control conduit is controlled to run along the transformed dynamic navigation path in the second branch of the target object. The second control module is further configured to:

[0034] Control the virtual camera to run in the registered 3D model according to the converted dynamic navigation path, and acquire real-time virtual images captured by the virtual camera;

[0035] Acquire real-time images of the duct as it travels along the transformed dynamic navigation path in the second branch of the target object.

[0036] Determine the similarity between real-time virtual images and real-time real images;

[0037] If the similarity is less than the preset similarity value, the control conduit stops running in the second branch of the target object;

[0038] If the similarity is not less than the preset similarity value, the control conduit runs in the second branch of the target object according to the transformed dynamic navigation path.

[0039] Secondly, this application provides a catheter control method. The method includes:

[0040] The system acquires the catheter radius, the real-time position of each catheter point within the target object during its operation, the real-time respiratory amplitude of the target object, and the first three-dimensional model of the target object under a preset respiratory amplitude; the target object includes a first branch and a second branch.

[0041] Based on the real-time location, determine the radius of the target object at each target location point within the target object where the conduit travels;

[0042] If the difference between the radius of the target object and the radius of the catheter is greater than a preset difference, the first registration matrix is ​​determined based on the real-time position in the first branch and the first three-dimensional model; based on the first registration matrix and the first three-dimensional model, the catheter is controlled to run in the first branch of the target object.

[0043] If the difference between the radius of the target object and the radius of the catheter is not greater than a preset difference, a second three-dimensional model is determined based on the real-time respiratory amplitude, the preset respiratory amplitude, the real-time position, the first registration matrix, and the first three-dimensional model. The first and second three-dimensional models are registered to obtain the second registration matrix and the registered three-dimensional model. Based on the first registration matrix, the second registration matrix, the first three-dimensional model, and the registered three-dimensional model, the catheter is controlled to run in the second branch of the target object.

[0044] Thirdly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0045] The system acquires the catheter radius, the real-time position of each catheter point within the target object during its operation, the real-time respiratory amplitude of the target object, and the first three-dimensional model of the target object under a preset respiratory amplitude; the target object includes a first branch and a second branch.

[0046] Based on the real-time location, determine the radius of the target object at each target location point within the target object where the conduit travels;

[0047] If the difference between the radius of the target object and the radius of the catheter is greater than a preset difference, the first registration matrix is ​​determined based on the real-time position in the first branch and the first three-dimensional model; based on the first registration matrix and the first three-dimensional model, the catheter is controlled to run in the first branch of the target object.

[0048] If the difference between the radius of the target object and the radius of the catheter is not greater than a preset difference, a second three-dimensional model is determined based on the real-time respiratory amplitude, the preset respiratory amplitude, the real-time position, the first registration matrix, and the first three-dimensional model. The first and second three-dimensional models are registered to obtain the second registration matrix and the registered three-dimensional model. Based on the first registration matrix, the second registration matrix, the first three-dimensional model, and the registered three-dimensional model, the catheter is controlled to run in the second branch of the target object.

[0049] The aforementioned catheter control device, method, and storage medium determine the radius of the target object at each target position point by measuring the real-time position of the catheter at each position point within the catheter as it travels within the target object. When the catheter travels to the first branch with a larger target object radius, it controls the catheter's movement in the first branch based on a first registration matrix and a first three-dimensional model. Since the first registration matrix is ​​based on the real-time position of the catheter in the first branch, it helps improve the accuracy of catheter control in the first branch. When the catheter travels to the second branch with a smaller target object radius, a second three-dimensional model is determined based on the real-time respiratory amplitude of the target object, a preset respiratory amplitude, the real-time position of the catheter, the first registration matrix, and the first three-dimensional model. This second model reflects the real-time shape of the target object in the respiratory scenario. Registration is performed based on the second three-dimensional model, and the catheter's movement is controlled in the second branch, which helps improve the accuracy of catheter control in the second branch. Attached Figure Description

[0050] Figure 1 This is a diagram illustrating the application environment of the catheter control device in one embodiment;

[0051] Figure 2 This is a structural block diagram of the catheter control device in one embodiment;

[0052] Figure 3 This is a schematic diagram of multiple shape fiber optic positioning sensors installed throughout the entire length of a catheter in one embodiment;

[0053] Figure 4 This is a schematic diagram of the internal structure of the catheter in one embodiment;

[0054] Figure 5 This is a schematic diagram of a shape fiber optic positioning sensor in a conduit in one embodiment;

[0055] Figure 6 This is a schematic diagram of a patch-type breathing sensor in one embodiment;

[0056] Figure 7 This is a schematic diagram of a bandage-type breathing sensor in one embodiment;

[0057] Figure 8 This is a schematic diagram of a bronchial model in one embodiment;

[0058] Figure 9 This is a waveform diagram of the real-time respiratory amplitude during a respiratory cycle in one embodiment;

[0059] Figure 10 This is a schematic diagram of the second three-dimensional model in one embodiment;

[0060] Figure 11 This is a schematic diagram illustrating the registration of a first 3D model and a second 3D model in one embodiment.

[0061] Figure 12 This is a schematic diagram of the cross-sections and center lines in the first three-dimensional model in one embodiment;

[0062] Figure 13 This is a schematic diagram of updating the cross-section and centerline in one embodiment;

[0063] Figure 14 This is a schematic diagram illustrating the acquisition of real-time real images and real-time virtual images in one embodiment;

[0064] Figure 15 This is a flowchart illustrating a catheter control method in one embodiment;

[0065] Figure 16 This is a schematic diagram of the bronchial shape under different respiratory phases in one embodiment;

[0066] Figure 17 This is a schematic diagram of the model reconstruction method in one embodiment;

[0067] Figure 18 This is a schematic diagram of a preparation phase method in one embodiment;

[0068] Figure 19 This is a flowchart illustrating the catheter control method in another embodiment;

[0069] Figure 20 This is a schematic diagram illustrating the second three-dimensional model in one embodiment;

[0070] Figure 21 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0072] The catheter control device provided in this application embodiment can be applied to, for example... Figure 1The application environment shown is illustrated. The catheter control device, applied to the catheter control system, includes: catheter 120, positioning sensor 140, breathing sensor 160, and computer device 180. Positioning sensor 140 detects the real-time position of each catheter position point within the catheter 120 as it moves through the target object. Breathing sensor 160 detects the real-time respiratory amplitude of the target object. Computer device 180 includes a memory and a processor; the memory stores computer programs. In some embodiments, computer device 180 can be a terminal or a server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0073] In one embodiment, such as Figure 2 As shown, a catheter control device 200 is provided, which includes: an acquisition module 220, a determination module 240, a first control module 260, and a second control module 280.

[0074] The acquisition module 220 is used to acquire the catheter radius, the real-time position of each catheter position point in the catheter when the catheter is running in the target object, the real-time respiratory amplitude of the target object, and the first three-dimensional model of the target object under the preset respiratory amplitude; the target object includes a first branch and a second branch.

[0075] Here, the conduit radius refers to the radius of the conduit to be controlled. Computer equipment controls the conduit's movement within the target object, and the conduit radius is smaller than the target object's minimum radius.

[0076] Real-time position refers to the real-time position of the catheter at various points along its length as it moves through the target object. Positioning sensors are installed at multiple points along the entire length of the catheter. A computer acquires the real-time positions of these points as the catheter moves through the target object, as captured by the positioning sensors. In some embodiments, the positioning sensors may be shape-guided fiber optic positioning sensors. Figure 3 The diagram shows multiple shape-based fiber optic positioning sensors installed throughout the entire catheter segment. Figure 4The diagram shows the internal structure of the catheter. At least one guidewire is inserted inside the catheter; this guidewire can extend and shorten, allowing the catheter tip to bend in at least one direction. A working channel is provided in the middle of the catheter for the entry of equipment such as biopsy and ablation devices. An image acquisition device is installed at the catheter tip to observe the internal structure of the bronchus and capture real endoscopic images for catheter navigation. At least one illumination lamp is installed at the catheter tip to provide a light source. At least one shaped optical fiber is inserted inside the catheter to accurately sense the shape of the catheter within the patient's body in real time, locating the position and orientation of the catheter tip. For example, four shaped optical fibers can be evenly and symmetrically distributed inside the catheter. Figure 5 The diagram illustrates a shape fiber optic positioning sensor within a catheter. Each shape fiber extends along the entire length of the catheter, measuring its shape and strain changes throughout the entire length. Several Bragg fiber gratings are evenly distributed along each shape fiber, sensing catheter strain and reflecting changes in the wavelength of the reflected wave, which is then converted into spatial coordinate information for positioning. The discrete points between several sensors are interpolated to obtain their positional information, thus describing the three-dimensional shape of the entire fiber optic segment, i.e., the entire catheter segment. Due to the excellent flexibility of optical fibers and their immunity to electromagnetic interference, they can be used with preoperative CT (Computed Tomography) equipment, intraoperative CBCT (Cone Beam Computed Tomography), X-ray, and fluoroscopic imaging equipment. Installing shape fiber optic positioning sensors throughout the entire catheter segment not only allows for the positioning of the catheter tip but also describes the overall shape and orientation of the catheter, facilitating accurate real-time catheter positioning.

[0077] Real-time respiratory amplitude refers to the respiratory amplitude of a target object at each breathing moment, as collected by a respiratory sensor. The respiratory sensor, made of fiber optics or other electromagnetically unaffected sensors, is fabricated into a respiratory patch or bandage and is attached to or strapped to the patient's chest to detect respiratory amplitude. For example... Figure 6 The diagram shows a patch-type respiration sensor. Three respiration patches, made of optical fibers or other sensors resistant to electromagnetic interference, are attached to skeletal landmarks on the subject: one on the sternum and two on the ribs. The patches can also be placed on other skeletal landmarks. Together, they detect changes in the thoracic cavity caused by respiratory movements. Figure 7The diagram shows a bandage-type respiratory sensor. A respiratory bandage, made of fiber optic cable or other electromagnetically unaffected sensors, is attached to the chest of the subject to detect respiratory amplitude. The preset respiratory amplitude refers to the respiratory amplitude of the target object collected by the respiratory sensor at a specific breathing moment. A computer-controlled image scanning device captures multiple scanned images of the target object at that breathing moment, and performs 3D reconstruction on these images to obtain a first 3D model of the target object under the preset respiratory amplitude.

[0078] The target object includes a first branch and a second branch. In some embodiments, the radius of the target object in the first branch is larger than its radius in the second branch. For example, the target object is a bronchial model, the first branch is the main branch of the bronchial model, and the second branch is a non-main branch of the bronchial model. Figure 8 The diagram shows a schematic of a bronchial model. The shape of the first branch is less affected by respiratory movements, while the shape of the second branch is more affected. In other words, the first branch represents the target object whose shape is less affected by respiratory movements, while the second branch represents the target object whose shape is more affected by respiratory movements.

[0079] The determination module 240 is used to determine the radius of the target object at each target location point in the target object when the conduit runs to the target object, based on the real-time position.

[0080] The radius of the target object refers to the radius at each target location point within the target object. The conduit operates within the target object, and because positioning sensors are installed throughout its entire length, the real-time position of the conduit within the target object can determine the positional information of each point within the target object's internal space, thus obtaining the radius at each target location point.

[0081] The first control module 260 is used to determine a first registration matrix based on the real-time position in the first branch and the first three-dimensional model when the difference between the radius of the target object and the radius of the catheter is greater than a preset difference; and to control the catheter to run in the first branch of the target object based on the first registration matrix and the first three-dimensional model.

[0082] Since the radius of the conduit is smaller than the radius of the target object, the computer device calculates the difference between the radius of the target object and the radius of the conduit, and compares the resulting difference with a preset difference value.

[0083] If the difference between the radius of the target object and the radius of the conduit is greater than a preset difference, it indicates that the conduit has reached the first branch of the target object. The computer device determines the first registration matrix based on the real-time position in the first branch and the first 3D model. Since the real-time position is the location information collected by the positioning sensor, and the first 3D model is obtained based on the scanned image, registering the real-time position and the first 3D model yields the first registration matrix, which is used to characterize the transformation relationship between the real-time position space and the 3D model space.

[0084] When the computer device controls the conduit to run in the first branch of the target object, it controls the conduit in real-time position space. The computer device pre-plans the navigation path of the conduit in the first three-dimensional model, and converts the first three-dimensional model and the planned navigation path to the real-time position space according to the first registration matrix and the first three-dimensional model, thereby controlling the conduit to run in the first branch of the target object.

[0085] The second control module 280 is used to determine a second three-dimensional model based on real-time respiratory amplitude, preset respiratory amplitude, real-time position, first registration matrix, and first three-dimensional model, provided that the difference between the radius of the target object and the radius of the catheter is not greater than a preset difference; to perform registration based on the first three-dimensional model and the second three-dimensional model to obtain a second registration matrix and a registered three-dimensional model; and to control the catheter to run in the second branch of the target object based on the first registration matrix, the second registration matrix, the first three-dimensional model, and the registered three-dimensional model.

[0086] If the difference between the radius of the target object and the radius of the conduit is not greater than a preset difference, it indicates that the conduit has reached the second branch of the target object.

[0087] The computer equipment determines a second three-dimensional model based on real-time respiratory amplitude, preset respiratory amplitude, real-time position, a first registration matrix, and a first three-dimensional model. The second three-dimensional model is a dynamically changing three-dimensional model of the second branch of the target object, reflecting its real-time respiratory status. For example... Figure 9 The diagram shows the waveform of real-time respiratory amplitude changing over time during a respiratory cycle. The horizontal axis represents time, and the vertical axis represents real-time respiratory amplitude. At the end of inspiration, the real-time respiratory amplitude is at its peak, and at the end of expiration, it is at its trough. The intermediate phase is any respiratory phase between the peak and the trough. The second three-dimensional model of each respiratory moment in a respiratory cycle changes in real time, and the second three-dimensional model of the same respiratory moment in multiple respiratory cycles is the same or similar. Figure 10 The diagram shows a schematic of the second 3D model. With respiratory movement, the second branch of the target object moves from the solid line position to the dashed line position. Since the second branch of the target object is significantly affected by breathing, using the target object's real-time breathing data is beneficial for obtaining an accurate 3D model of the target object.

[0088] like Figure 11 The diagram illustrates the registration of the first and second 3D models. Registering the first and second 3D models yields a second registration matrix and the registered 3D model. Since the second 3D model dynamically changes with respiration, the resulting second registration matrix also changes dynamically. The registered 3D model is a 3D model of the target object that dynamically changes with respiratory motion. Based on the first registration matrix, the second registration matrix, the first 3D model, and the registered 3D model, the computer device converts the navigation path in the registered 3D model into a real-time position space, thereby controlling the catheter to operate in the second branch of the target object.

[0089] The aforementioned catheter control device determines the radius of the target object at each target position point by measuring the real-time position of the catheter as it travels within the target object. In the first branch, where the target object radius is larger, the device controls the catheter's movement based on a first registration matrix and a first 3D model. Since the first registration matrix is ​​derived from the real-time position of the catheter in the first branch, it improves the accuracy of catheter control in this area. When the catheter reaches the second branch, where the target object radius is smaller, a second 3D model is determined based on the target object's real-time respiratory amplitude, a preset respiratory amplitude, the catheter's real-time position, the first registration matrix, and the first 3D model. This second 3D model reflects the real-time shape of the target object in the respiratory scenario. Registration is performed based on this second 3D model, and the catheter's movement is controlled in the second branch, further improving the accuracy of catheter control in this area.

[0090] In one embodiment, a first registration matrix is ​​determined based on the real-time position in the first branch and the first three-dimensional model. The first control module is further configured to: obtain a third three-dimensional model of the target object through three-dimensional reconstruction based on the real-time position; and obtain the first registration matrix through rigid body registration based on the third three-dimensional model corresponding to the first branch and the first three-dimensional model corresponding to the first branch.

[0091] In this system, because the entire catheter is equipped with positioning sensors, the real-time position of the catheter as it moves within the target object is collected and forms a point cloud in the real-time position space. The point cloud is then reconstructed in three dimensions to obtain a third-dimensional model of the target object. In some embodiments, the cross-sections of the target object where the catheter reaches each target position point are determined, and the coordinates of points on each cross-section in the third-dimensional model are determined. Based on the coordinates of the points on each cross-section, the radius of the target object at each target position point is determined.

[0092] Rigid body registration refers to the transformation process of mapping one model to another, generating a registration matrix. Rigid body registration does not change the distance between any two points in the model and generally includes translation and rotation transformations. The first registration matrix is ​​used to characterize the mapping relationship between the third 3D model and the first 3D model in the first branch. In some embodiments, the real-time position is acquired by a shape fiber positioning sensor. The conduit has at least one shape fiber, each extending along the entire length of the conduit, and each shape fiber has its own corresponding coordinate system zero point. For example, the coordinate system zero point is physically fixed in a fixed position of the robotic arm, so that the coordinate systems of multiple shape fibers can be mechanically registered. Each fiber can independently measure its spatial position and calculate the relative positions between the measurement points of multiple shape fibers. Based on the coordinate system zero point of the shape fiber, a fiber coordinate system is established. Based on the first 3D model, a model coordinate system is established. The first registration matrix is ​​used to characterize the transformation relationship between the fiber coordinate system and the model coordinate system.

[0093] In this embodiment, the real-time position of the catheter facilitates the acquisition of an accurate third-dimensional model of the target object. A first registration matrix is ​​obtained by rigidly registering the third-dimensional model corresponding to the first branch with the first-dimensional model corresponding to the first branch. Since navigation planning is often performed preoperatively using the first-dimensional model, obtaining the first registration matrix based on the third-dimensional model of the target object facilitates coordinate transformation during catheter control, thereby improving the accuracy of catheter control.

[0094] In one embodiment, the first control module controls the conduit to run in the first branch of the target object according to the first registration matrix and the first three-dimensional model. The first control module is further configured to: determine an initial navigation path according to the first three-dimensional model; perform coordinate transformation on the initial navigation path according to the first registration matrix to obtain a transformed navigation path; and control the conduit to run in the first branch of the target object according to the transformed navigation path.

[0095] The initial navigation path is the navigation path of the conduit determined in the first 3D model. The initial navigation path is the navigation path in the model coordinate system.

[0096] The computer device multiplies the initial navigation path by a first registration matrix, thereby transforming the initial navigation path to a fiber optic coordinate system, resulting in a transformed navigation path. The computer device controls the catheter to move along the transformed navigation path in the first branch of the target object. If the lesion area is in the first branch, and the initial navigation path is also located in the first branch, the catheter will only move within the first branch. If the lesion area is in the second branch, and the initial navigation path is located in both the first and second branches, the catheter will automatically enter the second branch after completing its movement in the first branch, following the navigation path's indication.

[0097] In this embodiment, an initial navigation path is determined in the first three-dimensional model. Based on the first registration matrix, the initial navigation path is transformed to obtain a transformed navigation path. The duct is then controlled to run along the transformed navigation path in the first branch of the target object. Transforming the initial navigation path using the first registration matrix, thereby controlling the duct to run along the transformed navigation path, improves the accuracy of duct control.

[0098] In one embodiment, a second three-dimensional model is determined based on the real-time respiratory amplitude, the preset respiratory amplitude, the real-time position, the first registration matrix, and the first three-dimensional model. The second control module is further configured to: determine the first real-time position of each catheter position point when the real-time respiratory amplitude is equal to the preset respiratory amplitude; determine the second real-time position of each catheter position point at each respiratory moment when the real-time respiratory amplitude is not equal to the preset respiratory amplitude; and determine the second three-dimensional model based on the second real-time position of each catheter position point at each respiratory moment, the first real-time position of each catheter position point, the first registration matrix, and the first three-dimensional model.

[0099] During catheter operation control, a respiratory sensor continuously monitors the real-time respiratory amplitude of the target object. If the real-time respiratory amplitude equals the preset respiratory amplitude, it indicates that the third-dimensional model of the target object is identical to the first-dimensional model at the current moment. The computer then determines the first real-time position of each catheter location point based on the equality of the real-time and preset respiratory amplitudes.

[0100] The discrepancy between the real-time respiratory amplitude and the preset respiratory amplitude indicates that the third-dimensional model of the target object at the current moment differs from the existing three-dimensional model. When the real-time respiratory amplitude differs from the preset respiratory amplitude, the computer equipment determines the second real-time position of each duct location at each respiratory moment.

[0101] The computer equipment determines the second three-dimensional model based on the second real-time position of each catheter location at each respiratory moment, the first real-time position of each catheter location, the first registration matrix, and the first three-dimensional model. Determining the second three-dimensional model based on the real-time positions of each catheter location at each respiratory moment improves the accuracy of catheter control for target objects significantly affected by respiration.

[0102] In this embodiment, by determining the first real-time position and the second real-time position of each catheter location point when the real-time respiratory amplitude is equal to or unequal to the preset respiratory amplitude, and by determining the second real-time position of each catheter location point, the first real-time position of each catheter location point, the first registration matrix, and the first three-dimensional model at each respiratory moment, the second three-dimensional model is determined, which can ensure improved accuracy of catheter control in target objects that are greatly affected by breathing.

[0103] In one embodiment, a second three-dimensional model is determined based on the second real-time position of each catheter position point at each respiratory moment, the first real-time position of each catheter position point, the first registration matrix, and the first three-dimensional model. The second control module is further configured to: for the current respiratory moment, determine the position offset vector of each catheter position point at the current respiratory moment based on the second real-time position of each catheter position point and the first real-time position of each catheter position point at the current respiratory moment; perform coordinate transformation on the position offset vector according to the first registration matrix to obtain the transformed offset vector; and determine the second three-dimensional model based on the transformed offset vector of each catheter position point at the current respiratory moment and the first three-dimensional model.

[0104] Here, "current breathing time" refers to any breathing moment. For each breathing moment, the computer device subtracts the second real-time position of each catheter location from its corresponding first real-time position to obtain the position offset vector at each catheter location. The computer device then multiplies the position offset vector by a first registration matrix, thereby transforming the position offset vector into a three-dimensional model space to obtain the transformed offset vector.

[0105] The computer equipment determines the second three-dimensional model based on the converted offset vectors at each catheter position point at the current respiratory moment and the first three-dimensional model. For each respiratory moment, the second three-dimensional model at that specific respiratory moment is obtained using the method described above.

[0106] In this embodiment, by determining the position offset vectors of the second real-time positions of each catheter position point relative to the first real-time position at the current breathing time, and converting the position offset vectors into the three-dimensional model space, a second three-dimensional model is determined based on the first three-dimensional model. The second three-dimensional model is obtained based on the breathing situation at each breathing time, which is beneficial to improving the control accuracy of the catheter in target objects that are greatly affected by breathing.

[0107] In one embodiment, a second three-dimensional model is determined based on the converted offset vectors at each catheter position point at the current respiratory time and the first three-dimensional model. The second control module is further configured to: determine the cross-section and the corresponding centerline of each catheter position point in the first three-dimensional model; update the corresponding cross-section and the corresponding centerline based on the converted offset vectors at each catheter position point at the current respiratory time to obtain the updated cross-section and the updated centerline; and determine the second three-dimensional model based on the updated cross-section and the updated centerline at each catheter position point at each respiratory time.

[0108] Here, a cross-section refers to the cross-section containing any point within any branch of the first 3D model of the target object. A centerline refers to the centerline within any branch of the first 3D model of the target object. Within the same branch, the cross-section containing any point is perpendicular to the centerline. For example... Figure 12 The diagram shows the cross-sections and center lines in the first three-dimensional model.

[0109] The computer equipment transforms each catheter location point into a three-dimensional model space through a first registration matrix, thereby determining the corresponding cross-section and centerline of each catheter location point in the first three-dimensional model.

[0110] The computer equipment determines the coordinates of points on each cross-section and the coordinates of points on the centerline. The coordinates of each cross-section point and the coordinates of the centerline point are then added to the corresponding transformed offset vector to obtain updated cross-section points and updated centerline points. From each updated cross-section point, an updated cross-section is fitted; from each updated centerline point, an updated centerline is fitted. For example... Figure 13 The diagram shows the updated cross-section and centerline. The updated cross-section and centerline at each catheter location point at the current respiratory time are used to determine the second 3D model at the current respiratory time. The same method is used to obtain the second 3D model at each respiratory time.

[0111] In this embodiment, the cross-section and centerline at the corresponding catheter location point are updated by the converted offset vector to obtain the second three-dimensional model. The second three-dimensional model changes dynamically with the respiratory motion at each breathing moment, which helps to improve the accuracy of catheter control.

[0112] In one embodiment, based on a first registration matrix, a second registration matrix, a first 3D model, and a registered 3D model, the catheter is controlled to run in the second branch of the target object. The second control module is further configured to: determine a calibration cycle; each calibration cycle includes multiple respiratory cycles; for the first respiratory cycle of each calibration cycle, determine a dynamic navigation path for each respiratory moment in the first respiratory cycle based on the position offset vector of each catheter position point at each respiratory moment in the first respiratory cycle and the initial navigation path; the initial navigation path is obtained based on the first 3D model; for non-first respiratory cycles of each calibration cycle, determine a dynamic navigation path for each respiratory moment in the non-first respiratory cycle based on the second registration matrix and the initial navigation path; perform coordinate transformation on the dynamic navigation path for each respiratory moment in each calibration cycle based on the first registration matrix to obtain a transformed dynamic navigation path; and control the catheter to run in the second branch of the target object according to the transformed dynamic navigation path based on the registered 3D model.

[0113] The calibration cycle refers to the period during which the second 3D model is calibrated. For example, calibration can be performed every preset time interval, or every preset distance the catheter travels. Each calibration cycle includes multiple respiratory cycles.

[0114] For the first respiratory cycle of each calibration period, the initial navigation path corresponding to each catheter position point at the current respiratory moment is added to the corresponding position offset vector to obtain the dynamic navigation path at the current respiratory moment, thus obtaining the dynamic navigation path at each respiratory moment in the first respiratory cycle. The position offset vector of each catheter position point in the first respiratory cycle of the calibration period is obtained based on the first real-time position and the second real-time position of each catheter position point. The first real-time position is the position of each catheter position point when the real-time respiratory amplitude is equal to the preset respiratory amplitude, and the second real-time position is the position of each catheter position point when the real-time respiratory amplitude is not equal to the preset respiratory amplitude. In some embodiments, when the real-time respiratory amplitude is detected to be equal to the preset respiratory amplitude, the catheter is controlled to stop running in the target object, and after one respiratory cycle, the catheter is controlled to continue running.

[0115] The initial navigation path is obtained based on the first 3D model. For each calibration cycle, not the first respiratory cycle, the computer device multiplies the initial navigation path by the second registration matrix to obtain the dynamic navigation path at each respiratory moment in the non-first respiratory cycle.

[0116] The computer equipment performs coordinate transformation on the dynamic navigation path at each breathing moment in each calibration cycle based on the first registration matrix, obtaining the transformed dynamic navigation path. Since the registered 3D model changes continuously with respiratory motion, the computer equipment controls the catheter to run in the second branch of the target object according to the transformed dynamic navigation path, achieving precise control of the catheter and avoiding damage to the target object caused by inaccurate catheter control during respiratory motion.

[0117] In this embodiment, by determining the calibration cycle, the catheter is controlled to stop running for one respiratory cycle in each calibration cycle, thereby calibrating and updating the second three-dimensional model and further improving the accuracy of catheter control.

[0118] In one embodiment, based on the registered 3D model, the control conduit runs along the converted dynamic navigation path in the second branch of the target object. The second control module is further configured to: control a virtual camera to run along the converted dynamic navigation path in the registered 3D model and acquire real-time virtual images captured by the virtual camera; acquire real-time real images captured by the conduit while running along the converted dynamic navigation path in the second branch of the target object; determine the similarity between the real-time virtual image and the real-time real image; if the similarity is less than a preset similarity value, control the conduit to stop running in the second branch of the target object; if the similarity is not less than the preset similarity value, control the conduit to run along the converted dynamic navigation path in the second branch of the target object.

[0119] In this system, computer equipment generates a virtual camera that simulates the real image acquisition device, based on the device's location and shooting parameters within the target object. The real image acquisition device, located within a conduit, captures real-time images as it navigates along a converted dynamic navigation path within the second branch of the target object. The virtual camera, simulating the real device, captures real-time virtual images by navigating the registered 3D model along the same path. Figure 14 The diagram shows the acquisition of real-time images and real-time virtual images.

[0120] The computer equipment compares the similarity of real-time virtual images and real-time images at the same location points to obtain the similarity score. If the similarity score is less than a preset similarity value, it indicates a significant difference between the real-time virtual image and the real-time image. In this case, the computer equipment controls the conduit to stop moving in the second branch of the target object to prevent further damage. If the similarity score is not less than the preset similarity value, it indicates a high similarity between the real-time virtual image and the real-time image. The resulting registered 3D model and dynamic navigation path have a high degree of matching with the real scene. In this case, the computer equipment controls the conduit to run according to the converted dynamic navigation path in the second branch of the target object, improving the accuracy of conduit control.

[0121] In this embodiment, by comparing the similarity between the real-time virtual image captured by the virtual camera and the real-time real image captured by the real image acquisition device, if the similarity is less than a preset similarity value, the conduit is controlled to stop running in the second branch of the target object, which can avoid errors in conduit control and damage to the target object; if the similarity is not less than the preset similarity value, the conduit is controlled to run in the second branch of the target object according to the converted dynamic navigation path, which improves the accuracy of conduit control.

[0122] Each module in the aforementioned catheter control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0123] In one embodiment, such as Figure 15 As shown, a catheter control method is provided, which is applied to a catheter control system. Figure 1 The following steps are used as an example to illustrate the computer equipment 108 of the central guide tube control system:

[0124] Step 1502: Obtain the catheter radius, the real-time position of each catheter position point in the catheter when the catheter is running in the target object, the real-time respiratory amplitude of the target object, and the first three-dimensional model of the target object under the preset respiratory amplitude; the target object includes the first branch and the second branch.

[0125] Step 1504: Based on the real-time location, determine the radius of the target object at each target location point in the target object where the conduit runs.

[0126] Step 1506: When the difference between the radius of the target object and the radius of the catheter is greater than a preset difference, determine the first registration matrix based on the real-time position in the first branch and the first three-dimensional model; control the catheter to run in the first branch of the target object based on the first registration matrix and the first three-dimensional model.

[0127] Step 1508: If the difference between the radius of the target object and the radius of the catheter is not greater than a preset difference, determine the second three-dimensional model based on the real-time respiratory amplitude, the preset respiratory amplitude, the real-time position, the first registration matrix, and the first three-dimensional model; perform registration based on the first three-dimensional model and the second three-dimensional model to obtain the second registration matrix and the registered three-dimensional model; based on the first registration matrix, the second registration matrix, the first three-dimensional model, and the registered three-dimensional model, control the catheter to run in the second branch of the target object.

[0128] In the aforementioned catheter control method, the radius of the target object at each target position point is determined by the real-time position of the catheter at each position point within the catheter as it travels through the target object. In the first branch where the target object radius is larger, the catheter is controlled to move within the first branch based on the first registration matrix and the first 3D model. Since the first registration matrix is ​​based on the real-time position of the catheter in the first branch, it helps improve the accuracy of catheter control in the first branch. When the catheter moves to the second branch where the target object radius is smaller, the second 3D model, determined based on the real-time respiratory amplitude of the target object, the preset respiratory amplitude, the real-time position of the catheter, the first registration matrix, and the first 3D model, reflects the real-time shape of the target object in the respiratory scenario. Registration is performed based on the second 3D model, and the catheter movement is controlled in the second branch, which helps improve the accuracy of catheter control in the second branch.

[0129] In one embodiment, determining the first registration matrix based on the real-time position in the first branch and the first three-dimensional model includes: obtaining a third three-dimensional model of the target object through three-dimensional reconstruction based on the real-time position; and obtaining the first registration matrix through rigid body registration based on the third three-dimensional model corresponding to the first branch and the first three-dimensional model corresponding to the first branch.

[0130] In one embodiment, controlling the conduit to run in the first branch of the target object according to the first registration matrix and the first three-dimensional model includes: determining an initial navigation path according to the first three-dimensional model; performing coordinate transformation on the initial navigation path according to the first registration matrix to obtain a transformed navigation path; and controlling the conduit to run in the first branch of the target object according to the transformed navigation path.

[0131] In one embodiment, determining a second three-dimensional model based on real-time respiratory amplitude, preset respiratory amplitude, real-time position, first registration matrix, and first three-dimensional model includes: determining the first real-time position of each catheter position point when the real-time respiratory amplitude is equal to the preset respiratory amplitude; determining the second real-time position of each catheter position point at each respiratory moment when the real-time respiratory amplitude is not equal to the preset respiratory amplitude; and determining the second three-dimensional model based on the second real-time position of each catheter position point at each respiratory moment, the first real-time position of each catheter position point, the first registration matrix, and the first three-dimensional model.

[0132] In one embodiment, determining the second three-dimensional model based on the second real-time position of each catheter position point at each respiratory time, the first real-time position of each catheter position point, the first registration matrix, and the first three-dimensional model includes: for the current respiratory time, determining the position offset vector of each catheter position point at the current respiratory time based on the second real-time position of each catheter position point at the current respiratory time and the first real-time position of each catheter position point; performing coordinate transformation on the position offset vector according to the first registration matrix to obtain the transformed offset vector; and determining the second three-dimensional model based on the transformed offset vector of each catheter position point at the current respiratory time and the first three-dimensional model.

[0133] In one embodiment, determining a second three-dimensional model based on the converted offset vectors at each catheter position point at the current respiratory time and a first three-dimensional model includes: determining the cross-section and centerline corresponding to each catheter position point in the first three-dimensional model; updating the corresponding cross-section and centerline based on the converted offset vectors at each catheter position point at the current respiratory time to obtain updated cross-sections and updated centerlines; and determining the second three-dimensional model based on the updated cross-sections and updated centerlines at each catheter position point at each respiratory time.

[0134] In one embodiment, controlling the catheter to operate in the second branch of the target object based on a first registration matrix, a second registration matrix, a first three-dimensional model, and a registered three-dimensional model includes: determining a calibration cycle; each calibration cycle includes multiple respiratory cycles; for the first respiratory cycle of each calibration cycle, determining a dynamic navigation path at each respiratory moment of the first respiratory cycle based on the position offset vector of each catheter position point at each respiratory moment in the first respiratory cycle and the initial navigation path; the initial navigation path is obtained based on the first three-dimensional model; for non-first respiratory cycles of each calibration cycle, determining a dynamic navigation path at each respiratory moment of the non-first respiratory cycle based on the second registration matrix and the initial navigation path; performing coordinate transformation on the dynamic navigation path at each respiratory moment of each calibration cycle based on the first registration matrix to obtain a transformed dynamic navigation path; and controlling the catheter to operate in the second branch of the target object according to the transformed dynamic navigation path based on the registered three-dimensional model.

[0135] In one embodiment, based on the registered 3D model, controlling the conduit to run in the second branch of the target object according to the converted dynamic navigation path includes: controlling a virtual camera to run in the registered 3D model according to the converted dynamic navigation path and acquiring real-time virtual images captured by the virtual camera; acquiring real-time real images captured by the conduit while running in the second branch of the target object according to the converted dynamic navigation path; determining the similarity between the real-time virtual image and the real-time real image; stopping the conduit in the second branch of the target object if the similarity is less than a preset similarity value; and controlling the conduit to run in the second branch of the target object according to the converted dynamic navigation path if the similarity is not less than the preset similarity value.

[0136] To illustrate the catheter control method and its effects in this solution in detail, a specific embodiment is described below:

[0137] The catheter control method is applied to a catheter control system. The system includes: a catheter, a positioning sensor, a breathing sensor, and a computer device; the positioning sensor is used to detect the real-time position of the catheter at various points in the catheter as it moves through the target object; the breathing sensor is used to detect the real-time respiratory amplitude of the target object; the computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the catheter control method.

[0138] The computer device acquires the radius of the catheter, the real-time position of each point within the catheter as it moves through the target object, the real-time respiratory amplitude of the target object, and a first three-dimensional model of the target object under a preset respiratory amplitude. The target object includes a first branch and a second branch. In some embodiments, the target object can be a bronchus, the first branch is a main bronchial branch, and the second branch is a non-main bronchial branch. In scenarios where the catheter moves through the bronchus, the shape of the bronchus continuously changes with respiratory movements. Figure 16 The diagram shows the bronchial shape at different respiratory phases. It illustrates that the bronchi undergo different shape changes at different respiratory phases. Each respiratory phase in the respiratory signal graph corresponds to a specific bronchial respiratory motion state. The bronchial shape is most stretched at the end of inspiration, most relaxed and constricted at the end of expiration, and the bronchial shape at intermediate phases falls between the inspiration and expiration phases. Figure 17The diagram illustrates the model reconstruction process during catheter control. The model reconstruction method mainly includes: global registration of the main branch point cloud, reconstruction of the first respiratory motion, reconstruction of the second respiratory motion, and reconstruction of the third respiratory motion. In the global registration of the main branch point cloud: the catheter advances spirally within the first branch of the target object. Shape fiber optic positioning sensors record the real-time position point cloud of the entire catheter segment at all times, and perform rigid body registration with the first 3D model of the target object to obtain the first registration matrix between the fiber optic coordinate system and the model coordinate system. During the reconstruction of the first respiratory motion: the respiratory phase time that is the same as the preset respiratory phase of the first 3D model is recorded as Phase 0. Shape fiber optic positioning sensors on the entire catheter segment capture the real-time position information of the catheter, calculate its position in the model coordinate system, and the centerline point and the cross-sectional point perpendicular to the centerline corresponding to each real-time position point. Since the position of the respiratory sensors remains unchanged, the real-time respiratory motion at each moment can indicate the same respiratory state as the first 3D model. During the second respiratory motion reconstruction process: As the catheter remains within the target object, shape fiber optic positioning sensors along the entire catheter segment record the positional offset of each phase relative to phase 0. This offset vector is transformed from the fiber optic coordinate system to the model coordinate system and added to the corresponding cross-section, forming a new second 3D model in the new phase. During the third respiratory motion reconstruction process: The target object branch of the navigation path of the first 3D model is elastically registered with the corresponding branch of the second 3D model to obtain the registered 3D model. The registered 3D model is then refreshed on the visualization interface, displaying the dynamic respiratory motion within the target object in two-dimensional or three-dimensional form.

[0139] like Figure 18 The diagram illustrates the preparation phase. First, a respiratory sensor is attached to the patient's chest or placed near the mouth and nose to detect respiratory movements. Then, PET-CT (Positron Emission Computed Tomography-Computed Tomography), CT, or CBCT are used to acquire images of the patient's chest, with the respiratory sensor recording the respiratory phase (Phase 0) at the time of image acquisition. Based on the real-time location, the computer performs 3D reconstruction to obtain a third-dimensional model of the target object. Based on the coordinates of each point in the third-dimensional model, the radius of the target object at each target location point within the catheter is determined.

[0140] If the difference between the radius of the target object and the radius of the conduit is greater than a preset difference, a first registration matrix is ​​obtained through rigid body registration based on the third 3D model corresponding to the first branch and the first 3D model corresponding to the first branch. Based on the first 3D model, an initial navigation path is determined. According to the first registration matrix, the initial navigation path is transformed to obtain a transformed navigation path. The conduit is then controlled to run along the transformed navigation path within the first branch of the target object.

[0141] If the difference between the radius of the target object and the radius of the catheter is not greater than a preset difference, a second three-dimensional model is determined based on the real-time respiratory amplitude, the preset respiratory amplitude, the real-time position, the first registration matrix, and the first three-dimensional model. Specifically, if the real-time respiratory amplitude is equal to the preset respiratory amplitude, the first real-time position of each catheter position point is determined. If the real-time respiratory amplitude is not equal to the preset respiratory amplitude, the second real-time position of each catheter position point at each respiratory moment is determined. For the current respiratory moment, based on the second real-time position and the first real-time position of each catheter position point at the current respiratory moment, the position offset vector of each catheter position point at the current respiratory moment is determined. The position offset vector is transformed according to the first registration matrix to obtain the transformed offset vector, and the corresponding cross-section and centerline of each catheter position point in the first three-dimensional model are determined. The corresponding cross-section and centerline are updated according to the transformed offset vector of each catheter position point at the current respiratory moment to obtain the updated cross-section and centerline. Based on the updated cross-section and centerline of each catheter position point at each respiratory moment, the second three-dimensional model is determined.

[0142] like Figure 19The diagram illustrates the catheter control method. Initially, the catheter is placed at the center of the first branch of the target object. The real-time position of the catheter in fiber optic coordinates is transformed to the model coordinate system using a first registration matrix. In the first branch of the target object, the first registration matrix is ​​the global registration matrix obtained through global registration. In the second branch of the target object, where the radius of the target object is not significantly different from the radius of the catheter, the second registration matrix is ​​calculated in real-time based on respiratory motion. Based on the updated centerline or updated navigation path of the third 3D model, a new catheter motion direction and amplitude are planned in real-time. The advance direction and amplitude in the model coordinate system are converted to the advance direction and amplitude in the fiber optic coordinate system using the first registration matrix. The catheter motion control receives commands from the computer and executes corresponding actions. The real-time position of the catheter is updated in the model coordinate system using the first registration matrix, generating a real-time virtual image for catheter motion observation. The similarity between the real-time virtual endoscopic image and the real-time real image is judged. If they are not similar, it is considered that there is a deviation in the direction and position of the catheter automatic control. The movement control operation of the catheter is stopped and manual intervention is performed. If they are similar, it is considered that the catheter automatic control is proceeding smoothly according to the plan. Automatic control continues until the planned target position is reached.

[0143] In addition, the computer equipment registers the first and second 3D models to obtain a second registration matrix and a registered 3D model. A calibration cycle is determined, with each cycle comprising multiple respiratory cycles. For the first respiratory cycle of each calibration cycle, the dynamic navigation path is determined based on the position offset vectors of each catheter position point at each respiratory moment and the initial navigation path. The initial navigation path is obtained based on the first 3D model. For non-first respiratory cycles of each calibration cycle, the dynamic navigation path is determined based on the second registration matrix and the initial navigation path. The dynamic navigation path at each respiratory moment of each calibration cycle is then transformed according to the first registration matrix to obtain the transformed dynamic navigation path. Based on the registered 3D model, the catheter is controlled to run along the transformed dynamic navigation path in the second branch of the target object.

[0144] During navigation, the navigation page displays a real-time three-dimensional dynamic respiratory motion model of the target object in the form of a two-dimensional image, showing the real-time shape and posture of the catheter within the target object, and the location of the lesion, thus providing the operator with an intuitive understanding. For example... Figure 20 The image shown is a schematic diagram of the second 3D model. This second 3D model can also be displayed to the operator in virtual reality form using head-mounted VR (Virtual Reality) glasses, showing the model and the camera movement process.

[0145] The aforementioned catheter control method determines the radius of the target object at each target location point by measuring the real-time position of the catheter at each location point within the catheter as it travels through the target object. In the first branch, where the target object radius is larger, the catheter is controlled based on a first registration matrix and a first 3D model. Since the first registration matrix is ​​derived from the real-time position of the catheter in the first branch, it improves the accuracy of catheter control in this area. When the catheter travels to the second branch, where the target object radius is smaller, a second 3D model is determined based on the real-time respiratory amplitude of the target object, a preset respiratory amplitude, the real-time position of the catheter, the first registration matrix, and the first 3D model. This second 3D model reflects the real-time shape of the target object in the respiratory scenario. Registration is performed based on this second 3D model, and the catheter is controlled in the second branch, further improving the accuracy of catheter control in this area.

[0146] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0147] Based on the same inventive concept, this application also provides a catheter control device for implementing the catheter control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the catheter control device provided below can be found in the limitations of the catheter control method described above, and will not be repeated here.

[0148] In one embodiment, a computer device is provided; the computer device may be a terminal, and its internal structure diagram may be as shown in the figure below. Figure 21As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a conduit control method.

[0149] Those skilled in the art will understand that Figure 21 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0150] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0151] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0152] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0153] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0154] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0155] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A catheter control device, characterized in that, The device includes: The acquisition module is used to acquire the catheter radius, the real-time position of each catheter position point in the catheter when the catheter is running in the target object, the real-time respiratory amplitude of the target object, and the first three-dimensional model of the target object under a preset respiratory amplitude; the target object includes a first branch and a second branch; The determining module is used to determine the radius of the target object at each target location point in the target object where the conduit runs, based on the real-time position; The first control module is configured to, when the difference between the radius of the target object and the radius of the catheter is greater than a preset difference, determine a first registration matrix based on the real-time position in the first branch and the first three-dimensional model; and control the catheter to run in the first branch of the target object based on the first registration matrix and the first three-dimensional model. The second control module is used to determine a second three-dimensional model based on the real-time respiratory amplitude, the preset respiratory amplitude, the real-time position, the first registration matrix, and the first three-dimensional model, provided that the difference between the radius of the target object and the radius of the catheter is not greater than a preset difference; to perform registration based on the second three-dimensional model and the first three-dimensional model to obtain a second registration matrix and a registered three-dimensional model; and to control the catheter to run in the second branch of the target object based on the first registration matrix, the second registration matrix, the first three-dimensional model, and the registered three-dimensional model.

2. The apparatus according to claim 1, characterized in that, The first control module is further configured to: determine the first registration matrix based on the real-time position in the first branch and the first 3D model; Based on the real-time location, a third 3D model of the target object is obtained through 3D reconstruction; Based on the third 3D model corresponding to the first branch and the first 3D model corresponding to the first branch, the first registration matrix is ​​obtained through rigid body registration.

3. The apparatus according to claim 1, characterized in that, The first control module, which controls the conduit to run in the first branch of the target object based on the first registration matrix and the first three-dimensional model, is further configured to: Based on the first 3D model, determine the initial navigation path; Based on the first registration matrix, the initial navigation path is transformed to obtain the transformed navigation path; The control conduit runs along the transformed navigation path in the first branch of the target object.

4. The apparatus according to claim 1, characterized in that, The second control module is further configured to determine the second three-dimensional model based on the real-time respiratory amplitude, the preset respiratory amplitude, the real-time position, the first registration matrix, and the first three-dimensional model, and to: When the real-time respiratory amplitude is equal to the preset respiratory amplitude, the first real-time position at each catheter location point is determined; When the real-time respiratory amplitude is not equal to the preset respiratory amplitude, the second real-time position of each catheter position point at each respiratory moment is determined; based on the second real-time position of each catheter position point at each respiratory moment, the first real-time position of each catheter position point, the first registration matrix and the first three-dimensional model, the second three-dimensional model is determined.

5. The apparatus according to claim 4, characterized in that, The second control module is further configured to determine the second three-dimensional model based on the second real-time position of each catheter position point at each respiratory time, the first real-time position of each catheter position point, the first registration matrix, and the first three-dimensional model. For the current breathing moment, the position offset vector of each catheter position point is determined based on the second real-time position of each catheter position point and the first real-time position of each catheter position point at the current breathing moment; The position offset vector is transformed according to the first registration matrix to obtain the transformed offset vector; The second three-dimensional model is determined based on the converted offset vectors at each catheter position point at the current breathing time and the first three-dimensional model.

6. The apparatus according to claim 5, characterized in that, The second control module is further configured to: determine the second three-dimensional model based on the converted offset vectors at each catheter position point at the current breathing time and the first three-dimensional model; Determine the cross-section and centerline of each catheter location point in the first three-dimensional model; The corresponding cross section and the corresponding center line are updated based on the converted offset vector at each catheter position point at the current breathing time to obtain the updated cross section and the updated center line; The second three-dimensional model is determined based on the updated cross-sections and updated centerlines at each catheter location point at each breathing time.

7. The apparatus according to claim 1, characterized in that, Based on the first registration matrix, the second registration matrix, the first 3D model, and the registered 3D model, the conduit is controlled to run in the second branch of the target object. The second control module is further configured to: Determine the calibration cycle; each calibration cycle includes multiple respiratory cycles. For the first respiratory cycle of each calibration cycle, the dynamic navigation path is determined based on the position offset vector of each catheter position point at each respiratory moment in the first respiratory cycle and the initial navigation path. The initial navigation path is obtained based on the first three-dimensional model; For each non-first respiratory cycle of each calibration cycle, a dynamic navigation path is determined at each respiratory moment in the non-first respiratory cycle based on the second registration matrix and the initial navigation path; Based on the first registration matrix, coordinate transformation is performed on the dynamic navigation path at each breathing time in each calibration cycle to obtain the transformed dynamic navigation path. Based on the registered 3D model, the control catheter runs in the second branch of the target object according to the transformed dynamic navigation path.

8. The apparatus according to claim 7, characterized in that, Based on the registered 3D model, the second control module controls the conduit to run along the transformed dynamic navigation path in the second branch of the target object, and the second control module is further configured to: The virtual camera is controlled to run in the registered 3D model according to the transformed dynamic navigation path, and real-time virtual images captured by the virtual camera are acquired. Acquire real-time images of the catheter as it runs along the transformed dynamic navigation path in the second branch of the target object; Determine the similarity between the real-time virtual image and the real-time real image; If the similarity is less than a preset similarity value, the control conduit stops operating in the second branch of the target object; If the similarity is not less than a preset similarity value, the control conduit runs in the second branch of the target object according to the transformed dynamic navigation path.

9. A catheter control system, characterized in that, The system includes: a catheter, a positioning sensor, a breathing sensor, and a computer device; the positioning sensor is used to detect the real-time position of the catheter at various points within the catheter as it moves through the target object; the breathing sensor is used to detect the real-time respiratory amplitude of the target object; the computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps: The following data are obtained: catheter radius, real-time position of each catheter point in the catheter during operation within the target object, real-time respiratory amplitude of the target object, and a first three-dimensional model of the target object under a preset respiratory amplitude; the target object includes a first branch and a second branch. Based on the real-time location, determine the radius of the target object at each target location point in the target object where the conduit travels; If the difference between the radius of the target object and the radius of the catheter is greater than a preset difference, a first registration matrix is ​​determined based on the real-time position in the first branch and the first three-dimensional model; based on the first registration matrix and the first three-dimensional model, the catheter is controlled to run in the first branch of the target object; If the difference between the radius of the target object and the radius of the catheter is not greater than a preset difference, a second three-dimensional model is determined based on the real-time respiratory amplitude, the preset respiratory amplitude, the real-time position, the first registration matrix, and the first three-dimensional model; registration is performed based on the second three-dimensional model and the first three-dimensional model to obtain a second registration matrix and a registered three-dimensional model; based on the first registration matrix, the second registration matrix, the first three-dimensional model, and the registered three-dimensional model, the catheter is controlled to run in the second branch of the target object.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the following steps: The following data are obtained: catheter radius, real-time position of each catheter point in the catheter during operation within the target object, real-time respiratory amplitude of the target object, and a first three-dimensional model of the target object under a preset respiratory amplitude; the target object includes a first branch and a second branch. Based on the real-time location, determine the radius of the target object at each target location point in the target object where the conduit travels; If the difference between the radius of the target object and the radius of the conduit is greater than a preset difference, a first registration matrix is ​​determined based on the real-time position in the first branch and the first three-dimensional model. Based on the first registration matrix and the first three-dimensional model, control the conduit to run in the first branch of the target object; If the difference between the radius of the target object and the radius of the catheter is not greater than a preset difference, a second three-dimensional model is determined based on the real-time respiratory amplitude, the preset respiratory amplitude, the real-time position, the first registration matrix, and the first three-dimensional model. The second 3D model and the first 3D model are registered to obtain a second registration matrix and a registered 3D model; based on the first registration matrix, the second registration matrix, the first 3D model and the registered 3D model, the conduit is controlled to run in the second branch of the target object.

Images