Catheter robot and method of registration thereof

Abstract

The application relates to a catheter robot and a registration method thereof, and the method comprises the following steps: generating a first anatomical model based on a first medical image in a first state; generating a second anatomical model based on a second medical image in a second state; acquiring body surface data sensed by a second sensor, determining a breathing coefficient of a current state based on the body surface data; acquiring a third anatomical model associated with the breathing coefficient based on the breathing coefficient and the first anatomical model; acquiring a plurality of actual path points sensed by the first sensor in the anatomical structure, and acquiring a simulated path point cloud based on the plurality of actual path points; acquiring a model point cloud, wherein the model point cloud comprises a plurality of skeleton points of a pipeline center line of the third anatomical model and / or vertexes of a plurality of pipeline walls; and registering the simulated path point cloud with the model point cloud, so that the third anatomical model can better reflect the real state of the patient's anatomical structure, and the success rate of surgery is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical devices, in particular to a catheter robot and a registration method thereof. BACKGROUND

[0002] Minimally invasive medical techniques are intended to reduce the amount of tissue that is damaged during a medical procedure to reduce patient recovery time, discomfort, and harmful side effects. Such minimally invasive techniques can be performed through natural orifices in the anatomy or through one or more surgical incisions. For example, when the anatomy is a bronchus, a commonly used pulmonary medical procedure is to use a bronchoscope technique, the process of which is to insert a medical device through the natural cavity of the patient or through one or more minimally invasive surgical incisions, and to reach the target tissue position, i.e., the lesion position, by relying on the camera at the front end of the bronchoscope and under the cooperation of an intraoperative navigation system.

[0003] In the intraoperative navigation process, an anatomical model (a virtual bronchus 3D model) needs to be established according to the CT image of the patient before the operation. Since the CT image taken by the patient before the operation is the shape of the patient's bronchus at a certain moment, the size and shape of the patient's bronchus are easily affected by breathing and dynamically change. In addition, the movement of the catheter also squeezes the bronchus, causing the bronchus to deform, such as bending.

[0004] In the prior art, some processing methods only consider the influence of breathing on the shape of the bronchus, ignoring the squeezing or driving effect of the medical device on the bronchus. Since the bronchus is small near the lesion, the movement of the catheter will affect the shape of the bronchus. If the lesion is located inside or linked to the bronchus, the deformation of the bronchus will drive the lesion to move together, and the success rate of puncture biopsy according to the original planned path will increase, and the positive rate of biopsy will decrease. Some processing methods obtain the data of multiple sensors on the catheter in real time and calculate the shape of the catheter and the influence on the bronchus, but ignore the influence of breathing on the bronchus and the lesion. Breathing can cause the bronchus to contract and dilate, and the lesion to move and deviate. In particular, when the lesion is located outside the bronchus, if the puncture biopsy is still performed according to the planned path when the patient's breathing amplitude is large, the success rate will also decrease.

[0005] Therefore, the prior art does not comprehensively consider the influence of breathing and the device on the shape of the bronchus. If there is a large difference between the anatomical model and the actual anatomical structure, it will affect the doctor's judgment of the orientation and distance between the catheter and the target tissue, and have a great influence on the success rate of the operation. SUMMARY

[0006] Therefore, it is necessary to provide a catheter robot and a registration method thereof to solve the problem of the influence of breathing and movement of a catheter on the shape of an anatomical structure during intraoperative navigation, and to improve the success rate of surgery by dynamically modifying an anatomical model to make the anatomical model better reflect the real state of the patient's anatomical structure.

[0007] In a first aspect, the present application provides a catheter robot, comprising:

[0008] a catheter;

[0009] a first sensor arranged on the catheter and configured to sense the position of the distal end of the catheter;

[0010] a second sensor arranged on the surface of the patient and configured to obtain surface data of the patient; and

[0011] a control device coupled to the first sensor and configured to:

[0012] obtain a first medical image of the patient's anatomical structure in a first state and generate a first anatomical model based on the first medical image;

[0013] obtain a second medical image of the patient's anatomical structure in a second state and generate a second anatomical model based on the second medical image;

[0014] obtain surface data sensed by the second sensor, and determine a breathing coefficient in the current state based on the surface data;

[0015] obtain a third anatomical model associated with the breathing coefficient based on the breathing coefficient and the first anatomical model, or obtain a third anatomical model associated with the breathing coefficient based on the breathing coefficient and the second anatomical model;

[0016] obtain a plurality of actual path points sensed by the first sensor in the anatomical structure, and obtain a simulated path point cloud based on the plurality of actual path points;

[0017] obtain a model point cloud, the model point cloud comprising a plurality of skeleton points of a pipeline center line of the third anatomical model and / or a plurality of vertexes of a pipeline wall;

[0018] register the simulated path point cloud and the model point cloud.

[0019] In a second aspect, a catheter robot comprises:

[0020] a catheter;

[0021] a first sensor arranged on the catheter and configured to sense the position of the distal end of the catheter;

[0022] a second sensor configured to be arranged on a surface of the patient and configured to acquire surface data of the patient; and

[0023] a control device coupled to the first sensor and configured to:

[0024] acquire a first medical image of an anatomical structure of the patient in a first state and generate a first anatomical model based on the first medical image;

[0025] acquire a second medical image of the anatomical structure of the patient in a second state and generate a second anatomical model based on the second medical image;

[0026] acquire the surface data sensed by the second sensor, and determine a respiration coefficient of a current state based on the surface data;

[0027] acquire a third anatomical model associated with the respiration coefficient based on the respiration coefficient and the first anatomical model, or acquire a third anatomical model associated with the respiration coefficient based on the respiration coefficient and the second anatomical model.

[0028] In a third aspect, the present application provides a registration method of a catheter robot, comprising:

[0029] acquiring a first medical image of an anatomical structure of the patient in a first state and generating a first anatomical model based on the first medical image;

[0030] acquiring a second medical image of the anatomical structure of the patient in a second state and generating a second anatomical model based on the second medical image;

[0031] acquiring the surface data sensed by the second sensor, and determining a respiration coefficient of a current state based on the surface data;

[0032] acquiring a third anatomical model associated with the respiration coefficient based on the respiration coefficient and the first anatomical model, or acquiring a third anatomical model associated with the respiration coefficient based on the respiration coefficient and the second anatomical model.

[0033] acquiring a plurality of actual path points sensed by the first sensor in the anatomical structure, and acquiring a simulated path point cloud based on the plurality of actual path points;

[0034] acquiring a model point cloud, the model point cloud comprising a plurality of skeleton points of a centerline of a duct of the third anatomical model and / or a plurality of vertexes of duct walls;

[0035] registering the simulated path point cloud and the model point cloud.

[0036] In a fourth aspect, the present application provides a computer readable storage medium, characterized in that the computer readable storage medium stores a computer program configured to be loaded and executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0037] The catheter robot and the registration method thereof have the following beneficial effects:

[0038] By eliminating the influence of respiration and movement of the catheter on the anatomical structure and the target tissue, the present application can obtain a dynamic and accurate anatomical model, improve the accuracy of the registration algorithm, make the position of the catheter tip displayed on the anatomical model closer to the actual position, and higher accuracy is beneficial to judge the distance between the catheter tip and / or the instrument tip (such as a biopsy needle) and the target tissue (for example, a lesion), reduce the number of DR or CBCT shooting in surgery, and reduce the radiation of doctors and patients. On the other hand, the present application can also accurately know the shape of the catheter and the anatomical structure in real time, provide guidance for the movement direction of the subsequent catheter, better adjust the angle of the catheter, align the center of the target tissue, and improve the success rate of the surgery. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 A structural schematic diagram of a catheter robot according to an embodiment of the present application is provided;

[0040] Figure 2 A structural schematic diagram of a catheter instrument and a power part according to an embodiment of the present application is provided;

[0041] Figure 3 A flowchart of a registration method of a catheter robot according to an embodiment of the present application is provided;

[0042] Figure 4 A second sensor position diagram of a patient's body surface according to an embodiment of the present application is provided;

[0043] Figure 5 A respiratory coefficient diagram obtained from EM data of at least one respiratory cycle sensed by a second sensor according to an embodiment of the present application is provided;

[0044] Figure 6 A flowchart of a third anatomical model obtained by acquiring a respiratory coefficient according to an embodiment of the present application is provided;

[0045] Figure 7 A geodesic diagram of a first anatomical model according to an embodiment of the present application is provided;

[0046] Figure 8 An effect comparison diagram of a third anatomical model obtained by contracting a first anatomical model in a first state according to an embodiment of the present application is provided;

[0047] Figure 9 An affine transformation flowchart of a first anatomical model as a whole according to an embodiment of the present application;

[0048] Figure 10 A path point cloud deduplication effect diagram of a catheter end according to an embodiment of the present application;

[0049] Figure 11 A flowchart of registering a simulation path point cloud with a model point cloud according to an embodiment of the present application;

[0050] Figure 12 A comparison effect diagram of deleting simulation path point clouds belonging to the first region and retaining simulation path point clouds belonging to the second region according to an embodiment of the present application;

[0051] Figures 13-14 A comparison effect diagram of registering a simulation path point cloud with a model point cloud using a second transformation matrix according to an embodiment of the present application;

[0052] Figure 15 A simplified user interface diagram of a doctor's operation according to an embodiment of the present application;

[0053] Figure 16 A principle diagram of a control device of a catheter robot according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings. The preferred embodiments of the present application are shown in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided so that the disclosure of the present application can be understood more thoroughly and completely.

[0055] It is to be noted that when an element is referred to as being "on" another element, it can be directly on the other element or intervening elements can also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements can also be present. When an element is referred to as being "coupled" to another element, it can be directly coupled to the other element or intervening elements can also be present. The terms "vertical", "horizontal", "left", "right" and similar expressions as used herein are for illustration purposes only and are not intended to be limiting. The terms "distal" and "proximal" as used herein are directional terms that are commonly used in the field of interventional medical devices, wherein "distal" refers to the end of the device that is closest to the patient during a procedure and "proximal" refers to the end of the device that is farthest from the patient during a procedure. The terms "first", "second", and the like as used herein refer to one element and a class of elements that have common characteristics.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application, the term "and / or" includes any and all combinations of one or more of the associated listed items. As used in this application, the term "each" includes one and more than one. As used in this application, the term "plurality" means two or more.

[0057] A catheter robot and a method for improving registration thereof according to embodiments of the application are described below in conjunction with the accompanying drawings.

[0058] Figure 1 A catheter system 1000 according to an embodiment of the application is shown. The catheter system 1000 includes an image cart 100, a trolley 200 and a master controller 300 connected to the image cart 100 respectively, a catheter instrument 400 which can be coupled to the trolley 200, a sensor system 500 connected to the trolley 200, and a control device 600 for realizing control among the catheter instrument 400, the master controller 300, the sensor system 500 and the image cart 100, etc. The master controller 300 can be connected to the trolley 200 in wired or wireless manner. When an operator performs various procedures on a patient beside the trolley 200, the operator can trigger a control instruction by operating the master controller 300, and the catheter instrument 400 can be controlled to advance, retract and bend, etc. by driving of the trolley 200.

[0059] The trolley 200 can be generally moved to the side of the operating bed for engaging the catheter instrument 400 and controlling the catheter instrument 400 to be lifted in the vertical direction, or to be translated in the horizontal direction, or to be moved in a direction other than the vertical and horizontal directions, under the control of the control instruction, so as to provide a better preoperative preparation angle for the operation of the catheter instrument 400. The control instruction can be triggered by the operator through the operation of the master controller 300, or can be triggered by the operator directly through clicking or pressing the keys arranged on the trolley 200. Of course, in other embodiments, the control instruction can also be a voice control or a force feedback mechanism triggered instruction.

[0060] As shown in Figure 1 Further, the trolley 200 can include a base 210, a sliding seat 220 that can be lifted and moved along the base 210, and two mechanical arms 230 fixedly connected with the sliding seat 220. The mechanical arm 230 can include a plurality of arm segments coupled at joints, which provide the mechanical arm 230 with a plurality of degrees of freedom, for example, seven degrees of freedom corresponding to seven arm segments. The distal end of the mechanical arm 230 is provided with a power unit (not shown in the figure), which is used to engage the catheter instrument 400 and control the distal end of the catheter instrument 400 to bend and turn correspondingly under the driving action of the power unit. The two mechanical arms 230 can be identical or partially identical in structure, one mechanical arm 230 is used to engage the inner catheter instrument 410, and the other mechanical arm 230 is used to engage the outer catheter instrument 420. When installed, the outer catheter instrument 420 can be installed first, and then the catheter of the inner catheter instrument 410 is inserted into the catheter of the outer catheter instrument 420 after the installation of the outer catheter instrument 420 is completed.

[0061] The sensor system 500 has one or more subsystems for receiving information about the catheter instrument 400. The subsystems can include a position sensor system for determining the position, orientation, speed, rate, pose, and / or shape of the distal end of the catheter instrument 400 and / or one or more segments of the catheter that can constitute the catheter instrument 400; and / or a visualization system for capturing images from the distal end of the catheter instrument 400.

[0062] The cart 100 can be provided with a display system 110 and a flushing system (not shown) and the like. The display system 110 is used to display images of the surgical site and the catheter instrument 400 generated by the subsystems of the sensor system 500. Real-time images of the surgical site and the catheter instrument 400 captured by the visualization system can also be displayed. Image data from imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), optical coherence tomography (OCT), and ultrasound and the like can also be used to present images of the preoperatively or intraoperatively recorded surgical site. Preoperative or intraoperative image data can be presented as two-dimensional, three-dimensional or four-dimensional (e.g., time-based or velocity-based information) images and / or as images from models created from preoperative or intraoperative image data sets, and virtual navigation images can also be displayed. In the virtual navigation images, the actual position of the catheter instrument 400 is registered with the preoperative images to present the operator with a virtual image of the catheter instrument 400 within the surgical site from the outside.

[0063] The control device 600 includes at least one memory and at least one processor. It can be appreciated that the control device 600 can be integrated into the trolley 200 or the cart 100, or can be independently provided. The control device 600 can support wireless communication protocols such as IEEE 802.11, IrDA, Bluetooth, HomeRF, DECT, and wireless telemetry and the like. The control device 600 can transmit signals indicative of one or more of the catheter instrument 400 moved by the power unit. The catheter instrument 400 can extend to a surgical position in the body via an opening of a natural cavity of the patient or a surgical incision.

[0064] Further, the control device 600 can include a mechanical control system (not shown in the figure) for controlling the movement of the catheter instrument 400, and thus can be integrated into the trolley 200, and an image processing system (not shown in the figure) for virtual navigation path planning, and thus can be integrated into the image vehicle 100. Of course, the various subsystems of the control device 600 are not limited to the specific cases listed above, but can be reasonably arranged according to actual conditions. Among them, the image processing system can use the above imaging technology to image the surgical site based on the images of the surgical site recorded before or during the operation. The image processing system can also use software combined with manual input to convert the recorded images into two-dimensional or three-dimensional composite images of part or the entire anatomical organ or section. During the virtual navigation procedure, the sensor system 500 can be used to calculate the position of the catheter instrument 400 relative to the patient's anatomical structure, which can be used to generate external tracking images and internal virtual images of the patient's anatomical structure, realize the registration of the actual position of the catheter instrument 400 with the preoperative images, and thus the virtual image of the catheter instrument 400 within the surgical site can be presented to the operator from the outside.

[0065] The internal catheter instrument 410 and the external catheter instrument 420 have substantially the same structure and composition, respectively, and have an elongated flexible internal catheter 41 and an external catheter 42, wherein the diameter of the external catheter 42 is slightly larger than that of the internal catheter 41, so that the internal catheter 41 can pass through the external catheter 42 and provide certain support for the internal catheter 41, so that the internal catheter 41 can reach the target position in the patient's body to facilitate tissue or cell sampling and other operations from the target position.

[0066] Some movements of the master controller 300 can cause corresponding movements of the catheter instrument 400. For example, when the operator moves the direction lever of the master controller 300 upward or downward, the movement of the direction lever of the master controller 300 can be mapped to the corresponding pitch movement of the tip of the catheter instrument 400; when the operator moves the direction lever of the master controller 300 to the left or to the right, the movement of the direction lever of the master controller 300 can be mapped to the corresponding yaw movement of the tip of the catheter instrument 400. In this embodiment, the master controller 300 can control the tip of the catheter instrument 400 to move within a 360° spatial range.

[0067] Figure 2A catheter instrument 400 provided by an embodiment of the present application is shown. The catheter instrument 400 is configured to be engaged with the powered part 240 of the mechanical arm 230, and the catheter instrument 400 includes an instrument box 45 configured to be engaged with the powered part 240 and a catheter 48 connected with the instrument box 45. Wherein, the "engagement" refers to a state that when the instrument box 45 is mounted to the powered part 240, the driving force of the powered part 240 can be transmitted to the instrument box 45 and can make the catheter 48 normally move. For example, under the action of the driving force of the powered part 240, the end of the catheter 48 can be bent and turned, etc.

[0068] In the present application, the end can also be referred to as the distal end, which refers to the end away from the instrument box 45; the front end can also be referred to as the proximal end, which refers to the end close to the instrument box 45.

[0069] The processor of the control device 600 is configured to perform the following steps to realize the registration method of the catheter robot provided by an embodiment of the present application. As shown in Figure 3 The registration method of the catheter robot includes:

[0070] Step S11, acquiring a first medical image of an anatomic structure of a patient in a first state and generating a first anatomic model, and acquiring a second medical image of the anatomic structure of the patient in a second state and generating a second anatomic model.

[0071] In the present embodiment, the doctor needs to obtain the medical image of the anatomic structure of the patient through CT, MRI, OCT or ultrasonic scanning (or shooting) before the operation, wherein the anatomic structure can be a kind of anatomic structure such as bronchus, urinary tract, (cardiovascular) and intestinal tract, etc. through natural cavity.

[0072] In an embodiment, when the medical image is a CT image, a first medical CT image in a first state of the patient and a second medical CT image in a second state of the patient are acquired, the first state including an inhale state of the patient, and the second state including an exhale state of the patient. For example, taking the bronchus as an example of an anatomical structure, when the first state is a full inhale state of the patient, the first medical CT image in the inhale phase is taken, and the patient needs to take a deep breath and then hold his breath until the CT is taken, because the bronchus is the largest in the inhale state of the human body, and therefore the fine branches at the end of the bronchus are more easily imaged on the CT. For another example, when the second state is a full exhale state of the patient, the second medical CT image in the exhale phase is taken, and the patient needs to exhale all the gas in the body and then hold his breath until the CT is taken. The first medical image is segmented and reconstructed to obtain a first anatomical model, and the second medical image is segmented and reconstructed to obtain a second anatomical model. For example, the medical image of the patient can be segmented by a segmentation algorithm such as region growing and convolutional neural network to segment the bronchus of the lung, and then the segmented image is reconstructed into a three-dimensional model by a moving cube algorithm to obtain the first anatomical model and the second anatomical model. The three-dimensional model includes a three-dimensional mesh model, a point cloud model, etc. The three-dimensional mesh model is a collection of vertices, edges, and faces to represent the shape of a three-dimensional object, and usually contains geometric information and topological information.

[0073] The first anatomical model and the second anatomical model include a pipeline centerline. In order to facilitate intraoperative navigation, the pipeline centerline is extracted from the first anatomical model and the second anatomical model by a centerline extraction algorithm. The pipeline centerline is the centerline of the anatomical model, which can also be referred to as the skeleton of the anatomical model, is a curve used to describe part of the geometric features of the anatomical model, is located in the middle part of the anatomical model, has the same topological structure as the anatomical model, and generally has a width of a single pixel.

[0074] The pipeline centerline is a three-dimensional curve, and in practical applications, the pipeline centerline is often stored and used in the form of multiple three-dimensional points, in other words, the pipeline centerline includes these three-dimensional points, which can be referred to as skeleton points, and the set of skeleton points can be referred to as a skeleton point set or a skeleton point cloud. The pipeline centerline can provide the information required for intraoperative navigation, and compared with the original anatomical model, the data amount is significantly reduced, the processing is more convenient, and it is beneficial to real-time navigation.

[0075] In step S12, body surface data sensed by a second sensor arranged on the body surface of the patient is acquired, and a breathing coefficient in the current state is determined based on the body surface data.

[0076] In some embodiments, the catheter robot includes a catheter, and the catheter is provided with a first sensor for collecting the position of the end of the catheter, and the first sensor can include a position sensor and / or a shape sensor.

[0077] The position sensor can be a component of an electromagnetic positioning system. The electromagnetic positioning system can further include a magnetic field generating component and a magnetic field detecting component. The magnetic field generating component is configured to generate a magnetic field. The position sensor is an EM sensor (i.e., an electromagnetic sensor). The position sensor induces a change in the magnetic field. The magnetic field detecting component detects the change in the magnetic field and determines the pose of the position sensor relative to the magnetic field / magnetic field generating component. The position of the catheter tip relative to the magnetic field / magnetic field generating component can be determined based on a preconfigured or calibrated coordinate transformation between the position sensor and the catheter tip. The position of the catheter tip in the world coordinate system can be determined based on a coordinate transformation between the magnetic field / magnetic field generating component and the world coordinate system.

[0078] For another example, a shape sensor can be used to obtain a point cloud of the catheter tip inserted into the anatomical structure. For example, the shape sensor can include an optical fiber aligned with the catheter. A fiber optic bend sensor formed by the optical fiber can feed back the shape of the catheter. Based on this, the position of the catheter tip relative to the base of the shape sensor can be calculated. The position of the catheter tip in the world coordinate system can be calculated based on the position of the base of the shape sensor in the world coordinate system.

[0079] The catheter robot further includes a second sensor configured on the surface of the patient. The second sensor is at least three and is configured to obtain surface data of the patient. The second sensor can be a position sensor or a pose sensor. The second sensor is generally configured on the surface of the patient corresponding to the anatomical structure, i.e., the second sensor is generally exposed on the surface of the patient. The second sensor can be an EM sensor or an optical positioning sensor.

[0080] In some embodiments, for example, when the anatomical structure is a bronchus, due to the different respiratory amplitudes of different parts of the lung, the second sensor at the middle of the main bronchus actually measures the respiratory amplitude of the front end of the main bronchus, which is relatively small, and the second sensors at the left and right sides of the end of the bronchus are mainly used to measure the respiratory amplitudes of the left and right ends of the lung, and the movement amplitude of the diaphragm at the lower end of the lung segment is the largest during respiration. Therefore, the second sensors are arranged at a position on the surface of the patient's body that is a position with little change in the middle of the main bronchus of the lung and a position with large changes at the left and right sides of the end of the bronchus. With the change of the patient's respiratory state, the anatomical structure changes in six dimensions of up and down, left and right, front and back. In theory, the more sensors, the more respiratory state changes obtained, that is, the more second sensors, the more areas of the patient's body surface covered, and the more accurate the sensing of the respiratory movement of the anatomical structure. For example, at least three second sensors are arranged on the surface of the patient's chest. Among them, one second sensor can be arranged in the middle of the patient's chest, and the other two second sensors can be arranged at positions corresponding to the seventh rib on the left side of the patient's chest and the seventh rib on the right side of the patient's chest, respectively, and the second sensors need to be fixed during the operation using adhesive tape or the like to avoid sliding during the operation.

[0081] In some embodiments, the step S12 of determining the respiratory coefficient in the current state can be implemented by the following steps:

[0082] (1) Obtain the patient's body surface data through the second sensor, and the sampling period of the patient's body surface data includes at least one respiratory cycle. For the bronchus, the sampling period can include at least one respiratory cycle, for example, one, two or more, and the duration of a normal respiratory cycle is usually 3-5 seconds.

[0083] (2) Determine the area of the figure enclosed by the second sensor according to the patient's body surface data, and the area includes a first area, a second area and a third area. The first area includes the area of the figure enclosed by the second sensor in the first state, the second area includes the area of the figure enclosed by the second sensor in the second state, and the third area is the area S of the figure enclosed by the second sensor in the current state. When the first state is the fully inhaling state, the first area is the maximum area S max enclosed by the second sensor; when the second state is the fully exhaling state, the second area is the minimum area S min enclosed by the second sensor.

[0084] As Figure 4As shown, three second sensors are exemplified to determine the breathing coefficient in the current state. The position points of the three second sensors are P1 (X1, Y1, Z1), P2 (X2, Y2, Z2), and P1 (X3, Y3, Z3), respectively. The distances between the second sensors, i.e., the side lengths A, B, and C, are calculated.

[0085] The lengths of A, B, and C are calculated by using the Euclidean distance, for example, the length of side A is calculated as follows:

[0086]

[0087] The lengths of B and C are calculated in the same way as A, which will not be described here.

[0088] The semi-perimeter P of the figure enclosed by the second sensors is calculated as follows:

[0089] P = (A + B + C) / 2 Formula (2)

[0090] The area S of the figure enclosed by the second sensors in the current state is calculated as follows:

[0091]

[0092] The breathing coefficient F in the current state is determined according to the maximum area S max , the minimum area S min , and the area S of the figure enclosed by the second sensors in the current state.

[0093]

[0094] In formula (4), Fmax represents the maximum value of the designed breathing coefficient, and Fmin represents the minimum value of the designed breathing coefficient. For example, in some embodiments, if the formula is designed according to the range of the breathing coefficient in the interval [-1, 1], the breathing coefficient F is:

[0095]

[0096] For another example, in some embodiments, if the formula is designed according to the range of the breathing coefficient in the interval [-100, 100], the breathing coefficient F is:

[0097]

[0098] For example, Figure 5The horizontal coordinate represents the number of EM data acquired by the second sensor with the change of the breathing time or the breathing cycle. The vertical coordinate from top to bottom represents the area of the graph enclosed by the second sensor, the area slope, and the breathing coefficient. The area of the graph enclosed by the second sensor represents the size of the area of the graph enclosed by the second sensor with the change of the breathing state of the patient. The area slope represents the change rate of the area. The breathing coefficient represents the different breathing states of the patient.

[0099] The breathing coefficient F can be used to intuitively and quickly determine the breathing state of the patient, and enable or disable the movement of the catheter in the anatomical structure based on the breathing state. For example, if the breathing coefficient F is positive, it indicates that the human thoracic cavity is in the first state of the inspiration state, the thoracic cavity is full, and the bronchus is in the diastolic state. The larger the value, the more full, and the movement of the catheter in the anatomical structure is enabled. Or, when the breathing coefficient F is negative, it is determined that the breathing state of the patient is the expiration state, and the movement of the catheter in the anatomical structure is disabled. For example, based on the breathing coefficient F, the breathing state of the patient is determined, and a prompt is generated based on the breathing state to determine whether the movement of the catheter in the anatomical structure is allowed. When the breathing coefficient F is positive, it is determined that the breathing state of the patient is the inspiration state, and a prompt is generated to allow the movement of the catheter in the anatomical structure. Or, when the breathing coefficient F is negative, it is determined that the breathing state of the patient is the expiration state, and a prompt is generated to disable the movement of the catheter in the anatomical structure. For example, when the anatomical structure is the bronchus, the visualization system assisting the doctor in operation can detect that the surgical instrument (for example, a needle) at the end of the catheter reaches the vicinity of the target tissue position, and detect that the patient is in the inspiration state. The system allows the doctor to be prompted to perform a percutaneous puncture operation, and the puncture is safer in the inspiration state of the patient. If the breathing coefficient is negative, it indicates that the human thoracic cavity is in the second state of the expiration state, and the thoracic cavity and the bronchus are in the contraction state. The system allows the doctor to be prompted to disable the operation.

[0100] When the first state is the full inspiration state, the first anatomic model corresponds to a respiration coefficient of 1, and when the second state is the full expiration state, the second anatomic model corresponds to a respiration coefficient of -1. Since the first anatomic model in the full inspiration state and the second anatomic model in the full expiration state have been obtained based on the first medical image and the second medical image before the operation, other respiration coefficients, such as 0, need to be obtained by affine transformation based on the first anatomic model or the second anatomic model in the first state or the second state. In some embodiments, in order to reduce the amount of calculation, when used, for example, five classic respiration coefficients {-1, -0.5, 0, 0.5, 1} are usually selected, and the respiration coefficient in the previous state is calculated to be closer to which classic respiration coefficient, then the first anatomic model or the second anatomic model associated with the classic respiration coefficient is selected for display. For example, when the respiration coefficient in the previous state is calculated to be 0.9, the first anatomic model with a respiration coefficient of 1 is selected for display.

[0101] S13, obtaining a third anatomic model associated with the respiration coefficient based on the respiration coefficient and the first anatomic model, or obtaining a third anatomic model associated with the respiration coefficient based on the respiration coefficient and the second anatomic model.

[0102] In this embodiment, since the bronchus of the human body is the largest in the first state, i.e., the inspiration state, the fine branches at the end of the bronchus are more easily imaged on the CT, and the bronchus is the smallest in the second state, i.e., the expiration state, resulting in that many bronchial ends cannot be imaged on the CT due to contraction. Therefore, the affine transformation in this application is based on the first anatomic model in the first state to scale and translate to obtain third anatomic models associated with different respiration coefficients.

[0103] In step S13, obtaining a third anatomic model associated with the respiration coefficient, as shown in FIG. 13, can include: Figure 6

[0104] S131, obtaining a first geodesic distance of the first anatomic model and a second geodesic distance of the second anatomic model.

[0105] ​In this embodiment, when the anatomical structure is a bronchus (usually referring to a pulmonary bronchus), a first geodesic distance from a main carina point to a lower lobe end point in the first anatomical model and a second geodesic distance from the main carina point to the lower lobe end point in the second anatomical model are calculated respectively. The main carina point refers to the bifurcation point of the left and right bronchus, and the lower lobe end point refers to the point closest to the human diaphragm in the bronchus model. In some embodiments, when the first anatomical model is a three-dimensional mesh model, the geodesic distance is the shortest path distance along the mesh surface from the main carina point to the lower lobe end point in the first anatomical model; in another embodiment, when the first anatomical model is a three-dimensional point cloud model, a graph similar to the surface structure of the mesh is constructed using all the points in the three-dimensional point cloud, and the geodesic distance is calculated by finding the shortest path from the main carina point to the lower lobe end point in the graph. The first anatomical model or the second anatomical model includes the target tissue, such as Figure 7 As shown, the geodesic line 10 formed by the main carina point to the lower lobe end point in the first anatomical model is a line on the mesh surface, and the length of the geodesic line 10 is the geodesic distance. The target tissue 20 is exactly at the end point of the lower lobe.

[0106] In S132, the maximum shrinkage ratio of the first anatomical model or the second anatomical model is determined according to the first geodesic distance and the second geodesic distance.

[0107] In some embodiments, for example, the first geodesic distance GDIn of the first anatomical model in the first state is obtained before surgery, GDIn is 166 mm; the second geodesic distance GDEx of the second anatomical model in the second state is obtained before surgery, GDEx is 127 mm; then the maximum shrinkage ratio of the first anatomical model or the second anatomical model is determined according to the ratio of the first geodesic distance and the second geodesic distance: ShrinkMax = GDEx / GDIn, at this time, ShrinkMax is about 0.76.

[0108] In S133, the scaling ratio of the first anatomical model or the second anatomical model is determined according to the respiration coefficient and the maximum shrinkage ratio, and the first anatomical model or the second anatomical model is scaled based on the scaling ratio.

[0109] The scaling ratio Shrink of the first anatomical model or the second anatomical model is determined according to the respiration coefficient F and the maximum shrinkage ratio ShrinkMax of the model, and the calculation formula is:

[0110]

[0111] Because the bronchus in expiration state not only contracts as a whole, but also some of the end-branch bronchus is too narrow to be imaged in medical CT image space, in some embodiments, the affine transformation is performed based on the first anatomic model in the first state of the patient, i.e., based on the first anatomic model in full inspiration state. For example, when the breathing coefficient is -1, it can be known from formula (7) that Shrink is equal to ShrinkMax about 0.76, i.e., based on the first anatomic model in full inspiration state, the third anatomic model in full expiration state is obtained by affine transformation with a shrinkage ratio of 0.76. The effect is shown in Figure 8 the left picture is the second anatomic model 63 acquired according to the second medical image in the second state (full expiration state), and the right picture is the third anatomic model 64 obtained by affine transformation based on the first anatomic model in the first state (full inspiration state) with a shrinkage ratio of 0.76, at this time the third anatomic model 64 is in full expiration state. As can be seen from the effect picture, the third anatomic model 64 and the second anatomic model 63 are basically the same in size, and the length of the bronchus trunk is also basically the same. However, the third anatomic model 64 well preserves the shape of the end-branch bronchus, solving the problem that the end-branch bronchus cannot be segmented due to unclear imaging (container effect, it is more difficult to image the small bronchus) of the second medical CT image in expiration phase. In addition, the complete and clear end-branch bronchus has a great help to improve the navigation accuracy, and can better judge whether the path of the catheter is correct.

[0112] In some embodiments, the scaling of the first anatomic model or the second anatomic model can be realized by affine transformation of a matrix, i.e., multiplying the scaling matrix S to obtain the third anatomic model, and the coordinates of the third anatomic model are multiplied by the matrix Transform for each vertex of the first anatomic model:

[0113] Transform = T2(Xn, Yn, Zn) · S(x, y, z) · T1(-Xn, -Yn, -Zn) Formula (8)

[0114] Wherein, S represents the scaling matrix, T1 represents the first translation matrix, and T2 represents the second translation matrix.

[0115]

[0116]

[0117]

[0118] In step S133, as Figure 9As shown, taking the first anatomical model as an example, the affine transformation process of the first anatomical model as a whole, that is, the scaling process is as follows:

[0119] S1331, obtaining a first centroid of the first anatomical model and a second centroid of the second anatomical model.

[0120] The centroid can also be referred to as the center of shape, that is, the center of shape, and the coordinates can be the average of the coordinates of all skeleton points constituting the center line of the pipeline. Since the center line of the pipeline is obtained by processing the medical image, the density can be considered to be uniform, so the center of the center line of the pipeline is the centroid. When the anatomical model is a three-dimensional mesh model, the first centroid can be obtained by calculating the average of all vertex coordinate values in the first anatomical model, and the second centroid can be obtained by calculating the average of all vertex coordinate values in the second anatomical model.

[0121] S1332, obtaining a scaling matrix S, a first translation matrix T1 and a second translation matrix T2.

[0122] S1333, according to the first translation matrix T1, moving the first anatomical model at the initial position to the origin direction with the first centroid as the reference, so that the first centroid coincides with the coordinate origin of the first medical image.

[0123] The coordinate origin in the first medical CT image is at the top left corner, the x-axis is horizontal to the right, and the y-axis is horizontal downward, while the first anatomical model image is in the right lower region of the coordinate system. When affine transformation is performed, the vertices of the first anatomical model are moved to the coordinate origin of the first medical image, and the first centroid of the first anatomical model is moved to the origin, so that the first centroid coincides with the coordinate origin. The value of the first centroid moving to the origin is the value of all vertices of the first anatomical model needing to be translated. For example, in the coordinate system of the CT image, the first centroid coordinate value of the first anatomical model is (Ox, Oy, Oz), and after moving to the origin by the T1 translation matrix (-Ox, -Oy, -Oz), the centroid coordinate value of the first anatomical model coincides with the coordinate value (0, 0, 0) of the origin.

[0124] S1334, determining the scaling ratio Shrink of the first anatomical model according to the respiratory coefficient F and the maximum shrinkage ratio ShrinkMax of the first anatomical model.

[0125] S1335, taking the first centroid of the first anatomical model as the center, or taking the coordinate origin of the first medical image as the center, and performing equal scaling on the first anatomical model based on the scaling ratio, that is, Sx, Sy and Sz are all equal to Shrink, indicating that the scaling ratio of the first anatomical model in three directions is the same, that is, equal scaling, at this time, the scaling matrix S is the product of the scaling ratio Shrink and the unit matrix.

[0126] S1336, moving the first anatomic model after scaling by the second translation matrix back to the initial position. The scaled first anatomic model is moved back to the initial position with the first centroid as the center by using the T2 translation matrix (Ox, Oy, Oz), so as to realize scaling of the first anatomic model.

[0127] The scaling of the second anatomic model can refer to the whole scaling process of the first anatomic model, which is not described herein again.

[0128] S134, translating the scaled first anatomic model or the second anatomic model to obtain a third anatomic model associated with the breathing coefficient.

[0129] Since breathing not only causes scaling of an anatomic structure, but also causes movement of the anatomic structure and a target tissue. For example, breathing not only causes scaling of a bronchus, but also causes movement of the bronchus and a lesion, so that the relative position of the target tissue and the scaled first anatomic model (a bronchus model caused by breathing) or the second anatomic model relative to other tissues such as bones changes. Therefore, it is necessary to translate the first anatomic model or the second anatomic model from the position before scaling to the position after scaling. In the present application, the first anatomic model or the second anatomic model includes a target tissue. In an embodiment, the movement offset vector between the first anatomic model and the second anatomic model can be determined according to the first centroid and the second centroid, and the movement offset vector includes the size and direction of translation.

[0130] The movement offset vector is obtained by: obtaining a first centroid of the first anatomic model and a second centroid of the second anatomic model, and determining a movement offset vector between the first anatomic model and the second anatomic model according to the first centroid and the second centroid. For example, the first centroid of the first anatomic model in the first state (the inhalation model) is CentroidIn (Xn, Yn, Zn), the second centroid of the first anatomic model in the second state (the exhalation model) is CentroidEx (Xm, Ym, Zm), and the movement offset vector MoveTotal from the inhalation model to the exhalation model is calculated, wherein MoveTotal = {Xm-Xn, Ym-Yn, Zm-Zn}.

[0131] According to the current breathing coefficient F and the movement offset vector MoveTotal, the vector Move that the target tissue and the scaled first anatomic model or the second anatomic model as a whole need to be translated can be calculated, and the calculation formula is as follows:

[0132]

[0133] The first or second section of the scaled anatomical model is translated according to the translation vector Move to obtain a third anatomical model associated with the breathing coefficient in the current state.

[0134] Therefore, the third anatomical model associated with the breathing coefficient in the current state can be obtained through the steps S131 to S134.

[0135] In an embodiment, when the anatomical structure is the urinary system, the steps S11 to S13 can be applied to the percutaneous nephrolithotomy puncture of the urinary catheter robot under the guidance of the CT image. In the treatment of urinary stones, if the stones in the kidney are large and hard, a percutaneous nephrolithotomy puncture surgery is usually required, and the puncture needle is inserted into the target calyx through the skin. The puncture is divided into intercostal puncture and subcostal puncture, and the biggest challenge of the intercostal puncture is that the puncture and dilation are easy to cause damage to the pleura and adjacent organs (liver, pancreas). The distance between the pleura and the diaphragm and the calyx is different in different breathing states. In the inspiratory state, the pleura is at the lowest position and is closest to the kidney, and at this time, the risk of pleural and pulmonary lobe injury is also the highest when the intercostal puncture is performed. Therefore, the percutaneous nephrolithotomy puncture is usually performed in the expiratory stage, and it is very important to calculate and judge the real-time breathing state of the patient and calculate the distance between the pleura and the kidney to improve the safety of the puncture.

[0136] Similar to steps S11 to S13, the urinary catheter robot performs the percutaneous nephrolithotomy puncture under the guidance of the CT image. Due to the influence of respiration, the scaling of the anatomical structure (pleura and diaphragm and other organs) can include the following steps:

[0137] (1) The target region of the patient is scanned before the operation to obtain a medical image such as a CT image, and a three-dimensional model is reconstructed by segmenting the pleura, diaphragm, kidney, and ureter and other organ tissues from the CT image;

[0138] (2) A second electromagnetic sensor is installed on the chest of the patient to collect EM data in real time, and the second electromagnetic sensor is at least three;

[0139] (3) The first sensor is arranged in the catheter and is in the world coordinate system. The catheter moves to the calyx through the ureter, the electromagnetic data of the catheter end is collected to obtain the motion path of the catheter, and the CT image and the world coordinate system are registered. The registration data is the motion path of the catheter and the three-dimensional model of the ureter (including the pipe center line) in the CT image. After registration, the three-dimensional spatial relationship between the puncture needle and the organ tissue can be displayed in real time in the CT image space, and the spatial relationship such as the distance between the puncture needle and the pleura and other tissues can be calculated in the CT image space.

[0140] (4) Obtain the real-time electromagnetic data of the second electromagnetic sensor and calculate the respiratory coefficient F, and change the size and position of the three-dimensional model of the pleura and diaphragm. Here, the change of the size and position of the three-dimensional model of the pleura and diaphragm is similar to the anatomical model associated with the calculation of the respiratory coefficient F in the above steps S131 to S134, which will not be repeated here, and the difference is that the calculation method of the maximum contraction ratio of the pleura and diaphragm is to calculate the ratio of the surface area of the three-dimensional model surface in the exhalation state and the inhalation state, that is, to use the surface area instead of the geodesic distance.

[0141] (5) According to the respiratory coefficient F, the respiratory state is judged, if the respiratory coefficient is less than 0, that is, the patient is in the exhalation state, the operation software prompts the doctor to perform percutaneous puncture operation; if the respiratory coefficient of the patient is greater than 0, that is, in the inhalation state, the operation software prompts the doctor to suspend the operation.

[0142] (6) During the puncture, the operation software real-time prompts the distance between the puncture needle and the pleura, kidney, and the puncture needle is too close to the pleura, such as less than 5mm, the software prompts the doctor to suspend the operation, and improves the safety and accuracy of the puncture in the operation process.

[0143] Therefore, by the method of the present application, the influence of the respiratory on the anatomical structure is eliminated, which can be applied to the percutaneous renal mirror puncture of the urinary catheter robot guided by the CT image.

[0144] Another embodiment, continuing to refer to Figure 3 After step S13, the method further comprises:

[0145] Step S14, obtaining a plurality of actual path points sensed by the first sensor in the anatomical structure, and obtaining a simulated path point cloud based on the plurality of actual path points.

[0146] By collecting and storing a plurality of actual path points of the catheter tip in the anatomical structure by the first sensor, the motion state of the catheter during the intraoperative navigation is obtained, which includes the shape of the catheter and the position of the catheter tip. During the intraoperative navigation, the EM (electromagnetic) data of the catheter tip is sensed by the first sensor in real time, which can also be called an actual path point. One actual path point reflects the position of the catheter tip / instrument tip when the first sensor feeds back, and the actual path points obtained by the same first sensor multiple times are arranged according to the feedback time, and the moving path of the catheter tip and / or instrument tip can be obtained. The set of these actual path points can be called a path point set or a path point cloud.

[0147] According to the first transformation matrix, the simulated path points corresponding to the actual path points of the catheter in the first or second anatomical model are obtained. The first transformation matrix can be a transformation matrix between the world coordinate system and the anatomical model. According to the first transformation matrix, the actual path points can be transformed from the world coordinate system to the coordinate system of the anatomical model to obtain the simulated path points. That is, according to the first transformation matrix, the simulated path points corresponding to the actual path points of the catheter in the first or second anatomical model are obtained. Based on the plurality of simulated path points, the simulated path point cloud can be obtained. The simulated path points reflect the position of the catheter tip / instrument tip in the first anatomical model. In combination with the display parameters of the surgical site, the position of the catheter tip / instrument tip in the displayed surgical site can be obtained, so as to fuse the displayed surgical site and the catheter / instrument therein, and realize intraoperative navigation.

[0148] The target path point cloud of the planned path is obtained, the planned path being a navigation path of the catheter to the target tissue, and the target path point cloud including a plurality of target path points. The simulated path point cloud is de-duplicated based on the target path point cloud to obtain an effective simulated path point cloud. Since the catheter may have repeated motion in the anatomical structure during the process of reaching the target tissue, in order to reduce the amount of calculation, the simulated path point cloud needs to be preprocessed, which includes data de-duplication processing, noise removal and the like.

[0149] In some embodiments, the process of data de-duplication processing includes: acquiring the simulated path point cloud of the catheter tip by the first sensor, finding out the matching points in the target path point cloud, the simulated path point cloud including a plurality of simulated path point subsets; if any simulated path point subset matches the same matching point on the planned path, the mean value operation is performed on the simulated path point subset to obtain the effective simulated path point cloud. For example, if a simulated path point subset on the catheter tip is point Q1(X1, Y1, Z1) and point Q2(X2, Y2, Z2), and the closest corresponding point on the planned path is K point, that is, the matching points of the simulated path point subset Q1 and Q2 in the target path point cloud are both K points, then the mean value of Q1 and Q2 is obtained as the effective simulated path point C, where C = 1 / 2*(X1+X2, Y1+Y2, Z1+Z2). As shown in Figure 10 The left side of the figure is the simulated path point cloud 60 of the repeated motion of the catheter, and the right side of the figure is the effective simulated path point cloud 61, that is, the effective simulated path point cloud 61 is the motion path of the catheter after de-duplication. The shape of the catheter can be simulated through the motion path of the catheter. The simulated path point cloud of the catheter tip can be fitted in real time based on a space-time sampling algorithm to simulate the shape of the catheter.

[0150] In step S15, the model point cloud is obtained, the model point cloud including a plurality of skeleton points of the pipeline center line of the third anatomical model and / or a plurality of vertexes of the pipeline wall.

[0151] Obtaining a model point cloud from the third anatomical model, the model point cloud comprising a plurality of skeleton points of a centerline of a tube of the third anatomical model and / or a plurality of vertices of a tube wall of the third anatomical model. The skeleton points can be obtained by, for example, extracting the centerline of the tube from the third anatomical model and then discretizing the centerline to obtain the skeleton points. The vertices of the tube wall can be obtained by, for example, extracting the tube wall from the third anatomical model and then discretizing the tube wall to obtain the vertices of the tube wall.

[0152] At step S16, the simulated path point cloud is registered with the model point cloud.

[0153] In step S16, as shown in FIG. 6, registering the simulated path point cloud with the model point cloud comprises: Figure 11

[0154] S161, identifying simulated path points in the simulated path point cloud that belong to the first region, deleting the simulated path points that belong to the first region, and retaining the simulated path points that belong to the second region.

[0155] In some embodiments, taking the bronchus as an example of the anatomical structure, the bronchus comprises a first region and a second region, the first region comprises a main bronchus region, and the second region comprises a terminal branch bronchus region. For example, the entire third anatomical model can be partitioned, and the tube with a diameter greater than 5 mm is divided into the first region, and the tube with a diameter less than 5 mm is divided into the second region.

[0156] The method for identifying whether the simulated path point cloud of the catheter tip belongs to the first region or the second region is as follows: determining the matching point on the third anatomical model corresponding to the simulated path point cloud of the catheter tip, and if the corresponding matching point on the third anatomical model belongs to the main bronchus region, then the simulated path point cloud of the catheter tip belongs to the first region. As shown in FIG. 6, the left side of the figure is the effective simulated path point cloud 61 after the simulated path point cloud 60 of the catheter is de-duplicated, the effective simulated path point cloud 61 comprises the effective simulated path point cloud 611 of the first region and the effective simulated path point cloud 612 of the second region, and the right side of the figure is the effective simulated path point cloud 612 of the second region after the effective simulated path point cloud 611 of the first region is deleted from the catheter. Figure 12

[0157] S162, obtaining the model point cloud of the second region based on the third anatomical model, that is, obtaining the model point cloud of the terminal branch bronchus region based on the third anatomical model.

[0158] S163, determining the second transformation matrix between the model point cloud and the simulated path point cloud.

[0159] ​​The ICP algorithm is used to determine the second transformation matrix between the model point cloud and the simulation path point cloud. The ICP algorithm is an iterative closest point (ICP) algorithm in point cloud registration, and its core idea is to minimize the distance between two point sets and make the two point sets close to each other through iteration. The principle of ICP is briefly introduced below.

[0160] The basic method of ICP includes two steps: 1. Matching the point clouds Q and P to find the corresponding point pairs between them; 2. Calculating the transformation matrix between the point clouds Q and P according to the corresponding point pairs. A corresponding point pair consists of two points, one from the point cloud P and the other from the point cloud Q. These two points are the matching points of each other, and they are considered to be corresponding, that is, the two points are essentially identical.

[0161] If the true and accurate corresponding point pairs can be directly found, the above process only needs to be performed once, and a sufficiently accurate transformation matrix can be directly calculated. However, in actual application, it is difficult to directly find accurate corresponding point pairs, so ICP will match according to the principle of the nearest distance (generally the Euclidean distance), that is, for each point in the point cloud P, find the nearest point in the point cloud Q as its matching point. Then a transformation matrix is calculated according to these point pairs. After completing a round of calculation, ICP will judge whether the iteration stopping condition is met. If not, the transformation matrix obtained in this round of calculation is used to update the point cloud P, and then the point cloud Q and the transformed point cloud P are used to repeat the above process until the iteration is stopped. An exemplary default stopping condition for iteration is that the difference between the errors of the previous two iterations is less than 0.01, but sometimes the difference between the point clouds is too large to meet the convergence condition, so an upper limit number of iterations needs to be set, for example, it can be set to 500.

[0162] In this embodiment, the model point cloud P of the third anatomical model and the simulation path point cloud Q of the catheter are matched, and the ICP algorithm can be used to determine the second transformation matrix MatrixBrToTube between the model point cloud and the simulation path point cloud.

[0163] In S164, the simulation path point cloud and the model point cloud are registered according to the second transformation matrix, so that the model point cloud and the simulation path point cloud coincide.

[0164] In an embodiment, the second region is easily deformed by being squeezed by the catheter due to its small diameter, softness and similar size to the catheter body, and the first region is less affected by the movement of the catheter, so the second transformation matrix between the simulation path point cloud of the second region and the model point cloud of the second region can be obtained, and the simulation path point cloud of the second region and the model point cloud of the second region are registered so that the model point cloud of the second region and the simulation path point cloud of the second region coincide.

[0165] As shown in Figure 13 and Figure 14 , the effective simulated path point cloud 612 of the second region of the catheter, the model point cloud 614 of the second region in the third anatomical model, the model point cloud 614 includes a plurality of skeleton points 613 of the tube centerline. As shown in Figure 13 As shown in the left side of the figure, due to the movement of the catheter during the operation, the effective simulated path point cloud 612 of the second region of the catheter deviates from the plurality of skeleton points 613 of the tube centerline of the second region in the third anatomical model. Therefore, it is necessary to use the second transformation matrix MatrixBrToTube to rigidly transform the plurality of skeleton points 613 of the tube centerline of the second region in the third anatomical model, that is, in the three-dimensional model, each vertex of the second region in the third anatomical model is translated and rotated by using the second transformation matrix MatrixBrToTube. As shown in Figure 13 As shown in the right side of the figure, the plurality of skeleton points 613 of the tube centerline of the second region in the third anatomical model are made to substantially coincide with the effective simulated path point cloud 612 of the second region of the catheter after applying the second transformation matrix. As shown in Figure 14 As shown in the left side of the figure, due to the movement of the catheter during the operation, the effective simulated path point cloud 612 of the second region of the catheter deviates from the model point cloud 614 of the second region in the third anatomical model. After transforming the model point cloud 614 of the second region in the third anatomical model according to the second transformation matrix MatrixBrToTube, as shown in Figure 14 As shown in the right side of the figure, the effective simulated path point cloud 612 of the second region of the catheter is substantially within the second region in the third anatomical model, that is, the model point cloud of the second region in the third anatomical model substantially coincides with the simulated path point cloud of the second region in the catheter. Therefore, it can be understood that according to the effective simulated path point cloud 612 of the second region of the catheter, the position and shape of the model point cloud 614 of the second region in the third anatomical model are dynamically modified, so that the third anatomical model better reflects the true state of the patient's anatomical structure.

[0166] S17, determine whether the catheter reaches the vicinity of the target tissue position, if the catheter does not reach the vicinity of the target tissue position, repeat the above steps S13-S16; otherwise, execute S18 to end the navigation.

[0167] The method of the present application has the following beneficial effects:

[0168] In one aspect, the third anatomic model can be obtained dynamically and accurately by eliminating the influence of respiration and movement of the catheter on the anatomic structure and the target tissue. The doctor can also accurately know the shape of the catheter and the anatomic structure in real time, so that the third anatomic model better reflects the real state of the patient's anatomic structure and improves the success rate of the operation. In another aspect, the third anatomic model of the present application can replace part of the DR image to assist the doctor in judging the distance between the catheter and the target tissue, reduce the number of DR or CBCT shooting, and reduce the amount of radiation taken by the patient. In addition, the third anatomic model of the present application can also provide guidance for the movement direction of the subsequent catheter, better adjust the angle of the catheter, align the center of the target tissue, and improve the success rate of the operation.

[0169] The present application introduces a respiration coefficient F to represent the respiration state of the patient: based on the area measured by the second sensor, the area is dynamically converted into a respiration coefficient in the interval [-1, 1] in real time, -1 represents the second state of the fully exhaled state, and 1 represents the second state of the fully inhaled state. The degree of inhalation or exhalation can be represented by the coefficient. Through the respiration coefficient, the dynamic third anatomic model between inhalation and exhalation can be more accurately interpolated.

[0170] By eliminating the influence of respiration and movement of the catheter on the anatomic structure and the target tissue, the present application can obtain a dynamic and accurate third anatomic model, improve the accuracy of the registration algorithm, and make the position of the catheter tip and / or instrument tip displayed on the third anatomic model closer to the actual position. Higher accuracy is beneficial to judge whether the catheter tip or instrument tip (such as a biopsy needle) reaches the lesion, reduces the number of DR or CBCT shooting during the operation, and reduces the radiation taken by the doctor and the patient.

[0171] In another aspect, the shape of the catheter and the anatomic structure can also be accurately known in real time, which can provide guidance for the movement direction of the subsequent catheter, better adjust the angle of the catheter, align the center of the target tissue, and improve the success rate of the operation. In addition, in an embodiment of the present application, only three general electromagnetic sensors, such as NDI or OME, are needed for the second sensor, and only one electromagnetic sensor is needed for the catheter shape estimation, which reduces the cost of hardware facilities compared with the prior art, and achieves the effect of low hardware facility cost, easy acquisition and promotion.

[0172] The application of the present application is as follows Figure 15 The doctor guides the catheter to be inserted into the end of the bronchus and / or the instrument tip to reach the target tissue through the user interface 800 of the visual system. The user interface 800 can display a camera view window 810, a local anatomic model view 820, a global anatomic model view 830, an indicator window 840, and a control window 850.

[0173] The camera view window 810 displays camera data taken by a visualization system of the medical instrument. For example, the camera data can include camera image views or camera video data captured by a stereoscopic camera or a monoscopic camera mounted on an endoscope at the distal end of the catheter.

[0174] One or more indicator windows 840 can display current operation tips and screen recording buttons, and can also display the state of the catheter, such as displaying the inner or outer bending angle information of the catheter, or prompting the surgeon that the surgical operation can be performed or paused.

[0175] The control window 850 can display the connection status of each device, such as the operation handle, the catheter, the magnetic navigation, the suction, the water flushing, and the locking or activation status.

[0176] In some embodiments, the local anatomy model view 820 displays images from a local anatomy model corresponding to the distal end of the catheter to simulate the view image of the endoscope. When the anatomy model is a bronchus, the local anatomy model can be a local virtual bronchus model, and the virtual bronchus model image data can be generated by a virtual visualization system using, for example, preoperative CT images. The virtual bronchus model image data can display the real-time position of the catheter inside the patient's bronchus.

[0177] In some embodiments, the global anatomy model view 830 displays a complete anatomy model observed by the surgeon from a global perspective, for example, the surgeon can observe the position of the catheter in the bronchus model from the overall bronchus model. The catheter device and the person skilled in the art and the method for improving registration can realize real-time transformation of the end posture of the third anatomy model in the global anatomy model view 830 during the intraoperative navigation process, so that the third anatomy model better reflects the real state of the patient's anatomical structure, and improves the success rate of the operation.

[0178] Although five simultaneously viewable windows on a single screen are respectively depicted in Figure 15 , it should be understood that the user interface 800 can display a suitable number of windows according to user needs.

[0179] The embodiment of the present application also provides a control device of a catheter robot. Please refer to Figure 16 , which shows a structural schematic diagram of a control device of a catheter system provided by an embodiment of the present application. As Figure 16 shown, the control device 600 comprises a processor 70, a memory 71, a bus 72 and a communication interface 73, the processor 70, the communication interface 73 and the memory 71 are connected through the bus 72; the memory 71 stores computer program instructions executable by the processor 70, and the processor 70 executes the computer program instructions and can specifically execute the related steps in the above method embodiments.

[0180] The memory 71 can include a high-speed random access memory (RAM) and can also include a non-volatile memory such as at least one disk memory. The communication connection between the apparatus network element and at least one other network element is realized through at least one communication interface 73 (which can be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. can be used.

[0181] The bus 72 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 71 is used to store a program, and the processor 70 executes the program after receiving an execution instruction. The registration method of the catheter robot disclosed in any of the embodiments of the present application can be applied to the processor 70 or implemented by the processor 70.

[0182] The processor 70 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready-to-program gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps, and logic block diagrams in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The control device includes one or more processors, which can be the same type of processor, such as one or more CPUs, or one or more GPUs; or different types of processors, such as one or more CPUs and one or more GPUs.

[0183] The control device of the catheter robot provided in the embodiments of the present application and the registration method of the catheter robot provided in the embodiments of the present application have the same beneficial effects as the methods they employ, run or implement.

[0184] The embodiments of the present application further provide a computer readable storage medium corresponding to the registration method of the catheter robot provided by the foregoing embodiments, and computer program instructions are stored on the computer readable storage medium. The computer program instructions, when executed by a processor, implement the registration method of the catheter robot provided by any of the foregoing embodiments. It should be noted that examples of the computer readable storage medium can include, but are not limited to, optical discs, phase change memories (PRAM), static random access memories (SRAM), dynamic random access memories (DRAM), other types of random access memories (RAM), read-only memories (ROM), electrically erasable programmable read-only memories (EEPROM), flash memories, or other optical or magnetic storage media, which are not listed one by one here.

[0185] The computer readable storage medium provided by the foregoing embodiments of the present application has the same beneficial effects as the method adopted, run or implemented by the application program stored therein, for the same inventive concept as the registration method of the catheter robot provided by the embodiments of the present application.

[0186] The embodiments of the present application provide a computer program product, which, when running on a mobile terminal, causes the mobile terminal to implement the steps in the above-mentioned various method embodiments.

[0187] It should be noted that:

[0188] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known structures and techniques are not shown in detail in order not to obscure the understanding of the present specification.

[0189] Similarly, it should be understood that, in order to simplify the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, various features of the present application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting the intention that the claimed present application requires more features than those explicitly recited in each claim. Rather, the inventive aspects are based on less than all of the features of the single embodiments disclosed above, as reflected in the claims below. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, in which each claim itself is a separate embodiment of the present application.

[0190] Furthermore, those skilled in the art will recognize that, while certain embodiments have been specifically described herein, alternative embodiments can be constructed using similar processes and materials, and that the embodiments described herein have been made by way of example and not as limitations to the scope of the application. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0191] The above descriptions are only preferred specific embodiments of the present application, and the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed by the present application can be easily conceived by those skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0192] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example for description, and in actual application, the above functions can be completed by different functional units and modules based on needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of mutual distinction, and are not used to limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0193] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can refer to the relevant description of other embodiments.

[0194] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized in electronic hardware, or in a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0195] In the embodiments of the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other manners. For example, the embodiments of the apparatus / terminal device described above are merely schematic, and the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0196] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected based on actual needs to achieve the purpose of the embodiments.

[0197] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated into a unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0198] The integrated module / unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method in the above embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include appropriate contents based on the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, based on legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0199] It should be understood that the term "includes" when used in the specification and the appended claims herein is used to indicate included, but not to the exclusion of, one or more additional features, integers, steps, operations, elements, components, and / or groups thereof.

[0200] It should also be understood that the term "and / or" when used in the specification and the appended claims herein is intended to mean one or the other and / or both. Similarly, the terms "comprises", "comprising", "includes", "including" and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense that may be implied by use of these terms in the specification and the appended claims herein.

[0201] As used in the description of the application and the appended claims herein, the term "if' can be construed to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if a described condition or event occurs" can be construed to mean "upon determining," or "in response to determining" or "upon detecting," or "in response to detecting" the described condition or event, depending on the context.

[0202] Additionally, the terms "first", "second", "third", etc. as used in the description and the appended claims herein are used only to differentiate one element from another, and do not imply a relative importance or a specific order.

[0203] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in a various embodiment" or "in some embodiments" or "in other embodiments" in various places throughout this specification are not necessarily all referring to the same embodiment, unless otherwise specifically stated. The terms "comprising," "including," "containing," and "having" and their variations are meant to be construed as "including but not limited to," unless otherwise indicated.

[0204] The above-described embodiments are merely intended to illustrate the technical solutions of the present application, but not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A catheter robot, comprising: a catheter; a first sensor disposed on the catheter and configured to sense a position of a distal end of the catheter; a second sensor configured to obtain surface data of a patient; and a control device coupled to the first sensor and configured to: obtain a first medical image of an anatomical structure of the patient in a first state and generate a first anatomical model based on the first medical image; obtain a second medical image of the anatomical structure of the patient in a second state and generate a second anatomical model based on the second medical image; obtain the surface data sensed by the second sensor and determine a breathing coefficient of a current state based on the surface data; obtain a third anatomical model associated with the breathing coefficient based on the breathing coefficient and the first anatomical model, or obtain a third anatomical model associated with the breathing coefficient based on the breathing coefficient and the second anatomical model; obtain a plurality of actual path points sensed by the first sensor in the anatomical structure and obtain a simulated path point cloud based on the plurality of actual path points; obtain a model point cloud comprising a plurality of skeleton points of a centerline of a tube of the third anatomical model and / or a plurality of vertexes of tube walls; register the simulated path point cloud with the model point cloud; and wherein the obtaining the third anatomical model associated with the breathing coefficient comprises: obtaining a first geodesic distance of the first anatomical model and a second geodesic distance of the second anatomical model; determining a maximum shrinkage ratio of the first anatomical model or the second anatomical model based on the first geodesic distance and the second geodesic distance; determining a scaling ratio of the first anatomical model or the second anatomical model based on the breathing coefficient and the maximum shrinkage ratio; scaling the first anatomical model or the second anatomical model based on the scaling ratio; and translating the scaled first anatomical model or the second anatomical model to obtain the third anatomical model associated with the breathing coefficient. 2.The catheter robot of claim 1, wherein: the first state comprises an inhale state, and the second state comprises an exhale state. 3.The catheter robot of claim 1, wherein: the second sensor comprises at least three sensors, the surface data comprises surface data sensed by the second sensor in at least one breathing cycle, and the determining the breathing coefficient of the current state comprises: determining an area of a figure enclosed by the second sensor based on the surface data, the area comprising a first area, a second area, and a third area, the first area comprising an area of a figure enclosed by the second sensor in the first state, the second area comprising an area of a figure enclosed by the second sensor in the second state, and the third area comprising an area of a figure enclosed by the second sensor in the current state; and determining the breathing coefficient of the current state based on the area. 4.The catheter robot of claim 3, wherein: the first area comprises a maximum area of the figure enclosed by the second sensor in the first state, and the second area comprises a minimum area of the figure enclosed by the second sensor in the second state. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 5. The catheter robot of claim 1, wherein: the scaling the first or second anatomical model comprises: obtaining a first centroid of the first anatomical model and a second centroid of the second anatomical model; scaling the first anatomical model based on the scaling ratio with the first centroid as a center, or scaling the second anatomical model based on the scaling ratio with the second centroid as a center.

6. The catheter robot of claim 1, wherein: the translating the scaled first or second anatomical model comprises: obtaining a first centroid of the first anatomical model and a second centroid of the second anatomical model; determining a motion offset vector between the first and second anatomical models according to the first and second centroids; determining a translation vector of the scaled first or second anatomical model according to the breathing coefficient and the motion offset vector; translating the scaled first or second anatomical model according to the translation vector.

7. The catheter robot of claim 1, wherein: the obtaining the simulated path point cloud based on the plurality of actual path points comprises: obtaining a plurality of actual path points of the catheter in the anatomical structure sensed by the first sensor; obtaining simulated path points corresponding to the actual path points of the catheter in the first or second anatomical model according to a first transformation matrix, and obtaining a simulated path point cloud based on the plurality of simulated path points.

8. The catheter robot of claim 7, wherein: the simulated path point cloud further comprises an effective simulated path point cloud, and the registering the simulated path point cloud with the model point cloud comprises: registering the effective simulated path point cloud with the model point cloud, wherein the obtaining the model point cloud comprises: obtaining a target path point cloud of a planned path, the planned path being a navigation path of the catheter to reach a target tissue, the target path point cloud comprising a plurality of target path points; de-duplicating the simulated path point cloud based on the target path point cloud to obtain the effective simulated path point cloud.

9. The catheter robot of claim 1, wherein: the registering the simulated path point cloud with the model point cloud comprises: determining a second transformation matrix between the model point cloud and the simulated path point cloud; registering the simulated path point cloud with the model point cloud according to the second transformation matrix, so that the model point cloud coincides with the simulated path point cloud.

10. The catheter robot of claim 9, wherein: the anatomical structure comprises a bronchus, the bronchus comprising a first region and a second region, the first region comprising a main bronchus region, and the second region comprising a terminal branch bronchus region, and the registering the simulated path point cloud with the model point cloud comprises: identifying simulated path points of the simulated path point cloud belonging to the first region; deleting the simulated path points belonging to the first region and retaining simulated path points belonging to the second region. obtaining a model point cloud of the second region based on the third anatomical model; obtaining a second transformation matrix between the simulated path point cloud of the second region and the model point cloud of the second region; registering the simulated path point cloud of the second region and the model point cloud of the second region according to the second transformation matrix, so that the model point cloud of the second region coincides with the simulated path point cloud of the second region.

11. The catheter robot of claim 1, wherein the control device is further configured to: determine a breathing state of the patient based on the breathing coefficient, and enable or disable movement of the catheter in the anatomical structure based on the breathing state.

12. The catheter robot of claim 11, wherein the determining a breathing state of the patient based on the breathing coefficient, and enabling or disabling movement of the catheter in the anatomical structure based on the breathing state comprises: when the breathing coefficient is positive, determining that the breathing state is an inhale state, and enabling movement of the catheter in the anatomical structure; or, when the breathing coefficient is negative, determining that the breathing state is an exhale state, and disabling movement of the catheter in the anatomical structure.

13. The catheter robot of claim 1, wherein the control device is further configured to: determine a breathing state of the patient based on the breathing coefficient, and generate a prompt of whether to allow movement of the catheter in the anatomical structure based on the breathing state.

14. The catheter robot of claim 13, wherein the determining a breathing state of the patient based on the breathing coefficient, and generating a prompt of whether to allow movement of the catheter in the anatomical structure based on the breathing state comprises: when the breathing coefficient is positive, determining that the breathing state is an inhale state, and generating a prompt of allowing movement of the catheter in the anatomical structure; or, when the breathing coefficient is negative, determining that the breathing state is an exhale state, and generating a prompt of disabling movement of the catheter in the anatomical structure.

15. A catheter robot, comprising: a catheter; a first sensor disposed on the catheter for sensing a position of a distal end of the catheter; a second sensor configured on a body surface of a patient for obtaining body surface data of the patient; and a control device coupled to the first sensor and configured to: obtain a first medical image of an anatomical structure of the patient in a first state and generate a first anatomical model based on the first medical image; obtain a second medical image of the anatomical structure of the patient in a second state and generate a second anatomical model based on the second medical image; obtain the body surface data sensed by the second sensor and determine a breathing coefficient in a current state based on the body surface data; obtain a third anatomical model associated with the breathing coefficient based on the breathing coefficient and the first anatomical model, or obtain a third anatomical model associated with the breathing coefficient based on the breathing coefficient and the second anatomical model; the obtaining the third anatomical model associated with the breathing coefficient comprises: ​ ​ obtaining a first geodesic distance of the first anatomical model and a second geodesic distance of the second anatomical model; determining a maximum shrinkage ratio of the first anatomical model or the second anatomical model according to the first geodesic distance and the second geodesic distance; determining a scaling ratio of the first anatomical model or the second anatomical model according to the breathing coefficient and the maximum shrinkage ratio; scaling the first anatomical model or the second anatomical model based on the scaling ratio; translating the scaled first anatomical model or the second anatomical model to obtain a third anatomical model associated with the breathing coefficient.

16. A computer readable storage medium, characterized in that, the computer readable storage medium stores a computer program configured to be loaded and executed by a processor to implement the following steps: obtaining a first medical image of a patient's anatomical structure in a first state and generating a first anatomical model based on the first medical image; obtaining a second medical image of the patient's anatomical structure in a second state and generating a second anatomical model based on the second medical image; obtaining body surface data sensed by a second sensor and determining a breathing coefficient of a current state based on the body surface data; obtaining a third anatomical model associated with the breathing coefficient based on the breathing coefficient and the first anatomical model, or obtaining a third anatomical model associated with the breathing coefficient based on the breathing coefficient and the second anatomical model; obtaining a plurality of actual path points sensed by a first sensor in the anatomical structure and obtaining a simulated path point cloud based on the plurality of actual path points; obtaining a model point cloud, the model point cloud including a plurality of skeleton points of a pipeline centerline of the third anatomical model and / or a plurality of vertexes of a pipeline wall; registering the simulated path point cloud with the model point cloud; the obtaining of the third anatomical model associated with the breathing coefficient includes: obtaining a first geodesic distance of the first anatomical model and a second geodesic distance of the second anatomical model; determining a maximum shrinkage ratio of the first anatomical model or the second anatomical model according to the first geodesic distance and the second geodesic distance; determining a scaling ratio of the first anatomical model or the second anatomical model according to the breathing coefficient and the maximum shrinkage ratio; scaling the first anatomical model or the second anatomical model based on the scaling ratio; translating the scaled first anatomical model or the second anatomical model to obtain a third anatomical model associated with the breathing coefficient.

17. A computer readable storage medium, characterized in that, the computer readable storage medium stores a computer program configured to be loaded and executed by a processor to implement the following steps: obtaining a first medical image of a patient's anatomical structure in a first state and generating a first anatomical model based on the first medical image; obtaining a second medical image of the patient's anatomical structure in a second state and generating a second anatomical model based on the second medical image; obtaining body surface data sensed by a second sensor and determining a breathing coefficient of a current state based on the body surface data; obtaining a third anatomical model associated with the respiration coefficient based on the respiration coefficient and the first anatomical model, or obtaining a third anatomical model associated with the respiration coefficient based on the respiration coefficient and the second anatomical model; the obtaining of the third anatomical model associated with the respiration coefficient comprises: obtaining a first geodesic distance of the first anatomical model and a second geodesic distance of the second anatomical model; determining a maximum shrinkage ratio of the first anatomical model or the second anatomical model according to the first geodesic distance and the second geodesic distance; determining a scaling ratio of the first anatomical model or the second anatomical model according to the respiration coefficient and the maximum shrinkage ratio; scaling the first anatomical model or the second anatomical model based on the scaling ratio; performing translation on the scaled first anatomical model or the second anatomical model to obtain the third anatomical model associated with the respiration coefficient.

Images