Catheter movement control device, method and storage medium

Abstract

This application relates to a catheter movement control device, method, and storage medium. The catheter movement control device includes: a first acquisition module for acquiring point cloud data collected from the catheter, the point cloud data being used to characterize the catheter's position within a target object; a determination module for determining the current mapped position of the catheter in a centerline model of the target object based on the point cloud data; and a control module for adjusting the catheter's direction and curvature based on the current mapped position; and controlling the catheter to move a preset distance after adjusting its direction and curvature based on the catheter's path planning data. Using this device, the direction and curvature of the catheter can be adjusted based on point cloud data before each movement, thereby precisely achieving automatic catheter movement control and improving the efficiency of catheter insertion.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a device, method and storage medium for controlling the movement of a catheter. Background Technology

[0002] A bronchoscope is a medical device inserted through the mouth or nose into a patient's lower respiratory tract for observation, biopsy sampling, bacteriological and cytological examination of lesions in the lobes, segments and subsegments of the lungs.

[0003] In related technologies, bronchoscopy insertion primarily relies on manual insertion by the physician. The physician plans the bronchoscope's insertion path based on preoperative computed tomography (CT) images. During the procedure, the physician uses real-time images obtained through the bronchoscope to determine the real-time position of the bronchoscope's catheter, matching it with the planned insertion path and continuously adjusting and moving the bronchoscope manually.

[0004] However, the time and effort required for doctors to plan and manually move the bronchoscope's insertion path results in low efficiency. Summary of the Invention

[0005] Therefore, it is necessary to provide a device, method, and storage medium for controlling the movement of a bronchoscope to improve the insertion efficiency of the bronchoscope, thereby addressing the aforementioned technical problems.

[0006] In a first aspect, this application provides a catheter movement control device for controlling catheter movement. The device includes:

[0007] The first acquisition module is used to acquire point cloud data collected by the catheter, and the point cloud data is used to characterize the position of the catheter in the target object;

[0008] The determination module is used to determine the current mapping position of the conduit in the centerline model of the target object based on the point cloud data;

[0009] The control module is used to adjust the direction and curvature of the catheter according to the current mapping position of the catheter; and to control the catheter to move a preset distance after adjusting the direction and curvature according to the path planning data of the catheter.

[0010] In one embodiment, the control module includes:

[0011] A centerline direction determination unit is used to determine the direction of the target centerline corresponding to the current mapping position of the conduit;

[0012] A movement direction determination unit is used to determine the movement direction of the duct based on the point cloud data;

[0013] The adjustment unit is used to adjust the orientation and curvature of the catheter according to the direction of movement of the catheter and the direction of the target centerline.

[0014] In one embodiment, the centerline direction determining unit is specifically used to determine a first plane of the catheter, the first plane being the head plane of the catheter; and to determine the tangent direction between the first plane and the target centerline as the direction of the target centerline.

[0015] In one embodiment, the movement direction determination unit is specifically used to determine the center curve of the conduit based on the point cloud data; determine the target tangent of the center curve of the conduit at the head centerline point of the conduit; and determine the direction of the target tangent as the movement direction of the conduit.

[0016] In one embodiment, the adjustment unit is specifically used to adjust the direction of the catheter based on the angle difference between the direction of movement of the catheter and the direction of the target centerline on a second plane, wherein the second plane is the tail plane of the catheter; and to adjust the degree of curvature of the catheter based on the angle between the direction of movement of the catheter and the direction of the target centerline.

[0017] In one embodiment, the adjustment unit is specifically used to determine the angle difference as the rotation angle of the catheter if the angle difference between the moving direction of the catheter and the direction of the target centerline on the second plane is greater than a first threshold; and to adjust the direction of the catheter according to the rotation angle of the catheter.

[0018] In one embodiment, the adjustment unit is specifically configured to keep the direction of the catheter unchanged if the angle difference between the direction of movement of the catheter and the direction of the center line corresponding to the current mapping position of the catheter on the target plane is less than or equal to a first threshold.

[0019] In one embodiment, the adjustment unit is specifically configured to: determine the angle to be bent as the angle between the direction of movement of the catheter and the direction of the target centerline if the angle is greater than a second threshold; adjust the degree of bending of the catheter according to the degree of bending of the catheter; and keep the degree of bending of the catheter unchanged if the angle between the direction of movement of the catheter and the direction of the target centerline is less than or equal to the second threshold.

[0020] Secondly, this application provides a method for controlling the movement of a catheter. The method includes:

[0021] Acquire point cloud data collected by the catheter, the point cloud data being used to characterize the position of the catheter in the target object;

[0022] Based on the point cloud data, determine the current mapping position of the conduit in the centerline model of the target object;

[0023] Adjust the direction and curvature of the catheter according to its current mapped position;

[0024] Based on the catheter path planning data, the catheter moves a preset distance after the direction and degree of curvature are controlled and adjusted.

[0025] In a third aspect of catheter movement control, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the catheter movement control method described in the second aspect above.

[0026] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the catheter movement control method described in the first aspect above.

[0027] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the catheter movement control method described in the first aspect above.

[0028] The aforementioned catheter movement control device, method, and storage medium for catheter movement control first acquire point cloud data collected from the catheter, which characterizes the catheter's position within the target object. Second, based on the point cloud data, the current mapped position of the catheter is determined in the centerline model of the target object. Third, based on the current mapped position, the direction and curvature of the catheter are adjusted. Finally, based on the catheter's path planning data, the catheter moves a preset distance after the direction and curvature are adjusted. Because the direction and curvature of the bronchoscope catheter are adjusted based on the point cloud data before each movement, precise automatic movement control of the bronchoscope is achieved without manual intervention, thus improving the efficiency of bronchoscope insertion. Attached Figure Description

[0029] Figure 1 An application environment diagram for a catheter movement control method provided in an embodiment of this application;

[0030] Figure 2 An application environment diagram for another catheter movement control method provided in this application embodiment;

[0031] Figure 3 A schematic flowchart illustrating a catheter movement control method provided in an embodiment of this application;

[0032] Figure 4 A schematic diagram of a bronchoscope provided for an embodiment of this application;

[0033] Figure 5 This is a schematic diagram illustrating a pre-collection location point cloud data collection method provided in an embodiment of this application.

[0034] Figure 6 A schematic diagram of coordinate system transformation provided in an embodiment of this application;

[0035] Figure 7 A schematic diagram illustrating real-time coordinate registration as provided in an embodiment of this application;

[0036] Figure 8 A schematic diagram of bronchial centerline extraction provided in an embodiment of this application;

[0037] Figure 9 A diagram showing the positional relationship between a bronchoscope and the bronchial centerline, provided for an embodiment of this application;

[0038] Figure 10 A schematic diagram showing the rotation angle of a bronchoscope as provided in an embodiment of this application;

[0039] Figure 11 A schematic diagram illustrating a bronchoscope whose rotation angle remains constant, provided as an embodiment of this application;

[0040] Figure 12 A schematic diagram showing the bending angle of a bronchoscope as provided in an embodiment of this application;

[0041] Figure 13 A schematic diagram illustrating a bronchoscope whose bending angle remains unchanged, provided as an embodiment of this application;

[0042] Figure 14 A schematic diagram illustrating the conversion of the bending angle provided in this application embodiment;

[0043] Figure 15 This application provides a schematic diagram of a bronchoscope path planning method.

[0044] Figure 16 A flowchart illustrating an automated bronchoscope insertion method provided in this application embodiment;

[0045] Figure 17 A schematic diagram of another catheter movement control provided in an embodiment of this application;

[0046] Figure 18 A structural block diagram of a catheter movement control device provided in an embodiment of this application;

[0047] Figure 19 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] The catheter movement control method provided in this application embodiment can be applied to, for example... Figure 1 and Figure 2 The application environment shown.

[0050] like Figure 1 As shown, before the bronchoscope is inserted, the terminal device 101 is connected to the computed tomography (CT) device 102 to acquire CT images of the human chest obtained by the CT device 102. Subsequently, the terminal device 101 processes the CT images to extract the bronchial centerline, thereby establishing a bronchial centerline model.

[0051] like Figure 2 As shown, during bronchoscope 103 insertion, terminal device 101 is connected to bronchoscope 103. Terminal device 101 can configure the position of bronchoscope 103 and the bronchial centerline model to control the automatic insertion of bronchoscope 103. Subsequently, based on path planning data, terminal device 101 controls bronchoscope 103 to move a small distance each time until bronchoscope 103 reaches the target insertion position indicated by the path planning data. During each movement, terminal device 101 adjusts the direction and curvature of bronchoscope 103 based on the point cloud data collected by bronchoscope 103.

[0052] In some embodiments, the terminal device 101 can also display the endoscopic images collected by the bronchoscope 103 when the bronchoscope 103 is inserted, so that the doctor can observe the movement of the bronchoscope 103.

[0053] It should be understood that the embodiments of this application do not limit the type of terminal device 101, and it may be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices and portable wearable devices, etc.

[0054] In some embodiments, the bronchoscope 103 can be combined with an electromagnetic generator and a drive assembly to form an endoscope insertion device. The drive assembly, under the control of the terminal device 101, drives the bronchoscope to move and adjusts its orientation and curvature. The electromagnetic generator acquires point cloud data during the bronchoscope's movement to determine its real-time pose. The bronchoscope 103 can be a catheter, which may include a monocular endoscope and an electromagnetic patch. The monocular endoscope is used to acquire endoscopic images, and the electromagnetic patch is used to acquire the coordinate information of the catheter relative to the magnetic field generator to generate the aforementioned point cloud data.

[0055] In one embodiment, such as Figure 3 As shown, a method for controlling the movement of a catheter is provided, which can be applied to... Figure 1 and Figure 2 The following explanation will be based on the terminal equipment in the example, including S201-S202:

[0056] S201. Acquire point cloud data collected by the bronchoscope. The point cloud data is used to characterize the position of the bronchoscope in the bronchus of the target object.

[0057] In this application, when the terminal device controls the bronchoscope to advance, the terminal device can control the bronchoscope to move multiple times until the bronchoscope reaches the target insertion position. During each movement, the terminal device can acquire point cloud data collected by the bronchoscope, thereby determining the position of the bronchoscope and adjusting its direction and curvature based on the point cloud data.

[0058] The point cloud data includes the position and orientation data of the bronchoscope within the bronchus of the target object. The target object can be a patient about to undergo bronchoscopy. The orientation can include the position and orientation of the bronchoscope.

[0059] It should be understood that the embodiments of this application do not limit how the point cloud data collected by the bronchoscope is acquired. In some embodiments, the point cloud data can be acquired through an electromagnetic patch inside the bronchoscope. For example, Figure 4 As shown, two electromagnetic patches can be installed at both the head and tail of the bronchoscope tube, including electromagnetic patch 1, electromagnetic patch 2, electromagnetic patch 3, and electromagnetic patch 4. Electromagnetic patches 1 and 2 at the tube head can collect the head coordinate information of the bronchoscope tube relative to the head of the magnetic field generator, while electromagnetic patches 3 and 4 at the tube tail can collect the tail coordinate information of the bronchoscope tube relative to the tail of the magnetic field generator. Subsequently, the terminal device can use the collected head and tail coordinate information as point cloud data acquired by the bronchoscope at the current location.

[0060] S202. Based on the point cloud data, determine the current mapping position of the bronchoscope in the bronchial centerline model of the target object.

[0061] In the steps, after the terminal device acquires the point cloud data collected by the bronchoscope, it can determine the current mapping position of the bronchoscope in the bronchial centerline model of the target object based on the point cloud data.

[0062] It should be understood that the embodiments of this application do not limit how the current mapping position of the bronchoscope is determined in the bronchial centerline model of the target object based on point cloud data. In some embodiments, if the point cloud data is acquired through an electromagnetic patch inside the bronchoscope, the position data of the bronchoscope is in electromagnetic coordinates. In this case, the terminal device can determine the relative position of the bronchoscope's location and the pre-acquisition position in the bronchus of the target object in electromagnetic coordinates, and thus determine the current mapping position of the bronchoscope in the bronchial centerline model based on the relative position and the mapping position of the pre-acquisition position in the centerline model.

[0063] It should be understood that the embodiments of this application do not limit the above-mentioned pre-collection locations. For example, the pre-collection locations include the bifurcation of the left and right bronchi, the left secondary bifurcation point, and the right secondary bifurcation point.

[0064] It should be noted that the coordinates of the aforementioned pre-acquisition locations in electromagnetic coordinates can be acquired during the registration of the coordinate systems of the bronchial centerline model and the point cloud data. The mapping position of the pre-acquisition locations in the centerline model can be determined based on the annotation information or through automatic identification.

[0065] In some embodiments, before determining the current mapping position of the bronchoscope in the bronchial centerline model of the target object, the terminal device acquires point cloud data collected by the bronchoscope at a pre-acquisition position in the bronchus of the target object. Based on the point cloud data collected at the pre-acquisition position and the mapping position of the pre-acquisition position in the centerline model, the coordinate system of the bronchial centerline model and the point cloud data are registered.

[0066] It should be understood that the embodiments of this application do not limit how point cloud data of the pre-collection location in the bronchus of the target object is collected. In some embodiments, before the automatic insertion of the bronchoscope, the doctor can manually insert the endoscope to the aforementioned pre-collection location to collect point cloud data.

[0067] For example, such as Figure 5 As shown, the bronchoscope can be moved into the trachea first, reaching the bifurcation of the left and right bronchi, and point cloud data of the bifurcation can be collected. Then, the bronchoscope is controlled to enter the left main bronchus and reach the left secondary bifurcation point, where point cloud data of the left secondary bifurcation point is collected. Finally, the bronchoscope is controlled to enter the right main bronchus and reach the right secondary bifurcation point, where point cloud data of the right secondary bifurcation point is collected.

[0068] like Figure 6 As shown, since the point cloud data for the pre-acquisition location is acquired via the electromagnetic patch of the bronchoscope, this point cloud data is in an electromagnetic coordinate system. Since the bronchial centerline model is established based on chest CT images, it is in a CT coordinate system. Therefore, before automatic bronchoscope insertion, it is necessary to register the bronchial centerline model with the electromagnetic coordinate system of the point cloud data to determine the mapping position of the bronchoscope within the bronchial centerline model.

[0069] like Figure 7 As shown, after acquiring point cloud data at the pre-acquisition location, the global registration matrix T corresponding to the electromagnetic coordinate system and CT coordinate system of the point cloud data can be calculated based on the acquired point cloud data. Subsequently, the bronchial centerline model and the electromagnetic coordinate system of the point cloud data are registered in real time using the global registration matrix T.

[0070] The following is an explanation of the bronchial centerline model of the target object.

[0071] It should be understood that the embodiments of this application do not limit how the bronchial centerline model of the target object is generated. In some embodiments, the terminal device can acquire a chest CT image of the target object and then extract the bronchial centerline of the target object from the chest CT image. Subsequently, a bronchial centerline model of the target object is established based on the bronchial centerline.

[0072] In this process, extracting the bronchial centerline of the target object from the aforementioned chest CT image can be achieved by image segmentation followed by reconstruction of the bronchial model of the target object. In some embodiments, the terminal device can input a chest computed tomography (CT) image into an image processing model and obtain a segmented bronchial image output by the model. Subsequently, the terminal device reconstructs the bronchial model of the target object based on the segmented bronchial image. Finally, the terminal device extracts the bronchial centerline of the target object based on the bronchial model.

[0073] It should be understood that the embodiments of this application do not limit the image processing model described above, and it can be any type of deep learning model. For example, historical chest CT models can be labeled to form a training set for the image processing model, and this training set can be used to train the image processing model. After training is complete, inputting the chest CT image of the target object into the image processing model yields a segmented bronchial image of the target object.

[0074] An exemplary embodiment of this application provides a network structure (not shown) for an image processing model. The encoding unit in this image processing model can downsample the input chest CT image to obtain image features with different receptive fields at different scales. Subsequently, the encoding unit feeds the image features with different receptive fields at different scales to the decoding unit in the image processing model, preventing feature loss due to an overly deep model. The decoding unit can upsample the obtained image features to obtain image features associated with the bronchi.

[0075] For example, Figure 8 This is a schematic diagram illustrating the extraction of the bronchial centerline, provided as an embodiment of this application. Figure 8 As shown, after obtaining the bronchial segmentation image, a 3D model reconstruction of the bronchial model can be performed using bronchial segmentation images from multiple locations. After the 3D model reconstruction, a 3D skeleton extraction algorithm (itk.BinaryThinningImageFilter3D) is used to extract the bronchial centerline from the bronchial model.

[0076] In some embodiments, when extracting the bronchial centerline, the terminal device may also display the reconstructed bronchial model and / or the extracted bronchial centerline.

[0077] S203. Adjust the direction and curvature of the bronchoscope according to its current mapping position.

[0078] In this step, after the terminal device determines the current mapping position of the bronchoscope in the bronchial centerline model of the target object, it can adjust the direction and curvature of the bronchoscope according to the current mapping position and the attitude data of the bronchoscope.

[0079] It should be understood that the embodiments of this application do not limit how the direction and curvature of the bronchoscope are adjusted. In some embodiments, the terminal device may first determine the direction of the target bronchial centerline corresponding to the current mapped position of the bronchoscope. Subsequently, the terminal device determines the direction of movement of the bronchoscope based on point cloud data. Finally, the terminal device adjusts the orientation and curvature of the bronchoscope according to the direction of movement of the bronchoscope and the direction of the target bronchial centerline.

[0080] In some embodiments, the terminal device may first determine the center curve of the bronchoscope tube based on point cloud data. Then, it determines the target tangent line of the center curve of the bronchoscope tube at the centerline point of the bronchoscope tube tip. Finally, the terminal device determines the direction of the target tangent line as the direction of movement of the bronchoscope.

[0081] For example, continue to refer to Figure 4Two electromagnetic patches can be installed at both the tip and tail of the bronchoscope tube. The point cloud data mentioned above can be used to represent the coordinates of electromagnetic patches 1, 2, 3, and 4 in the electromagnetic coordinate system. Since the tip and tail of the bronchoscope tube are on the same plane, line segment L... 12 and line segment L 34 The perpendicular line is the point intersecting this plane. Correspondingly, with this point as the center, the arc between the head centerline point A and the tail centerline point B of the bronchoscope tube is the center curve of the bronchoscope tube. Subsequently, the target tangent I of the bronchoscope tube's center curve at the head centerline point, and the tangent J of the bronchoscope tube's center curve at the tail centerline point can be determined. At this point, the direction of the target tangent I can be determined as the direction of bronchoscope movement, and the tangent J at the tail centerline point is the tail direction vector.

[0082] In some embodiments, the terminal device may determine a first plane of the bronchoscope, which is the tip plane of the bronchoscope's catheter. Subsequently, the tangential direction between the first plane and the centerline of the target bronchus is determined as the direction of the centerline of the target bronchus.

[0083] For example, Figure 9 This is a diagram illustrating the positional relationship between a bronchoscope and the bronchial centerline, provided as an embodiment of this application. Figure 9 As shown, the head plane of the bronchoscope tube is the first plane, and the tail plane of the bronchoscope tube is the second plane. A spatial rectangular coordinate system xyz is established with the tail direction vector J of the bronchoscope tube as the z-axis, the line connecting the two electromagnetic patches on the tail plane as the x-axis, and the perpendicular line from the line connecting the two electromagnetic patches on the second plane to the tail midline point as the y-axis. In this spatial rectangular coordinate system xyz, the tangent direction at the intersection of the first plane and the target bronchial centerline is the centerline direction P, i.e., the direction of the target bronchial centerline.

[0084] If so Figure 10 In the spatial rectangular coordinate system xyz shown, when the direction of movement I of the bronchoscope is not on the same plane and the direction is not consistent with the direction of the target bronchial centerline, continuing to control the movement of the bronchoscope will cause the bronchoscope to deviate further from the planned path. At this time, the bronchoscope needs to be adjusted.

[0085] It should be understood that the embodiments of this application do not limit how the orientation and curvature of the bronchoscope are adjusted. In some embodiments, the terminal device can adjust the orientation of the bronchoscope based on the angular difference between the direction of movement of the bronchoscope and the direction of the target bronchial centerline on a second plane. Subsequently, the curvature of the bronchoscope is adjusted based on the angle between the direction of movement of the bronchoscope and the direction of the target bronchial centerline.

[0086] In some embodiments, if the angle difference between the direction of movement of the bronchoscope and the direction of the centerline of the target bronchus on the second plane is greater than a first threshold, the terminal device determines the angle difference as the rotation angle of the bronchoscope. The terminal device can adjust the direction of the bronchoscope according to the rotation angle of the bronchoscope. If the angle difference between the direction of movement of the bronchoscope and the direction of the centerline corresponding to the current mapped position of the bronchoscope on the target plane is less than or equal to the first threshold, the direction of the bronchoscope remains unchanged.

[0087] For example, Figure 10 This is a schematic diagram showing the rotation angle of a bronchoscope as provided in an embodiment of this application. Figure 10 As shown, a rectangular coordinate system xoy is established with the line connecting the two electromagnetic patches on the tail end plane as the x-axis and the perpendicular line connecting the two electromagnetic patches on the second plane passing through the tail midline point as the y-axis. In the rectangular coordinate system xoy, the bronchoscope's movement direction I is projected onto the second plane xoy to obtain projection I1, with an angle α0 between I1 and the y-axis. The direction P of the target bronchial centerline is projected onto the second plane xoy to obtain projection P1, with an angle α1 between P1 and the y-axis. Correspondingly, the angular difference between the bronchoscope's movement direction and the target bronchial centerline on the second plane is Δα = α0 - α1.

[0088] at this time, Figure 11 As shown, if Δα is less than or equal to the first threshold, it can be determined that the direction of bronchoscope movement and the centerline of the target bronchus are not significantly offset, and the direction of the bronchoscope can be kept unchanged, and the bronchoscope can continue to be advanced. If Δα is greater than the first threshold, it can be determined that the direction of bronchoscope movement and the centerline of the target bronchus are significantly offset, and the angle difference Δα can be determined as the angle to be rotated of the bronchoscope, thereby adjusting the direction of the bronchoscope.

[0089] It should be noted that the embodiments of this application do not limit the first threshold mentioned above, and can be set according to the specific control precision, such as 5°, 10°, etc.

[0090] It should be understood that when adjusting the direction of the bronchoscope according to the rotation angle, the direction of adjustment can be determined by the sign of Δα. If Δα > 0, the bronchoscope is rotated by an angle Δα closer to the y-axis; if Δα < 0, it is rotated by an angle Δα away from the y-axis. After adjustment, the direction of movement of the bronchoscope and the direction of the centerline of the target bronchus are on the same plane.

[0091] In some embodiments, if the angle between the direction of movement of the bronchoscope and the direction of the target bronchial centerline is greater than a second threshold, the angle is determined as the bending angle of the bronchoscope. The degree of bending of the bronchoscope is adjusted according to the bending angle. If the angle between the direction of movement of the bronchoscope and the direction of the target bronchial centerline is less than or equal to the second threshold, the degree of bending of the bronchoscope remains unchanged.

[0092] For example, Figure 12 This is a schematic diagram showing the bending angle of a bronchoscope as provided in an embodiment of this application. Figure 12 As shown, a rectangular coordinate system poz is established with the direction vector J of the bronchoscope's distal end as the z-axis and the direction P of the target bronchial centerline as the p-axis. After rotating by the desired angle and adjusting the bronchoscope's direction, both the bronchoscope and the target bronchial centerline lie on the plane poz. Given the rotated bronchoscope's movement directions I2 and P, the angle between the rotated bronchoscope's movement direction I2 and the z-axis can be determined as θ0, and the angle between the target bronchial centerline direction P and the z-axis as θ1. The angle between the line connecting the bronchoscope's tip and the origin o and the z-axis is β0, and the angle between the line connecting the target bronchial centerline's distal end and the origin o and the z-axis is β1. Accordingly, the angle Δβ between the bronchoscope's movement direction and the target bronchial centerline's direction is Δβ = β0 - β1.

[0093] At this time, as Figure 13 As shown, if Δβ is less than or equal to the second threshold, it can be determined that the curvature of the bronchoscope is not significantly different from the curvature of the target bronchus centerline, and the curvature of the bronchoscope can be kept constant, continuing the insertion. If Δβ ​​is greater than the second threshold, it can be determined that the curvature of the bronchoscope is significantly different from the curvature of the target bronchus centerline, and the angle difference Δβ can be determined as the desired curvature angle of the bronchoscope, thereby adjusting the direction of the bronchoscope.

[0094] It should be noted that the embodiments of this application do not limit the second threshold mentioned above, and it can be set according to the specific control precision, for example, it can be set to 5°, 10°, etc.

[0095] It should be understood that when adjusting the curvature of the bronchoscope according to the desired curvature angle, the direction of adjustment can be determined based on the sign of Δβ. If Δβ ​​> 0, the bronchoscope is controlled to bend further away from the z-axis, i.e., the curvature is increased; if Δβ < 0, the curvature is decreased, moving further away from the z-axis.

[0096] For example, Figure 14 This is a schematic diagram illustrating the conversion of the bending angle provided in an embodiment of this application. For example... Figure 14As shown, a rectangular coordinate system poz is established with the direction vector J of the bronchoscope tube's tail end as the z-axis and the direction P of the target bronchial centerline as the p-axis. Given that I is the arc from point O to point B, and OA and OB are tangents to the arc at points O and B respectively, and since the distance from the intersection of two tangents to the point of tangency is equal, we know OA = AB. Since OA = AB, △AOB is an isosceles triangle, therefore ∠AOB = ∠ABO = β0. Since the exterior angle of a triangle is equal to the sum of its two non-adjacent interior angles, θ0 = ∠AOB = ∠ABO = 2β0, i.e., β0 = θ0 / 2. Therefore, At this point, the terminal device controls the bronchoscope to bend by Δβ, so that the direction of movement of the bronchoscope is parallel to the direction of the target bronchial centerline.

[0097] S204. Based on the bronchoscope's path planning data, control and adjust the direction and curvature of the bronchoscope to move a preset distance.

[0098] In this step, after the terminal device adjusts the direction and curvature of the bronchoscope, it can control the bronchoscope to move a preset distance after adjusting the direction and curvature according to the bronchoscope's path planning data.

[0099] The aforementioned bronchoscope path planning data can be the path taken by the bronchoscope to the target insertion point. This target insertion point can be the location requiring puncture, the site to be treated, or the location of the lesion, etc.

[0100] For example, such as Figure 15 The diagram shown is a schematic diagram of a bronchoscope path planning method, which illustrates the bronchus that needs to be traversed to reach the target insertion position.

[0101] It should be understood that the embodiments of this application do not limit how the bronchoscope path planning data is determined. In some embodiments, the terminal device can automatically plan and generate bronchoscope path planning data based on the target insertion position input by the doctor, combined with the reconstructed bronchial model and the chest CT scan of the target patient. In other embodiments, the bronchoscope path planning data can be manually input by the doctor.

[0102] It should be understood that the above path planning data is used to indicate which fork in the bronchoscope should enter when the bronchoscope enters a fork in the road.

[0103] It should be understood that the embodiments of this application do not limit the above-mentioned preset distance, such as 1 mm, 2 mm, 5 mm, etc.

[0104] In this application, point cloud data of the real-time pose of the bronchoscope, obtained during the movement of the bronchoscope, is used to calculate the relationship between the position of the bronchoscope and the center line of the target bronchus, thereby adjusting the direction and curvature of the bronchoscope to ensure the accuracy of the duct movement control.

[0105] The catheter movement control method provided in this application first acquires point cloud data collected by a bronchoscope, which characterizes the position of the bronchoscope in the bronchus of the target object. Second, based on the point cloud data, the current mapping position of the bronchoscope is determined in the bronchial centerline model of the target object. Third, based on the current mapping position of the bronchoscope, the direction and curvature of the bronchoscope are adjusted. Finally, based on the bronchoscope's path planning data, the bronchoscope is moved a preset distance after the direction and curvature are adjusted. Because the direction and curvature of the bronchoscope are adjusted based on the point cloud data before each movement, precise automatic movement control of the bronchoscope is achieved without manual intervention, thus improving the efficiency of bronchoscope insertion.

[0106] The complete bronchoscopy insertion process is explained below. Figure 16 This is a schematic flowchart illustrating an automated bronchoscope insertion method provided in an embodiment of this application. Figure 16 As shown, the automatic insertion method of this bronchoscope includes:

[0107] S301. Obtain the bronchoscope path planning data, which includes the target insertion position of the bronchoscope.

[0108] S302. Acquire point cloud data collected by the bronchoscope. The point cloud data is used to characterize the position of the bronchoscope in the bronchus of the target object.

[0109] S303. Based on the point cloud data, determine the current mapping position of the bronchoscope in the bronchial centerline model of the target object.

[0110] S304. Adjust the direction and curvature of the bronchoscope according to its current mapping position.

[0111] S305. Based on the bronchoscope's path planning data, control and adjust the direction and curvature of the bronchoscope to move a preset distance.

[0112] S306. Determine if the bronchoscope has reached the target insertion position.

[0113] If yes, then execute S307; otherwise, execute S302.

[0114] S307. End the bronchoscopy insertion.

[0115] In this embodiment, before each controlled movement of the bronchoscope during automated bronchoscope insertion, real-time point cloud data is used for registration, adjusting the direction and curvature of the bronchoscope to avoid deviation from the planned path due to poor preoperative registration. Simultaneously, the automatic control of the bronchoscope insertion by the terminal device allows for movement without physician intervention, reducing surgical difficulty.

[0116] The following explains the process of adjusting the direction and curvature of the bronchoscope. Figure 17 This is a schematic diagram illustrating another catheter movement control process provided in an embodiment of this application. Figure 17 As shown, the movement control of the catheter includes:

[0117] S401, Control the bronchoscope to move a preset distance.

[0118] S402. Determine if the bronchoscope has reached the target insertion position.

[0119] If yes, then execute S411; otherwise, execute S403.

[0120] S403. Acquire point cloud data collected by bronchoscopy.

[0121] S404. Based on the point cloud data, determine the current mapping position of the bronchoscope in the bronchial centerline model of the target object.

[0122] S405. Based on the current mapping position of the bronchoscope, determine the angle difference between the direction of movement of the bronchoscope and the direction of the center line of the target bronchus on the second plane.

[0123] S406. Determine whether the angle difference is greater than the first threshold.

[0124] If yes, then execute S407; otherwise, execute S408.

[0125] S407. Adjust the direction of the bronchoscope.

[0126] S408. Based on the current mapping position of the bronchoscope, determine the angle between the direction of movement of the bronchoscope and the direction of the center line of the target bronchus.

[0127] S409. Determine whether the included angle difference is greater than the second threshold.

[0128] If yes, then execute S410; otherwise, execute S401.

[0129] S410. Adjust the curvature of the bronchoscope.

[0130] After S410, execute S401.

[0131] S411. End the bronchoscopy insertion.

[0132] The catheter movement control method provided in this application first acquires point cloud data collected by a bronchoscope, which characterizes the position of the bronchoscope in the bronchus of the target object. Second, based on the point cloud data, the current mapping position of the bronchoscope is determined in the bronchial centerline model of the target object. Third, based on the current mapping position of the bronchoscope, the direction and curvature of the bronchoscope are adjusted. Finally, based on the bronchoscope's path planning data, the bronchoscope is moved a preset distance after the direction and curvature are adjusted. Because the direction and curvature of the bronchoscope are adjusted based on the point cloud data before each movement, precise automatic movement control of the bronchoscope is achieved without manual intervention, thus improving the efficiency of bronchoscope insertion.

[0133] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0134] Based on the same inventive concept, this application also provides a catheter movement control device for implementing the catheter movement control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more catheter movement control device embodiments provided below can be found in the limitations of the catheter movement control method described above, and will not be repeated here.

[0135] In one embodiment, such as Figure 18 As shown, a catheter movement control device 500 is provided, comprising: a first acquisition module 501, a determination module 502, and a control module 503, wherein:

[0136] The first acquisition module 501 is used to acquire point cloud data collected by the catheter, and the point cloud data is used to characterize the position of the catheter in the target object;

[0137] The determination module 502 is used to determine the current mapping position of the conduit in the centerline model of the target object based on the point cloud data;

[0138] The control module 503 is used to adjust the direction and curvature of the catheter according to the current mapping position of the catheter; and to control the catheter to move a preset distance after adjusting the direction and curvature according to the path planning data of the catheter.

[0139] In one embodiment, the control module 503 includes:

[0140] The centerline direction determination unit is used to determine the direction of the target centerline corresponding to the current mapping position of the duct.

[0141] The movement direction determination unit is used to determine the movement direction of the duct based on point cloud data;

[0142] The adjustment unit is used to adjust the orientation and curvature of the catheter according to the direction of the catheter's movement and the direction of the target centerline.

[0143] In one embodiment, the centerline direction determination unit is specifically used to determine the first plane of the catheter, the first plane being the head plane of the catheter; and to determine the tangent direction between the first plane and the target centerline as the direction of the target centerline.

[0144] In one embodiment, the movement direction determination unit is specifically used to determine the center curve of the catheter based on point cloud data; determine the target tangent of the center curve of the catheter at the head centerline point of the catheter; and determine the direction of the target tangent as the movement direction of the catheter.

[0145] In one embodiment, the adjustment unit is specifically used to adjust the direction of the catheter based on the angle difference between the direction of movement of the catheter and the direction of the target centerline on a second plane, the second plane being the tail plane of the catheter; and to adjust the degree of curvature of the catheter based on the angle between the direction of movement of the catheter and the direction of the target centerline.

[0146] In one embodiment, the adjustment unit is specifically configured to determine the angle difference as the angle to be rotated of the catheter if the angle difference between the direction of movement of the catheter and the direction of the target centerline on the second plane is greater than a first threshold; and adjust the direction of the catheter according to the angle to be rotated of the catheter.

[0147] In one embodiment, the adjustment unit is specifically configured to keep the direction of the catheter unchanged if the angle difference between the direction of the catheter's movement and the direction of the centerline corresponding to the current mapped position of the catheter on the target plane is less than or equal to a first threshold.

[0148] In one embodiment, the adjustment unit is specifically configured to determine the angle to be bent as the angle between the direction of movement of the catheter and the direction of the target centerline if the angle is greater than a second threshold; adjust the degree of bending of the catheter according to the degree of bending of the catheter; and keep the degree of bending of the catheter unchanged if the angle between the direction of movement of the catheter and the direction of the target centerline is less than or equal to the second threshold.

[0149] Each module in the aforementioned catheter movement control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0150] In one embodiment, catheter movement control is provided using a computer device, which may be a terminal, and its internal structure diagram may be as follows. Figure 19 As shown. The computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the aforementioned catheter movement control method or the aforementioned automatic bronchoscope insertion method.

[0151] The display unit of this computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of this computer device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad set on the casing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0152] Those skilled in the art will understand that Figure 19The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0153] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method for controlling the movement of the conduit.

[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described method for controlling the movement of a catheter.

[0155] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described catheter movement control method.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0157] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0158] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0159] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A device for controlling the movement of a catheter, characterized in that, The device includes: The first acquisition module is used to acquire point cloud data collected by the catheter, and the point cloud data is used to characterize the position of the catheter in the target object; The determination module is used to determine the current mapping position of the conduit in the centerline model of the target object based on the point cloud data; The control module is used to adjust the direction and curvature of the catheter according to its current mapping position; and to control the catheter to move a preset distance after adjusting its direction and curvature according to the path planning data of the catheter. The control module includes: a centerline direction determination unit, used to determine a first plane of the catheter, the first plane being the head plane of the catheter; and to determine the tangent direction between the first plane and the target centerline as the direction of the target centerline corresponding to the current mapping position of the catheter; A movement direction determination unit is used to determine the movement direction of the duct based on the point cloud data; The adjustment unit is used to adjust the orientation and curvature of the catheter according to the direction of movement of the catheter and the direction of the target centerline.

2. The apparatus according to claim 1, characterized in that, The movement direction determination unit is specifically used to determine the center curve of the conduit based on the point cloud data; determine the target tangent of the center curve of the conduit at the head centerline point of the conduit; and determine the direction of the target tangent as the movement direction of the conduit.

3. The apparatus according to claim 1, characterized in that, The adjustment unit is specifically used to adjust the direction of the catheter based on the angle difference between the direction of movement of the catheter and the direction of the target centerline on a second plane, wherein the second plane is the tail plane of the catheter; and to adjust the degree of curvature of the catheter based on the angle between the direction of movement of the catheter and the direction of the target centerline.

4. The apparatus according to claim 3, characterized in that, The adjustment unit is specifically used to determine the angle difference as the angle to be rotated of the catheter if the angle difference between the moving direction of the catheter and the direction of the target centerline on the second plane is greater than a first threshold. Adjust the direction of the catheter according to the angle to be rotated.

5. The apparatus according to claim 3, characterized in that, The adjustment unit is specifically used to keep the direction of the catheter unchanged if the angle difference between the direction of movement of the catheter and the direction of the center line corresponding to the current mapping position of the catheter on the target plane is less than or equal to a first threshold.

6. The apparatus according to claim 3, characterized in that, The adjustment unit is specifically used to determine the angle to be bent as the angle between the direction of movement of the catheter and the direction of the target centerline if the angle is greater than a second threshold; adjust the degree of bending of the catheter according to the degree of bending of the catheter; and keep the degree of bending of the catheter unchanged if the angle between the direction of movement of the catheter and the direction of the target centerline is less than or equal to the second threshold.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The steps implemented when the computer program is executed by the processor include: Acquire point cloud data collected by the catheter, the point cloud data being used to characterize the position of the catheter in the target object; Based on the point cloud data, determine the current mapping position of the conduit in the centerline model of the target object; Adjust the direction and curvature of the catheter according to its current mapped position; Based on the catheter path planning data, the catheter is moved a preset distance after the direction and degree of curvature are controlled and adjusted; The step of adjusting the direction and curvature of the conduit based on its current mapped position includes: determining a first plane of the conduit, wherein the first plane is the head plane of the conduit; determining the tangent direction between the first plane and the target centerline as the direction of the target centerline corresponding to the current mapped position of the conduit; determining the movement direction of the conduit based on the point cloud data; and adjusting the orientation and curvature of the conduit based on the movement direction of the conduit and the direction of the target centerline.

8. The computer-readable storage medium according to claim 7, characterized in that, Determining the direction of movement of the conduit based on the point cloud data includes: The center curve of the duct is determined based on the point cloud data; Determine the target tangent of the center curve of the catheter at the center line point of the catheter head; determine the direction of the target tangent as the direction of movement of the catheter.

9. The computer-readable storage medium according to claim 7, characterized in that, The step of adjusting the orientation and curvature of the catheter according to the direction of movement of the catheter and the direction of the target centerline includes: The direction of the catheter is adjusted according to the angle difference between the direction of movement of the catheter and the direction of the target centerline on the second plane, where the second plane is the tail plane of the catheter; The degree of curvature of the catheter is adjusted according to the angle between the direction of movement of the catheter and the direction of the target centerline.

10. The computer-readable storage medium according to claim 9, characterized in that, The step of adjusting the direction of the catheter based on the angle difference between the direction of movement of the catheter and the direction of the target centerline on the second plane includes: If the angle difference between the direction of movement of the catheter and the direction of the target centerline on the second plane is greater than a first threshold, then the angle difference is determined as the angle to be rotated of the catheter. Adjust the direction of the catheter according to the angle to be rotated.

11. The computer-readable storage medium according to claim 9, characterized in that, The step of adjusting the direction of the catheter based on the angle difference between the direction of movement of the catheter and the direction of the target centerline on the second plane includes: If the angle difference between the direction of movement of the catheter and the direction of the centerline corresponding to the current mapped position of the catheter on the target plane is less than or equal to a first threshold, then the direction of the catheter remains unchanged.

12. The computer-readable storage medium according to claim 9, characterized in that, Adjusting the curvature of the catheter based on the angle between the direction of movement of the catheter and the direction of the target centerline includes: If the angle between the direction of movement of the catheter and the direction of the target centerline is greater than the second threshold, then the angle is determined as the bending angle of the catheter; and the degree of bending of the catheter is adjusted according to the bending angle of the catheter. If the angle between the direction of movement of the catheter and the direction of the target centerline is less than or equal to the second threshold, the degree of curvature of the catheter remains unchanged.

Images