Robot-assisted transauditory meatus inner ear injection sampling operation path planning method

Abstract

The embodiment of the invention discloses a robot-assisted transauditory meatus inner ear injection sampling operation path planning method. The method comprises the following steps: acquiring original image data of a patient; performing auditory meatus target structure segmentation on the CT sequence image by using an artificial intelligence segmentation model; performing three-dimensional reconstruction on the ear canal target structure; selecting a starting point, a tympanic membrane punching point and a round window membrane needle inserting point according to the three-dimensional reconstruction image; performing path search by using an improved A * path planning algorithm; and carrying out smoothing processing on the path by using cubic spline interpolation so as to obtain a final planned path. Operation path planning is simplified, objective and scientific, subjective operation path deviation caused by clinical experience is reduced, the learning curve of a doctor is shortened, and the operation success rate is increased.

Description

Technical Field

[0001] The present invention relates to the field of medicine, and in particular to a robot-assisted trans-aural intra-ear injection sampling surgical path planning method. Background Art

[0002] In traditional surgery, surgeons usually use two-dimensional CT image sequences to develop surgical plans. Due to the lack of depth information, it is difficult to accurately determine the relative position of the lesion tissue and surrounding structures, and it is impossible to pre-verify the planned surgical path, resulting in high surgical risks and failure rates. With the development of computer graphics, computer-assisted surgical planning (such as surgical path, perforation planning, needle insertion posture, etc.) has gradually been applied to clinical applications and has shown great advantages, such as three-dimensional visualization information of the surgical area and the ability to pre-verify the surgical path.

[0003] In the trans-auricular injection sampling surgery, the anatomical structure of the ear canal is long and complex, the round window membrane is difficult to identify, and the endoscope is prone to damage important human structures such as the ossicles, cochlea, and facial nerve during the operation. Therefore, the safety of preoperative surgical path planning is a key issue in robot-assisted endoscopic ear canal injection sampling surgery and has important clinical application value. Summary of the invention

[0004] The purpose of the present invention is to solve the problems raised in the above-mentioned background technology. The present invention provides a robot-assisted trans-auricular intra-ear injection sampling surgical path planning method, which simplifies the surgical path planning and makes it objective and scientific, thereby improving the safety and efficiency of clinical surgery.

[0005] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0006] A robot-assisted trans-auricular intra-auricular injection sampling surgical path planning method, comprising:

[0007] S1. Collect original imaging data of patients;

[0008] S2. Use artificial intelligence segmentation model to segment the ear canal target structure of CT sequence images;

[0009] S3, three-dimensional reconstruction of the target structure of the ear canal;

[0010] S4, selecting the starting point, the tympanic membrane perforation point, and the round window membrane needle insertion point according to the three-dimensional reconstructed image;

[0011] S5. Use the improved A* path planning algorithm to search for a path;

[0012] S6. Use cubic spline interpolation to smooth the path to obtain the final planned path.

[0013] Beneficial effects:

[0014] The present invention obtains the three-dimensional model of the patient's ear canal and other related structures by segmenting and reconstructing the CT image based on the patient's individualized imaging data, obtains the preliminary path using an improved path planning algorithm, and smoothes the path using cubic spline interpolation to obtain the final surgical planning path. The surgical path planning is simplified, objective, and scientific, reducing the deviation of the subjective surgical path caused by clinical experience, shortening the doctor's learning curve, and improving the success rate of surgery.

[0015] The present invention realizes accurate segmentation of the target structure; the improved A* path planning algorithm of the present invention improves the search efficiency, and makes the planned path meet the constraints in the operation, improves the safety, and smoothes the path through the cubic B-spline interpolation curve, so that the planned obstacle avoidance path is safer and more efficient, and improves the accuracy and smoothness of the operation of the continuum robot during the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is a flow chart of a method for planning a surgical path for robot-assisted intra-auricular injection sampling through the ear canal according to an embodiment of the present invention;

[0018] Figure 2 This is a diagram of the nn-UNet network model according to an embodiment of the present invention;

[0019] Figure 3 Schematic diagram of the structure of the A* algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0021] As shown in the figure, an embodiment of the present invention discloses a robot-assisted trans-aural intra-ear injection sampling surgical path planning method, comprising:

[0022] S1. Collect original imaging data of patients;

[0023] S2. Use artificial intelligence segmentation model to segment the ear canal target structure of CT sequence images;

[0024] S3, three-dimensional reconstruction of the target structure of the ear canal;

[0025] S4, selecting the starting point, the tympanic membrane perforation point, and the round window membrane needle insertion point according to the three-dimensional reconstructed image;

[0026] S5. Use the improved A* path planning algorithm to search for a path;

[0027] S6. Use cubic spline interpolation to smooth the path to obtain the final planned path.

[0028] The present invention can obtain a safe surgical path from the ear canal entrance point to the surgical area. The surgical path planning method of the present invention can simplify the surgical path planning and make it objective and scientific, thereby improving the safety and efficiency of clinical surgery.

[0029] In one embodiment of the present invention, in step S2, the nn-UNet deep neural network segmentation model is used to segment the target structure of the ear canal in the CT sequence images, and the collected CT sequence images of the patient's head are manually annotated. The annotated structures include but are not limited to the external auditory canal, tympanic membrane, auditory ossicles, facial nerve, round window membrane, cochlear bony labyrinth and membranous labyrinth, etc. A data set is made from the annotated images, and the data set is input into the nn-UNet deep neural network segmentation model for training. The training results are inferred and evaluated to obtain a segmentation model, and the segmentation results of each training segmentation model are evaluated until the optimal segmentation model is obtained. The automatic segmentation of the ear canal structure by the trained segmentation model can improve the quality and accuracy of the segmentation results.

[0030] In one embodiment of the present invention, in step S3, the segmented ear canal model is surface rendered using the vtkMarchingCubes function in VTK.

[0031] In one embodiment of the present invention, in step S4, the starting point, the tympanic membrane perforation point and the round window membrane insertion point are selected according to the reconstructed three-dimensional model. The starting point is the midpoint of the external auditory canal, the tympanic membrane perforation point is generally the center point of the tympanic membrane, and the perforation point of individual patients needs to be selected by the doctor based on the three-dimensional model of the ear canal. The end point is the round window.

[0032] In one embodiment of the present invention, in step S5, an improved A* algorithm is used to perform path search, and a binary three-dimensional grid map is constructed based on the three-dimensional model. If the pixel gray value is 0, the point is considered to be a free space point; if the pixel gray value is 255, the point is regarded as an obstacle point, and the grid position is abstracted as a point and represented by a three-dimensional coordinate point; according to the improved A* algorithm, a global path composed of free grids is obtained, and the global path consists of the three-dimensional coordinates of the free grids.

[0033] In one embodiment of the present invention, in step S5, the cost function of the A* algorithm is divided into two parts. The first part is from the starting point to the tympanic membrane perforation point. Since the external auditory canal is a straight channel structure similar to bone rigidity, the cost function of this part is designed as:

[0034] f(n)=g(n)+h(n)

[0035] Among them, the cost function f(n) is the sum of the actual cost and the estimated cost, g(n) is the actual cost between the starting position and the node n, h(n) is the estimated cost from the current node n to the target point g, and n is the extended node;

[0036] The estimated cost h(n) is the distance parameter. Using Euclidean distance as the heuristic distance, the estimated cost h(n) is:

[0037]

[0038] Among them, (x s ,y s , z s ) is the starting position coordinate, (x g ,y g , z g ) is the target point position coordinate,

[0039] The second part is from the tympanic membrane puncture point to the target point. Since the middle ear is a narrow and tortuous rigid bone structure, in order to meet the clinical surgical requirements, the continuum robot must first search for a path downward after entering the tympanic membrane puncture position, and then reach the target position at the desired angle. Therefore, the downward cost factor and the angle cost factor are added to the cost function of the second part. The cost function of the second part is:

[0040]

[0041] Among them, (x t ,y t , z t ) is the coordinate of the tympanic membrane perforation point, (x n ,y n , z n ) is the current position coordinate, (x g ,y g , z g ) is the target point position coordinate, θ e is the expected angle, the angle that the robot expects to reach with the horizontal direction, θr is the actual angle, the angle between the robot's current node and the target node and the horizontal direction, C down is the downward cost factor, C angle is the angle cost factor.

[0042] The cost function of the first part evaluates the quality of the path from the starting point to the tympanic membrane perforation point. The cost function of the second part evaluates the quality of the path from the tympanic membrane perforation point to the target point.

[0043] In one embodiment of the present invention, in step S5, the three-dimensional A* algorithm search step is to search in the "open" list to find the node with the smallest F value; if the "open" list is empty, then exit the loop; if the "open list" is not empty, then put the current node into the "closed" list, and regard the node with the smallest F value as the current node; check the nodes adjacent to the current node, if these nodes are obstacles, then search for child nodes again;

[0044] If these nodes are not obstacles, they are not in the "closed" list but in the "open" list, and then calculate their new G, H, and F values; if their new F values ​​are less than the old F values, update their G, H, F values ​​and their parent node information;

[0045] After jumping out of the path search loop, if the open list is empty, the path search fails and no suitable path is found; if the current node coordinates are equal to the target node coordinates, the path search is successful, and the found path can be recorded, starting from the target node, along the parent node information, and returning to the starting node to obtain the entire path.

[0046] The present invention creatively improves the A* algorithm, which can improve search efficiency, make the planned path meet the constraints in surgery, and improve safety.

[0047] In one embodiment of the present invention, in step S3, the segmented binary mask image information is read to extract the volume pixel points of the two-dimensional image sequence of adjacent layers; the scalar field gradient value is calculated to obtain the vertex coordinates and normal vector of the pixel point; the combination method is obtained according to the index table to generate the isosurface of the three-dimensional model.

[0048] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A robot-assisted trans-auricular intra-auricular injection sampling surgical path planning method, characterized in that: include: S1. Collect original imaging data of patients; S2. Use artificial intelligence segmentation model to segment the ear canal target structure of CT sequence images; S3, three-dimensional reconstruction of the target structure of the ear canal; S4, selecting the starting point, the tympanic membrane perforation point, and the round window membrane needle insertion point according to the three-dimensional reconstructed image; S5. Use the improved A* path planning algorithm to search for a path; S6. Use cubic spline interpolation to smooth the path to obtain the final planned path.

2. A robot-assisted trans-auricular intra-auricular injection sampling surgical path planning method according to claim 1, characterized in that: In step S2, the nn-UNet deep neural network segmentation model is used to segment the ear canal target structure of the CT sequence images, and the collected patient head CT sequence images are manually annotated. The annotated structures include the external auditory canal, tympanic membrane, auditory ossicles, facial nerve, round window membrane, cochlear bony labyrinth and membranous labyrinth. A data set is made from the annotated images, and the data set is input into the nn-UNet deep neural network segmentation model for training. The training results are inferred and evaluated to obtain a segmentation model, and the segmentation results of each training segmentation model are evaluated until the optimal segmentation model is obtained.

3. The robot-assisted trans-auricular intra-auricular injection sampling surgical path planning method according to claim 1, characterized in that: In step S3, the segmented ear canal model is surface rendered using the vtkMarchingCubes function in VTK.

4. The robot-assisted trans-auricular intra-auricular injection sampling surgical path planning method according to claim 1, characterized in that: In step S4, the starting point, tympanic membrane perforation point and round window membrane insertion point are selected according to the reconstructed three-dimensional model. The starting point is the midpoint of the external auditory canal, the tympanic membrane perforation point is generally the center point of the tympanic membrane, and the perforation point of individual patients needs to be selected by the doctor based on the three-dimensional model of the ear canal. The end point is the round window.

5. The robot-assisted trans-auricular intra-auricular injection sampling surgical path planning method according to claim 1, characterized in that: In step S5, an improved A* algorithm is used to perform path search, and a binary three-dimensional grid map is constructed based on the three-dimensional model. If the pixel gray value is 0, the point is considered to be a free space point; if the pixel gray value is 255, the point is considered to be an obstacle point, and the grid position is abstracted as a point and represented by a three-dimensional coordinate point; according to the improved A* algorithm, a global path composed of free grids is obtained, and the global path consists of the three-dimensional coordinates of the free grids.

6. The robot-assisted trans-auricular intra-auricular injection sampling surgical path planning method according to claim 5, characterized in that: In step S5, the cost function of the A* algorithm is divided into two parts. The first part is from the starting point to the eardrum perforation point. The cost function of this part is designed as: f(n)=g(n)+h(n) Among them, the cost function f(n) is the sum of the actual cost and the estimated cost, g(n) is the actual cost between the starting position and the node n, h(n) is the estimated cost from the current node n to the target point g, and n is the extended node; The estimated cost h(n) is the distance parameter. Using Euclidean distance as the heuristic distance, the estimated cost h(n) is: Among them, (x s ,y s , z s ) is the starting position coordinate, (x g ,y g , z g ) is the target point position coordinate, The second part is from the eardrum perforation point to the target point. The cost function of the second part is: Among them, (x t ,y t , z t ) is the coordinate of the tympanic membrane perforation point, (x n ,y n , z n ) is the current position coordinate, (x g ,y g , z g ) is the target point position coordinate, θ e is the expected angle, the angle that the robot expects to reach with the horizontal direction, θr is the actual angle, the angle between the robot's current node and the target node and the horizontal direction, C down is the downward cost factor, C angle is the angle cost factor.

7. The robot-assisted trans-auricular intra-auricular injection sampling surgical path planning method according to claim 6, characterized in that: In step S5, the three-dimensional A* algorithm search step is to search in the "open" list to find the node with the smallest F value; if the "open" list is empty, then exit the loop; if the "open list" is not empty, then put the current node into the "closed" list, and regard the node with the smallest F value as the current node; check the nodes adjacent to the current node, if these nodes are obstacles, then search for child nodes again; If these nodes are not obstacles, they are not in the "closed" list but in the "open" list, and then calculate their new G, H, and F values; if their new F values ​​are less than the old F values, update their G, H, and F values ​​and their parent node information; After jumping out of the path search loop, if the open list is empty, the path search fails and no suitable path is found; if the current node coordinates are equal to the target node coordinates, the path search is successful, and the found path can be recorded, starting from the target node, along the parent node information, and returning to the starting node to obtain the entire path.

8. The robot-assisted trans-auricular intra-auricular injection sampling surgical path planning method according to claim 3, characterized in that: In step S3, the segmented binary mask image information is read to extract the volume pixel points of the two-dimensional image sequence of the adjacent layers; the scalar field gradient value is calculated to obtain the vertex coordinates and normal vector of the pixel point; the combination method is obtained according to the index table to generate the isosurface of the three-dimensional model.

Images