Path Planning Method and Device with Multiple Condition Constraints

Abstract

A path planning method and device with multi-condition constraints can ensure the accuracy of lesion localization. The method includes: (1) The doctor interactively selects and determines the target area of the lesion, uses its centroid as the path planning target, and constructs multi-condition constraints based on the surgical entrance range limitation, obstacle avoidance limitation, and surgical instrument shape limitation; (2) Iteratively search for the tangent points of obstacles passing through the current path point, update the path point with the non-obstacle points near it until there are no obstacles between it and the target; traverse the surgical entrance range, use the reverse tangent point search method to search for path points, calculate the point cloud state of the surgical instrument coupling model at the path points, and check whether it meets the surgical entrance range constraint by detecting whether it collides with the surgical entrance boundary area, and check whether it meets the obstacle avoidance constraint by detecting whether it collides with obstacles; (3) Use the linear weighted method to solve the global optimal path, and use the main objective method to solve the best path under different objectives.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a path planning method with multiple condition constraints and a path planning device with multiple condition constraints. Background Art

[0002] Different from the conventional path planning problem, the starting point of the transnasal optic canal decompression path planning is not fixed. During the operation, the surgical instrument enters from the nasal cavity entrance, and the surgical entrance range constraint is the starting point range, which can be extracted by automatic segmentation. In practical applications, the doctor can also interactively select the surgical entrance range.

[0003] For the path planning of transnasal optic canal decompression, although the obstacle constraints of important tissues such as brain tissue, intraorbital tissue, nasal septum and inferior turbinate have been determined when constructing the environmental map, in practice, before performing transnasal optic nerve decompression, in order to expand the surgical field of view and operating space, the doctor will use adrenaline to contract the mucous membranes of the nasal septum and inferior turbinate and other parts multiple times. At present, there is no quantitative study on the contraction distance of nasal mucosa under the action of adrenaline.

[0004] During transnasal optic canal decompression, under direct vision of the endoscope, the surgeon inserts the scalpel from the surgical entrance into the lesion area of the optic canal, removes the bone on the medial wall of the optic canal to achieve decompression, and at the same time avoids damaging other tissues. The main surgical instruments used during the operation are nasal endoscopes and surgical drills (with shaving or grinding heads). The nasal endoscope is generally a rigid tube endoscope with a diameter of 2.7 - 4.0 mm. Different nasal endoscopes have different field of view angles (the maximum angle range that the endoscope can observe) and viewing angles (the angle between the geometric axis of the front end of the endoscope insertion part and its objective optical axis, which is convenient for oblique or side viewing), and can provide a good lighting environment for the surgical field of view. The surgical drill shaving knife is generally straight or curved rod-shaped, with common diameters of 2.5 mm, 2.9 mm, and 3.0 mm, and the shape of the knife head is complex and diverse. Therefore, the path planning of transnasal optic canal decompression is to plan the best path for the endoscope and the scalpel from the surgical entrance to the lesion area of the optic canal.

[0005] The path planning of the endoscope and the scalpel can be regarded as a two-robot cooperation path planning, but the two-robot planning is relatively complex, and the positions of the endoscope and the scalpel are relatively fixed. Summary of the Invention

[0006] To overcome the defects of the prior art, the technical problem to be solved by the present invention is to provide a path planning method with multiple condition constraints, which can ensure the accuracy of lesion localization.

[0007] The technical solution of the present invention is: This path planning method with multiple condition constraints includes the following steps:

[0008] (1) The doctor interactively selects and determines the target lesion area, uses its centroid as the path planning target, and constructs multi - condition constraints based on the surgical entrance range limitation, obstacle avoidance limitation, and surgical instrument shape limitation;

[0009] (2) Iteratively search for the tangent points of obstacles passing through the current path point, update the path point with non - obstacle points near it until there are no obstacles between it and the target; traverse the surgical entrance range, use the reverse tangent point search method to search for path points, calculate the point cloud state of the surgical instrument coupling model at the path points, and check whether it meets the surgical entrance range constraint by detecting whether it collides with the surgical entrance boundary area, and check whether it meets the obstacle avoidance constraint by detecting whether it collides with obstacles, so as to solve the feasible surgical entrance area and the set of feasible paths under multi - condition constraints;

[0010] (3) Use the linear weighted method to solve the global optimal path, and use the main - objective method to solve the best paths under different objectives, including the path farthest from the brain tissue, the shortest - length path, and the path with the least amount of tissue included.

[0011] The present invention first constructs the surgical entrance range, obstacle avoidance, and surgical instrument shape constraints. Then it traverses the surgical entrance range, uses the reverse tangent point search algorithm to search for the next path point, calculates the point cloud state of the surgical instrument coupling model, fuses the cylindrical envelope and KD - tree for collision detection, and obtains the feasible surgical entrance area and the set of feasible paths. Furthermore, it uses the multi - objective optimization method to solve the global optimal path and the best paths under different objectives, so as to ensure the accuracy of lesion localization.

[0012] A path planning device with multi - condition constraints is also provided, which includes:

[0013] A multi - condition constraint construction module, which is configured to allow the doctor to interactively select and determine the target lesion area, use its centroid as the path planning target, and construct multi - condition constraints based on the surgical entrance range limitation, obstacle avoidance limitation, and surgical instrument shape limitation;

[0014] A reverse tangent point search module, which is configured to iteratively search for the tangent points of obstacles passing through the current path point, update the path point with non - obstacle points near it until there are no obstacles between it and the target; traverse the surgical entrance range, use the reverse tangent point search method to search for path points, calculate the point cloud state of the surgical instrument coupling model at the path points, and check whether it meets the surgical entrance range constraint by detecting whether it collides with the surgical entrance boundary area, and check whether it meets the obstacle avoidance constraint by detecting whether it collides with obstacles, so as to solve the feasible surgical entrance area and the set of feasible paths under multi - condition constraints;

[0015] A solution module configured to use the linear weighted method to solve the global optimal path and the main objective method to solve the best paths under different objectives, including the path farthest from the brain tissue, the shortest length path, and the path with the least amount of tissue included. Description of the Drawings

[0016] Figure 1 is a flowchart of the multi-condition constrained path planning method according to the present invention. Detailed Implementation Manner

[0017] As Figure 1 shown, this multi-condition constrained path planning method includes the following steps:

[0018] (1) The doctor interactively selects and determines the lesion target area, uses its centroid as the path planning target, and constructs multi-condition constraints based on the surgical entrance range limitation, obstacle avoidance limitation, and surgical instrument shape limitation.

[0019] (2) Iteratively search for the tangent points of the obstacles passing through the current path point, update the path point with the non-obstacle points near it until there are no obstacles between it and the target; traverse the surgical entrance range, use the reverse tangent point search method to search for path points, calculate the point cloud state of the surgical instrument coupling model at the path points, and check whether it meets the surgical entrance range constraint by detecting whether it collides with the surgical entrance boundary area, and check whether it meets the obstacle avoidance constraint by detecting whether it collides with the obstacles, so as to solve the feasible surgical entrance area and the set of feasible paths under multi-condition constraints.

[0020] (3) Use the linear weighted method to solve the global optimal path and the main objective method to solve the best paths under different objectives, including the path farthest from the brain tissue, the shortest length path, and the path with the least amount of tissue included.

[0021] The present invention first constructs the surgical entrance range, obstacle avoidance, and surgical instrument shape constraints. Then traverse the surgical entrance range, use the reverse tangent point search algorithm to search for the next path point, calculate the point cloud state of the surgical instrument coupling model, fuse the cylindrical envelope and the KD tree for collision detection, and obtain the feasible surgical entrance area and the set of feasible paths. Furthermore, use the multi-objective optimization method to solve the global optimal path and the best paths under different objectives, so as to ensure the accuracy of lesion localization.

[0022] Preferably, in the step (1), for the surgical entrance range limitation,

[0023] The CT image is sliced into sagittal planes along the X-axis. First, the first skin-soft tissue edge point is detected from the front to the back, and starting from this edge point, possible edges are detected and traced backward and downward until they are all air points whether moving backward or downward. If there is no nasal cavity entrance, continuously tracing the soft tissue edge rightward and downward will eventually exceed the image range. Based on this, the nasal cavity entrance area is segmented on the X-axis.

[0024] Then, select the lowest edge point of the nasal margin contour as the front edge point of the nasal cavity entrance area. Detect the air-above - skin-below edge point backward and downward from this point, and select the edge point closest to the front edge point as the back edge point of the nasal cavity entrance area. All the voxels between the connections of these two edge points are located in the nasal cavity entrance area, and the set of voxels between the connections of the corresponding edge points of all sagittal planes of the nasal cavity entrance area is the required entrance area.

[0025] Under the action of external force, the maximum distance of the overall rightward displacement of the skin at the tip of the nose is d r , and the maximum distance of the overall upward displacement is d u . Extend and expand the edge of the nasal cavity entrance area rightward and upward by d r , d u distance respectively; take d r = 1 cm, d u = 1 cm, and extend and expand the nasal cavity entrance area to obtain the constraint of the surgical entrance range. Then, perform three-dimensional visualization on the surgical entrance range for easy observation and subsequent processing.

[0026] Preferably, in the step (1), for the obstacle avoidance limit,

[0027] Adjust the partial edge of the nasal septum mucosa on the horizontal plane, and contract it from left to right to the neighborhood of the soft tissue - bone edge. If no edge is detected, it is flush with the edge of the adjacent area; simulate the comparison of the nasal septum before and after the contraction of the nasal mucosa.

[0028] Adjust the contour edge of the inferior turbinate on the coronal plane, and contract it from right to left to the neighborhood of the soft tissue - bone edge or the soft tissue - air edge. If no edge is detected, it is flush with the edge of the adjacent area; simulate the comparison of the inferior turbinate before and after the contraction of the nasal mucosa.

[0029] Preferably, in the step (1), for the shape limit of the surgical instrument,

[0030] Establish a coupling model of the endoscope and the scalpel and perform path planning for it. When performing path planning for the mass point, the state of the mass point at each path node is its position coordinate. When performing path planning for the surgical instrument, expand the state of the surgical instrument at the path node from the three-dimensional coordinates of the mass point to a high-dimensional point cloud set.

[0031] Preferably, in the step (1), for the shape limit of the surgical instrument,

[0032] The initial state of the specified coupling model is: the center axes of the shaving knife and the endoscope are parallel to the Y-axis, perpendicular to the X-axis, and perpendicular to the Z-axis, and the corresponding pose angles are 0. At this time, the mathematical model of the shaving knife is:

[0033]

[0034] In the formula, (x t , y t , z t ) are the coordinates of the front vertex of the shaving knife, r s = d s / 2 is the radius of the shaving knife; the mathematical model of the beveled cylinder at the front end of the endoscope is:

[0035]

[0036] In the formula, r e = d e / 2 is the radius of the endoscope, h es is the height difference between the center of the front end face of the endoscope and the vertex of the shaving knife: r e + r s < h es < d w , l es is the difference in the front-back distance between the two: When d w = 10 mm, take h es = 6 mm, l es = 8 mm; it is further specified that the pose angles are: rotating around the initial center axis of the shaving knife as the rotation axis, the rotation angle is β; rotating around the Z-axis with the vertex of the shaving knife as the rotation center, the rotation angle is α; rotating around the X-axis with the vertex of the shaving knife as the rotation center, the rotation angle is γ; according to the right-hand system rule, the thumb points to the positive direction of the coordinate axis, and the direction of the four fingers is the positive direction of rotation; the vertex of the surgical instrument coupling model is P t = (x t , y t , z t ) and the pose angles are (α, β, γ), its point cloud set is:

[0037] S(x t , y t , z t , α, β, γ) = {(M - P t ) · R(α, β, γ) + P t : M ∈ M s ∪ M e} (3)

[0038] The state of the surgical instrument coupling model at the path node is a high-dimensional point cloud set jointly determined by three-dimensional vertex coordinates and three-dimensional pose angles;

[0039] In path planning, the surgical instrument advances and penetrates along the path points, and the vertex of the scalpel coincides with the path point; the surgical instrument moves straight from the current path node P c to the next path node P n . Then P t = P n . The central axis of the surgical shaver coincides with the line connecting P c and P n . α is the angle between the positive direction of the Y-axis and the projection on the XOY plane, and γ is the angle between the positive direction of the Y-axis and the projection on the YOZ plane; β is approximately 0.

[0040] Preferably, the step (2) includes the following sub-steps:

[0041] (2.1) Set the distance d t from the tangent point when updating the path point, and set the starting point as the current path point q s ;

[0042] (2.2) Detect all pixel points on the line connecting q s and the target point q g . If there is no obstacle point, the line connecting q s and q g is a feasible path, and jump to (6). If there is an obstacle, search for the tangent point q s of the first obstacle surface passing through q t on the line. When there are multiple tangent points, the tangent point with the minimum distance to the target can be selected;

[0043] (2.3) In the plane where q s , q t , and q g are located, calculate the point q t on the perpendicular line on the side away from the obstacle with a distance of d t from q gt ;

[0044] (2.4) Use q s as the starting point and q gt as the target to perform sub-path planning according to the above steps until there is no obstacle between q s and q gt , and the line connecting the two is the feasible path of the sub-path planning;

[0045] (2.5) Update q s to q gt , and use q s as the starting point and qg Perform sub-path planning for the target point according to the above steps until there are no obstacles between q s and q g ;

[0046] (2.6) Combine all the feasible paths obtained from sub-path planning to form the globally planned path.

[0047] Preferably, the collision detection in step (2) includes the following steps:

[0048] (2.a) Traverse each point C on the central axis of the cylinder, and use the nearest neighbor search algorithm to find the obstacle point B in the KD tree that is closest to C;

[0049] (2.b) Use the cylinder envelope method to detect whether B collides. If there is a collision, the algorithm ends; otherwise, record the minimum distance;

[0050] (2.c) Traverse all the surface points of the non-cylindrical part, and use the nearest neighbor search algorithm to find the obstacle point closest to it in the KD tree. If the distance is less than the threshold 0.1, it is considered that the algorithm ends; otherwise, record the minimum distance;

[0051] (2.d) If no collision occurs in the above steps, then there is no collision between the surgical instrument and the obstacle, and the distance between the two is the minimum value of the above minimum distances.

[0052] Preferably, in step (3),

[0053] The multi-objective optimization problem with constraints is described as:

[0054] min x F(x) = [f 1 (x), f 2 (x), …, f K (x)], x ∈ D (4)

[0055] where D is the feasible region under conditional constraints. The linear weighted method sets weights according to the importance k of the objective f

[0056] and performs linear weighting:

[0057]

[0058] In the formula, λ k is the weight of the objective f k (x);

[0059] The main objective method selects the most important sub-objective as the optimization objective, and the remaining sub-objectives are used as constraint conditions and are bounded by constraints:

[0060] min x fp (x), x ∈ D, f k (x) ≤ ∈ k , k ≠ p (6)

[0061] where the boundary value ∈ k Generally, the upper bound value of the sub-objective function is taken;

[0062] For the multi-objective optimization of the transnasal optic canal decompression path planning, the linear weighted method and the main objective method are suitable; construct the objective function of the distance between the constructed path and the brain tissue:

[0063]

[0064] where d bi is the distance from path i to the brain tissue, d bmax is the maximum distance from all feasible paths to the brain tissue, d bmin is the minimum distance from all feasible paths to the brain tissue; construct the objective function of the distance between the constructed path and the intraorbital tissue f e (i), the distance function f ns (i) from the nasal septum, and the distance function f it (i) from the inferior turbinate, construct the objective function of the path length:

[0065]

[0066] where l i is the length of path i, l min is the shortest length of all feasible paths, l max is the longest length of all feasible paths; the amount of tissue included in the path is measured by the number of soft tissue and bone pixel points included in the path channel, and construct the objective function of the amount of target tissue included in the path:

[0067]

[0068] where t i is the amount of tissue included in path i, t max is the maximum amount of tissue included in all feasible paths, t min is the minimum amount of tissue included in all feasible paths;

[0069] Use the linear weighted method to determine the global optimal path, and assign different weights to different objective functions according to the importance of the objectives:

[0070]

[0071] In the formula, the distance between the path and the brain tissue is the most important, and its weight coefficient w b is the largest; the distance between the path and the intraorbital tissue and the amount of tissue included are relatively important, and their weight coefficients w e and wt is larger; the importance of the distance between the path and the nasal septum, the distance from the inferior turbinate, and the length is average, and their weight coefficients w ns , w it , w l is the smallest.

[0072] Those of ordinary skill in the art can understand that all or part of the steps in implementing the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps of the methods of the above embodiments, and the storage medium can be: ROM / RAM, magnetic disk, optical disc, memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also simultaneously includes a path planning device with multi-condition constraints. The device includes:

[0073] A multi-condition constraint construction module configured to interactively select and determine the lesion target area by a doctor, use its centroid as the path planning target, and construct multi-condition constraints based on surgical entrance range limitations, obstacle avoidance limitations, and surgical instrument shape limitations;

[0074] A reverse tangent point search module configured to iteratively search for the tangent points of obstacles passing through the current path point, update the path point with non-obstacle points near it until there are no obstacles between it and the target; traverse the surgical entrance range, use the reverse tangent point search method to search for path points, calculate the point cloud state of the surgical instrument coupling model at the path points, and check whether it meets the surgical entrance range constraint by detecting whether it collides with the surgical entrance boundary area, and check whether it meets the obstacle avoidance constraint by detecting whether it collides with obstacles, so as to solve the feasible surgical entrance area and the set of feasible paths under multi-condition constraints;

[0075] A solution module configured to use the linear weighted method to solve the global optimal path and use the main objective method to solve the best path under different objectives, including the path farthest from the brain tissue, the shortest length path, and the path with the least tissue volume.

[0076] The present invention will be described in more detail below.

[0077] 1. Multi-condition constraints

[0078] 1.1 Surgical entrance range constraint

[0079] Different from the conventional path planning problem, the starting point of the transnasal optic canal decompression path planning is not fixed. During the operation, the surgical instrument enters from the nasal cavity entrance, and the surgical entrance range constraint is the starting point range, which can be extracted by automatic segmentation. In practical applications, the doctor can also interactively select the surgical entrance range.

[0080] For transnasal left optic canal decompression, the left nasal cavity entrance is located within the region of interest below, behind, and to the left of the tip of the nose. The CT image is sliced into sagittal planes along the X-axis. First, the first skin and soft tissue edge point is detected from front to back. Starting from this edge point, possible edges are detected and traced backward and downward until air points are reached whether moving backward or downward. If there is no nasal cavity entrance, continuously tracing the soft tissue edge rightward and downward will eventually exceed the image range, based on which the nasal cavity entrance region can be segmented on the X-axis.

[0081] Then, select the lowermost edge point of the nasal margin contour as the anterior (upper) edge point of the nasal cavity entrance region. Detect the air (upper) - skin (lower) edge point backward and downward from this point, and select the edge point closest to the anterior edge point as the posterior (lower) edge point of the nasal cavity entrance region. The voxels between the connections of these two edge points are all within the nasal cavity entrance region, and the set of voxels between the corresponding edge point connections of all sagittal planes of the nasal cavity entrance region is the desired entrance region.

[0082] The nasal skin in the nasal cavity entrance region is very loose soft tissue and can easily undergo large deformations to the left and right sides or upward under external forces, even exposing tissues such as the nasal septum. Assume that under external forces, the maximum displacement distance of the skin at the tip of the nose to the right is d r , and the maximum displacement distance upward is d u . To be closer to the actual situation, extend and expand the nasal cavity entrance region edge to the right and upward by d r , d u distances respectively. Through multiple human experiments and measurements, for adults, d r , d u both exceed 1 cm. Therefore, take d r = 1 cm, d u = 1 cm, extend and expand the nasal cavity entrance region to obtain the surgical entrance range constraint. Then perform three-dimensional visualization of the surgical entrance range for easy observation and subsequent processing.

[0083] 1.2 Obstacle Constraint

[0084] For the path planning of transnasal optic canal decompression, although the obstacle constraints of important tissues such as brain tissue, intraorbital tissue, nasal septum, and inferior turbinate have been determined when constructing the environmental map, in practice, before transnasal optic nerve decompression, in order to expand the surgical field of view and operating space, doctors will use adrenaline to contract the mucosa of the nasal septum and inferior turbinate multiple times. Currently, there is no quantitative study on the contraction distance of nasal mucosa under the action of adrenaline. Therefore, simulate the contraction of nasal mucosa based on prior knowledge. In practical applications, if conditions permit, the CT image of the patient's nasal mucosa after contraction can be used for preoperative path planning.

[0085] On both sides of the front end of the nasal septum, there is a small amount of mucosa attached, and the middle is the nasal septum cartilage. For the left transnasal optic canal decompression surgery, adrenaline acts on the left side of the nasal septum. Adjust the partial edge of the nasal septum mucosa in the horizontal plane, and contract it from left to right to the neighborhood of the soft tissue (left) - bone (right) edge. If the edge is not detected, it is flush with the edge of the adjacent area. In the simulation experiment, the maximum rightward contraction of the nasal septum contour edge was about 4.06 mm, which is more in line with the actual situation. Compare the nasal septum before and after simulating the contraction of the nasal mucosa.

[0086] The surface of the inferior turbinate is covered with thick mucosa, and it contracts more significantly under the action of adrenaline, contracting from the outer surface inward to near the bone. For the left transnasal optic canal decompression surgery, adrenaline acts on the right side, upper and lower parts of the inferior turbinate. Adjust the contour edge of the inferior turbinate in the coronal plane, and contract it from right to left to the neighborhood of the soft tissue (right) - bone (left) edge or the soft tissue (right) - air (left) edge. If the edge is not detected, it is flush with the edge of the adjacent area. In the simulation experiment, the maximum inward contraction of the inferior turbinate edge contour was about 8.13 mm, which is more in line with the actual situation. Compare the inferior turbinate before and after simulating the contraction of the nasal mucosa.

[0087] The three-dimensional spatial area through which surgical instruments can pass between the brain tissue, intraorbital tissue, nasal septum and inferior turbinate has been significantly enlarged.

[0088] 1.3 Shape constraints of surgical instruments

[0089] In the transnasal optic canal decompression surgery, under direct vision with an endoscope, the doctor inserts a scalpel from the surgical entrance into the lesion area of the optic canal, removes the bone on the medial wall of the optic canal to achieve decompression, and at the same time avoids damaging other tissues. The main surgical instruments used in the surgery are nasal endoscopes and surgical drills (with shaving or grinding heads). Nasal endoscopes are generally rigid tube endoscopes with a diameter of 2.7 - 4.0 mm. Different nasal endoscopes have different field of view angles (the maximum angle range that the endoscope can observe) and viewing angles (the angle between the geometric axis of the front end of the insertion part of the endoscope and its objective lens optical axis, which is convenient for oblique or lateral viewing), which can provide a good lighting environment for the surgical field. The shaving knife of the surgical drill is generally straight or curved rod-shaped, with common diameters of 2.5 mm, 2.9 mm, and 3.0 mm, and the shape of the knife head is complex and diverse. Therefore, the path planning of the transnasal optic canal decompression surgery is to plan the best path for the endoscope and the scalpel from the surgical entrance to the lesion area of the optic canal.

[0090] The path planning of the endoscope and the scalpel can be regarded as the path planning of two - machine cooperation. However, the two - machine planning is relatively complex, and the positions of the endoscope and the scalpel are relatively fixed. Therefore, this paper establishes a coupling model of the endoscope and the scalpel and conducts path planning for it. When conducting path planning for a particle, the state of the particle at each path node is its position coordinates; when conducting path planning for a surgical instrument, the shape and size constraints of the surgical instrument need to be satisfied, and it cannot be regarded as a particle. Therefore, the state of the surgical instrument at the path node is extended from the three - dimensional coordinates of a particle to a high - dimensional point cloud set.

[0091] During the operation, the nasal endoscope generally illuminates from above, and the scalpel works within its lower working distance. At this time, the imaging is relatively clear. Measuring (or referring to the instrument parameters) gives the diameter d of the commonly used nasal endoscope e = 4mm, the length l e = 175mm, and the working distance is d w = 10mm (at this time, the visual magnification is more than one - fold), and the viewing angle θ = 30°; the diameter d of the surgical shaver s = 2.9mm, the length l s = 110mm. The head of the surgical shaver is approximately spherical. Ideally, its front vertex is located at the working distance of the endoscope. During actual operation, the central axis of the nasal endoscope and the central axis of the surgical shaver are approximately in the same plane, and the included angle between their central axes is very small and approximately parallel.

[0092] It is stipulated that the initial state of the coupling model is: the central axes of the shaver and the endoscope are parallel to the Y - axis, perpendicular to the X - axis, and perpendicular to the Z - axis, and the corresponding pose angles are: (α = 0, β = 0, γ = 0). At this time, the mathematical model of the shaver is:

[0093]

[0094] In the formula, (x t , y t , z t ) are the coordinates of the front vertex of the shaver, and r s = d s / 2 is the radius of the shaver. The front - end beveled cylinder of the endoscope can be approximated as a cylinder, and its mathematical model is:

[0095]

[0096] In the formula, r e = d e / 2 is the radius of the endoscope, h es is the height difference between the center of the front end face of the endoscope and the vertex of the shaver: r e + r s < h es < d w , l es is the difference in the front - to - back distance between the two: d w When d = 10 mm, h can be taken as es h = 6 mm, and l es = 8 mm. Then, the pose angles are defined as follows: taking the initial central axis of the planing tool (parallel to the Y-axis) as the rotation axis and rotating around it by an angle β; taking the vertex of the planing tool as the rotation center and rotating around the Z-axis by an angle α; taking the vertex of the planing tool as the rotation center and rotating around the X-axis by an angle γ. According to the right-hand system rule, the thumb points to the positive direction of the coordinate axis, and the direction of the four fingers is the positive direction of rotation. Therefore, the vertex of the surgical instrument coupling model is P t =(x t , y t , z t ), and when the pose angles are (α, β, γ), its point cloud set is:

[0097] S(x t , y t , z t , α, β, γ) = {(M - P t ) · R(α, β, γ) + P t : M ∈ M s ∪M e} (3)

[0098] As can be seen from the above formula, the state of the surgical instrument coupling model at the path node is a high-dimensional point cloud set jointly determined by the three-dimensional vertex coordinates and the three-dimensional pose angles.

[0099] In path planning, the surgical instrument advances and penetrates along the path points, and the vertex of the scalpel coincides with the path point. The surgical instrument moves straight from the current path node P c to the next path node P n . Then, P t = P n , the central axis of the surgical planing tool coincides with the line connecting P c and P n . α is the angle between the positive direction of the Y-axis and the projection of in the XOY plane, and γ is the angle between the positive direction of the Y-axis and the projection of in the YOZ plane. Although β is not determined, it is restricted by obstacles and the left and right boundaries and can only vary within a very limited angle range and can be approximated as 0. Therefore, when the path node is determined, the above coupling model is discretely rasterized in the environmental map space, and the point cloud state of the surgical instrument at this path point is also correspondingly determined. Due to the shape and size constraints of the surgical instrument, when performing path planning for it, whether it is collision detection or calculating the distance between it and important tissues, the point cloud state of it needs to be solved.

[0100] 2. Reverse tangent point search

[0101] To achieve efficient search for path nodes in a three-dimensional environmental map under multiple condition constraints, referring to the ideas and strategies of divide-and-conquer algorithms and dynamic programming algorithms, a Reverse Tangent Points Search (RTS) path planning algorithm is proposed. It iteratively searches for the tangent points of the obstacle surface passing through the current path point, and updates the path point with the non-obstacle points near it until there are no obstacles between the current path point and the target.

[0102] In path planning, the simplest case is that there are no obstacles between the starting point and the target point, and the line segment connecting the two is the shortest path. If there is only one obstacle between the starting point and the target point, solve for the tangent and the corresponding tangent point of the obstacle surface contour passing through the starting point. There are no obstacles between the starting point and the tangent point, and between the tangent point and the target point. Then the line connecting the starting point and the tangent point is the first segment of the path, and the line connecting the tangent point and the target point is the second segment of the path. The global path planning problem is solved by dividing it into two parts, and merging the two segments of the path is the feasible path. If there are multiple tangent points, there are multiple starting point - tangent point - target point paths, and the shortest path or the path with the least curvature can be further selected from them.

[0103] Similarly, in general, there may be multiple obstacles between the starting point and the target point. Then the global path planning between the starting point and the target point can be continuously decomposed into sub-path planning between the starting point and the tangent point, and between the tangent point and the target point. Taking the starting point as the current path point, if the line connecting the current path point and the target point does not collide with an obstacle, the line is the feasible path. If a collision occurs, search for the tangent and the tangent point of the first obstacle on the line passing through the current path point, and perform sub-path planning for the current path point and the tangent point: if the line connecting the current path point and the tangent point does not collide with an obstacle, the line is the feasible path for sub-path planning, update the current path point to the tangent point and continue to perform sub-path planning for it and the target point; if there is a collision, search for the tangent and the tangent point of the first obstacle on the line passing through the current path point, and perform sub-path planning for the current path point and the tangent point... Iteratively search like this until there are no obstacles between the current path point and the target point. Merging the feasible paths of all sub-path planning is the feasible path of the global path planning. Since the tangent point is on the obstacle surface, to avoid collision, a non-obstacle point at a certain distance from the tangent point can be taken as the path point each time the path point is updated.

[0104] In the path planning of transnasal optic canal decompression, there are more obstacles near the target area than near the starting point, and the environmental map is more complex. Therefore, starting from the target and performing reverse search towards the starting point can reduce the search for useless tangent points and improve the search efficiency. To sum up, the processing flow of the RTS algorithm is as follows:

[0105] (1) Set the distance d from the tangent point when updating the path point t , and set the starting point as the current path point q s;

[0106] (2) For q s and the target point q g detect all the pixel points on the connection line. If there is no obstacle point, then q s and q g the connection line is the feasible path and jump to (6). If there is, search for the tangent point q s on the surface of the first obstacle on the connection line passing through q t . When there are multiple tangent points, the tangent point with the minimum distance to the target can be selected (or other strategies can be adopted);

[0107] (3) In the plane where q s , q t , and q g are located, calculate the point q t on the perpendicular line on the side away from the obstacle of the tangent line at a distance of d t from q gt ;

[0108] (4) Using q s as the starting point and q gt as the target, perform sub-path planning according to the above steps until there is no obstacle between q s and q gt . The connection line between the two is the feasible path of the sub-path planning;

[0109] (5) Update q s to q gt . Using q s as the starting point and q g as the target point, perform sub-path planning according to the above steps until there is no obstacle between q s and q g ;

[0110] (6) Merge all the feasible paths of the sub-path planning, which is the planned global path.

[0111] The reverse tangent point search algorithm needs to iteratively search for the tangent point passing through the current path point on the surface of the first obstacle on the connection line from the current path point to the target point, and is applicable to path planning with known global environmental information. When there is an obstacle between the starting point and the target point, the planned path is a path at a certain distance from the obstacle, approximately the shortest path close to the obstacle boundary. The fewer obstacles there are in the environmental map, the higher the search efficiency of the reverse tangent point search algorithm. The 4 obstacles in the transnasal optic nerve canal decompression path planning have been determined in the previous chapter, and the reverse tangent point search algorithm can achieve efficient search of path points in this path planning problem.

[0112] 3. Collision Detection by Fusing Cylindrical Envelope and KD Tree

[0113] In the process of path planning under multiple conditional constraints, to meet the shape constraints of surgical instruments, it is necessary to calculate the point cloud state of the instrument coupling model at the path points. To determine whether the path meets the obstacle avoidance constraint, it is necessary to perform collision detection on the point cloud of the coupling model and the obstacles. To determine whether the path meets the surgical entrance range constraint, it is necessary to perform collision detection on the point cloud of the coupling model and the boundary area of the surgical entrance: perform dilation processing on the upper, lower, left, and right edges of the surgical entrance range to obtain the dilated area adjacent to the surgical entrance boundary. If the point cloud of the surgical instrument collides with the dilated area, it exceeds the surgical entrance range and the corresponding path is infeasible; otherwise, it is feasible. The dilation structuring element can be a square structuring element with a size of 3×3.

[0114] Collision detection is the core link to determine whether the planned path meets multiple conditional constraints. However, usually, the point cloud of the instrument coupling model contains tens of thousands of voxels, and the number of voxels in obstacles such as brain tissue ranges from tens of thousands to millions. For collision detection between large-scale point clouds, if the traversal method is used to detect by judging whether the positions overlap, hundreds of millions of operations are required, which is very inefficient and time-consuming. To solve the problem of low efficiency, this paper combines the cylindrical envelope and the KD tree for collision detection.

[0115] The main parts of the surgical shaver and the nasal endoscope are cylindrical. The minimum cylindrical bounding box that can envelope them can be calculated using the bounding box algorithm, and then collision detection is performed. Inspired by this, the cylindrical bounding box algorithm is improved. For the cylindrical part of the surgical instrument or the robotic arm, the central axis of the cylindrical envelope is further extracted to replace the surface for collision detection and distance calculation.

[0116] Assume that the radius of the cylindrical part is r, and its central axis C is extracted 1 C 2 . For any point A or B in space, calculate the distance d between A and the line C 1 C 2 , ∠AC A C 1 C 2 , ∠AC 2 C 1 :

[0117] (1) When d A > r, A does not collide with the cylinder. If ∠AC 1 C 2 > 90° or ∠AC 2 C 1 > 90° (at this time A is at the position of B), then the distance between the two Otherwise, d = d A - r;

[0118] (2) When d A ≤ r, if ∠AC 1 C2 ≤ 90° and ∠AC 2 C 1 ≤ 90°, then the two will collide and d = 0; otherwise, no collision occurs.

[0119] The spatial position coordinates of the surface of the obstacle and the boundary area of the surgical entrance in the environmental map are fixed. They can be segmented according to certain rules, and data structures are used to accelerate the calculation. During collision detection, only adjacent structural units need to be detected. The KD-tree is a special type of spatial binary tree, where each node is a K-dimensional point, and the hyperplane where it is located divides the space into left and right parts. A typical creation method is as follows: As the depth of the tree increases, each dimension is alternately selected as the splitting axis in the order of decreasing data variance, and the data at the median of its coordinates is used as the subtree node. The splitting plane at this point divides the remaining data into left and right subtrees, and continues to divide until the leaf nodes. The nearest neighbor search process for finding the node in the KD-tree closest to point P is as follows:

[0120] (1) Starting from the root node, search recursively downward. If P is on the left side of the splitting plane, search in the left subtree; otherwise, search in the right subtree until reaching the leaf node, and mark it as the nearest node.

[0121] (2) Recursively backtrack upward. If the node passed through is closer to P than the nearest node, update it as the nearest node. If the splitting plane of the subtree passed through is closer to P than the nearest node, perform the nearest neighbor search in that subtree; otherwise, backtrack upward.

[0122] (3) Backtrack upward until the root node. The nearest node at this time is the node closest to P.

[0123] In summary, for the main cylindrical part of the surgical instrument, construct a cylindrical envelope and extract the central axis; for other non-cylindrical parts (such as the cutting head of the shaving knife), extract the surface points; for the obstacle, extract the surface points to construct a KD-tree; for the surgical entrance, extract the boundary expansion area to construct a KD-tree. The algorithm process for collision detection of the surgical instrument and the obstacle (or the boundary of the surgical entrance) by fusing the cylindrical envelope and the KD-tree is as follows:

[0124] (1) Traverse each point C on the central axis of the cylinder, and use the nearest neighbor search algorithm to find the obstacle point B in the KD-tree that is closest to C.

[0125] (2) Use the cylindrical envelope method to detect whether B collides. If it collides, the algorithm ends; otherwise, record the minimum distance. (3) Traverse all the surface points of the non-cylindrical parts, and use the nearest neighbor search algorithm to find the obstacle point in the KD-tree that is closest to it. If the distance is close to 0 (such as less than the threshold 0.1), it is considered that the algorithm ends; otherwise, record the minimum distance.

[0126] (4) If no collision occurs in the above steps, it means that the surgical instrument does not collide with the obstacle, and the distance between the two is the minimum value of the above minimum distances.

[0127] Traverse the starting points within the surgical entrance area, use the reverse tangent point search algorithm to search for path points, calculate the point cloud state of the surgical instrument coupling model, and use the method of fusing cylinder envelope and KD tree for collision detection. If the path still does not collide with the obstacle and does not collide with the boundary area of the surgical entrance when reaching the target, then this path is a feasible path, and the corresponding starting point is a feasible surgical entrance point.

[0128] 4. Multi-objective path optimization

[0129] To meet the clinical needs, after solving the set of feasible paths, it is necessary to further find the optimal path that achieves certain goals from them, such as the shortest path, the farthest distance from important tissues, the least damage, etc. There may be conflicts among these goals. The optimization of one goal often comes at the expense of the degradation of other goals, and it is difficult to determine the unique optimal solution. It is necessary to balance and coordinate these goals to make the overall global goal reach the optimal as much as possible. Therefore, the solution of the optimal path in the transnasal optic nerve canal decompression path planning is a multi-objective optimization problem under the constraints of avoiding obstacles of important tissues, the range of surgical entrance, the shape of surgical instruments, etc.

[0130] The constrained multi-objective optimization problem can be described as:

[0131] min x F(x) = [f 1 (x), f 2 (x), …, f K (x)], x ∈ D (4)

[0132] where D is the feasible domain under the conditional constraints, such as the set of feasible paths obtained by solving. There are many classic algorithms for multi-objective optimization, including the linear weighted method, the main objective method, the approximation objective method, the gradient descent method, the multi-task learning method, the genetic evolution algorithm, the particle swarm algorithm, etc. The linear weighted method sets weights for linear weighting according to the importance of the objective f k (x):

[0133]

[0134] In the formula, λ k is the weight of the objective f k (x). The main objective method (also known as the ∈-constraint method) selects the most important sub-objective as the optimization objective, and the remaining sub-objectives are used as constraint conditions subject to boundary constraints:

[0135] min x f p (x), x ∈ D, fk (x) ≤ ∈ k , k ≠ p (6)

[0136] where the boundary value ∈ k Generally, the upper bound value of the sub-objective function is taken.

[0137] For the multi-objective optimization of the transnasal optic canal decompression path planning, the objective function is not continuously differentiable, and the relative importance of each objective is generally known, so it is suitable to use the linear weighted method and the main objective method. According to clinical needs, the distance between the path and important tissue structures, the path length, and the tissue volume included in the path are selected as the objective functions. The distance between the path and important tissue structures is the minimum distance between the surgical instrument and the important tissue structures when moving along the path. The farther this distance is, the lower the surgical risk. Construct the objective function of the distance between the path and the brain tissue:

[0138]

[0139] where d bi is the distance from path i to the brain tissue, d bmax is the maximum distance from all feasible paths to the brain tissue, d bmin is the minimum distance from all feasible paths to the brain tissue. Similarly, the objective function f e (i) of the distance between the path and the intraorbital tissue, the distance function f ns (i) from the nasal septum, and the distance function f it (i) from the inferior turbinate can be constructed. The shorter the path length, the smaller the bending degree, and the less tissue included, the easier it is to open the path channel, and the greater the surgical success rate. The path length is the sum of the distances between adjacent path points. Construct the objective function of the path length:

[0140]

[0141] where l i is the length of path i, l min is the shortest length of all feasible paths, l max is the longest length of all feasible paths. The tissue volume included in the path can be measured by the number of soft tissue and bone pixel points included in the path channel. Construct the objective function of the tissue volume included in the path:

[0142]

[0143] where t i is the tissue volume included in path i, t max is the maximum tissue volume included in all feasible paths, t min is the minimum tissue volume included in all feasible paths.

[0144] The global optimal path can be determined using the linear weighted method. Different weights are assigned to different objective functions according to the importance of the objectives:

[0145]

[0146] Combined with clinical experience, the distance between the path and the brain tissue is the most important in the formula, and its weight coefficient w b is the largest; the distances between the path and the intraorbital tissues and the amount of included tissues are relatively important, and their weight coefficients w e and w t are relatively large;

[0147] The distances between the path and the nasal septum, the distance from the inferior turbinate, and the length are of general importance, and their weight coefficients w ns and

[0148] w it and w l are the smallest. Among all the paths, the path with the smallest value of the weighted function f is the global optimal path. To provide more surgical approaches for doctors to refer to and select, the above objectives can be used as the main objectives respectively, and the main objective method can be used to solve the best path under this objective function. If there is more than one best path under this objective, the linear weighted method is used for other objectives, and the path with the smallest sum of the weighted functions is selected.

[0149] The above are only the preferred embodiments of the present invention, and do not impose any form of limitation on the present invention. Any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A path planning device with multi - condition constraints, characterized in that it includes: A multi - condition constraint construction module, configured to determine the lesion target area through doctor - patient interaction selection, use its centroid as the path planning target, and construct multi - condition constraints based on surgical entry range limitation, obstacle avoidance limitation, and surgical instrument shape limitation; A reverse tangent point search module, configured to iteratively search for the tangent points of obstacles passing through the current path point, and update the path point with non - obstacle points near it until there are no obstacles between it and the target; Traverse the surgical entry range, use the reverse tangent point search method to search for path points, calculate the point cloud state of the surgical instrument coupling model at the path points, check whether it meets the surgical entry range constraint by detecting whether it collides with the surgical entry boundary area, and check whether it meets the obstacle avoidance constraint by detecting whether it collides with obstacles, so as to solve the feasible surgical entry area and the set of feasible paths under multi - condition constraints, including the following 6 steps: (1) Set the distance d from the tangent point when updating the path point t , and set the starting point as the current path point q s ; (2) For q s Detect all pixel points on the line connecting to the target point q g If there is no obstacle point, the line connecting q s and q g is the feasible path, jump to (6). If there is an obstacle, search for the tangent point q s on the surface of the first obstacle on the line passing through q t When there are multiple tangent points, the tangent point with the minimum distance to the target can be selected; (3) At q s 、q t 、q g In the plane where they are located, calculate the point q t on the perpendicular line on the side away from the obstacle of the tangent line, at a distance d t from q gt ; (4) Starting from q s as the starting point and q gt as the target, perform sub-path planning according to the above steps until there are no obstacles between q s and q gt . The line connecting the two is the feasible path of sub-path planning; (5) Update q s to q gt , and perform sub-path planning starting from q s and targeting q g according to the above steps until there are no obstacles between q s and q g . (6) Merge the feasible paths of all sub - path planning as the planned global path; A solution module, configured to use the linear weighted method to solve the global optimal path and use the main objective method to solve the best paths under different objectives, including the path farthest from the brain tissue, the shortest length path, and the path with the least amount of tissue included.

2. The path planning device with multi - condition constraints according to claim 1, characterized in that The main objective method selects the most important sub - objective as the optimization objective, and the remaining sub - objectives are used as constraint conditions subject to boundary constraints, satisfying the following constraint conditions: min x f p (x), x ∈ D, f k (x) ≤ ∈ k , k ≠ p where the threshold value ∈ k Take the upper bound value of the sub-objective function. D is the feasible region under the conditional constraints, and f k (x) and f p (x) are sub-objective functions; For the multi - objective optimization of the transnasal optic canal decompression path planning, construct the objective function of the distance between the path and the brain tissue: where d bi is the distance from path i to the brain tissue, and d bmax is the maximum distance from all feasible paths to the brain tissue, and d bmin is the minimum distance from all feasible paths to the brain tissue; construct the objective function f for the distance between the path and the intraorbital tissue e (i), the distance function f from the nasal septum ns (i), the distance function f from the inferior turbinate it (i), construct the objective function for the path length: where l i is the length of path i, l min is the shortest length of all feasible paths, and l max is the longest length of all feasible paths; the amount of tissue included in the path is measured by the number of soft tissue and bone pixels included in the path channel, and the objective function for constructing the amount of target tissue included in the path is: where t i is the tissue volume included in path i, and t max is the maximum tissue volume included in all feasible paths, and t min is the minimum tissue volume included in all feasible paths; Use the linear weighted method to determine the global optimal path, and assign different weights to different objective functions according to the importance of the objectives: Among them, the distance between the path and the brain tissue is the most important, and its weight coefficient w b is the largest; the distances between the path and the intraorbital tissues and the tissue volume involved are relatively important, and their weight coefficients w e and w t are relatively large; the distances between the path and the nasal septum, the inferior turbinate, and the length are of less importance, and their weight coefficients w ns and w it and w l are the smallest.

Images