Soft tissue cutting method in a virtual surgery simulation system

Abstract

A soft tissue cutting method in a virtual surgery simulation system, comprising: 1) constructing a surgical instrument and a soft tissue model; 2) determining a starting point of contact between the surgical instrument and the soft tissue surface, judging whether an external force applied by the surgical instrument to the soft tissue reaches a critical value, if yes, executing the next step of cutting, otherwise, continuing deformation; 3) splitting the vertex of the cutting point, recording the cutting point generated in the cutting and all vertex point sets of the reconstructed triangular grid unit; 4) determining an ending point of the path of the surgical instrument and the soft tissue surface; 5) according to the starting point and the ending point of the cutting surface path and the point sets recorded in step 3), a Bezier curve method is used to draw and form a smooth cutting wound; 6) calculating the force feedback generated in the cutting process and feeding back to the operator. The improved vertex splitting method is used to determine the cutting point, connect the cutting points and reconstruct the triangular grid topology, and the Bezier curve is used to redraw the incision, so that the smoothness of the incision is optimized and the effect is more realistic.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of virtual reality in medical technology, and particularly relates to the cutting problem between a soft tissue model and a surgical instrument in a virtual surgery simulation system. BACKGROUND

[0002] In a virtual surgery system, common cutting algorithms include a surface mesh-based cutting algorithm and a polyhedral-based cutting method.

[0003] The common surface mesh-based cutting algorithms include a cell removal method and a vertex split method, which are commonly used to simulate the cutting effect of soft tissue. The cell removal method, as a cutting algorithm proposed by Bro-Nielsen et al. (Bro-Nielsen M. Finite element modeling in surgery simulation [J]. Proceedings of the IEEE, 1998, 86(3): 489-503.) relatively early, generates a cut to simulate the cutting effect by removing the cells of the soft tissue in contact with the surgical instrument. This method is relatively simple to implement, but may not be fine enough when dealing with complex cutting paths or when it is necessary to maintain the smoothness of the cutting surface. The vertex split method is more fine, and according to the cutting path of the interaction between the surgical instrument and the soft tissue, the cutting point of the model is split into two new vertices symmetric about the cutting path, and then the new vertices are connected by a new connecting line, thereby simulating a smoother and more realistic cutting effect. To prevent the generation of low-quality reconstructed triangular facets during cutting, the vertex split algorithm often replaces the intersection point with a new vertex to maintain the reasonable topological structure of the model after cutting. These two algorithms have their own advantages and disadvantages and are suitable for different application scenarios. The cell removal method is more suitable for situations where the cutting precision requirement is not high or the computing resources are limited, while the vertex split method is more suitable for application scenarios that require high-precision cutting simulation and good visual effect.

[0004] Polyhedron-based cutting algorithms often use tetrahedrons or hexahedrons. Researchers have proposed a variety of algorithms based on the selection of different elements. Different algorithms differ in how they handle contact between surgical instruments and soft tissue, but all strive for higher simulation efficiency and better visual effects. Bielser et al. (Bielser D, Glardon P, Teschner M, et al. A state machine for real-time cutting of tetrahedral meshes [C]. Graphical Models, 2004: 398–417.) proposed a method that subdivides tetrahedral elements based on the contact points between surgical instruments and soft tissue, then forms a linked list, effectively improving simulation efficiency. This method accounts for differences in contact locations and enables more detailed simulation of the cutting process. Molino et al. (Molino N, Ba OZ, Fedkiw RA virtual node algorithm for changing mesh topology during simulation [J]. Acm Transactions on Graphics, 2005, 23 (3): 385-392.) proposed a virtual node algorithm to solve the problem of ill-conditioned voxels. By copying cutting units and assigning the remaining material to the copied units, the cutting simulation is stabilized. Pietron et al. (Pietron N, Ganovelli F, Cignoni P, et al. Splitting cubes: a fast and robust technique for virtual cutting [J]. Visual Computer, 2009, 25 (3): 227-239.) proposed a fast cube splitting technique based on hexahedrons. The technique embeds objects in a regular grid to simulate cutting and handle topological changes. The concept is powerful. Frisken et al. (Frisken-GIBSON S. Using linked volumes to model object collisions, deformation, cutting, carving, and joining [J]. IEEE Transactions on Visualization and Computer Graphics, 1999, 5(4): 333-348.) proposed a linked volume representation method, which decomposes the object into hexahedral elements, connects them through links, and disconnects them when cutting to represent the interaction state. The structural characteristics of the hexahedron are used to achieve stable and realistic simulation.

[0005] The soft tissue cutting method can also be classified according to different cutting processing opportunities, and is divided into non-progressive and progressive. Non-progressive cutting means that only data is collected during cutting, and the cutting effect is calculated and displayed after a single cutting is completed, thereby reducing the simulation operation amount, but the interaction delay between the surgical instrument and the model result is large, and it is difficult to meet the real-time visual feedback demand. In order to meet the demand of real-time cutting effect in virtual surgery simulation, the progressive cutting is usually selected. The progressive cutting includes two ways: one is to process the unit data and display the incision after completing a unit cutting, so as to realize the minimization of calculation, but the lag may be caused due to the unit edge length; the other is to subdivide the incision unit in real time and display the effect, although the delay is eliminated, but more time is spent, and is suitable for the dissection method cutting.

[0006] For the soft tissue cutting operation, the traditional body model cutting method can present a more realistic soft tissue incision, but a large number of body elements are also generated on the cutting path, thereby increasing the calculation demand and affecting the real-time performance of the cutting simulation process. The vertex splitting method in the face model cutting has fast calculation speed, but the simulation effect is poor, and the cutting wound drawing is relatively rough. Therefore, designing a cutting algorithm with high efficiency and good simulation effect for the virtual surgery system is a problem to be solved urgently at present. SUMMARY

[0007] Based on the problems existing in the prior art, the present application provides a soft tissue cutting method in a virtual surgery simulation system, and a new soft tissue cutting algorithm applied to the interactive scene of a soft tissue model and a surgical instrument in virtual surgery.

[0008] The present application is an optimized progressive cutting algorithm in virtual surgery, which calculates the coincident intersection of the cutting trajectory and the triangular facet, and reasonably splits the same, so as to accelerate the calculation efficiency and form the cutting wound. At the same time, in order to further optimize the real effect of the incision, the drawing of the surface incision is optimized by using the Bezier curve. It is suitable for various interactive scenes of surgical instruments and various soft tissue models.

[0009] The present application is realized by the following technical solutions.

[0010] The soft tissue cutting method in the virtual surgery simulation system according to the present application comprises the following steps:

[0011] Step 1, constructing a surgical instrument and a soft tissue model. First, a soft tissue model and a surgical instrument model are constructed by means of a three-dimensional object modeling software. Both the two models are based on basic triangular meshes, and the sizes of the meshes are different, but they jointly constitute the core skeleton of the model.

[0012] Step 2, determining the starting point of the contact between the surgical instrument and the soft tissue surface, judging whether the external force applied by the surgical instrument to the soft tissue reaches a critical value; if yes, the cutting is converted from deformation, and the next step is executed, otherwise, the deformation is continued.

[0013] As attached Figure 1 As shown in (a), when the scalpel tip contacts the first triangular patch on the soft tissue surface, that patch becomes the starting mesh unit, and the first force point on it is set as the starting point. As the external force applied by the scalpel on the soft tissue gradually increases, once a preset critical value is reached, the model will transition from deformation to cutting.

[0014] Step 3: During the cutting process, the cutting points are split into vertices, as shown in the attached figure. Figure 1 As shown, all cutting points generated during the cutting process and the point sets of all vertices of the reconstructed triangular mesh units are recorded.

[0015] As attached Figure 1 As shown in (b) and (c), cut points are set at the vertices and edge midpoints of the triangular mesh unit, rather than at the triangle center. This avoids computational issues caused by selecting points at the triangle center. If a triangular mesh unit encounters a cut and has two cut points, reconstruction is required. During reconstruction, the second cut point is split into two symmetrical points, which are then connected to the first split point to form symmetrical cut edges.

[0016] Search all vertices of the reconstructed triangular mesh unit to form a new point set P1; search all vertices of the reconstructed triangular mesh unit to form a new point set P2.

[0017] Step 4: Determine the end point of the path between the surgical instrument and the soft tissue surface.

[0018] As attached Figure 1 As shown in (d), when the tip of the surgical instrument completely leaves the soft tissue model, the triangular mesh unit that is in contact with it last becomes the termination mesh, and the force point on it serves as the termination point.

[0019] Step 5: Based on the starting point and the ending point of the cutting surface path and the point set recorded in step 3, a Bezier curve method is used to draw a smooth cutting wound.

[0020] As attached Figure 2 As shown, after the above steps, the progressive cutting path can be determined, and then the Bezier curve method is used to draw it to form a smooth cutting wound.

[0021] The mathematical expression of the Bezier curve is based on the vector position of n+1 control points

[0022] P i =(x i ,y i ,z i ), i = 0, 1, ..., n, these control points together generate the position vector P (t), which represents the distance from P0 to P nThe Bezier polynomial function path approximated between the two points is expressed by the following formula:

[0023]

[0024] The polynomial expansion of the Bezier curve is as follows:

[0025]

[0026] n is the polynomial degree, and the value of t in the function P(t) ranges from 0 to 1. When n = 2 is substituted into the above formula, the quadratic Bezier curve blending function set is obtained as follows:

[0027]

[0028] The quadratic Bezier equation is obtained as follows:

[0029]

[0030] Step 6, calculate the force feedback generated during the cutting process and feed back to the operator.

[0031] According to the biomechanical characteristics of the soft tissue model, when the surgical instrument cuts the soft tissue, the soft tissue first deforms, and when the cutting threshold is reached, the cutting operation occurs. At this time, the force exerted by the surgical instrument on the soft tissue includes the contact force of the knife tip on the model and the cutting force of the knife blade penetrating into the model. And with the progress of the cutting operation, the contact force will increase with the advancement of the surgical knife.

[0032] The present application provides a new soft tissue cutting algorithm, which is a surface mesh-based soft tissue gradual cutting method. The improved vertex splitting method is used to determine the cutting point, and the cutting point is connected and the triangular mesh topology is reconstructed. And use the Bezier curve to redraw the incision, further optimize the smoothness of the incision, and provide a more realistic effect for the virtual surgery cutting operation. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 It is a soft tissue surface mesh cutting diagram. Among them, (a) is the initial cutting, (b) and (c) are the cutting path, and (d) is the complete cutting path.

[0034] Figure 2 It is a schematic diagram of surface mesh incision.

[0035] Figure 3 It is a force feedback diagram during the cutting process. Among them, (a) is the surface contact force, and (b) is the internal contact force.

[0036] Figure 4 It is a comparison diagram of virtual incision and real incision. Among them, (a) is a virtual incision, and (b) is a real incision.

[0037] Figure 5 Flowchart of the present application. DETAILED DESCRIPTION

[0038] The present application will be further described by the following examples, which enable a person skilled in the art to carry out the application as described in the specification.

[0039] The soft tissue cutting method in virtual surgery described in this embodiment comprises the following steps:

[0040] Step 1: The virtual surgery simulation system is composed of a liver soft tissue model and a surgical instrument model. The liver soft tissue model is composed of 3127 triangular basic units, with a length, height and width of 32 cm, 28 cm and 31 cm respectively. First, the required model is built using modeling software, and the model is composed of basic triangular elements. It is assumed that ΔABC, ΔBCD, ΔCDE and ΔDEF are triangular mesh elements on the surface of the liver soft tissue model.

[0041] Step 2: As shown in FIG. 2, when the tip of the scalpel contacts the first triangular surface of the soft tissue, the surface is used as the starting mesh element, and the first force point on it is set as the starting point. As the external force applied by the scalpel to the soft tissue gradually increases, once the preset critical value is reached, the model will change from deformation to cutting. Figure 1 (a) shows the first cutting point M1, which is the midpoint of BC. According to the radius R of the tip of the scalpel, M1 is divided into N1 and N2, and AN1 and AN2 are connected, where M1N1 = M2N2 = R. At this time, the triangular mesh element ΔABC is reconstructed into two triangular mesh elements ΔABN1 and ΔACN2.

[0042] The triangular mesh element ΔABC is used as the starting mesh, and the vertex A is used as the starting point. M1, the midpoint of BC, is used as the second cutting point. According to the radius R of the tip of the scalpel, M1 is divided into N1 and N2, and AN1 and AN2 are connected, where M1N1 = M2N2 = R. At this time, the triangular mesh element ΔABC is reconstructed into two triangular mesh elements ΔABN1 and ΔACN2.

[0043] Step 3: In the cutting process, the intersection of the path drawn by the surgical instrument on the triangular mesh is the path point.

[0044] As shown in FIG. 3, the second cutting point M2 is the midpoint of CD. According to the radius R of the tip of the scalpel, N is divided into N3 and N4, and N1N3 and N2N4 are connected, where M2N3 = M2N4 = R. The triangular mesh element ΔBCD is divided into four new triangular mesh elements, namely triangular mesh element ΔCN2N4, triangular mesh element ΔBGN1, triangular mesh element ΔN1N3G and triangular mesh element ΔDGN3, where G is the midpoint of BD. Figure 1 (b) shows the second cutting point M2, which is the midpoint of CD. According to the radius R of the tip of the scalpel, N is divided into N3 and N4, and N1N3 and N2N4 are connected, where M2N3 = M2N4 = R. The triangular mesh element ΔBCD is divided into four new triangular mesh elements, namely triangular mesh element ΔCN2N4, triangular mesh element ΔBGN1, triangular mesh element ΔN1N3G and triangular mesh element ΔDGN3, where G is the midpoint of BD.

[0045] M3 cutting continues, as shown in FIG. 4. Figure 1(c) As shown, a cutting point M3 appears, which is the midpoint of DE edge. According to the radius R of the scalpel tip, M3 is split into N5 and N6. N3N5 and N4N6 are connected, where M3N5 = M3N6 = R. The triangle mesh cell ACDE is divided into four new triangle mesh cells, namely triangle mesh cell ΔDN3N5, triangle mesh cell ΔCHN4, triangle mesh cell ΔN4N6, and triangle mesh cell ΔEHN6, where H is the midpoint of CE.

[0046] Search all vertices of the reconstructed triangle mesh cell to form a new point set P1:

[0047] P1 = (A, N1, N2, N3, N4, N5, N6, F)

[0048] Search all vertices of the reconstructed triangle mesh cell to form a new point set P2:

[0049] P2 = (B, C, D, E, G, H)

[0050] Step 4: When the tip of the surgical instrument completely leaves the soft tissue model, the triangle mesh cell that last contacts the soft tissue model becomes the termination mesh, and the force point thereon is the termination point.

[0051] When the cutting operation is terminated, the termination point F is formed. This point does not need to be split and is directly connected to the symmetric point of the first cutting point split in the termination mesh to form a cutting opening. As shown in Fig. Figure 1 (d), point F is the vertex, i.e., the termination point, which is directly connected to FN5 and FN6. Then, the triangle mesh cell ΔDEF is reconstructed into two new triangle mesh cells, ΔDFN5 and ΔEFN6.

[0052] Step 5: As shown in Fig. Figure 2 , the progressive cutting path can be determined through the above steps, and then the Bezier curve method is used to draw a smooth cutting wound.

[0053] The mathematical expression formula of the Bezier curve is based on the vector positions of n+1 control points

[0054] P i = (x i , y i , z i ), i = 0, 1, …, n, which together generate a position vector P(t) representing the Bezier polynomial function path approximated from P0 to P n , and the expression formula is as follows:

[0055]

[0056] The polynomial expansion of the Bezier curve is:

[0057]

[0058] n is the polynomial degree, and the value of t in the function P(t) ranges from 0 to 1. When n = 2 is brought into the above formula, the quadratic Bezier curve blending function set can be obtained as:

[0059]

[0060] The quadratic Bezier equation can be obtained as:

[0061]

[0062] Connect the starting point A and the ending point F to form a straight line l. According to the relative position of all vertices in the P2 point set to the straight line, mark the vertices as positive or negative.

[0063] Traverse all vertices in the P2 point set to find the coordinates of the control points G and H.

[0064] Based on the vertices A, F, G and the vertices A, F, H, use the quadratic Bezier method to draw two curves respectively to form the smooth incision required for the cutting operation.

[0065] Step 6: According to the biomechanical properties of the soft tissue model, when the surgical instrument cuts the soft tissue, the soft tissue first deforms and reaches the cutting threshold, then the cutting operation occurs. As shown in FIG. 6, at this time, the force exerted by the surgical instrument on the soft tissue includes the contact force of the knife tip on the model and the cutting force of the knife blade penetrating into the model. And with the progress of the cutting operation, the contact force will increase with the advancement of the surgical knife. Figure 3

[0066] The horizontal and vertical decomposition forces of the cutting force are calculated as follows:

[0067] F1 = F c sinθ

[0068] F2 = F c cosθ

[0069] When the cutting force acts on the model, a reverse friction force will occur. When the cutting force is less than the soft tissue surface threshold, it is subject to static friction, and when the cutting force is greater than the reverse friction force, it is subject to sliding friction. The cutting force feedback calculation formula is:

[0070]

[0071] where F f is the soft tissue surface threshold; μ m is the static friction coefficient; μ s ​is the dynamic friction coefficient; d is the cutting depth.

[0072] Table 1 is the data change of different liver models before and after cutting according to the present application. The different liver models are constructed in the same way, and the vertex number and the triangular facet number are different.

[0073] Table 1

[0074]

Claims

1. A soft tissue cutting method in a virtual surgery simulation system, characterized by The following steps are involved: Step 1: Using 3D object modeling software, a soft tissue model and a surgical instrument model were constructed. Both models are based on basic triangle meshes, which together form the core skeleton of the model. Step 2: Determine the starting point of contact between the surgical instrument and the soft tissue surface, and judge whether the external force applied by the surgical instrument to the soft tissue reaches a critical value; if deformation turns into cutting, proceed to the next step; otherwise, continue deformation; When the scalpel tip contacts the first triangular patch on the soft tissue surface, that patch becomes the starting mesh unit, and the first force point on it is set as the starting point. As the external force applied by the scalpel on the soft tissue gradually increases, once the preset critical value is reached, the model will transition from deformation to cutting. Step 3: During the cutting process, the cutting points are split into vertices, and all the cutting points generated during the cutting process and the point sets of all vertices of the reconstructed triangular mesh units are recorded; Set cutting points at the vertices and midpoints of the mesh edges of the triangular mesh unit. If a triangular mesh unit encounters a cut and has two cutting points, it needs to be reconstructed. During reconstruction, the second cutting point is split into two symmetrical points, which are then connected to the first split point to form symmetrical cutting edges. Search all vertices of the reconstructed triangular mesh unit to form a new point set P1; search all vertices of the reconstructed triangular mesh unit to form a new point set P2; Step 4: When the surgical instrument tip completely leaves the soft tissue model, the triangular mesh unit that it last contacts becomes the termination mesh, and the force point on it becomes the termination point; Step 5, drawing a smooth cut wound using the Bezier curve method based on the starting point and the ending point of the cutting surface path and the point set recorded in step 3; The mathematical expression of the Bezier curve is based on the vector position of n+1 control points P i =(x i ,y i ,z i ), i = 0, 1, ..., n, these control points together generate the position vector P (t), which represents the distance from P0 to P n The Bessel polynomial function path approximated between is expressed as follows: The polynomial expansion in the Bezier curve is: n is the degree of the polynomial, and the value of t in the function P(t) ranges from 0 to 1. Substituting n=2 into the above formula, we can get the set of quadratic Bezier curve mixing functions: The quadratic Bessel equation can be obtained as: Step 6: Calculate the force feedback generated during the cutting process and feed it back to the operator.

Images