Soft tissue cutting method in virtual operation simulation system

By adopting a progressive cutting algorithm based on surface mesh and Bezier curve optimization technology in virtual surgical systems, the shortcomings of soft tissue cutting algorithm in the existing technology in complex path processing and real-time performance are solved, and efficient and real The cutting simulation effect improves the simulation quality of virtual surgery.

CN118975853AActive Publication Date: 2024-11-19NANCHANG UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411059503.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-04
Publication Date
2024-11-19
Estimated Expiration
2044-08-04

AI Technical Summary

Technical Problem

In the existing virtual surgical system, soft tissue cutting algorithms are difficult to achieve high precision and real-time performance when processing complex cutting paths and keeping the cutting surface smooth, resulting in poor simulation results and large computing requirements, which affects the real-time cutting simulation process. sex.

Method used

A progressive cutting algorithm based on surface mesh is used to calculate the overlapping intersection points of the cutting trajectory and the triangular face sheet for reasonable separation, and optimize the incision drawing using the Bezier curve to form a smooth cutting wound, which is suitable for a variety of surgeries. Interaction scenarios between instruments and soft tissue models.

Benefits of technology

It improves computing efficiency and the real effect of cutting wounds, reduces computing needs, enhances real-time and visual effects of cutting simulation, and provides a more realistic operating experience for virtual surgery.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118975853A_ABST
    Figure CN118975853A_ABST
Patent Text Reader

Abstract

The invention discloses a soft tissue cutting method in a virtual operation simulation system. The method comprises the following steps: 1) constructing a surgical instrument and soft tissue model; 2) determining a starting point of contact between the surgical instrument and the soft tissue surface, judging whether external force applied to the soft tissue by the surgical instrument reaches a critical value or not, if so, executing next cutting, otherwise, continuing deformation; (3) cutting point vertexes are split, and cutting points generated in cutting and all vertex point sets of the reconstructed triangular grid units are recorded; 4) determining an end point of a path between the surgical instrument and the soft tissue surface; (5) according to the starting point and the ending point of the cutting surface path and the point set recorded in the step (3), drawing by a Bezier curve method to form a smooth cutting wound; and (6) force feedback generated in the cutting process is calculated and fed back to an operator. According to the method, the cutting points are determined by using the improved vertex splitting method, the cutting points are connected, the triangular mesh topology is reconstructed, and the incision is redrawn by using the Bezier curve, so that the smoothness of the incision is optimized, and the effect is more vivid.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of virtual reality medical technology, and in particular relates to a cutting problem between a soft tissue model and a surgical instrument in a virtual surgery simulation system. Background Art

[0002] In virtual surgery systems, common cutting algorithms include surface mesh-based cutting algorithms and polyhedron-based cutting methods.

[0003] Common surface mesh-based cutting algorithms include unit removal and vertex splitting, which are commonly used to simulate soft tissue cutting effects. Unit removal, as a cutting algorithm proposed earlier by Bro-Nielsen et al. (Bro-Nielsen M. Finite element modeling in surgery simulation [J]. Proceedings of the IEEE, 1998, 86 (3): 489-503.), removes primitives where soft tissue contacts surgical instruments to produce incisions to simulate cutting effects. This method is relatively simple to implement, but it may not be sophisticated enough when dealing with complex cutting paths or when the cutting surface needs to be kept smooth. The vertex splitting rule is more sophisticated. Based on the cutting path where the surgical instrument interacts with the soft tissue, the cutting point of the model is split into two new vertices that are symmetrical about the cutting path, and then these new vertices are connected with new connecting lines to simulate a smoother and more realistic cutting effect. In order to prevent the generation of low-quality reconstructed triangular facets during cutting, the vertex splitting algorithm often replaces the intersection with a new vertex to maintain the reasonable topological structure of the model after cutting. These two algorithms have their own advantages and disadvantages and are suitable for different application scenarios. The unit removal method is more suitable for situations where the cutting accuracy is not high or the computing resources are limited, while the vertex splitting method is more suitable for application scenarios that require high-precision cutting simulation and good visual effects.

[0004] Polyhedron-based cutting algorithms often use tetrahedrons or hexahedrons. According to different element selections, researchers have proposed a variety of algorithms. Different algorithms differ in dealing with the contact between surgical instruments and soft tissues, but they all pursue higher simulation efficiency and better visual effects. Bielser et al. (Bielser D, Glardon P, Teschner M, et al. A state machine for real-time cutting of tetrahedral meshes [C]. Graphical Models, 2004: 398-417.) proposed a method that subdivides tetrahedral elements according to the contact position between surgical instruments and soft tissues, and then links them, thereby effectively improving simulation efficiency. This method takes into account the differences in contact positions and can simulate the cutting process more finely. Molino et al. (Molino N, Ba OZ, Fedkiw RA virtual node algorithm for changing mesh topology during simulation [J]. Acm Transactions on Graphics, 2005, 23 (3): 385-392.) proposed a virtual node algorithm to solve the problem of ill-conditioned voxels. By copying the cutting unit and assigning the remaining material to the copy unit, the simulation cutting can be stabilized. Pietron et al. (Pietron N, Ganovelli F, Cignoni P, et al. Splitting cubes: a fast and robust technique for virtual cutting [J]. Visual Computer, 2009, 25 (3): 227-239.) proposed a fast cube splitting technology based on hexahedron, which embeds objects into a regular grid to simulate cutting and handle topological changes. The concept is powerful. Frisken et al. (Frisken-GIBSON S. Using linked volumes to model object collisions, deformation, cutting, carving, and joining [J]. IEEE Transactions on Visualization and Computer Graphics, 1999, 5 (4): 333-348.) proposed a linked volume representation method, which decomposes the object into hexahedral elements, connects them through links, and disconnects them when cutting to represent the interaction state, and uses the structural characteristics of the hexahedron to achieve stable and realistic simulation.

[0005] Soft tissue cutting methods can also be classified according to the timing of cutting processing, which can be divided into non-progressive and progressive. Non-progressive cutting means that only data is collected during cutting. After a single cutting is completed, the cutting effect is calculated and displayed, which reduces the amount of simulation calculations, but the interaction delay between the surgical instrument and the model results is large, and it is difficult to meet the real-time visual feedback requirements. In order to meet the needs of real-time cutting effects in virtual surgical simulation, progressive cutting is usually selected. Progressive cutting includes two methods: one is to process the unit data and display the incision after completing a unit cutting, so as to minimize the calculation, but there may be a lag due to the length of the unit edge; the second is to subdivide the incision unit in real time and display the effect. Although the delay is eliminated, it takes more time and is suitable for cutting by dissection.

[0006] For soft tissue cutting operations, the traditional volume model cutting method can present a more realistic soft tissue incision, but the cutting path will also increase the computing demand due to the generation of a large number of voxels, thus affecting the real-time performance of the cutting simulation process. The vertex splitting method in surface model cutting has a fast calculation speed, but the simulation effect is poor, and the cut wound drawing is relatively rough. Therefore, designing an efficient cutting algorithm with good simulation effect for the virtual surgery system is an urgent problem to be solved today. Summary of the invention

[0007] Based on the problems existing in the prior art, the present invention proposes a soft tissue cutting method in a virtual surgery simulation system, a new soft tissue cutting algorithm applied to the interaction scene between soft tissue models and surgical instruments in virtual surgery.

[0008] The present invention is an optimized progressive cutting algorithm in virtual surgery. By calculating the overlapping intersections of the cutting trajectory and the triangular facets and splitting them reasonably, the calculation efficiency is accelerated to form the cutting wound. At the same time, in order to further optimize the real effect of the incision, the Bezier curve is used to optimize the drawing of the surface incision. It is suitable for the interaction scenes of various surgical instruments and various soft tissue models.

[0009] The present invention is achieved through the following technical solutions.

[0010] A soft tissue cutting method in a virtual surgery simulation system according to the present invention comprises the following steps:

[0011] Step 1: Construct surgical instrument and soft tissue models. First, the soft tissue model and surgical instrument model were constructed with the help of 3D object modeling software. Both models are based on basic triangular meshes, which vary in size but together form the core skeleton of the model.

[0012] Step 2, determine the starting point of contact between the surgical instrument and the soft tissue surface, and judge whether the external force applied by the surgical instrument to the soft tissue reaches a critical value; if the deformation turns into cutting, execute the next step, otherwise continue the deformation.

[0013] As attached Figure 1 As shown in (a), when the scalpel tip contacts the first triangular patch on the soft tissue surface, the patch is used as the starting mesh unit, and the first force point on it is set as the starting point. As the external force applied by the scalpel to the soft tissue gradually increases, once the preset critical value is reached, the model will change from deformation to cutting.

[0014] Step 3: During the cutting process, the cutting points are split into vertices, as shown in the following figure. Figure 1 As shown, all cutting points generated during the cutting process and the point sets of all vertices of the reconstructed triangular mesh units are recorded.

[0015] As attached Figure 1 As shown in (b) and (c), the cutting points are set at the vertices of the triangular mesh unit and the midpoints of the mesh edge instead of the triangle center, to avoid the calculation problems caused by selecting the point at the triangle center. When a triangular mesh unit encounters a cut, if there are two cutting points, it needs to be reconstructed. During reconstruction, the second cutting point is split into two symmetrical points, and then connected to the first split point to form a symmetrical cutting edge.

[0016] All vertices of the reconstructed triangular mesh unit are searched to form a new point set P1; all vertices of the reconstructed triangular mesh unit are searched to form a new point set P2.

[0017] Step 4, determine the end point of the path between the surgical instrument and the soft tissue surface.

[0018] As attached Figure 1 As shown in (d), when the tip of the surgical instrument completely leaves the soft tissue model, the triangular mesh unit that is in contact with it for the last time becomes the termination mesh, and the force point on it serves as the termination point.

[0019] Step 5, according to the starting point and the end point of the cutting surface path and the point set recorded in step 3, a Bezier curve method is used to draw to form a smooth cutting wound.

[0020] As attached Figure 2 As shown, after the above steps, the progressive cutting path can be determined, and then the Bezier curve method is used to draw it to form a smooth cutting wound.

[0021] The mathematical expression of the Bezier curve is based on the vector positions of n+1 control points.

[0022] P i =(x i ,y i ,z i ), i = 0, 1, ..., n, these control points together generate the position vector P (t), which represents the distance from P0 to P nThe Bessel polynomial function path approximated between is expressed as follows:

[0023]

[0024] The polynomial expansion in the Bezier curve is:

[0025]

[0026] n is the degree of the polynomial, and the value of t in the function P(t) ranges from 0 to 1. Substituting n=2 into the above formula, we can get the set of quadratic Bezier curve hybrid functions:

[0027]

[0028] The quadratic Bessel equation can be obtained as:

[0029]

[0030] Step 6, calculate the force feedback generated during the cutting process and feed it back to the operator.

[0031] According to the biomechanical characteristics of soft tissue models, when surgical instruments cut soft tissue, the soft tissue first deforms, and when the cutting threshold is reached, the cutting operation occurs. At this time, the force applied by the surgical instrument to the soft tissue includes the contact force of the knife tip on the model and the cutting force of the blade penetrating into the model. And as the cutting operation proceeds, the contact force will increase with the advancement of the scalpel.

[0032] The present invention provides a new soft tissue cutting algorithm, which is a progressive soft tissue cutting method based on surface mesh. The improved vertex splitting method is used to determine the cutting points, connect the cutting points and reconstruct the triangular mesh topology. The incision is redrawn using Bezier curves, which further optimizes the smoothness of the incision and provides a more realistic effect for virtual surgical cutting operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Soft tissue surface mesh cutting diagram, where (a) is the starting cutting, (b) and (c) are the cutting paths, and (d) is the complete cutting path.

[0034] Figure 2 Draw a schematic diagram for the surface mesh cutout.

[0035] Figure 3 Schematic diagram of force feedback during cutting process, where (a) is the surface contact force and (b) is the internal contact force.

[0036] Figure 4 Comparison diagram of virtual incision and real incision, where (a) is the virtual incision and (b) is the real incision.

[0037] Figure 5 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described by the following examples so that those skilled in the art can implement the invention with reference to the description.

[0039] The soft tissue cutting method in a virtual surgery described in this embodiment includes the following steps:

[0040] Step 1: The virtual surgery simulation system consists of a liver soft tissue model and a surgical instrument model. The liver soft tissue model consists of 3127 triangular basic units, with a length, height and width of 32cm, 28cm and 31cm respectively. First, the required model is built using modeling software. The model is composed of primitive triangles. It is assumed that ΔABC, ΔBCD, ΔCDE and ΔDEF are triangular mesh units on the surface of the liver soft tissue model.

[0041] Step 2: As attached Figure 1 As shown in (a), when the scalpel tip contacts the first triangular patch on the soft tissue surface, the patch is used as the starting mesh unit, and the first force point on it is set as the starting point. As the external force applied by the scalpel to the soft tissue gradually increases, once the preset critical value is reached, the model will change from deformation to cutting.

[0042] The triangular mesh unit ΔABC is used as the starting mesh, vertex A is used as the starting point, and the midpoint M1 of BC is used as the second cutting point. According to the radius R of the scalpel end point, the M1 point is split into N1 and N2, and AN1 and AN2 are connected, where M1N1 = M2N2 = R. At this time, the triangular mesh unit ΔABC is reconstructed and divided into two triangular mesh units ΔABN1 and ΔACN2.

[0043] Step 3: During the cutting process, the intersection of the path drawn by the surgical instrument on the triangular mesh is the path point.

[0044] As attached Figure 1 As shown in (b), the second cutting point M2 is the midpoint of the CD edge. Similarly, according to the radius R of the scalpel end point, point N is split into N3 and N4, and N1N3 and N2N4 are connected, where M2N3 = M2N4 = R. The triangular mesh unit ΔBCD is divided into four new triangular mesh units, namely, triangular mesh unit ΔCN2N4, triangular mesh unit ΔBGN1, triangular mesh unit ΔN1N3G and triangular mesh unit ΔDGN3, where G is the midpoint of BD.

[0045] M3 cutting continues, as shown in the attached Figure 1As shown in (c), a cutting point M3 appears, which is the midpoint of the DE edge. Similarly, according to the radius R of the scalpel endpoint, the M3 point is split into N5 and N6, and N3N5 and N4N6 are connected, where M3N5 = M3N6 = R. The triangular mesh unit ΔCDE is divided into four new triangular mesh units, namely, the triangular mesh unit ΔDN3N5, the triangular mesh unit ΔCHN4, the triangular mesh unit ΔN4N6H and the triangular mesh unit ΔEHN6, where H is the midpoint of CE.

[0046] Search all vertices of the reconstructed triangular mesh unit to form a new point set P1:

[0047] P1=(A,N1,N2,N3,N4,N5,N6,F)

[0048] Search all vertices of the reconstructed triangular mesh unit to form a new point set P2:

[0049] P2=(B,C,D,E,G,H)

[0050] Step 4: When the top of the surgical instrument completely leaves the soft tissue model, the triangular mesh unit that is in contact with it for the last time becomes the termination mesh, and the force point on it serves as the termination point.

[0051] When the cutting operation is terminated, the termination point F is formed. This point does not need to be split, and is directly connected to the symmetrical point of the first cutting point in the termination unit grid to form a cutting opening. Figure 1 As shown in (d), point F is the vertex, i.e., the termination point, which directly connects FN5 and FN6, and then the triangular mesh unit ΔDEF is reconstructed into two new triangular mesh units ΔDFN5 and ΔEFN6.

[0052] Step 5: As attached Figure 2 As shown, after the above steps, the progressive cutting path can be determined, and then the Bezier curve method is used to draw it to form a smooth cutting wound.

[0053] The mathematical expression of the Bezier curve is based on the vector positions of n+1 control points.

[0054] P i =(x i ,y i ,z i ), i = 0, 1, ..., n, these control points together generate the position vector P (t), which represents the distance from P0 to P n The Bessel polynomial function path approximated between is expressed as follows:

[0055]

[0056] The polynomial expansion in the Bezier curve is:

[0057]

[0058] n is the degree of the polynomial, and the value of t in the function P(t) ranges from 0 to 1. Substituting n=2 into the above formula, we can get the set of quadratic Bezier curve hybrid functions:

[0059]

[0060] The quadratic Bessel equation can be obtained as:

[0061]

[0062] Connect the starting point A and the ending point F to form a straight line l. Mark the vertices in the P2 point set as positive or negative according to their relative positions to the straight line.

[0063] Traverse all vertices in the P2 point set and find the position coordinates G and H of the control points.

[0064] Based on vertices A, F, G and vertices A, F, H, two curves are drawn respectively using the quadratic Bezier method to form a smooth cut required for the cutting operation.

[0065] Step 6: According to the biomechanical characteristics of the soft tissue model, when the surgical instrument cuts the soft tissue, the soft tissue first deforms, and the cutting operation occurs after reaching the cutting threshold. Figure 3 As shown in the figure, the force applied by the surgical instrument to the soft tissue at this time includes the contact force of the knife tip on the model and the cutting force of the blade penetrating into the model. And as the cutting operation proceeds, the contact force will increase with the advancement of the scalpel.

[0066] The horizontal and vertical decomposition of the cutting force is calculated as follows:

[0067] F1=F c sinθ

[0068] F2=F c ·cosθ

[0069] When the cutting force acts on the model, reverse friction will occur. When the cutting force is less than the threshold of the soft tissue surface, it is subject to static friction. When the cutting force is greater than the reverse friction, it is subject to sliding friction. The cutting force feedback calculation formula is:

[0070]

[0071] Among them, F f is the soft tissue surface threshold; μ m is the static friction coefficient; μ sis the coefficient of kinetic friction; d is the cutting depth.

[0072] Table 1 shows the data changes of different liver models before and after cutting. Different liver models are constructed in the same way, but with different numbers of vertices and triangles.

[0073] Table 1

[0074]

Claims

1. A soft tissue cutting method in a virtual surgery simulation system, Its characteristics are The following steps are involved: Step 1: Using 3D object modeling software, a soft tissue model and a surgical instrument model were constructed. Both models were based on basic triangle meshes, which together formed the core skeleton of the model. Step 2, determine the starting point of contact between the surgical instrument and the soft tissue surface, and judge whether the external force applied by the surgical instrument to the soft tissue reaches a critical value; if the deformation turns into cutting, execute the next step, otherwise continue to deform; When the tip of the scalpel contacts the first triangular patch on the surface of the soft tissue, the patch serves as the starting mesh unit, and the first force point on it is set as the starting point; as the external force applied by the scalpel to the soft tissue gradually increases, once the preset critical value is reached, the model will change from deformation to cutting. Step 3: During the cutting process, the cutting points are split into vertices, and all the cutting points generated during the cutting process and the point sets of all vertices of the reconstructed triangular mesh units are recorded; Set cutting points at the vertices and midpoints of the mesh edges of the triangular mesh unit. If a triangular mesh unit encounters a cut and has two cutting points, it needs to be reconstructed. During reconstruction, the second cutting point is split into two symmetrical points, and then connected to the first split point to form a symmetrical cutting edge. Search all vertices of the reconstructed triangular mesh unit to form a new point set P 1 ; Search all vertices of the reconstructed triangular mesh unit to form a new point set P 2 ; Step 4: When the surgical instrument tip completely leaves the soft tissue model, the triangular mesh unit that is in contact with it last becomes the termination mesh, and the force point on it serves as the termination point; Step 5, drawing a smooth cutting wound by using the Bezier curve method according to the starting point and the ending point of the cutting surface path and the point set recorded in step 3; The mathematical expression of the Bezier curve is based on the vector position P of n+1 control points. i =(x i ,y i ,z i ), i = 0, 1, ..., n, these control points together generate a position vector P (t), which represents the position from P 0 To P n The Bessel polynomial function path approximated between is expressed as follows: The polynomial expansion in the Bezier curve is: n is the degree of the polynomial, and the value of t in the function P(t) ranges from 0 to 1. Substituting n=2 into the above formula, we can get the set of quadratic Bezier curve hybrid functions: The quadratic Bessel equation can be obtained as: Step 6, calculate the force feedback generated during the cutting process and feed it back to the operator.

Citation Information

Patent Citations

  • Method for virtually simulating incision based on Bezier curve

    CN102207994A

  • Operation cutting training system and method based on force feedback and used for surgical robot

    CN105559887A

  • Methods and systems for surface mesh cutting and notching boundary optimization processing

    CN109754469A

  • Mesh optimization based virtual cutting algorithm of deformable object

    CN110880180A

  • Method for Generating Quartic Bezier Triangle Patches from Triangle Mesh for Computer Graphics

    US20150339849A1