Soft tissue suturing simulation methods, apparatuses, and media

Abstract

The application provides a soft tissue suturing simulation method, which comprises the following steps: determining a target pose offset of a suture in a current frame, and obtaining a stage mark, a model of the suture comprises a plurality of particle points arranged in a chain and a rod node connected between adjacent particle points; determining a temporary constraint function of the particle points according to at least the stage mark, the stage mark is used to represent a stage in which the current frame is located in a soft tissue suturing process, the stage comprises a puncture stage, a suturing stage and a tensioning stage, wherein if the stage mark is the suturing stage, the temporary constraint function of a target particle point located in the soft tissue comprises a path point constraint based on an associated path point in a path point sequence corresponding to the target particle point, the path point sequence is obtained by using a particle point interval of the suture to interpolate a movement path of a first control point in the soft tissue, and the first control point is a particle point directly connected to a needle tail of a suture needle; using model parameters of the suture in the current frame calculated based on position dynamics according to the target pose offset, wherein the constraint function of the particle points comprises the temporary constraint function; and displaying the suture according to the model parameters of the suture in the current frame. The application also provides a soft tissue suturing simulation device and a storage medium.

Description

Technical Field

[0001] The present application relates to the technical field of computer graphics, and in particular to a soft tissue suturing simulation method, device, and computer-readable storage medium. Background Art

[0002] Suturing is an essential step in surgical simulation. The suturing process involves multiple aspects, including simulation of suture threads, soft tissue, and surgical instruments like needles, forceps, and tweezers. It also includes interactions between these entities, such as the forceps holding the needle to pierce the surface of soft tissue and enter the interior, the needle driving the suture thread through the soft tissue, the suture thread tightening the soft tissue together, and finally tying the suture thread to secure the soft tissue.

[0003] Traditional methods often use the Follow the Leader (FTL) method to simulate sutures, the Mass-Spring Model (MSM) to simulate soft tissue, and the particle model to simulate surgical instruments. Because these three methods use different models, some model overlap is inevitable when handling their interactions. Furthermore, this approach is not convenient for scalable simulation of a wider range of organ and tissue forms, resulting in the entire surgical simulation consisting of multiple different frameworks, increasing the complexity of the simulation. Summary of the Invention

[0004] To solve the existing technical problems, the present application provides a soft tissue suturing simulation method, device and computer-readable storage medium that can simulate the soft tissue suturing process in real time and accurately.

[0005] In a first aspect of an embodiment of the present application, a soft tissue suturing simulation method is provided, comprising: determining a target pose offset of a suture line in a current frame, and obtaining a stage mark, wherein the model of the suture line includes a plurality of particle points arranged in a chain and rod nodes connected between adjacent particle points; determining a temporary constraint function of the particle point at least according to the stage mark, wherein the stage mark is used to indicate the stage in which the current frame is located in the soft tissue suturing process, wherein the stage includes a puncture stage, a suturing stage, and a tensioning stage, wherein if the stage mark is a suturing stage, the temporary constraint function of the target particle point located inside the soft tissue includes a path point constraint based on an associated path point corresponding to the target particle point in a path point sequence, wherein the path point sequence is obtained by interpolating a movement path of a first control point in the soft tissue using the particle point interval of the suture line, wherein the first control point is a particle point directly connected to the needle tail of the suture needle; obtaining model parameters of the suture line in the current frame using position dynamics-based calculation according to the target pose offset, wherein the constraint function of the particle point includes a temporary constraint function; and displaying the suture line according to the model parameters of the suture line in the current frame.

[0006] In a second aspect of an embodiment of the present application, a soft tissue suturing simulation device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, a soft tissue suturing simulation method as in any embodiment of the present application is implemented.

[0007] According to a third aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, a soft tissue suturing simulation method as in any embodiment of the present application is implemented.

[0008] In the above embodiment, the soft tissue suturing simulation method uses the particle point interval of the suture line to interpolate the movement path of the first control point in the soft tissue during the suturing stage to obtain a path point sequence, and then adds path point constraints to the target particle points based on the correspondence between the path point sequence and the target particle points in the suture line located inside the soft tissue, so that the part of the suture line located inside the soft tissue will not produce drastic length changes under the action of the path point constraint, thereby reducing the distortion that may be caused by this.

[0009] In the above embodiments, the soft tissue suturing simulation device and the computer-readable storage medium respectively belong to the same concept as the corresponding soft tissue suturing simulation method embodiments, and thus have the same technical effects as the corresponding soft tissue suturing simulation method embodiments, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 Schematic diagram of the model of the suture line with the introduction of elastic rods;

[0011] Figure 2 Flowchart for introducing position-based dynamics of elastic rods;

[0012] Figure 3 Schematic diagram of the stages of the soft tissue suturing process;

[0013] Figure 4 This is a flow chart of the first embodiment of the soft tissue suturing simulation method of the present application;

[0014] Figure 5 This is a schematic diagram of a process for determining whether a needle has penetrated the surface of soft tissue in one embodiment of the present application;

[0015] Figure 6 for Figure 4 Specific process diagram of S3;

[0016] Figure 7 A schematic diagram of a process for adding path point constraints to a target particle point in one embodiment of the present application;

[0017] Figure 8 for Figure 7Specific process diagram of S44;

[0018] Figure 9 This is a schematic diagram of a process for constraining the collision response between a suture line and soft tissue in one embodiment of the present application;

[0019] Figure 10 for Figure 9 Specific process diagram of S49;

[0020] Figure 11 This is a flow chart of a second embodiment of the soft tissue suturing simulation method of the present application;

[0021] Figure 12 A schematic diagram of a real-time process for obtaining a path point sequence in an embodiment of the present application;

[0022] Figure 13 for Figure 11 Specific process diagram of S102;

[0023] Figure 14 This is a flow chart of a third embodiment of the soft tissue suturing simulation method of the present application;

[0024] Figure 15 When the suture needle is treated as a rigid body Figure 14 Specific process diagram of S201;

[0025] Figure 16 for Figure 14 Specific process diagram of S203;

[0026] Figure 17 This is a flowchart of determining the sub-stages of the puncture stage in one embodiment of the present application;

[0027] Figure 18 This is a flow chart of a fourth embodiment of the soft tissue suturing simulation method of the present application;

[0028] Figure 19 A flowchart of adding a second collision response constraint function in an embodiment of the present application;

[0029] Figure 20 This is a structural diagram of the first embodiment of the soft tissue suturing simulation device of the present application;

[0030] Figure 21 This is a structural diagram of the second embodiment of the soft tissue suturing simulation device of the present application;

[0031] Figure 22 This is a schematic structural diagram of the third embodiment of the soft tissue suturing simulation device of the present application;

[0032] Figure 23This is a schematic structural diagram of a fourth embodiment of the soft tissue suturing simulation device of the present application;

[0033] Figure 24 This is a structural schematic diagram of an embodiment of the soft tissue suturing simulation device of the present application. DETAILED DESCRIPTION

[0034] The technical solution of this application is further elaborated in detail below with reference to the accompanying drawings and specific embodiments.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are intended only to describe specific embodiments and are not intended to limit the implementation of this application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0036] The various embodiments of the present application can be combined with each other without conflict. In the following description, when referring to "some embodiments", it describes a subset of possible embodiments, but it should be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0037] The order of steps given in the embodiments of the present application is for illustration only. If the steps can be performed in other orders to achieve the same effect, these orders are also within the scope of protection of the present application.

[0038] In computer graphics, common physical methods for simulating object deformation in real time typically first update acceleration using external forces, then integrate to determine velocity and position. PBD, however, directly corrects position using constraint functions, then uses the position to update velocity. The system equations in the PBD algorithm are constructed from a set of constraint function equations, which consists of multiple constraint function equations (referred to as simply constraint functions).

[0039] In PBD, the model of an object includes multiple particle points that make up the object, also known as mass points. The parameters of the particle points include their position, velocity and mass. The parameters of the constraint function include the position of these particle points. The constraint function is used to describe the relationship between the particle points that make up the object.

[0040] The PBD algorithm primarily consists of an initialization phase and a real-time simulation phase. In the initialization phase, the parameters of all the particle points and rod nodes that make up the object are initialized based on preset object parameters, such as initial position, initial posture, and number of particle points. The real-time simulation phase then begins, looping through the frame-by-frame process to simulate the object's deformation based on environmental influences (including any user commands). This process involves three steps: prediction, constraint modification, and variable update.

[0041] In the prediction portion of the current frame, the forces acting on the object from the external environment in the current frame are converted to displacements as much as possible. External forces that cannot be directly converted to displacements (such as gravity) are retained. The positions of each particle point in the previous frame are used as a basis, combining the converted displacements and external forces that cannot be directly converted to displacements to predict the positions of each particle point in the current frame.

[0042] The aforementioned prediction process often uses only integrals (such as explicit Euler integrals) to directly predict the positions of each particle point in the current frame, without considering the object's mechanical properties or its interaction with the environment. Therefore, this prediction cannot be directly used as the simulation result for the current frame. Instead, the particle point positions must be corrected using constraint functions. Constraint functions are used to constrain the relationships between particle points. They are mainly divided into two types: temporary constraint functions, which are typically generated when an interaction between an object and external objects in the environment is detected; and regular constraint functions, which are determined by the object's mechanical properties. The process of calculating a corrected position that closely satisfies the constraint functions based on the constraint functions and the current position is called constrained projection and is the core process of PBD. Constrained correction essentially involves iteratively optimizing the particle point positions based on a set of constraint function equations. This process typically involves iteratively executing constraint projections using the constraint functions in the set of constraint function equations until a set number of iterations is reached, resulting in a particle point position that closely satisfies all the constraint functions in the set of constraint function equations.

[0043] After the constraint correction is completed, the result is the position of each particle point in the current frame. Based on this, the velocity of each particle point in the current frame can be calculated, and the corresponding variables are updated for display and calculation of the next frame. The Δt used in PBD is generally the default frame interval, that is, the duration between two adjacent frames under normal circumstances.

[0044] For soft tissue, the model is a mesh model consisting of multiple vertices and tetrahedrons defined by the vertices. The vertices here are similar to particle points, and the constraint functions mainly include the distance constraints between adjacent vertices and the volume invariance constraints of each tetrahedron.

[0045] Surgical instruments, such as forceps, needle holders, tweezers, and suture needles, can be approximated as rigid bodies. Their models typically consist of multiple mass points / particles determined by their specific shapes. For example, the model of a suture needle might consist of multiple mass points arranged in a chain. Since surgical instruments other than suture needles are generally under direct user control, the PBD process can be omitted, and kinematic equations can be used directly to calculate their posture and the positions of each mass point in the model.

[0046] For suture lines, in order to more accurately describe their shape, rod nodes are further introduced, also known as elastic rods (rods). Figure 1 As shown, the suture model includes a plurality of particle points arranged in a chain (e.g. Figure 1 p1, p2, ..., p in n ) and the rod nodes connecting adjacent particle points (e.g. Figure 1 q1,q2,…,q in n-1 The extension directions in the figure are used only to specify the numbering order of the particle points and rod nodes and do not represent the actual extension directions. Rod node parameters include their attitude, angular velocity, and moment of inertia. Accordingly, the parameters of the constraint function also include the attitude of these rod nodes. For ease of calculation, the attitude is generally represented using unit quaternions. Of course, in other embodiments, the attitude can also be represented using transformation / rotation matrices.

[0047] In practical applications, sutures are often connected to an external object that controls their movement, also known as a control volume. For example, if the suture is a suture, the control volume is a needle, surgical forceps, or tweezers that directly drives the suture's movement. If the suture is hair, the control volume is the skin or hair accessories that hold the hair in place, or objects like fingers or combs that directly drive the hair's movement. The particle point in the suture that is directly connected to the control volume—that is, the particle point closest to the control volume—can be called a control point.

[0048] The process of introducing PBD with elastic rod is as follows Figure 2 As shown, the PBD process is described in detail below with reference to the accompanying drawings.

[0049] Figure 2 Steps 1-4 in the figure are the initial stage, in which the parameters of all the particle points and rod nodes that make up the object are initialized according to the preset object parameters, such as the initial position, initial posture, number of particle points, etc. Then the real-time simulation stage begins, in which the process of simulating the deformation of the object according to the environment (including possible user operation instructions) is executed in a loop in frames, that is, it is executed once for each frame. Figure 2 Steps 5-21 in the above process may specifically include three parts: prediction, constraint modification, and variable update.

[0050] In the prediction part of the current frame (corresponding to Figure 2 In steps 6-12), the external environmental forces acting on the object in the current frame are converted to displacements as much as possible, and external forces that cannot be directly converted to displacements (such as gravity) are retained. The external environmental torques acting on the object in the current frame are converted to angular displacements as much as possible, and external torques that cannot be directly converted to angular displacements are retained. The positions of the particle points in the previous frame are used as a basis, combined with the converted displacements and the external forces that cannot be directly converted, to predict the positions of the particle points in the current frame. The postures of the rod nodes in the previous frame are used as a basis, combined with the converted angular displacements and the external torques that cannot be directly converted, to predict the postures of the rod nodes in the current frame.

[0051] The aforementioned prediction process often uses only integrals (e.g., explicit Euler integrals) to directly predict the position of each particle point and the posture of each rod node in the current frame. This method fails to consider the object's mechanical properties and its interaction with the environment. Therefore, it cannot be directly used as the simulation result for the current frame. Instead, constraint functions are used to modify the positions of the particle points and the posture of the rod nodes. Constraint functions are used to constrain the relationships between particle points, rod nodes, and the relationships between them. There are two main types: temporary constraint functions, which are typically generated upon detecting an interaction between the object and external objects in the environment; and regular constraint functions, which are determined by the object's mechanical properties. For example, for sutures and similar linear objects, a corresponding regular constraint function can be derived based on their mechanical properties of being resistant to stretching and shearing but easily bending and twisting. The process of calculating a corrected position / posture that best satisfies the constraint function based on the constraint function and the current position / posture is called constraint projection and is the core process of PBD. Constraint correction is essentially an iterative optimization of the particle point position and the rod node posture based on the constraint function equation group. The specific implementation is generally an iterative execution of the constraint projection process using the constraint functions in the constraint function equation group in turn until the set number of iterations is reached, in order to find the particle point position and rod node posture that satisfies all the constraint functions in the constraint function equation group as much as possible. The constraint correction part corresponds to Figure 2 In steps 13-15, step 13 is used to perform collision detection to generate a temporary constraint function, and steps 14-15 are to iteratively optimize the position of the particle point and the posture of the rod node using the constraint function equation group.

[0052] Update some corresponding Figure 2 After completing the constraint correction in steps 16-21, the resulting values ​​are the positions of each particle point and the poses of each rod node in the current frame. Based on these values, the velocities of each particle point and the angular velocities of each rod node in the current frame can be calculated. The corresponding variables are then updated for display and calculations for the next frame. The Δt used in PBD is generally the default frame interval, which is the normal duration between two consecutive frames.

[0053] After executing the simulated object deformation process of the current frame, the model parameters of the current frame can be obtained, specifically including the parameters of the updated particle points and rod nodes. The updated positions of each particle point and the posture of each rod node can be used to determine the shape of the stitching line of the current frame for display.

[0054] It should be noted that the vertices, particle points and mass points in this application are actually equivalent in the PBD framework. In order to simplify the description, different names are selected for the particle points in different models, that is, the particle points in soft tissue are called vertices, the particle points in surgical instruments are called mass points, and the particle points in sutures are called particle points.

[0055] Combined with the above preliminary introduction to the soft tissue suturing simulation method, the following will combine Figures 3 to 19 The soft tissue suturing simulation method provided in the embodiment of the present application is further described in detail. Since object simulation is a real-time process, when the current frame is mentioned below, it refers to the specific process of object simulation within a frame using the current frame as an example. In fact, this process must be performed for each frame.

[0056] See also Figure 3 ,The soft tissue suturing process mainly includes the following stages: puncture stage, suturing stage, tightening stage and knotting stage.

[0057] From the perspective of user operation, during the puncture stage, the user uses a first clamping body (generally surgical forceps, needle holder, etc.) to clamp the middle and rear part of the suture needle and insert it into the soft tissue. After the needle tip of the suture thread penetrates the soft tissue, it passes through the soft tissue until the needle tip of the suture thread pierces the soft tissue. The user then moves the first clamping body to the front of the suture needle, and then continues to pull the suture needle until the needle tail of the suture needle (that is, the particle in the suture needle that is directly connected to the suture thread) enters the soft tissue.

[0058] The suturing phase generally begins when the tail of the suture needle drives the suture into the soft tissue and ends when the tension mark appears. There are two main tension marks: one is when there is a pre-tied knot at the tail of the suture, in which case the tension mark is when the pre-tied knot contacts the surface of the soft tissue; the other is when there is no pre-tied knot, in which case the tension mark is when the user directly grasps the tail of the suture with a second clamp (a surgical instrument different from the first clamp, such as forceps).

[0059] The tensioning stage generally starts from the appearance of a tensioning mark and ends when the suture thread collides with itself under the drive of the first clamping body and the second clamping body.

[0060] The knotting stage generally starts when the suture line collides with itself under the drive of the first clamping body and the second clamping body, and the specific end time can be determined according to needs, for example, by cutting the suture line with surgical scissors.

[0061] For ease of description, this application only takes one puncture as an example. In actual applications, multiple punctures may occur. In this case, each piece of soft tissue has its own puncture stage and suturing stage, which is not limited here.

[0062] See also Figure 4 The first embodiment of the soft tissue suturing simulation method provided in this application includes the following steps.

[0063] S1: Obtain models of soft tissue, sutures, and surgical instruments.

[0064] The soft tissue model is a mesh model consisting of multiple vertices and tetrahedrons defined by the vertices. The suture model consists of multiple particle points arranged in a chain and rod nodes connecting adjacent particle points. The surgical instrument model consists of multiple mass points arranged according to their shape, and the surgical instrument includes a suture needle. For more information about the models, please refer to the previous description.

[0065] S2: performing collision detection on the soft tissue, the suture, and the surgical instrument, and determining the stage of the suturing process at least based on the result of the collision detection.

[0066] The stages include puncture, suturing, and tensioning, and may also include knotting. The previous stages are described from the perspective of user operation. In the actual simulation of soft tissue suturing, the stage of the current frame in the suturing process can be determined based on the collision detection results between the soft tissue, suture thread, and surgical instruments.

[0067] Collision detection between soft tissue, sutures, and surgical instruments, in addition to collision detection between soft tissue and sutures, collision detection between soft tissue and surgical instruments, and collision detection between sutures and surgical instruments, can also include self-collision detection of sutures and / or collision detection between different surgical instruments.

[0068] In addition to the collision detection results, the stage of the current frame can also be determined by combining the historical records of the stage, the relative position relationship between the surgical instrument and the soft tissue, etc. The following example illustrates this.

[0069] The puncture phase mainly focuses on the collision detection between the suture needle and the soft tissue. Based on whether the surface of the soft tissue is pierced and the direction of the collision between the suture needle and the soft tissue surface, the puncture phase can be further divided into multiple sub-phases, as follows:

[0070] a. If the needle tip of the suture needle collides with the surface of the soft tissue from the outside to the inside but does not pierce the surface of the soft tissue, the suturing stage is the piercing of the soft tissue in the puncture stage.

[0071] In this sub-stage, since the suture needle is generally clamped by the first clamping body, it is possible to perform a collision response on the soft tissue instead of the suture needle.

[0072] b. If the suture needle has pierced the surface of the soft tissue and the needle head is located inside the soft tissue, the suturing stage is the puncture stage during which the soft tissue has been pierced.

[0073] c. If the needle collides with the surface of the soft tissue from the inside to the outside but does not pierce the surface of the soft tissue, the suturing stage is piercing the soft tissue during the puncture stage.

[0074] In this sub-stage, since the suture needle is generally clamped by the first clamping body, it is possible to perform a collision response on the soft tissue instead of the suture needle.

[0075] d. If the suture needle has pierced the surface of the soft tissue and the needle tip is located outside the soft tissue, the suturing stage is the stage where the soft tissue has been pierced during the puncture stage.

[0076] The needle tip of a suture needle can be a small area at the tip of the needle. The direction in which the suture needle collides with the surface of the soft tissue, namely, "from the outside of the soft tissue to the inside" and "from the inside of the soft tissue to the outside" mentioned above, can be implemented in a variety of ways. One is to consult the historical records of stage changes. If the puncture stage of the soft tissue did not appear in the previous stage, the direction is from the outside to the inside, otherwise the direction is from the inside to the outside; another is to refer to the relative position relationship between the suture needle and the soft tissue. If the other parts of the suture needle except the part that collides with the soft tissue are all located outside the soft tissue, the direction is from the outside to the inside. If the middle part of the suture needle is located inside the soft tissue, the direction is from the inside to the outside; another is to refer to the relative relationship between the direction of the suture needle's speed and the soft tissue. If the speed of the suture needle points to the inside of the soft tissue, the direction is from the outside to the inside. If the speed of the suture needle points to the outside of the soft tissue, the direction is from the inside to the outside.

[0077] like Figure 5 As shown, in one embodiment of the present application, determining whether the needle has pierced the surface of the soft tissue may specifically include:

[0078] S21: performing collision detection between the needle head of the suture needle and the surface of the soft tissue.

[0079] S22: If the number of frames in which the needle is detected to have continuously collided with the surface of the soft tissue is greater than a first threshold, the collision response displacement of the soft tissue is accumulated to obtain a cumulative displacement.

[0080] S23: If the accumulated displacement is greater than the second threshold, it is determined that the suture needle has pierced the surface of the soft tissue.

[0081] Since the suture needle is generally clamped by the first clamping body during the process of piercing the soft tissue surface, if the suture needle is regarded as a rigid body, the collision response may not be performed on the suture needle, but only on the soft tissue.

[0082] Collision response refers to the handling of colliding objects. This handling is generally designed to be as close to human understanding of physical laws as possible to avoid obvious abnormal interactions between colliding objects, such as mold penetration. When the needle of a suture needle punctures the surface of soft tissue, it generates a puncture force on the soft tissue and a resistance force from the soft tissue to the needle. Due to the elasticity of soft tissue, the soft tissue is not punctured immediately. Instead, it deforms at the puncture location along the direction of the needle's movement. As the puncture force continues to increase and exceeds the maximum resistance of the soft tissue, the soft tissue is punctured.

[0083] To simplify the puncture simulation process within the PBD framework, this embodiment forgoes the simulation of puncture force and soft tissue resistance. Instead, the cumulative displacement of the soft tissue's collision response is used to reflect the puncture force, and a second threshold is used to reflect the soft tissue's maximum resistance. This force comparison is converted into a comparison between the cumulative displacement and the second threshold, making it more adaptable to the PBD framework. The second threshold can be set based on the characteristics of different organs and tissues. For the same piece of soft tissue, the second threshold used for puncturing the soft tissue surface from the inside out and from the outside in can be the same or different.

[0084] To prevent the needle tip's collision with the soft tissue surface from being mistakenly interpreted as a puncture, this embodiment requires that the needle tip only confirms the start of puncture when continuous collisions with the soft tissue surface are detected, and the number of consecutive collision frames exceeds a first threshold. The determination of whether the needle has penetrated the soft tissue surface is then based on a comparison of the cumulative displacement with a second threshold. After the needle begins puncturing the soft tissue surface, if the cumulative displacement in the current frame is less than the second threshold, it is determined that the soft tissue surface has not been penetrated, and the cumulative displacement is retained for use in the next frame. To prevent jitter on the soft tissue surface from affecting the puncture determination, the collision response displacement is temporarily stored and accumulated in vector format. In other words, the cumulative displacement in the current frame is essentially the displacement between the position of the soft tissue surface in the current frame due to the collision response between the soft tissue and the needle tip and the position before the collision with the needle tip.

[0085] The start of the suturing stage is marked by the detection of the tail of the suture needle entering the soft tissue, or in other words, the tail of the needle and the soft tissue just colliding. The detection of the tensioning mark means that the suturing stage has entered the tensioning stage. If the tensioning mark is that the pre-tied knot contacts the surface of the soft tissue, the corresponding collision detection result is that the pre-tied knot has just been detected to have collided with the surface of the soft tissue. If the tensioning mark is that the user uses the second clamping body to directly clamp the tail of the suture thread, the corresponding collision detection result is that the tail of the suture thread, that is, the part located outside the soft tissue and on the side where the suture needle penetrates the soft tissue, has just been detected to have collided with the second clamping body.

[0086] The detection of the tightening mark means the start of the tightening phase, while the presence of at least two control points in the suture and the detection of a self-collision means the end of the tightening phase.

[0087] In order to simulate the process of soft tissue suturing under the framework of PBD, at least one stage mark can be designed to distinguish different stages, and the stage mark can be updated as the stage changes. During the real-time simulation of different types of objects, the stage mark can be read to perform corresponding operations, such as adding corresponding temporary constraints (collision response constraints, path point constraints, etc.), updating path points, etc. Since the stages that need to be paid attention to in the simulation of different types of objects are not necessarily the same, you can choose to set different stage marks for different types of objects, or you can set a unified stage mark for all objects. The latter is used as an example in this application.

[0088] S3: During the suturing stage, the particle point interval of the suture line is used to interpolate the movement path of the needle tail of the suture needle in the soft tissue to obtain a path point sequence.

[0089] As the needle tail moves within the soft tissue, its position in each frame, also known as a movement point, is arranged in chronological order to form a movement path. During the suturing phase, the suture generally moves along this movement path as it moves through the soft tissue. Therefore, the movement path constraint on the suture can be used as a collision response constraint between the suture and the soft tissue.

[0090] Since the distribution of moving points in the moving path is affected by the user's operation and is not necessarily uniform, if temporary constraints are added to the particle points located inside the soft tissue in the suture line, that is, the target particle points, directly according to the moving path, the spacing between adjacent target particle points after the update will be affected by the distribution of moving points and will no longer be uniform. There may be a large error with the initially set particle point interval, that is, the spacing between adjacent particle points in the suture line, which will cause a significant change in the length of the suture line.

[0091] In order to avoid obvious length changes of the suture line during its movement in the soft tissue and reduce the resulting distortion, in this embodiment, the particle point interval of the suture line is used to interpolate the movement path to obtain a path point sequence, and then the path point sequence is used to add temporary constraints to the target particle point.

[0092] like Figure 6 As shown, in one embodiment of the present application, S3 specifically includes:

[0093] S31: In the stitching stage, the position of the needle tail in the current frame is obtained.

[0094] For the suture needle, if it is treated as a rigid body, then when the suture needle is clamped by other surgical instruments, the PBD process can be omitted, and the kinematic equation can be directly used in combination with the posture of the surgical instrument holding the suture needle to calculate its posture and the position of each particle in the model; when the suture needle is not clamped by other surgical instruments, in order to prevent penetration of the mold, the PBD / extended position-based dynamics (XPBD) process can be used to calculate the position of each particle in the model. Specifically, the overall collision constraint of the suture needle can be determined based on the results of the collision detection between the suture needle and the soft tissue, and the position of each particle in the suture needle model can be obtained based on the overall collision constraint using position-based dynamics.

[0095] If the elasticity of the suture needle is taken into account, it cannot be treated as a rigid body. Regardless of whether it is clamped by other surgical instruments, the PBD / XPBD process must be used to calculate the position of each mass point in the model.

[0096] According to the above-mentioned method of calculating the position of each particle in the suture needle, the position of each particle in the suture needle in the current frame can be obtained. Combined with the position of the needle tail in the suture needle, the position of the needle tail in the current frame can be obtained.

[0097] S32: If the position of the needle tail in the current frame is within the soft tissue, determine whether a first distance between the position of the needle tail in the current frame and the current path point in the path point sequence is within a deviation interval determined based on the particle point interval.

[0098] The current pathpoint is the last pathpoint added to the pathpoint sequence, i.e., the newest pathpoint in the pathpoint sequence. Generally, the first pathpoint added to the pathpoint sequence is the location where the needle tip and soft tissue collision were first detected. The first distance is a scalar.

[0099] The deviation interval can be written in the form of [k1*rd, k2*rd], where rd is the particle point interval, 0 < k1 ≤ 1 ≤ k2. Generally, the differences between k1, k2 and 1 are all relatively small values, such as 0.9 and 1.1. The smaller the differences between k1, k2 and 1, the less likely the length of the suture line will change drastically during the suture stage, but the more likely it is to bring deviations between the path point sequence and the moving path. The values of k1 and k2 can be determined according to the setting of the elasticity of the suture line in stretching when establishing the suture line model. For example, if the suture line is set to be completely inextensible, then both k1 and k2 are 1, and the greater the elasticity in stretching, the greater k1 and k2 are.

[0100] If the first distance is within the deviation interval, jump to S33; if the first distance is greater than the upper limit of the deviation interval, jump to S34; if the first distance is less than the lower limit of the deviation interval, jump to S35.

[0101] S33: Take the position of the needle tail in the current frame as a new path point and add it to the path point sequence.

[0102] S34: Interpolate between the current path point and the position of the needle tail in the current frame using the particle point interval, and take the interpolation result as a new path point and add it to the path point sequence.

[0103] The interpolation result is on the line connecting the current path point and the position of the needle tail in the current frame, and the distance between it and the current path point is k*rd. The value range of k does not exceed [k1, k2]. The values of k in different frames can be the same, such as all being 1, or different, such as randomly selecting a value within the value range.

[0104] S35: Do not update the path point sequence.

[0105] If the position of the needle tail in the current frame is not within the soft tissue, it means that the needle tail has left the soft tissue, the moving path of the needle tail within the soft tissue has ended, and the path point sequence no longer needs to be updated.

[0106] S4: Use the position-based dynamics to calculate the model parameters of the soft tissue and the suture line frame by frame, and use the position-based dynamics and / or kinematic equations to calculate the model parameters of the surgical instrument frame by frame.

[0107] During the suture stage, the main temporary constraint considered is the path point constraint of the target particle points. The target particle points are the particle points of the suture line located inside the soft tissue. Each target particle point has an associated path point in the path point sequence. The path point constraint of each target particle point refers to the constraint between this target particle point and its associated path point.

[0108] During the suture stage, in the part of generating temporary constraints in PBD, path point constraints can be added for the target particle points, such as Figure 7As shown, in one embodiment of the present application, adding a path point constraint to a target particle point may specifically include:

[0109] S41: In the suturing stage, it is determined whether the needle tail in the current frame is located inside the soft tissue.

[0110] If the needle tail is located inside the soft tissue, jump to S42; if the needle tail is located outside the soft tissue, jump to S43.

[0111] S42: taking a particle point directly connected to the needle tail in the suture line as a reference point, and determining a position change of the reference point according to a position change of the needle tail.

[0112] The position delta is the difference between the position in the current frame and the position in the previous frame.

[0113] The particle point in the suture that is directly connected to the needle tail is the control point directly connected to the needle tail, referred to as the first control point in this application. The position change of the reference point can be directly calculated based on the position change of the needle tail. For example, if the needle tail and the first control point are set to overlap in space, the position change of the needle tail can be directly used as the position change of the reference point.

[0114] Jump to S44.

[0115] S43: The particle point closest to the soft tissue in the suture line is used as the reference point, and the position change of the reference point is calculated based on position dynamics.

[0116] The particle point closest to the soft tissue is the particle point located outside the soft tissue, on the side where the suture needle exits the soft tissue, and closest to the soft tissue surface. When the needle tail just leaves the soft tissue, the reference point remains the first control point. In other cases, the reference point is not the first control point, and its position is not directly affected by the suture needle like the first control point. Therefore, PBD is required to calculate the reference point's position change.

[0117] S44: Determine the target particle point in the current frame and the corresponding relationship between the target particle point and the path point in the path point sequence according to the position change of the reference point and the particle point interval.

[0118] According to the position change of the reference point and the particle point interval, it can be determined whether a new particle point has entered the soft tissue and become a target particle point and / or a new target particle point has entered the soft tissue and become a non-target particle point.

[0119] The position of the first control point in the suture line is directly determined by the position of the needle tail and does not participate in the correction and update of the PBD. Even if the first control point is located inside the soft tissue, it cannot be processed as a target particle point.

[0120] When the needle tail is located inside the soft tissue, the update of the path point sequence and the target particle point can be synchronized with the path points in the path point sequence, that is, a new path point and a target particle point are added in the same frame.

[0121] like Figure 8 As shown, in one embodiment of the present application, S44 may specifically include:

[0122] S441: Calculate the required position change of the reference point in the current frame according to the position change of the reference point in the current frame and the remaining reference point positions in the previous frame.

[0123] The position change of the reference point in the current frame and the position of the remaining reference points in the previous frame are both vectors. Generally, the vector sum of the two is calculated as the required position change.

[0124] S442: Determine whether the required position change of the reference point is greater than the particle point interval.

[0125] If so, it means that a new particle point has entered the soft tissue, and the target particle point and its correspondence with the path point sequence need to be updated, and the process goes to S443; otherwise, it means that no new particle point has entered the soft tissue, and the target particle point and its correspondence with the path point sequence do not need to be updated, and the process goes to S444.

[0126] S443: Determine the number of particle points where the suture line moves into the soft tissue and the remaining reference point positions in the current frame based on the ratio of the required position change of the reference point to the particle point interval. Update the target particle points in the previous frame and the corresponding relationship between them and the path points in the path point sequence based on the number of particle points where the suture line moves into the soft tissue to obtain the target particle points in the current frame and the corresponding relationship between them and the path points in the path point sequence.

[0127] Generally, the comparison value is rounded down to obtain the number of particle points n of the suture line moving into the soft tissue in the current frame. The remaining reference point position quantity has the same direction as the required position change quantity, and its size is the size of the reference point position quantity minus n*rd.

[0128] The first n particle points in the previous frame that have not yet entered the soft tissue and are closest to the soft tissue surface become the new target particle points. If the reference point in the previous frame is not the first control point, the reference point in the previous frame is moved n particle points away from the first control point and used as the new reference point for subsequent use. At the same time, the oldest n target particle points in the previous frame, i.e., the first n target particle points that entered the soft tissue, are no longer target particle points. In other words, the oldest n target particle points are removed from the target particle points, and the total number of target particle points remains unchanged when combined with the previously added n target particle points. If the reference point in the previous frame is the first control point, the number m of particle points other than the first control point that have left the soft tissue in the current frame is counted based on the required position change of the reference point. If m = 0, the reference point does not need to be updated and the target particle points do not need to be removed. If m > 0, the reference point in the previous frame is moved m particle points away from the first control point and used as the new reference point for subsequent use, and the oldest m target particle points are removed from the target particle points.

[0129] After updating the target particle points, if there are no new path points added to the path point sequence in the current frame, generally when the needle tail has left the soft tissue, the sequence number of the target particle point corresponding to each path point in the path point sequence in the previous frame is moved by n in the direction away from the first control point, which is the sequence number of the target particle point corresponding to each path point in the path point sequence in the current frame.

[0130] For example, assuming that the target particle points M, M+1,…M+k-1 in the previous frame correspond to path points k, k-1,…1 respectively, and the reference point M-1 is not the first control point. After calculation, it is found that N particle points move in and out of the soft tissue in the current frame. Then the reference point in the current frame is updated to M+N-1, and the target particle points M+N, M+N+1,…M+N+k-1 correspond to path points k, k-1,…1 respectively.

[0131] After updating the target particle points, if new path points are added to the path point sequence in the current frame, generally when the needle tail is still inside the soft tissue, the sequence number of the target particle point corresponding to each path point in the path point sequence in the previous frame is moved by n in the direction away from the first control point, and the target particle points that no longer have a corresponding relationship with the path points in the path point sequence of the previous frame after the movement are arranged in the direction pointing to the first control point, and the newly added path points in the current frame are arranged in the order of being added to the path point sequence, and the corresponding relationship between the two is established in sequence, thereby obtaining the sequence number of the target particle point corresponding to each path point in the path point sequence in the current frame.

[0132] For example, assuming that the target particle points M, M+1,…M+k-1 in the previous frame correspond to path points k, k-1,…1 respectively, and the reference point M-1 is the first control point. After calculation, N particle points have moved into the soft tissue and N new path points have been added in the current frame, and the reference point M-1 is still the first control point, then the target particle points M, M+1,…M+N+k-1 correspond to path points k+N, k+N-1,…1 respectively.

[0133] S444: The required position change of the reference point is used as the remaining reference point position, and the target particle point in the previous frame and the corresponding relationship between it and the path point in the path point sequence are used as the target particle point in the current frame and the corresponding relationship between it and the path point in the path point sequence.

[0134] S45: Add path point constraints to the target particle points based on the corresponding relationship.

[0135] Based on the corresponding relationship, the associated path point of each target particle point can be determined. A path point constraint generally constrains the position of the target particle point to the position of the associated path point. A path point constraint can be expressed as an equation, where the distance between the target path point and the associated path point is 0; or as an inequality, where the distance between the target path point and the associated path point is less than a small fixed value; or by comparing the distance between the target path point and the associated path point with a fixed value. If the distance is less than the fixed value, the path point constraint is not added; otherwise, a constraint is added where the distance between the target path point and the associated path point is 0.

[0136] In practical applications, the first control point may not be the endpoint of the suture line. In this case, the two sides of the first control point can independently update their respective target particle points and their correspondence with the path point sequence. That is, the same path point may correspond to two different target particle points at the same time.

[0137] To ensure real-time simulation during the suturing phase and reduce potential instabilities, in some embodiments, the collision response constraints between the suture and the soft tissue can be omitted during the suturing phase. This means that the soft tissue will not be tightened by the suture during the suturing phase, and the collision response constraints between the suture and the soft tissue will be applied during the subsequent tightening phase. In other embodiments, the collision response constraints between the suture and the soft tissue can be applied during the suturing phase.

[0138] like Figure 9 As shown, in one embodiment of the present application, performing collision response constraints between the suture line and the soft tissue may specifically include:

[0139] S46: In the tensioning stage, after calculating the model parameters of the stitching line in the current frame, a second distance between the target particle point and the nearest path point is calculated.

[0140] The position of the target particle point is calculated with the addition of a path point constraint. The target particle point can be determined frame by frame based on the movement of the reference point, or by performing collision detection on the suture and soft tissue. In the latter case, particle points that collide only with the soft tissue surface need to be excluded to avoid mistakenly identifying particle points that fall on the soft tissue surface as target particles.

[0141] S47: If the second distance is greater than the distance threshold, a collision response constraint is added to the collision tetrahedron where the nearest path point is located.

[0142] The second distance is greater than the distance threshold, that is, the distance between the target particle point and the nearest path point is still too far when the path point constraint is added, which means that the soft tissue has undergone significant deformation under the pull of the suture line and a collision response is required.

[0143] The specific collision response constraint can be to move the collision tetrahedron as a whole in the direction from the nearest path point to the particle point by a distance threshold. Since the constraint function of the soft tissue includes a vertex-based constraint function, the movement of the collision tetrahedron is generally converted into the movement of each vertex of the collision tetrahedron. For example, assuming that the calculated displacement is Δp, its direction is from the nearest path point to the particle point, and its size is the distance threshold, the overall displacement of the collision tetrahedron can be to add a temporary constraint of p=p+Δp to each of the four vertices of the collision tetrahedron. The aforementioned constraint method actually also limits the collision tetrahedron to be as non-deformed as possible, which may cause the surface of the soft tissue to be uneven. To reduce this phenomenon, the temporary constraint can be modified to an inequality, such as |p-(p+Δp)|<0.1|Δp|, thereby expanding the movable range of each vertex of the collision tetrahedron.

[0144] S48: Calculate model parameters of the soft tissue in the current frame using position-based dynamics based on the collision response constraints.

[0145] S49: Update the coordinates of the path points according to the model parameters of the soft tissue.

[0146] The coordinates of a waypoint can be updated based on the barycentric coordinates, also known as the mass center coordinates or volume coordinates. The waypoints here can be all the waypoints in the waypoint sequence.

[0147] like Figure 10 As shown, in one embodiment of the present application, S49 specifically includes:

[0148] S491: Get the coordinates of the center of gravity of the path point in the tetrahedron before the collision response constraint is added.

[0149] Barycentric coordinates, also known as mass coordinates or volume coordinates, are the coordinates defined by the four vertices of a tetrahedron. The coordinates of any point in a tetrahedron can be written as a weighted average of the coordinates of the four vertices of the tetrahedron. These four weights are the barycentric coordinates of the point.

[0150] S492: Calculate the coordinates of the updated path points by combining the coordinates of the vertices and the center of gravity of the tetrahedron in the soft tissue model parameters.

[0151] The centroid coordinates of the path points are used as weights to calculate the weighted average of the coordinates of the vertices of the tetrahedron in the soft tissue model parameters, and the result obtained is the coordinate of the updated path points.

[0152] S5: Soft tissues, sutures, and surgical instruments are displayed frame by frame according to their model parameters.

[0153] In each frame, after completing the calculation of the model parameters of the soft tissue, sutures and surgical instruments, the morphology of the soft tissue, sutures and surgical instruments can be determined, and on this basis the model can be rendered and then output to a display device for display.

[0154] S6: Calculate the feedback force according to the feedback force strategy corresponding to the stage.

[0155] If there is no feedback force function, this step can be omitted.

[0156] This step only needs to be executed after S2, and there is no restriction on the execution order between S3-S5.

[0157] If the stage is before the puncture stage, the feedback force is 0.

[0158] If the stage is the puncture stage of piercing into soft tissue, the feedback force is the product of the accumulated displacement and the first proportional coefficient.

[0159] If the stage is the soft tissue punctured stage, the feedback force is the first default value.

[0160] If the stage is the puncture stage of puncturing the soft tissue, the feedback force is the sum of the product of the accumulated displacement and the second proportional coefficient and the first default value.

[0161] If the stage is the puncture stage where the soft tissue has been punctured, the feedback force is the second default value.

[0162] If the stage is the suturing stage, the feedback force is the second default value.

[0163] The feedback force calculation strategy for the tensioning phase is to calculate the feedback force based on the second default value and the displacement of at least a portion of the tetrahedron in the soft tissue. For example, the displacement of the collision tetrahedron can be used to calculate the displacement of the nearest path point before and after the update. The feedback force is then calculated as the sum of the product of this displacement, the third proportionality factor, and the second default value.

[0164] Through the implementation of this embodiment, a unified position dynamics-based framework is used to simulate soft tissue, sutures, and suture needles, which can effectively reduce the cross-modeling between different models. In addition, during the suturing stage, the particle point interval of the suture line is used to interpolate the movement path of the needle tail in the soft tissue to obtain a path point sequence. Based on the correspondence between the path point sequence and the target particle points in the suture line located inside the soft tissue, path point constraints are added to the target particle points. This ensures that the portion of the suture line located inside the soft tissue does not undergo drastic length changes under the action of the path point constraints, thereby reducing the distortion that may be caused.

[0165] This embodiment describes the simulation of the entire suturing process. The following describes the real-time simulation processes of the suture thread, suture needle, and soft tissue respectively with reference to the accompanying drawings.

[0166] like Figure 11 As shown, the second embodiment of the soft tissue suturing simulation method provided by the present application includes the following steps.

[0167] S101: Determine the target pose offset of the stitching line in the current frame and obtain a stage flag.

[0168] This embodiment is a real-time simulation process of each suture line during the soft tissue suturing process. Parts that are identical or similar to the above content will not be described in detail. The suture line model includes multiple particle points arranged in a chain and rod nodes connecting adjacent particle points.

[0169] The initial number of control points on the suture line and the specific locations of each control point can be determined based on the actual application scenario. For example, if the suture line is not pre-attached to the suture needle, the threading operation before suturing is referenced, that is, the suture line is first passed through the hole at the end of the suture needle and then clamped to the front of the needle holder. The simulated threaded suture line actually has three control points: one connecting to the suture needle and two connecting to the needle holder. If the suture needle and suture line are pre-attached, the point directly connected to the suture needle is the suture line control point.

[0170] The number and / or location of control points may change based on user interaction. For example, during the suturing process, the user inserts the needle into the tissue using the needle holder and then releases the holder. This leaves only one control point for the threaded suture, the one connected to the needle. During the knotting phase, the user uses forceps to grasp the suture remaining outside the tissue on the needle entry side, creating a new control point for the forceps.

[0171] The target pose offset is the pose offset of the connection part (hereinafter referred to as the connection part) where the control body is directly connected to the suture line, including the target position offset and the target pose offset. According to the operation instructions given by the user, the pose of each connection part in the current frame can be calculated, and then the difference between the pose in the current frame and the pose in the previous frame is calculated as the command pose offset. If the pose is expressed by a unit quaternion, the pose difference refers to the angle difference between the two quaternions. For example, if the pose of the previous frame is q n-1 , the pose of the current frame is q n , then the difference between the posture of the current frame and the previous frame is The superscript -1 indicates inversion. Without considering other factors, the command pose offset can be directly used as the target pose offset. Of course, the target pose offset can be obtained by processing the command pose offset, as described in the subsequent embodiments.

[0172] Each joint has a corresponding control point, i.e., the directly connected control point. The target pose offset for each joint is transferred from the control volume to the seam line, acting on the corresponding control point and the corresponding rod node. The target position offset is applied to the control point, while the target pose offset is applied to the corresponding rod node. For ease of description, the following example uses a single control point as an example. This analogy can be used to describe situations where the number of control points is greater than one.

[0173] The rod node corresponding to the control point is the rod node directly connected to the control point. If the control point is the endpoint of the suture line, there is only one rod node directly connected to the control point as the rod node corresponding to the control point. Figure 1 For example, if Figure 1 If the particle point p1 is the control point, the corresponding rod node is q1. If the control point is not the endpoint of the suture line, there are two rod nodes directly connected to the control point. The rod node corresponding to the control point can be one of the two rod nodes, or both rod nodes can be used as the rod nodes corresponding to the control point. Figure 1 For example, if Figure 1 The particle point p2 is the control point, and the corresponding rod node can be at least one of q1 and q2.

[0174] The position of the control point in the current frame and the posture of the rod node corresponding to the control point are determined by the target posture offset. Specifically, the sum of the position of the control point in the previous frame and the target position offset is calculated as the position of the control point in the current frame, and the product of the posture of the rod node corresponding to the control point in the previous frame and the target posture offset is calculated as the posture of the rod node corresponding to the control point in the current frame. During the entire constraint correction process, the position of the control point is fixed to the calculated position in the current frame, and the posture of the rod node corresponding to the control point is fixed to the calculated posture in the current frame. They will not change like other particle points and rod nodes. In other words, the position of the control point and the posture of the rod node corresponding to the control point will not be updated by the constraint function equation group.

[0175] S102: Determine a temporary constraint function of the particle point at least according to the stage flag.

[0176] The stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, which includes the puncture stage, suturing stage and tensioning stage.

[0177] If the stage mark is the suturing stage, the temporary constraint function of the target particle point located inside the soft tissue includes a path point constraint based on the associated path point corresponding to the target particle point in the path point sequence. The path point sequence is obtained by interpolating the movement path of the first control point in the soft tissue using the particle point interval of the suture line. The first control point is the particle point directly connected to the needle tail of the suture needle.

[0178] like Figure 12 As shown, in one embodiment of the present application, the real-time process of obtaining a path point sequence may specifically include:

[0179] S121: Obtain the position of the first control point in the current frame.

[0180] Throughout the suturing process, the suture thread is connected to the needle and moves with it. Therefore, the target pose offset includes the target position offset of the needle tail. The position offset of the first control point in the current frame is determined based on the target pose offset of the current frame. The position of the first control point in the current frame is then calculated as the sum of the position of the first control point in the previous frame and the position offset of the first control point in the current frame.

[0181] S122: If the position of the first control point in the current frame is within the soft tissue, determine whether a first distance between the position of the first control point in the current frame and the current path point in the path point sequence is within a deviation interval determined based on the particle point interval.

[0182] If the first distance is within the deviation interval, jump to S123; if the first distance is greater than the upper limit of the deviation interval, jump to S124; if the first distance is less than the lower limit of the deviation interval, jump to S125.

[0183] S123: Add the position of the first control point in the current frame as a new path point to the path point sequence.

[0184] S124: Using the particle point interval, interpolate between the current path point and the position of the first control point in the current frame, and add the interpolation result as a new path point to the path point sequence.

[0185] S125: Do not update the waypoint sequence.

[0186] like Figure 13 As shown, in one embodiment of the present application, S102 may specifically include:

[0187] S131: If the stage flag is the suturing stage, determine whether the first control point in the current frame is located inside the soft tissue.

[0188] If the first control point is located inside the soft tissue, the process jumps to S132 ; if the first control point is located outside the soft tissue, the process jumps to S133 .

[0189] S132: Taking the first control point as a reference point, and determining the position change of the reference point according to the position change of the first control point.

[0190] Jump to S134.

[0191] S133: The particle point closest to the soft tissue in the suture line is used as a reference point, and the position change of the reference point is calculated using position dynamics.

[0192] S134: Determine the target particle point in the current frame and the corresponding relationship between the target particle point and the path point in the path point sequence according to the position change of the reference point and the particle point interval.

[0193] The specific process can be found in Figure 8 and description of corresponding embodiments.

[0194] S135: Adding path point constraints to the target particle points based on the corresponding relationship.

[0195] S103: Perform collision detection on the suture thread and soft tissue, and surgical instruments other than the suture needle.

[0196] S104: Determine whether to update the stage flag at least based on the result of the collision detection.

[0197] There is no restriction on the execution order of this step and S105 - S106 .

[0198] If the stage flag is the puncture stage and it is detected that the first control point collides with the soft tissue, the stage flag is updated to the suturing stage.

[0199] If the stage flag is the suturing stage and it is detected that the preset knot in the suture line collides with the surface of the soft tissue, the stage flag is updated to the tightening stage.

[0200] If the stage mark is the suturing stage and it is detected that the suture thread collides with the second clamping body, the stage mark is updated to the tensioning stage, and the second clamping body is a surgical instrument different from the suture needle and the first clamping body that clamps the suture needle.

[0201] S105: Obtain model parameters of the stitching line in the current frame using position-based dynamics calculation according to the target pose offset.

[0202] The constraint functions of particle points include temporary constraint functions.

[0203] S106: Displaying the seam line according to the model parameters of the seam line in the current frame.

[0204] like Figure 14 As shown, the third embodiment of the soft tissue suturing simulation method provided by the present application includes the following steps.

[0205] S201: Calculate the model parameters of the suture needle in the current frame.

[0206] This embodiment is a real-time simulation process of each frame of a suture needle during the soft tissue suturing process. The suture needle model includes multiple mass points arranged according to its shape.

[0207] like Figure 15 As shown, in one embodiment of the present application, the suture needle is treated as a rigid body, and S201 may specifically include:

[0208] S211: Determine whether the suture needle is in a clamped state in the current frame.

[0209] If the suture needle is in the clamped state, jump to S212; otherwise jump to S213.

[0210] S212: Acquire the posture of the first clamping body that clamps the suture needle, and calculate the model parameters of the suture needle in the current frame using the kinematic equation according to the posture of the first clamping body.

[0211] S213: Determine the overall collision constraint of the suture needle according to the result of the collision detection between the suture needle and the soft tissue, and calculate the model parameters of the suture needle in the current frame based on the overall collision constraint using position-based dynamics.

[0212] If the elasticity of the suture needle is taken into account, it cannot be treated as a rigid body. Regardless of whether it is clamped by other surgical instruments, the PBD / XPBD process must be used to calculate the position of each mass point in the model.

[0213] S202: Acquire the stage flag.

[0214] The stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, which includes the puncture stage, suturing stage and tensioning stage.

[0215] S203: If the stage mark is the suturing stage, the moving path of the needle tail of the suturing needle in the soft tissue is interpolated using the particle point interval of the suture line to determine whether to update the path point sequence.

[0216] like Figure 16 As shown, in one embodiment of the present application, S203 may specifically include:

[0217] S231: If the stage mark is the suturing stage, the position of the needle tail of the suturing needle in the current frame is determined according to the model parameters of the suturing needle in the current frame.

[0218] S232: If the position of the needle tail in the current frame is within the soft tissue, determine whether a first distance between the position of the needle tail in the current frame and the current path point in the path point sequence is within a deviation interval determined based on the particle point interval.

[0219] If the first distance is within the deviation interval, jump to S233; if the first distance is greater than the upper limit of the deviation interval, jump to S234; if the first distance is less than the lower limit of the deviation interval, jump to S235.

[0220] S233: Add the position of the needle tail in the current frame as a new path point to the path point sequence.

[0221] S234: interpolating between the current path point and the position of the needle tail in the current frame using the particle point interval, and adding the interpolation result as a new path point to the path point sequence.

[0222] S235: Do not update the waypoint sequence.

[0223] S204: Displaying the suture needle according to the model parameters of the suture needle in the current frame.

[0224] S205: Determine whether to update the stage flag based on at least the result of the collision detection between the soft tissue and the suture needle.

[0225] If PBD / XPBD is used to calculate the model parameters of the suture needle, the collision detection between the soft tissue and the suture needle is generally performed before or during S201. This step only needs to be performed after the collision detection between the soft tissue and the suture needle, and there is no restriction on the order with other steps.

[0226] If the stage flag is that the soft tissue has been pierced in the puncture stage and it is detected that the needle tail of the suture needle collides with the surface of the soft tissue, the stage flag is updated to the suturing stage.

[0227] The puncture phase can be divided into multiple sub-phases based on the needle's puncture direction and whether it penetrates the soft tissue surface. There are multiple ways to determine the puncture direction. The following example uses the historical record of the stage marker to determine the puncture direction.

[0228] like Figure 17 As shown, in one embodiment of the present application, the sub-stages of determining the puncture stage may specifically include:

[0229] S261: Determine whether a collision between the needle tip of the suture needle and the surface of the soft tissue is detected in the current frame.

[0230] If so, jump to S262; otherwise jump to S269.

[0231] S262: Add 1 to the cumulative number of frames and temporarily store the collision response displacement of the current frame.

[0232] The collision response displacement is the position of the puncture point that collides with the needle in the soft tissue model parameters of the current frame minus the position of the puncture point in the soft tissue model parameters of the previous frame.

[0233] S263: If the accumulated number of frames is greater than the first threshold, all temporarily stored collision response displacements are accumulated to obtain an accumulated displacement.

[0234] S264: Compare the accumulated displacement with a second threshold value to determine whether the needle has penetrated the surface of the soft tissue.

[0235] Specifically, if the cumulative displacement in the current frame is greater than a second threshold, it is determined that the needle has penetrated the surface of the soft tissue; otherwise, it is determined that the needle has not penetrated the surface of the soft tissue. Alternatively, if the cumulative displacement in the current frame is greater than or equal to the second threshold, it is determined that the needle has penetrated the surface of the soft tissue; otherwise, it is determined that the needle has not penetrated the surface of the soft tissue.

[0236] S265: If the needle has not pierced the surface of the soft tissue and the stage flag is empty, the stage flag is updated to piercing the soft tissue in the puncture stage.

[0237] S266: If the needle has pierced the surface of the soft tissue and the stage flag is empty or has penetrated the soft tissue during the puncture stage, the stage flag is updated to have penetrated the soft tissue during the puncture stage.

[0238] S267: If the needle has not pierced the surface of the soft tissue, and the stage mark is "entered into the soft tissue" in the puncture stage, then the stage mark is updated to "pierced out of the soft tissue" in the puncture stage.

[0239] S268: If the needle has pierced the surface of the soft tissue, and the stage mark is "pierced into the soft tissue" or "pierced out of the soft tissue" in the puncture stage, then the stage mark is updated to "pierced out of the soft tissue" in the puncture stage.

[0240] S269: Set the cumulative number of frames to 0 and clear all temporarily stored collision response displacements.

[0241] like Figure 18 As shown, the fourth embodiment of the soft tissue suturing simulation method provided by the present application includes the following steps.

[0242] S301: Obtain a stage flag in the current frame.

[0243] This embodiment is a real-time simulation process of each frame of soft tissue during the soft tissue suturing process. The stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, which includes the puncture stage, the suturing stage, and the tensioning stage.

[0244] S302: Perform collision detection on soft tissue, sutures, and surgical instruments.

[0245] The soft tissue model is a mesh model including a plurality of vertices and tetrahedrons defined by the vertices.

[0246] S303: Determine a temporary constraint function of the vertex according to the collision detection result and / or the stage flag.

[0247] If a collision is detected between the soft tissue surface and the suture needle tip in the surgical instrument, a first collision response constraint is added to the puncture point where the needle tip collides. This temporary constraint function is mainly used during the puncture phase.

[0248] The second collision response constraint function mainly acts on the tension stage. Figure 19 As shown, in one embodiment of the present application, the specific process of adding the second collision response constraint function may include:

[0249] S331: If the stage flag is the tensioning stage, after calculating the model parameters of the suture line in the current frame, calculate the second distance between the target particle point located inside the soft tissue and the nearest path point in the path point sequence.

[0250] The path point sequence is obtained by interpolating the movement path of the needle tail of the suture needle in the soft tissue using the particle point interval of the suture line.

[0251] S332: If the second distance is greater than the distance threshold, a second collision response constraint is added to the collision tetrahedron where the nearest path point is located.

[0252] Moves the collision tetrahedron by the threshold distance in the direction from the nearest path point to the target particle point.

[0253] S304: Obtain model parameters of the soft tissue in the current frame using position-based dynamics calculation.

[0254] The vertex constraint functions include temporary constraint functions. If the phase flag is the tension phase, the coordinates of each path point in the path point sequence can be updated based on the soft tissue model parameters in the current frame. Specifically, for each path point, the coordinates of the center of gravity of the path point in the tetrahedron before the second collision response constraint is added can be obtained. Then, the coordinates of each vertex of the tetrahedron and the center of gravity coordinates in the soft tissue model parameters can be combined to calculate the updated coordinates of the path point.

[0255] S305: Determine whether to update the phase flag at least according to the result of the collision detection.

[0256] This step only needs to be executed after S302, and there is no restriction on the execution order between S303 and subsequent steps.

[0257] If the stage flag is that the soft tissue has been pierced in the puncture stage and it is detected that the needle tail of the suture needle collides with the surface of the soft tissue, the stage flag is updated to the suturing stage.

[0258] Based on the collision detection results between the soft tissue surface and the needle, the sub-stages of the puncture stage can be distinguished. For details, please refer to Figure 17 and corresponding embodiments.

[0259] S306: Displaying the soft tissue according to the model parameters of the soft tissue in the current frame.

[0260] See also Figure 20 As shown, the first embodiment of the soft tissue suturing simulation device provided by the present application includes: an acquisition module 101 , a detection module 102 , an interpolation module 103 , a calculation module 104 and a display module 105 .

[0261] The acquisition module 101 is used to acquire models of soft tissue, sutures, and surgical instruments. The soft tissue model is a mesh model comprising a plurality of vertices and tetrahedrons defined by the vertices; the suture model comprises a plurality of particle points arranged in a chain-like manner and rod nodes connecting adjacent particle points; and the surgical instrument model comprises a plurality of mass points arranged according to the shape of the surgical instrument, which includes a suture needle.

[0262] The detection module 102 is used to perform collision detection between the soft tissue, the suture, and the surgical instrument, and determine the stage in the suturing process at least based on the result of the collision detection, wherein the stage includes the puncture stage, the suturing stage, and the tensioning stage.

[0263] The interpolation module 103 is used to interpolate the movement path of the needle tail of the suture needle in the soft tissue using the particle point interval of the suture line to obtain a path point sequence during the suturing stage.

[0264] The calculation module 104 is used to calculate the model parameters of the soft tissue and suture line frame by frame based on position dynamics, and to calculate the model parameters of the surgical instrument frame by frame based on position dynamics and / or kinematic equations, wherein the constraint function of the target particle point includes a path point constraint based on an associated path point in the path point sequence corresponding to the target particle point, and the target particle point is a particle point of the suture line located inside the soft tissue.

[0265] The display module 105 is configured to display the soft tissue, suture and surgical instrument frame by frame according to the model parameters of the soft tissue, suture and surgical instrument.

[0266] See also Figure 21 As shown, the second embodiment of the soft tissue suturing simulation device provided by the present application includes: an acquisition module 201 , a temporary constraint module 202 , a calculation module 203 and a display module 204 .

[0267] The acquisition module 201 is used to determine the target pose offset of the stitching line in the current frame and obtain the stage flag. The stitching line model includes a plurality of particle points arranged in a chain and rod nodes connecting adjacent particle points.

[0268] The temporary constraint module 202 is used to determine a temporary constraint function of the particle point based on at least a stage flag. The stage flag is used to indicate the stage of the current frame in the soft tissue suturing process, wherein the stages include the puncture stage, the suturing stage, and the tensioning stage. If the stage flag is the suturing stage, the temporary constraint function of the target particle point located inside the soft tissue includes a path point constraint based on an associated path point corresponding to the target particle point in a path point sequence, wherein the path point sequence is obtained by interpolating the movement path of the first control point within the soft tissue using the particle point interval of the suture line, wherein the first control point is a particle point directly connected to the needle tail of the suture needle.

[0269] The calculation module 203 is configured to obtain the model parameters of the stitching line in the current frame using a position dynamics-based calculation according to the target pose offset, wherein the constraint function of the particle point includes the temporary constraint function.

[0270] The display module 204 is configured to display the stitching line according to the model parameters of the stitching line in the current frame.

[0271] See also Figure 22 As shown, the third embodiment of the soft tissue suturing simulation device provided by the present application includes: a calculation module 301 , an acquisition module 302 , an interpolation module 303 and a display module 304 .

[0272] The calculation module 301 is used to calculate the model parameters of the suture needle in the current frame, where the model of the suture needle includes a plurality of mass points arranged according to its shape.

[0273] The acquisition module 302 is used to acquire a stage mark, where the stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, where the stages include the puncture stage, the suturing stage, and the tensioning stage.

[0274] The interpolation module 303 is configured to interpolate the movement path of the needle tail of the suture needle in the soft tissue using the particle point interval of the suture line to determine whether to update the path point sequence when the stage mark is the suturing stage.

[0275] The display module 304 is configured to display the suture needle according to the model parameters of the suture needle in the current frame.

[0276] See also Figure 23 As shown, the fourth embodiment of the soft tissue suturing simulation device provided by the present application includes: an acquisition module 401 , a detection module 402 , a temporary constraint module 403 , a calculation module 404 , an update module 405 and a display module 406 .

[0277] The acquisition module 401 is used to acquire a stage mark in the current frame, where the stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, where the stages include the puncture stage, the suturing stage, and the tensioning stage;

[0278] The detection module 402 is used to perform collision detection on the soft tissue, sutures, and surgical instruments. The model of the soft tissue is a mesh model including a plurality of vertices and tetrahedrons defined by the vertices.

[0279] The temporary constraint module 403 is configured to determine a temporary constraint function of the vertex according to the collision detection result and / or the stage flag.

[0280] The calculation module 404 is configured to obtain the model parameters of the soft tissue in the current frame using position-based dynamics calculation, wherein the constraint function of the vertex includes the temporary constraint function.

[0281] The updating module 405 is configured to determine whether to update the phase flag at least according to the result of the collision detection.

[0282] The display module 406 is configured to display the soft tissue according to the model parameters of the soft tissue in the current frame.

[0283] It should be noted that the soft tissue suturing simulation device provided in the above embodiment, in implementing the soft tissue suturing simulation method, is merely illustrated by the division of the aforementioned program modules. In actual applications, the aforementioned processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the method steps described above. Furthermore, the soft tissue suturing simulation device provided in the above embodiment and the soft tissue suturing simulation method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0284] See Figure 24 , which is a schematic diagram of the structure of a soft tissue suturing simulation device provided in accordance with an embodiment of the present application. The object simulation device provided in accordance with an embodiment of the present application includes a memory 502 and a processor 501. The memory 502 stores a computer program executable by the processor 501. When the computer program is executed by the processor 501, the soft tissue suturing simulation method according to any embodiment of the present application is implemented.

[0285] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the various processes of the above-mentioned soft tissue suturing simulation method embodiment are implemented, and the same technical effects are achieved. To avoid repetition, the description is omitted here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0286] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A soft tissue suturing simulation method, characterized in that: include: Determining a target pose offset of a stitching line in a current frame and obtaining a stage flag, wherein the stitching line model includes a plurality of particle points arranged in a chain and rod nodes connected between adjacent particle points; determining a temporary constraint function for the particle point based on at least a stage flag, the stage flag being used to indicate the stage of the current frame in the soft tissue suturing process, the stages including a puncture stage, a suturing stage, and a tensioning stage, wherein if the stage flag is the suturing stage, the temporary constraint function for the target particle point located within the soft tissue includes a path point constraint based on an associated path point corresponding to the target particle point in a path point sequence, the path point sequence being obtained by interpolating a movement path of a first control point within the soft tissue using a particle point interval of the suture line, the first control point being a particle point directly connected to the needle tail of the suture needle; Obtaining model parameters of the stitching line in the current frame using position dynamics-based calculation according to the target pose offset, wherein the constraint function of the particle point includes the temporary constraint function; The stitching line is displayed according to the model parameters of the stitching line in the current frame.

2. The soft tissue suturing simulation method according to claim 1, wherein: Also includes: Obtaining the position of the first control point in the current frame; If the position of the first control point in the current frame is within the soft tissue, determining whether a first distance between the position of the first control point in the current frame and the current path point in the path point sequence is within a deviation interval determined based on the particle point interval; If the first distance is within the deviation interval, adding the position of the first control point in the current frame as a new path point to the path point sequence; If the first distance is greater than the upper limit of the deviation interval, interpolating between the current path point and the position of the first control point in the current frame using the particle point interval, and adding the interpolation result as a new path point to the path point sequence; If the first distance is less than the lower limit of the deviation interval, the path point sequence is not updated.

3. The soft tissue suturing simulation method according to claim 2, wherein: The obtaining the position of the first control point in the current frame includes: Determine a position offset of the first control point in the current frame according to the target posture offset; The sum of the position of the first control point in the previous frame and the position offset of the first control point in the current frame is calculated as the position of the first control point in the current frame.

4. The soft tissue suturing simulation method according to claim 1, wherein: The determining of the temporary constraint function of the particle point at least according to the stage flag comprises: If the stage flag is the suturing stage, determining whether the first control point in the current frame is located inside the soft tissue; If the first control point is located inside the soft tissue, the first control point is used as a reference point, and the position change of the reference point is determined according to the position change of the first control point; if the first control point is located outside the soft tissue, the particle point in the suture line that is closest to leaving the soft tissue is used as a reference point, and the position change of the reference point is calculated using position dynamics; Determining the target particle point in the current frame and the corresponding relationship between the target particle point and the path points in the path point sequence according to the position change of the reference point and the particle point interval; The path point constraint is added to the target particle point based on the corresponding relationship.

5. The soft tissue suturing simulation method according to claim 4, wherein: The determining, based on the position change of the reference point and the particle point interval, of the target particle point in the current frame and the corresponding relationship between the target particle point and the path point in the path point sequence comprises: Calculating the required position change of the reference point in the current frame based on the position change of the reference point in the current frame and the remaining reference point positions in the previous frame; Determining whether the required position change of the reference point is greater than the particle point interval; If not, taking the required position change of the reference point as the remaining reference point position, and taking the target particle point in the previous frame and the corresponding relationship between it and the path points in the path point sequence as the target particle point in the current frame and the corresponding relationship between it and the path points in the path point sequence; If so, the number of particle points of the suture line moving into the soft tissue and the remaining reference point positions in the current frame are determined based on the ratio of the required position change of the reference point to the particle point interval. The target particle points in the previous frame and the corresponding relationship between them and the path points in the path point sequence are updated based on the number of particle points of the suture line moving into the soft tissue to obtain the target particle points in the current frame and the corresponding relationship between them and the path points in the path point sequence.

6. The soft tissue suturing simulation method according to claim 1, wherein: Also includes: performing collision detection on the suture thread, the soft tissue, and surgical instruments other than the suture needle; Whether to update the phase flag is determined at least according to the result of the collision detection.

7. The soft tissue suturing simulation method according to claim 6, wherein: The determining whether to update the stage flag at least according to the result of the collision detection includes: If the stage mark is the puncture stage and it is detected that the first control point collides with the soft tissue, the stage mark is updated to the suturing stage.

8. The soft tissue suturing simulation method according to claim 6, wherein: The determining whether to update the stage flag at least according to the result of the collision detection includes: If the stage mark is the suturing stage and it is detected that the preset knot in the suture line collides with the surface of the soft tissue, the stage mark is updated to the tightening stage.

9. The soft tissue suturing simulation method according to claim 6, wherein: The determining whether to update the stage flag at least according to the result of the collision detection includes: If the stage mark is the suturing stage and it is detected that the suture thread collides with the second clamping body, the stage mark is updated to the tensioning stage, and the second clamping body is a surgical instrument different from the suture needle and the first clamping body that clamps the suture needle.

10. A soft tissue suturing simulation device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the soft tissue suturing simulation method according to any one of claims 1 to 9 is implemented.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the soft tissue suturing simulation method according to any one of claims 1 to 9 is implemented.

Images