Soft tissue suturing simulation methods, apparatuses, and media

Abstract

The application provides a soft tissue suturing simulation method, which comprises the following steps: calculating model parameters of a suturing needle in a current frame, the model of the suturing needle comprising a plurality of mass points arranged according to the shape of the suturing needle; obtaining a stage mark, the stage mark being used to indicate a stage in which the current frame is located in a soft tissue suturing process, the stage comprising a puncture stage, a suturing stage and a tensioning stage; if the stage mark is the suturing stage, using a particle point interval of a suture to interpolate a moving path of a needle tail of the suturing needle in the soft tissue to determine whether to update a path point sequence; and displaying the suturing needle according to the model parameters of the suturing needle in the current frame. The application also provides a soft tissue suturing simulation device and a storage medium.

Description

Technical Field

[0001] This application relates to the field of computer graphics technology, and in particular to a method, apparatus and computer-readable storage medium for simulating soft tissue suturing. Background Technology

[0002] Suturing is an essential part of surgical simulation. The suturing process involves many aspects, such as the simulation of sutures, soft tissues, and surgical instruments such as suture needles, surgical forceps, and tweezers. It also includes the interaction between these entities, such as the surgical forceps clamping the suture needle to pierce the surface of the soft tissue and enter the soft tissue, the suture needle driving the suture through the soft tissue, the suture pulling the soft tissue together tightly, and the suture finally being knotted to fix the soft tissue.

[0003] Traditional methods often employ the Follow-The-Lead (FTL) approach to simulate sutures, the Mass-Spring Model (MSM) to simulate soft tissues, and particle models to simulate surgical instruments. Because these three models are different, some model overlap inevitably occurs when handling their interactions. Furthermore, this approach is not suitable for simulating more diverse organ and tissue types, resulting in the entire surgical simulation consisting of multiple different frameworks, thus increasing the simulation's complexity. Summary of the Invention

[0004] To address the existing technical problems, this application provides a soft tissue suturing simulation method, device, and computer-readable storage medium capable of simulating the soft tissue suturing process in real time and accurately.

[0005] In a first aspect, this application provides a soft tissue suturing simulation method, comprising: calculating model parameters of a suture needle in the current frame, wherein the model of the suture needle includes multiple particles arranged according to its shape; obtaining a stage marker, wherein the stage marker is used to indicate the stage in the soft tissue suturing process of the current frame, and the stage includes a puncture stage, a suturing stage, and a tightening stage; if the stage marker is a suturing stage, then interpolating the movement path of the suture needle tail in the soft tissue using the particle point interval of the suture line to determine whether to update the path point sequence; and displaying the suture needle according to the model parameters of the suture needle in the current frame.

[0006] A second aspect of this application provides a soft tissue suturing simulation device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, it implements the soft tissue suturing simulation method as described in any embodiment of this application.

[0007] In a third aspect of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the soft tissue suturing simulation method as described in any embodiment of this application.

[0008] In the above embodiments, during the suturing stage, the particle point interval of the suture line is used to interpolate the movement path of the needle tail in the soft tissue to obtain a path point sequence. Then, based on the correspondence between the path point sequence and the target particle points located inside the soft tissue in the suture line, path point constraints are added to the target particle points so that the part of the suture line located inside the soft tissue will not undergo drastic length changes under the action of path point constraints, thereby reducing the distortion that may result.

[0009] In the above embodiments, the soft tissue suturing simulation device and the computer-readable storage medium belong to the same concept as the corresponding soft tissue suturing simulation method embodiments, and thus have the same technical effects as the corresponding soft tissue suturing simulation method embodiments, which will not be repeated here. Attached Figure Description

[0010] Figure 1 A schematic diagram of a suture line for introducing an elastic rod;

[0011] Figure 2 A flowchart for introducing position-based dynamics of an elastic rod;

[0012] Figure 3 This is a schematic diagram of the various stages of the soft tissue suturing process;

[0013] Figure 4 This is a schematic flowchart of the first embodiment of the soft tissue suturing simulation method of this application;

[0014] Figure 5 This is a schematic diagram of the process for determining whether a needle has pierced the surface of soft tissue in one embodiment of this application;

[0015] Figure 6 for Figure 4 A detailed flowchart of the S3 process;

[0016] Figure 7 This is a schematic diagram of the process of adding path point constraints to a target particle point in one embodiment of this application;

[0017] Figure 8 for Figure 7 A detailed flowchart of the S44 process;

[0018] Figure 9 This is a schematic diagram illustrating the process of constraining the collision response between sutures and soft tissue in one embodiment of this application;

[0019] Figure 10 for Figure 9 A detailed flowchart of the S49 process;

[0020] Figure 11 This is a schematic flowchart of the second embodiment of the soft tissue suturing simulation method of this application;

[0021] Figure 12 This is a schematic diagram of the real-time process for obtaining a path point sequence in one embodiment of this application;

[0022] Figure 13 for Figure 11 A detailed flowchart of the process in S102;

[0023] Figure 14 This is a flowchart illustrating the third embodiment of the soft tissue suturing simulation method of this application;

[0024] Figure 15 When the suture needle is treated as a rigid body Figure 14 A detailed flowchart of the S201 process;

[0025] Figure 16 for Figure 14 A detailed flowchart of the S203 process;

[0026] Figure 17 This is a flowchart illustrating the sub-stage of determining the puncture stage in one embodiment of this application;

[0027] Figure 18 This is a flowchart illustrating the fourth embodiment of the soft tissue suturing simulation method of this application;

[0028] Figure 19 This is a flowchart of adding a second collision response constraint function in one embodiment of this application;

[0029] Figure 20 This is a schematic diagram of the structure of the first embodiment of the soft tissue suturing simulation device of this application;

[0030] Figure 21 This is a schematic diagram of the structure of the second embodiment of the soft tissue suturing simulation device of this application;

[0031] Figure 22 This is a schematic diagram of the structure of the third embodiment of the soft tissue suturing simulation device of this application;

[0032] Figure 23 This is a schematic diagram of the structure of the fourth embodiment of the soft tissue suturing simulation device of this application;

[0033] Figure 24 This is a schematic diagram of the structure of an embodiment of the soft tissue suturing simulation device of this application. Detailed Implementation

[0034] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to limit the ways in which this application may be implemented. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0036] The various embodiments of this application can be combined with each other without conflict. In the following description, the expression "some embodiments" is used, which describes a subset of possible embodiments. However, it should be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0037] The order of steps given in the embodiments of this application is only illustrative. If other orders can be used to perform these steps to achieve the same effect, these orders are also within the scope of protection of this application.

[0038] In the field of computer graphics, common physical methods for simulating object deformation in real time typically update acceleration using external forces and then integrate to obtain velocity and position. PBD, however, directly corrects position using constraint functions and then updates velocity using position. The system equations in the PBD algorithm are constructed from a set of constraint function equations, which includes multiple constraint function equations (which can be simply referred to as constraint functions).

[0039] In PBD, the model of an object consists of multiple particle points, also known as mass points, that make up the object. The parameters of a particle point include its position, velocity, and mass. The parameters of the constraint function include the positions of these particle points. The constraint function is used to describe the relationships between the particle points that make up the object.

[0040] The PBD algorithm mainly consists of an initialization phase and a real-time simulation phase. In the initialization phase, the parameters of all particle points and lever nodes constituting the object are initialized based on preset object parameters, such as initial position, initial orientation, and number of particle points. Then, the real-time simulation phase is initiated, cyclically executing frame-by-frame simulations of the object's deformation process based on environmental influences (including potential user commands). This phase specifically includes three parts: prediction, constraint correction, and variable update.

[0041] In the prediction section of the current frame, the forces exerted on objects by the external environment in the current frame are converted into displacements as much as possible. External forces that cannot be directly converted into displacements (such as gravity) are retained. Based on the positions of each particle point in the previous frame, the positions of each particle point in the current frame are predicted by combining the converted displacements and external forces that cannot be directly converted.

[0042] The aforementioned prediction process often simply uses integration (e.g., explicit Euler integral) to directly predict the positions of each particle in the current frame, without considering the object's mechanical properties or its interaction with the environment. Therefore, it cannot be directly used as the simulation result for the current frame; instead, constraint functions are used to correct the particle positions. Constraint functions are used to restrict the relationships between particle points and are mainly divided into two types: temporary constraint functions, which are generally generated temporarily when an interaction between an object and an external object in the environment is detected; and regular constraint functions, which are determined by the object's mechanical properties. The process of calculating the corrected position that best satisfies the constraint functions based on the constraint functions and the current position is called constraint projection, which is the core process of PBD. Constraint correction essentially involves iteratively optimizing the particle positions based on the constraint function equations. Specifically, this is typically achieved by iteratively executing constraint projection using the constraint functions in the constraint function equations until a predetermined number of iterations is reached, in order to find the particle positions that best satisfy all constraint functions in the constraint function equations.

[0043] After constraint correction, the result is the position of each particle point in the current frame. Based on this, the velocity of each particle point in the current frame can be calculated, and then the corresponding variables are updated for display and calculation in the next frame. The Δt used in PBD is generally the default frame interval, that is, the duration between two adjacent frames under normal circumstances.

[0044] For soft tissue, the model is a mesh model consisting of multiple vertices and tetrahedrons defined by the vertices. Here, the vertices are similar to particle points, and the constraint functions mainly include distance constraints between adjacent vertices and volume invariance constraints of each tetrahedron.

[0045] Surgical instruments, such as forceps, needle holders, tweezers, and suture needles, can be approximated as rigid bodies. Their models typically include multiple point masses / particles determined by their specific shapes. For example, a suture needle model may include multiple point masses arranged in a chain. Since surgical instruments other than suture needles are generally directly controlled by the user, the PBD (Programmable Analytical Design) process can be omitted, and their attitude and the positions of each point mass in the model can be directly calculated using kinematic equations.

[0046] For sutures, to more accurately describe their morphology, rod nodes, also known as elastic rods, are further introduced. For example... Figure 1 As shown, the suture model includes multiple particle points arranged in a chain (e.g. Figure 1p1, p2, ..., p n ) and rod nodes connecting adjacent particle points (e.g. Figure 1 In q1, q2, ..., q n-1 The extension directions shown in the diagram are only used to specify the numbering order of particle points and lever nodes and do not represent the actual extension directions. The parameters of the lever nodes include their attitude, angular velocity, and moment of inertia. Correspondingly, the parameters of the constraint function also include the attitude of these lever nodes. For ease of calculation, the attitude is generally represented using a unit quaternion; however, in other embodiments, a transformation / rotation matrix can also be used to represent the attitude.

[0047] In practical applications, sutures are often connected to an external object, which controls the movement of the suture and can be called the control body. For example, if the suture is the suture itself, then the control body is the suture needle, surgical forceps, tweezers, etc., that directly move the suture. If the suture is hair, the control body is the skin or hair ornament that holds the hair in place, and the object that directly moves the hair is a finger or comb. The particle point in the suture that is directly connected to the control body, that is, the particle point closest to the control body, can be called the control point.

[0048] The process of introducing a flexible rod in a PBD is as follows: Figure 2 As shown in the attached diagram, the PBD process will be explained in detail below.

[0049] Figure 2 Steps 1-4 constitute the initial stage. In this stage, based on preset object parameters such as initial position, initial posture, and number of particle points, the parameters of all particle points and rod nodes that make up the object are initialized. Then, the real-time simulation stage begins, simulating the object's deformation process frame-by-frame based on environmental influences (including potential user commands). Figure 2 Steps 5-21 can specifically include three parts: prediction, constraint correction, and variable update.

[0050] In the prediction part of the current frame (corresponding to) Figure 2 In steps 6-12), the forces exerted on the object by the external environment in the current frame are converted into displacements as much as possible; external forces that cannot be directly converted into displacements (such as gravity) are retained. Similarly, the torques exerted on the object by the external environment in the current frame are converted into angular displacements as much as possible; external torques that cannot be directly converted into angular displacements are retained. Based on the positions of each particle point in the previous frame, the positions of each particle point in the current frame are predicted by combining the converted displacements and external forces that cannot be directly converted. Similarly, based on the attitudes of each link node in the previous frame, the attitudes of each link node in the current frame are predicted by combining the converted angular displacements and external torques that cannot be directly converted.

[0051] The aforementioned predictions often simply use integration (e.g., explicit Euler integral) to directly predict the positions of each particle and the orientation of each link node in the current frame, without considering the object's mechanical properties or its interaction with the environment. Therefore, they cannot be directly used as the simulation results for the current frame. Instead, constraint functions are used to correct the positions of the particle points and the orientations of the link nodes. Constraint functions are used to restrict the relationships between particle points, link nodes, and between them. There are two main types: temporary constraint functions, which are generally generated temporarily when an interaction between an object and an external object in the environment is detected; and regular constraint functions, which are determined by the object's mechanical properties. For example, for sutures and similar linear objects, regular constraint functions can be given based on their mechanical properties of being difficult to stretch and shear, but easy to bend and twist. The process of calculating the corrected position / or orientation that best satisfies the constraint functions based on the current position / or orientation is called constraint projection, and it is the core process of PBD (Problem-Based Design). Constraint correction essentially involves iteratively optimizing the particle positions and link poses based on a set of constraint function equations. Specifically, this typically involves iteratively projecting constraints using the constraint functions in the set of equations until a predetermined number of iterations is reached. The goal is to find particle positions and link poses that satisfy as many constraints as possible from the set of equations. The constraint correction section corresponds to... Figure 2 Steps 13-15 in the process, where step 13 is used to perform collision detection to generate temporary constraint functions, and steps 14-15 are used to iteratively optimize the position of the particle point and the pose of the rod node using the constraint function equations.

[0052] Updated section corresponding Figure 2 Steps 16-21 in the process, after constraint correction, yield the positions of each particle point and the attitude of each link node in the current frame. Based on this, the velocities of each particle point and the angular velocities of each link node in the current frame can be calculated, and then the corresponding variables are updated for display and calculation in the next frame. The Δt used in PBD is generally the default frame interval, that is, the duration between two adjacent frames under normal circumstances.

[0053] After completing the simulated object deformation process in the current frame, the model parameters for the current frame can be obtained, specifically including the updated parameters of particle points and rod nodes. The updated positions of each particle point and the orientation of each rod node can be used to determine the shape of the stitching line in the current frame for display.

[0054] It should be noted that vertices, particle points, and mass points in this application are actually equivalent in the PBD framework. However, different names have been chosen for particle points in different models for the sake of simplifying the description. That is, particle points in soft tissue are called vertices, particle points in surgical instruments are called mass points, and particle points in sutures are called particle points.

[0055] Based on the above preliminary introduction to soft tissue suturing simulation methods, the following will combine... Figures 3 to 19 The soft tissue suturing simulation method provided in the embodiments of this application will be further described in detail. Since object simulation is a real-time process, when the current frame is mentioned below, it refers to describing the specific process of object simulation within a frame using the current frame as an example. In fact, this process is executed in every frame.

[0056] Please see Figure 3 The soft tissue suturing process mainly includes the following stages: puncture stage, suturing stage, tightening stage, and knotting stage.

[0057] From the user's perspective, during the puncture stage, the user uses the first clamp (usually surgical forceps, needle holder, etc.) to hold the middle and rear part of the suture needle and insert it into the soft tissue. After the needle tip of the suture needle enters the soft tissue, it travels through the soft tissue until the needle tip of the suture needle pierces out of the soft tissue. Then the user moves the first clamp to the front part of the suture needle and continues to pull the suture needle until the needle tail (i.e. the mass point in the suture needle that is directly connected to the suture) enters the soft tissue.

[0058] The suturing stage generally begins when the needle tip guides the suture into the soft tissue and ends with the appearance of a tightening mark. There are two main types of tightening marks: one is when there is a pre-tied knot at the end of the suture, in which case the tightening mark is when the pre-tied knot contacts the surface of the soft tissue; the other is when there is no pre-tied knot, in which case the tightening mark is when the user directly clamps the end of the suture with a second clamp (another surgical instrument different from the first clamp, such as forceps).

[0059] The tightening phase generally begins with the appearance of the tightening mark and ends when the suture line self-collides under the action of the first and second clamping bodies.

[0060] The knotting stage generally begins when the suture collides with itself under the action of the first and second clamps. The exact end time can be determined as needed, such as cutting the suture with surgical scissors.

[0061] For ease of description, this application only uses a single puncture as an example. In actual application, multiple punctures may occur. In this case, each soft tissue has its own puncture stage and suturing stage, which is not limited here.

[0062] Please see Figure 4 The first embodiment of the soft tissue suturing simulation method provided in this application includes the following steps.

[0063] S1: Obtain models of soft tissue, sutures, and surgical instruments.

[0064] The soft tissue model is a mesh model consisting of multiple vertices and tetrahedrons defined by those vertices. The suture model consists of multiple particle points arranged in a chain and rod nodes connecting adjacent particle points. The surgical instrument model consists of multiple particles arranged according to their shape, and the surgical instrument includes a suture needle. Detailed information about the models can be found in the preceding descriptions.

[0065] S2: Conduct collision detection between soft tissue, sutures, and surgical instruments, and determine the stage of the suturing process based on the collision detection results.

[0066] The stages include the puncture stage, the suturing stage, and the tightening stage, and may also include the knotting stage. The preceding stages are described from the user's perspective. In the actual simulation of soft tissue suturing, the stage of the current frame in the suturing process can be determined based on the collision detection results between the soft tissue, suture, and surgical instruments.

[0067] Collision detection between soft tissue, sutures, and surgical instruments, in addition to collision detection between soft tissue and sutures, soft tissue and surgical instruments, and sutures and surgical instruments, may also include self-collision detection of sutures and / or collision detection between different surgical instruments.

[0068] In addition to collision detection results, the stage of the current frame can be determined by combining historical stage information, the relative positions of surgical instruments and soft tissue, etc. An example is given below.

[0069] The puncture phase primarily focuses on detecting the collision between the suture needle and the soft tissue. Based on whether the soft tissue surface has been punctured and the direction of the collision, the puncture phase can be further divided into several sub-phases, as follows:

[0070] a. If the needle tip of the suture needle collides with the surface of the soft tissue from the outside in but does not pierce the surface of the soft tissue, then the suturing stage is the puncture stage in which the needle penetrates the soft tissue.

[0071] In this sub-stage, since the suture needle is generally held by the first clamping body, there is no need to respond to the impact of the suture needle, but only to the impact of the soft tissue.

[0072] b. If the suture needle has pierced the surface of the soft tissue and the needle tip is inside the soft tissue, then the suturing stage is the puncture stage in which the needle has pierced the soft tissue.

[0073] c. If the needle collides with the surface of the soft tissue from the inside out but does not pierce the surface of the soft tissue, then the suturing stage is the puncture stage in which the needle pierces the soft tissue.

[0074] In this sub-stage, since the suture needle is generally held by the first clamping body, there is no need to respond to the impact of the suture needle, but only to the impact of the soft tissue.

[0075] d. If the suture needle has pierced the surface of the soft tissue and the needle tip is outside the soft tissue, then the suturing stage is the soft tissue that has been pierced in the puncture stage.

[0076] The needle tip can refer to a small area of ​​the pointer. The direction in which the suture needle collides with the soft tissue surface, i.e., "from the outside of the soft tissue to the inside" and "from the inside of the soft tissue to the outside" as mentioned above, can be achieved in several ways. One is to consult the historical record of the puncture stage; if the soft tissue was not punctured in a previous stage, the direction is from the outside to the inside, and vice versa. Another is to refer to the relative position of the suture needle and the soft tissue; if the part of the suture needle other than the part that collides with the soft tissue is located on the outside of the soft tissue, the direction is from the outside to the inside; if the middle part of the suture needle is located on the inside of the soft tissue, the direction is from the inside to the outside. Yet another is to refer to the relative relationship between the direction of the suture needle's velocity and the soft tissue; if the suture needle's velocity points towards the inside of the soft tissue, the direction is from the outside to the inside; if the suture needle's velocity points towards the outside of the soft tissue, the direction is from the inside to the outside.

[0077] like Figure 5 As shown, in one embodiment of this application, determining whether the needle has pierced the surface of soft tissue may specifically include:

[0078] S21: Collision detection is performed between the needle tip of the suture needle and the surface of the soft tissue.

[0079] S22: If the number of frames in which the needle continuously collides with the surface of the soft tissue is greater than the first threshold, the collision response displacement of the soft tissue is accumulated to obtain the cumulative displacement.

[0080] S23: If the cumulative displacement is greater than the second threshold, it is determined that the suture needle has pierced the surface of the soft tissue.

[0081] Since the suture needle is usually held by the first clamping body during the process of piercing the soft tissue surface, if the suture needle is regarded as a rigid body, then no collision response is required for the suture needle, but only for the soft tissue.

[0082] Collision response refers to the handling of objects that collide. This handling generally aims to closely resemble human understanding of physical laws, avoiding obviously abnormal interactions between the colliding objects, such as clipping. When a suture needle punctures the surface of soft tissue, a puncturing force is generated between the needle and the soft tissue. Due to the elasticity of soft tissue, it is not immediately punctured. Instead, it deforms along the direction of needle movement at the puncture site. Only when the puncturing force increases beyond the maximum resistance that the soft tissue can generate is it punctured.

[0083] To simplify the simulation of the puncture process within the PBD framework, this embodiment abandons the simulation of puncture force and soft tissue resistance. Instead, it uses the cumulative displacement of the soft tissue's impact response displacement to reflect the magnitude of the puncture force, and a second threshold to reflect the maximum resistance that the soft tissue can generate. The comparison of forces is transformed into a comparison of cumulative displacement and the second threshold, making it more suitable for the PBD framework. The second threshold can be set according to the characteristics of different organs and tissues. For the same soft tissue, the second threshold used for puncturing the soft tissue surface from the inside out and from the outside in can be the same or different.

[0084] To prevent the rubbing of the needle against the soft tissue surface from being incorrectly identified as puncture, this embodiment only confirms that the needle has begun to puncture the soft tissue surface when continuous collisions between the needle and the soft tissue surface are detected, and the number of consecutive collision frames is greater than a first threshold. At this point, the comparison between the cumulative displacement and a second threshold is used to determine whether the needle has punctured the soft tissue surface. After the needle begins to puncture the soft tissue surface, if the cumulative displacement in the current frame is less than the second threshold, it is determined that the soft tissue surface has not yet been punctured, and the cumulative displacement is retained for use in the next frame. To prevent the jitter of the soft tissue surface from affecting the puncture determination result, the collision response displacement is temporarily stored and accumulated in vector format. That is, the cumulative displacement in the current frame is essentially the displacement between the position of the soft tissue surface in the current frame under the collision response between the soft tissue and the needle, and its position before the collision with the needle.

[0085] The suturing stage begins when the needle tip is detected entering the soft tissue, or in other words, when the needle tip just collides with the soft tissue. Detecting the tension indicator signifies the transition from the suturing stage to the tensioning stage. If the tension indicator is a pre-tied knot contacting the soft tissue surface, the corresponding collision detection result is that the pre-tied knot has just collided with the soft tissue surface. If the tension indicator is the user directly clamping the tail of the suture with a second clamp, the corresponding collision detection result is that the tail of the suture, i.e., the portion outside the soft tissue and located on the side where the needle has pierced the soft tissue, has just collided with the second clamp.

[0086] The detection of the tension mark signifies the start of the tensioning phase, while the presence of at least two control points in the suture and the immediate detection of a self-collision signifies the end of the tensioning phase.

[0087] To simulate the soft tissue suturing process within the PBD framework, at least one stage marker can be designed to distinguish different stages. This stage marker can be updated as the stage changes. During real-time simulations of different types of objects, the stage marker can be read to perform corresponding operations, such as adding temporary constraints (collision response constraints, path point constraints, etc.) and updating path points. Since the stages that need to be considered in the simulation of different types of objects may not be the same, different stage markers can be set for different types of objects, or a uniform stage marker can be set for all objects. This application uses the latter as an example.

[0088] S3: During the suturing stage, the path point sequence is obtained by interpolating the movement path of the needle tail within the soft tissue using the particle point spacing of the suture thread.

[0089] During the movement of the needle tail within the soft tissue, the position of the needle tail in each frame, also known as the movement point, is arranged chronologically to form the movement path. During the suturing stage, the suture generally moves along this movement path within the soft tissue. Therefore, the constraint of the movement path on the suture can be used as a collision response constraint between the suture and the soft tissue acting on the suture.

[0090] Because the distribution of movement points in the movement path is affected by the user's operation and is not necessarily uniform, if temporary constraints are added directly to the particle points located inside the soft tissue in the suture line according to the movement path, i.e. the target particle points, the spacing between adjacent target particle points after the update will no longer be uniform due to the influence of the movement point distribution and may have a large error compared to the initially set particle point spacing, i.e. the spacing between adjacent particle points in the suture line, thus causing a significant change in the length of the suture line.

[0091] To avoid significant length changes in the suture as it moves through soft tissue and to reduce the resulting distortion, this embodiment uses the particle point interval of the suture to interpolate the movement path to obtain a path point sequence, and then uses the path point sequence to add temporary constraints to the target particle points.

[0092] like Figure 6 As shown, in one embodiment of this application, S3 specifically includes:

[0093] S31: During the stitching stage, obtain the position of the needle tail in the current frame.

[0094] For a suture needle, if it is treated as a rigid body, in the case where the suture needle is clamped by other surgical instruments, the PBD process can be omitted, and the kinematic equation can be directly used to calculate its pose and the positions of each mass point in the model in combination with the pose of the surgical instrument clamping the suture needle; in the case where the suture needle is not clamped by other surgical instruments, to prevent penetration of the model, the PBD / Extended Position-Based Dynamics (XPBD) process can be used to calculate the positions of each mass point in the model. Specifically, the overall collision constraint of the suture needle can be determined according to the result of the collision detection between the suture needle and the soft tissue, and the positions of each mass point in the suture needle model can be calculated using position-based dynamics based on the overall collision constraint.

[0095] If the elasticity of the suture needle is considered, it cannot be treated as a rigid body. Whether it is clamped by other surgical instruments or not, the PBD / XPBD process needs to be used to calculate the positions of each mass point in the model.

[0096] According to the above calculation method of the positions of each mass point in the suture needle, the positions of each mass point in the suture needle in the current frame can be obtained. Combining the position of the needle tail in the suture needle, the position of the needle tail in the current frame can be obtained.

[0097] S32: If the position of the needle tail in the current frame is within the soft tissue, it is judged whether the first distance between the position of the needle tail in the current frame and the current path point in the path point sequence is within the deviation interval determined based on the particle point interval.

[0098] The current path point is the currently last path point added to the path point sequence, that is, the latest path point in the path point sequence. Generally, the first path point added to the path point sequence is the position where the needle tail is first detected to collide with the soft tissue. The first distance is a scalar.

[0099] The deviation interval can be written in the form of [k1*rd, k2*rd], where rd is the particle point interval, 0 < k1 ≤ 1 ≤ k2. Generally, the differences between k1, k2 and 1 are all relatively small values, such as 0.9 and 1.1. The smaller the differences between k1, k2 and 1, the less likely the length of the suture line will change violently during the suturing stage, but the more likely it is to bring a deviation between the path point sequence and the moving path. The values of k1 and k2 can be determined according to the setting of the elasticity of the suture line in terms of stretching when establishing the model of the suture line. For example, if the suture line is set to be completely inextensible, then both k1 and k2 are 1, and the greater the elasticity in stretching, the greater k1 and k2 are.

[0100] If the first distance is within the deviation interval, it jumps to S33; if the first distance is greater than the upper limit of the deviation interval, it jumps to S34; if the first distance is less than the lower limit of the deviation interval, it jumps to S35.

[0101] S33: Add the position of the needle tail in the current frame as a new path point to the path point sequence.

[0102] S34: Use particle point intervals to interpolate between the current path point and the position of the needle tail in the current frame, and add the interpolation result as a new path point to the path point sequence.

[0103] The interpolation result lies on the line connecting the current path point and the position of the needle tail in the current frame, and the distance between the current path point and the current path point is k*rd. The value of k does not exceed [k1, k2]. The value of k in different frames can be the same, for example, all of them are 1, or they can be different, for example, a value is randomly selected within the range of values.

[0104] S35: Do not update the path point sequence.

[0105] If the position of the needle tail in the current frame is not within the soft tissue, it means that the needle tail has left the soft tissue, the movement path of the needle tail within the soft tissue has ended, and the path point sequence no longer needs to be updated.

[0106] S4: Calculate the model parameters of soft tissue and sutures frame by frame using position dynamics, and calculate the model parameters of surgical instruments frame by frame using position dynamics and / or kinematic equations.

[0107] During the suturing stage, the main temporary constraint to consider is the path point constraint of the target particle point. The target particle point is the particle point of the suture line located inside the soft tissue. Each target particle point has a corresponding associated path point in the path point sequence. The path point constraint of each target particle point refers to the constraint between the target particle point and its associated path point.

[0108] During the stitching phase, in the temporary constraint generation section of PBD, path point constraints can be added to the target particle points, such as... Figure 7 As shown, in one embodiment of this application, adding path point constraints to the target particle point may specifically include:

[0109] S41: During the suturing phase, determine whether the needle tail is located inside the soft tissue in the current frame.

[0110] If the needle tip is inside the soft tissue, proceed to S42; if the needle tip is outside the soft tissue, proceed to S43.

[0111] S42: Use the particle point in the suture that is directly connected to the needle tail as the reference point, and determine the position change of the reference point based on the position change of the needle tail.

[0112] The change in position refers to the difference between the position in the current frame and the position in the previous frame.

[0113] The particle point in the suture that is directly connected to the needle tail is the control point directly connected to the needle tail, which is referred to as the first control point in this application. The position change of the reference point can be directly calculated based on the position change of the needle tail. For example, if the needle tail and the first control point are set to overlap in space, the position change of the needle tail can be directly used as the position change of the reference point.

[0114] Jump to S44.

[0115] S43: Using the particle point in the suture that has most recently left the soft tissue as a reference point, calculate the positional change of the reference point using positional dynamics.

[0116] The particle point that most recently left the soft tissue refers to the particle point located outside the soft tissue, on the side where the suture needle exits the soft tissue, and closest to the soft tissue surface. When the needle tail just leaves the soft tissue, the reference point is still the first control point. In other cases, the reference point is not the first control point, and its position is not directly affected by the suture needle like the position of the first control point. PBD is required to calculate the change in the position of the reference point.

[0117] S44: Based on the position change of the reference point and the particle point interval, determine the target particle point in the current frame and its correspondence with the path points in the path point sequence.

[0118] Based on the change in the position of the reference point and the particle interval, it can be determined whether a new particle point has entered the soft tissue and become a target particle point, and / or whether a new target particle point has entered the soft tissue and become a non-target particle point.

[0119] The position of the first control point in the suture is directly determined by the position of the needle tail and does not participate in the PBD correction update. Even if the first control point is located inside the soft tissue, it cannot be treated as a target particle point.

[0120] When the needle tail is located inside soft tissue, the update of the path point sequence and the target particle point can be synchronized with the path points in the path point sequence, that is, a new path point and a target particle point are added in the same frame.

[0121] like Figure 8 As shown, in one embodiment of this application, S44 may specifically include:

[0122] S441: Calculate the required change in the position of the reference point in the current frame based on the change in the position of the reference point in the current frame and the remaining position of the reference point in the previous frame.

[0123] The position change of the reference point in the current frame and the position of the remaining reference point in the previous frame are both vectors. Generally, the vector sum of the two is calculated as the required position change.

[0124] S442: Determine whether the required position change of the reference point is greater than the particle point interval.

[0125] If yes, it means that new particle points have entered the soft tissue, and the target particle points and their correspondence with the path point sequence need to be updated. Jump to S443. Otherwise, it means that no new particle points have entered the soft tissue, and the target particle points and their correspondence with the path point sequence do not need to be updated. Jump to S444.

[0126] S443: Determine the number of particle points that the suture line moves into the soft tissue and the remaining reference point positions in the current frame based on the ratio of the required position change of the reference point to the particle point interval. Update the target particle points in the previous frame and their correspondence with the path points in the path point sequence based on the number of particle points that the suture line moves into the soft tissue to obtain the target particle points in the current frame and their correspondence with the path points in the path point sequence.

[0127] Generally, the comparison value is rounded down to obtain the number of particle points n in the current frame where the suture line moves into the soft tissue. The remaining reference point position quantity is in the same direction as the required position change quantity, and its size is the size of the reference point position quantity minus n*rd.

[0128] In the previous frame, the n particle points that have not yet entered the soft tissue and are closest to the soft tissue surface become the new target particle points. If the reference point in the previous frame is not the first control point, the reference point in the previous frame is moved n particle points away from the first control point to become the new reference point for subsequent use. At the same time, the oldest n target particle points in the previous frame, i.e., the earliest ones to enter the soft tissue, are no longer target particle points. That is, the oldest n are removed from the target particle points. Combined with the previously added n target particle points, the total number of target particle points remains unchanged. If the reference point in the previous frame is the first control point, the number m of other particle points (excluding the first control point) that have left the soft tissue in the current frame is counted based on the change in the required position of the reference point. If m = 0, the reference point does not need to be updated and no target particle points need to be removed. If m > 0, the reference point in the previous frame is moved m particle points away from the first control point to become the new reference point for subsequent use. At the same time, the oldest m are removed from the target particle points.

[0129] After updating the target particle points, if no new path points are added to the path point sequence in the current frame (generally when the needle tail has left the soft tissue), the index of the target particle point corresponding to each path point in the path point sequence of the previous frame is moved n in the direction away from the first control point. This is the index of the target particle point corresponding to each path point in the path point sequence of the current frame.

[0130] For example, suppose that in the previous frame, the target particle points M, M+1, ..., M+k-1 correspond to path points k, k-1, ..., 1 respectively, and the reference point M-1 is not the first control point. After calculation, it is found that N particle points move into and out of the soft tissue in the current frame. Then, the reference point in the current frame is updated to M+N-1, and the target particle points M+N, M+N+1, ..., M+N+k-1 correspond to path points k, k-1, ..., 1 respectively.

[0131] After updating the target particle points, if new path points are added to the path point sequence in the current frame (generally when the needle tail is still inside the soft tissue), the index of the target particle point corresponding to each path point in the path point sequence of the previous frame is moved n in a direction away from the first control point. The target particle points that no longer correspond to the path points in the path point sequence of the previous frame after the movement are arranged in a direction pointing towards the first control point. The newly added path points in the current frame are arranged in the order they were added to the path point sequence, and the correspondence between the two is established in turn, thereby obtaining the index of the target particle point corresponding to each path point in the path point sequence of the current frame.

[0132] For example, suppose that in the previous frame, target particle points M, M+1, ..., M+k-1 correspond to path points k, k-1, ..., 1 respectively, and reference point M-1 is the first control point. After calculation, in the current frame, N particle points have moved into the soft tissue and N new path points have been added, and reference point M-1 is still the first control point. Then, target particle points M, M+1, ..., M+N+k-1 correspond to path points k+N, k+N-1, ..., 1 respectively.

[0133] S444: Use the change in the required position of the reference point as the remaining reference point position, and use the target particle points in the previous frame and their correspondence with the path points in the path point sequence as the target particle points in the current frame and their correspondence with the path points in the path point sequence.

[0134] S45: Add path point constraints to the target particle points based on the correspondence.

[0135] Based on the correspondence, the associated path points for each target particle can be determined. Path point constraints generally constrain the position of the target particle to the position of its associated path points. Path point constraints can be written as an equation, meaning the distance between the target path point and its associated path points is 0; or as an inequality, meaning the distance between the target path point and its associated path points is less than a very small fixed value; or the distance between the target path point and its associated path points can be compared with a fixed value, and if it is less than the fixed value, no path point constraint is added; otherwise, a constraint is added that the distance between the target path point and its associated path points is 0.

[0136] In practical applications, the first control point may not be the endpoint of the suture line. In this case, the target particle points on both sides of the first control point can be updated independently, as well as their correspondence with the path point sequence. That is to say, the same path point may correspond to two different target particle points at the same time.

[0137] To ensure the real-time simulation effect during the suturing stage and reduce potential instability, in some embodiments, the collision response constraint between the suture and the soft tissue can be omitted during the suturing stage. Externally, the soft tissue is not tightened by the suture during the suturing stage; the collision response constraint between the suture and the soft tissue is then applied during the subsequent tightening stage. Of course, in other embodiments, the collision response constraint between the suture and the soft tissue can be applied during the suturing stage.

[0138] like Figure 9 As shown, in one embodiment of this application, the collision response constraint between the suture and the soft tissue may specifically include:

[0139] S46: During the tightening phase, after calculating the model parameters of the stitching line in the current frame, calculate the second distance between the target particle point and the nearest path point.

[0140] The position of the target particle point is calculated with path point constraints added. The target particle point can be determined by updating the reference point frame by frame, or by collision detection of the suture line and soft tissue. If it is the latter, particle points that only collide with the soft tissue surface need to be excluded to avoid incorrectly identifying particle points that fall on the soft tissue surface as target particle points.

[0141] S47: If the second distance is greater than the distance threshold, then add a collision response constraint to the collision tetrahedron containing the nearest path point.

[0142] The second distance is greater than the distance threshold, meaning that even with path point constraints added, the distance between the target particle point and the nearest path point is still too far. This indicates that the soft tissue has undergone significant deformation under the pull of the sutures, and a collision response is required.

[0143] Specific collision response constraints can be defined as moving the colliding tetrahedron as a whole by a threshold distance in the direction from the nearest path point to the particle point. Since the constraint functions of soft tissue include vertex-based constraint functions, the movement of the colliding tetrahedron is generally converted into the movement of each vertex of the colliding tetrahedron. For example, assuming the calculated displacement is Δp, its direction is from the nearest path point to the particle point, and its magnitude is the distance threshold, the overall displacement of the colliding tetrahedron can be achieved by adding a temporary constraint of p = p + Δp to each of the four vertices of the colliding tetrahedron. The aforementioned constraint method also limits the deformation of the colliding tetrahedron, which may lead to unevenness on the soft tissue surface. To reduce this phenomenon, the form of the temporary constraint can be modified to an inequality, such as |p-(p+Δp)|<0.1|Δp|, thereby expanding the movable range of each vertex of the colliding tetrahedron.

[0144] S48: Based on collision response constraints, use position dynamics to calculate model parameters of soft tissue in the current frame.

[0145] S49: Update the coordinates of the path points based on the model parameters of the soft tissue.

[0146] The coordinates of path points can be updated based on the centroid coordinates, also known as the center of mass coordinates or volume coordinates. Here, path points can be any path point in the path point sequence.

[0147] like Figure 10 As shown, in one embodiment of this application, S49 specifically includes:

[0148] S491: Obtain the centroid coordinates of the path point in the tetrahedron it is located before adding collision response constraints.

[0149] The centroid coordinates, also known as the center of mass coordinates or volume coordinates, are coordinates defined by the four vertices of a tetrahedron. The coordinates of any point in a tetrahedron can be written as the weighted average of the coordinates of the four vertices of the tetrahedron; these four weights are the centroid coordinates of that point.

[0150] S492: By combining the coordinates of each vertex and the centroid coordinates of the tetrahedron in the soft tissue model parameters, the updated coordinates of the path points are calculated.

[0151] Using the centroid coordinates of the path points as weights, the weighted average of the coordinates of each vertex of the tetrahedron in the soft tissue model parameters is calculated, and the result is the updated coordinates of the path points.

[0152] S5: Displays soft tissue, sutures, and surgical instruments frame by frame based on the model parameters of the soft tissue, sutures, and surgical instruments.

[0153] In each frame, after calculating the model parameters of soft tissue, sutures, and surgical instruments, the shape of soft tissue, sutures, and surgical instruments can be determined. Based on this, the model can be rendered and then output to the display device for display.

[0154] S6: Calculate the feedback force according to the feedback force strategy corresponding to the stage.

[0155] If the system does not have a feedback function, this step can be omitted.

[0156] This step only needs to be executed after S2, and there are no restrictions on the execution order between it and S3-S5.

[0157] If the stage is before the puncture stage, the feedback force is 0.

[0158] If the stage is the puncture stage where the needle penetrates into the soft tissue, then the feedback force is the product of the cumulative displacement and the first proportional coefficient.

[0159] If the stage is the puncture stage where the soft tissue has been punctured, the feedback force is the first default value.

[0160] If the stage is the puncture stage where the needle pierces the soft tissue, then the feedback force is the sum of the product of the cumulative displacement and the second proportional coefficient and the first default value.

[0161] If the stage is the puncture stage where soft tissue has been punctured, then the feedback force is the second default value.

[0162] If the stage is the stitching stage, then the feedback force is the second default value.

[0163] The feedback force calculation strategy for the tensioning phase is to calculate the feedback force based on the second default value and the displacement of at least some tetrahedrons in the soft tissue. For example, the displacement of the nearest path point before and after the update can be calculated based on the displacement of the colliding tetrahedron, and then the product of this displacement and the third proportionality coefficient, plus the second default value, is calculated as the feedback force.

[0164] This embodiment employs a unified positional dynamics-based framework to simulate soft tissue, sutures, and suture needles, effectively reducing model crossover between different models. Furthermore, during the suturing stage, the particle point intervals of the suture are used to interpolate the movement path of the suture needle tail within the soft tissue to obtain a path point sequence. Then, based on the correspondence between the path point sequence and the target particle points located within the soft tissue in the suture, path point constraints are added to the target particle points. This prevents the portion of the suture located within the soft tissue from undergoing drastic length changes under the influence of path point constraints, reducing potential distortion.

[0165] This embodiment describes the simulation of the overall suturing process. The following, in conjunction with the accompanying drawings, describes the real-time simulation process of the suture thread, suture needle, and soft tissue.

[0166] like Figure 11 As shown, the second embodiment of the soft tissue suturing simulation method provided in this application includes the following steps.

[0167] S101: Determine the target pose offset of the stitching line in the current frame and obtain the stage flag.

[0168] This embodiment describes the real-time simulation process of each frame of the suture during soft tissue suturing. Details identical or similar to those described above will not be described in detail. The suture model includes multiple particle points arranged in a chain and rod nodes connecting adjacent particle points.

[0169] The initial number of control points on the suture line and the specific location of each control point can be determined according to the actual application scenario. For example, if the suture line is not pre-fixed to the suture needle, refer to the threading operation before suturing, that is, first pass the suture line through the hole at the end of the suture needle and then clamp the threaded line at the front end of the needle holder. The simulated threaded suture line actually has 3 control points, 1 connecting to the suture needle and 2 connecting to the needle holder. If the suture needle and suture line are pre-fixed together, the part directly connected to the suture needle is the control point of the suture line.

[0170] The number and / or location of control points may vary depending on the user's actions. For example, during suturing, the user will release the needle holder after inserting the suture needle into the tissue, leaving only one control point for the threaded suture connected to the needle. During the knotting stage, the user will use forceps to hold the suture thread outside the tissue on the needle insertion side, adding a control point for connecting the forceps.

[0171] The target pose offset is the pose offset of the connection points (hereinafter referred to as connection points) directly connecting the control volume to the suture line, including the target position offset and the target attitude offset. Based on the user-provided operation commands, the poses of each connection point in the current frame can be calculated. Then, the difference between the pose in the current frame and the pose in the previous frame is calculated as the commanded pose offset. If the attitude is represented by a unit quaternion, the attitude difference refers to the angle difference between two quaternions. For example, if the attitude in the previous frame is q... n-1 The pose of the current frame is q. n Then the difference in pose between the current frame and the previous frame is The superscript -1 indicates inversion. Without considering other factors, the commanded pose offset can be directly used as the target pose offset. Of course, the target pose offset can be obtained by processing the commanded pose offset, as described in the subsequent embodiments.

[0172] Each connection point has a corresponding control point, i.e., a directly connected control point. The target pose offset of each connection point is transmitted from the control volume to the stitching line, and acts on the corresponding control point and the corresponding link node. The target position offset acts on the control point, and the target pose offset acts on the link node corresponding to the control point. For ease of description, the following explanation uses the case with 1 control point as an example, and so on, to illustrate the case with more than 1 control points.

[0173] The link node corresponding to a control point is the link node directly connected to the control point. If the control point is the endpoint of the suture line, then only one link node directly connected to the control point serves as the link node corresponding to the control point. (Combined) Figure 1 For example, if Figure 1 If particle point p1 is a control point, then the corresponding link node is q1. If the control point is not an endpoint of the suture line, then there are two link nodes directly connected to the control point. The link node corresponding to the control point can be one of these two link nodes, or both link nodes can be used as the link nodes corresponding to the control point. (Combined...) Figure 1 For example, if Figure 1 If particle point p2 is a control point, then the corresponding rod node can be at least one of q1 and q2.

[0174] The position of the control point and the pose of the corresponding link node in the current frame are determined by the target pose offset. Specifically, the sum of the control point's position in the previous frame and the target position offset is calculated as the position of the control point in the current frame, and the product of the pose of the corresponding link node in the previous frame and the target pose offset is calculated as the pose of the corresponding link node in the current frame. Throughout the constraint correction process, the position of the control point remains fixed at the calculated position in the current frame, and the pose of the corresponding link node remains fixed at the calculated pose in the current frame. Unlike other particle points and link nodes, these positions do not change. In other words, the position of the control point and the pose of the corresponding link node are not updated by the constraint function equations.

[0175] S102: Determine the temporary constraint function for the particle point based at least on the stage marker.

[0176] Stage markers are used to indicate the current stage of the soft tissue suturing process. Stages include the puncture stage, suturing stage, and tightening stage.

[0177] If the stage is marked as the suturing stage, the temporary constraint function of the target particle point located inside the soft tissue includes path point constraints based on the associated path points corresponding to the target particle point in the path point sequence. The path point sequence is obtained by interpolating the movement path of the first control point in the soft tissue using the particle point interval of the suture line. The first control point is the particle point directly connected to the needle tail of the suture needle.

[0178] like Figure 12 As shown, in one embodiment of this application, the real-time process for obtaining the path point sequence may specifically include:

[0179] S121: Get the position of the first control point in the current frame.

[0180] Throughout the suturing process, the suture thread is connected to the suture needle and moves under the influence of the needle. Therefore, the target pose offset includes the target position offset of the needle tail. Based on the target pose offset of the current frame, the position offset of the first control point in the current frame can be determined. Then, the sum of the position of the first control point in the previous frame and the position offset of the first control point in the current frame is calculated as the position of the first control point in the current frame.

[0181] S122: If the position of the first control point in the current frame is within the soft tissue, then determine whether the first distance between the position of the first control point in the current frame and the current path point in the path point sequence is within the deviation range determined based on the particle point interval.

[0182] If the first distance is within the deviation range, jump to S123; if the first distance is greater than the upper limit of the deviation range, jump to S124; if the first distance is less than the lower limit of the deviation range, jump to S125.

[0183] S123: Add the position of the first control point in the current frame as a new path point to the path point sequence.

[0184] S124: Use particle point intervals to interpolate between the current path point and the position of the first control point in the current frame, and add the interpolation result as a new path point to the path point sequence.

[0185] S125: Do not update the path point sequence.

[0186] like Figure 13 As shown, in one embodiment of this application, S102 may specifically include:

[0187] S131: If the stage marker is the suturing stage, determine whether the first control point in the current frame is located inside the soft tissue.

[0188] If the first control point is located inside the soft tissue, then jump to S132; if the first control point is located outside the soft tissue, then jump to S133.

[0189] S132: Use the first control point as a reference point, and determine the position change of the reference point based on the position change of the first control point.

[0190] Jump to S134.

[0191] S133: Using the particle point in the suture that is most recently removed from the soft tissue as a reference point, calculate the positional change of the reference point using positional dynamics.

[0192] S134: Based on the position change of the reference point and the particle point interval, determine the target particle point in the current frame and its correspondence with the path points in the path point sequence.

[0193] For details, please refer to [the relevant documentation]. Figure 8 And a description of the corresponding embodiments.

[0194] S135: Add path point constraints to the target particle points based on the correspondence.

[0195] S103: Collision detection of sutures with soft tissue and surgical instruments other than suture needles.

[0196] S104: Determine whether to update the stage flag based at least on the results of the collision detection.

[0197] There are no restrictions on the execution order of this step and S105-S106.

[0198] If the stage marker is puncture stage and a collision between the first control point and soft tissue is detected, the stage marker is updated to suture stage.

[0199] If the stage marker is suture stage and a pre-defined knot in the suture is detected colliding with the surface of the soft tissue, the stage marker is updated to tension stage.

[0200] If the stage marker is suturing stage and a collision between the suture and the second clamp is detected, the stage marker is updated to tension stage. The second clamp is a surgical instrument that is different from the suture needle and the first clamp holding the suture needle.

[0201] S105: Use position dynamics-based calculations to obtain the model parameters of the stitching line in the current frame according to the target pose offset.

[0202] The constraint functions for particle points include temporary constraint functions.

[0203] S106: Display the suture line according to the model parameters of the suture line in the current frame.

[0204] like Figure 14 As shown, the third embodiment of the soft tissue suturing simulation method provided in this application includes the following steps.

[0205] S201: Calculate the model parameters of the suture needle in the current frame.

[0206] This embodiment is a real-time simulation of the suture needle process for each frame during soft tissue suturing. The suture needle model includes multiple particles arranged according to their shape.

[0207] like Figure 15 As shown, in one embodiment of this application, the suture needle is treated as a rigid body, and S201 may specifically include:

[0208] S211: Determine whether the suture needle is in a clamped state in the current frame.

[0209] If the suture needle is in a clamped state, jump to S212; otherwise, jump to S213.

[0210] S212: Obtain the pose of the first clamping body holding the suture needle, and calculate the model parameters of the suture needle in the current frame using kinematic equations based on the pose of the first clamping body.

[0211] S213: Determine the overall collision constraints of the suture needle based on the collision detection results between the suture needle and soft tissue, and calculate the model parameters of the suture needle in the current frame based on the overall collision constraints using position dynamics.

[0212] If the elasticity of the suture needle is taken into account, it cannot be treated as a rigid body. Regardless of whether it is held by other surgical instruments, the position of each mass point in the PBD / XPBD process calculation model must be used.

[0213] S202: Acquire phase marker.

[0214] Stage markers are used to indicate the current stage of the soft tissue suturing process. Stages include the puncture stage, suturing stage, and tightening stage.

[0215] S203: If the stage is marked as the suture stage, the particle point interval of the suture line is used to interpolate the movement path of the needle tail in the soft tissue to determine whether the path point sequence should be updated.

[0216] like Figure 16 As shown, in one embodiment of this application, S203 may specifically include:

[0217] S231: If the stage marker is the stitching stage, determine the position of the needle tail in the current frame based on the model parameters of the stitch needle in the current frame.

[0218] S232: If the position of the needle tail in the current frame is within the soft tissue, then determine whether the first distance between the position of the needle tail in the current frame and the current path point in the path point sequence is within the deviation range determined based on the particle point interval.

[0219] If the first distance is within the deviation range, jump to S233; if the first distance is greater than the upper limit of the deviation range, jump to S234; if the first distance is less than the lower limit of the deviation range, jump to S235.

[0220] S233: Add the position of the needle tail in the current frame as a new path point to the path point sequence.

[0221] S234: Use particle point intervals to interpolate between the current path point and the position of the needle tail in the current frame, and add the interpolation result as a new path point to the path point sequence.

[0222] S235: Do not update the path point sequence.

[0223] S204: Display the suture needle according to the model parameters of the suture needle in the current frame.

[0224] S205: Determine whether to update the stage marker based at least on the results of collision detection between soft tissue and suture needle.

[0225] If PBD / XPBD is used to calculate the model parameters of the suture needle, the collision detection between the soft tissue and the suture needle is generally performed before or during S201. This step only needs to be performed after the collision detection between the soft tissue and the suture needle, and there is no restriction on the order between it and other steps.

[0226] If the stage marker is in the puncture stage and the needle tip of the suture needle is detected to collide with the surface of the soft tissue, then the stage marker is updated to the suturing stage.

[0227] The puncture process can be divided into several sub-stages based on the direction of needle insertion and whether the soft tissue surface is punctured. There are several ways to determine the puncture direction; the following explanation uses historical records of stage markers as an example.

[0228] like Figure 17 As shown in one embodiment of this application, the sub-stage of determining the puncture stage may specifically include:

[0229] S261: Determine whether a collision between the tip of a suture needle and the surface of soft tissue is detected in the current frame.

[0230] If yes, then jump to S262; otherwise, jump to S269.

[0231] S262: Increment the cumulative frame count by 1 and temporarily store the collision response displacement of the current frame.

[0232] The collision response displacement is the position of the puncture point that collides with the needle in the soft tissue model parameters of the current frame minus the position of the puncture point in the soft tissue model parameters of the previous frame.

[0233] S263: If the cumulative number of frames is greater than the first threshold, then the cumulative displacement is obtained by summing all the temporarily stored collision response displacements.

[0234] S264: Compare the cumulative displacement with a second threshold to determine whether the needle has pierced the surface of soft tissue.

[0235] Specifically, if the cumulative displacement in the current frame is greater than the second threshold, it is determined that the needle has pierced the surface of the soft tissue; otherwise, it is determined that the needle has not yet pierced the surface of the soft tissue. Alternatively, if the cumulative displacement in the current frame is greater than or equal to the second threshold, it is determined that the needle has pierced the surface of the soft tissue; otherwise, it is determined that the needle has not yet pierced the surface of the soft tissue.

[0236] S265: If the needle has not yet pierced the surface of the soft tissue and the stage marker is empty, update the stage marker to "pierced into the soft tissue" in the puncture stage.

[0237] S266: If the needle has pierced the surface of the soft tissue and the stage marker is empty or in the puncture stage, then update the stage marker to "pierced soft tissue" in the puncture stage.

[0238] S267: If the needle has not yet pierced the surface of the soft tissue, and the stage marker is "pierced into soft tissue" in the puncture stage, then update the stage marker to "pierced out of soft tissue" in the puncture stage.

[0239] S268: If the needle has pierced the surface of the soft tissue, and the stage marker is either "pierced into the soft tissue" or "pierced out of the soft tissue" in the puncture stage, then update the stage marker to "pierced out of the soft tissue" in the puncture stage.

[0240] S269: Set the cumulative frame count to 0 and clear all temporary collision response displacements.

[0241] like Figure 18 As shown, the fourth embodiment of the soft tissue suturing simulation method provided in this application includes the following steps.

[0242] S301: Obtain the stage flag in the current frame.

[0243] This embodiment describes the real-time simulation process of each frame of soft tissue during soft tissue suturing. Stage markers are used to indicate the current stage of the soft tissue suturing process, including the puncture stage, suturing stage, and tensioning stage.

[0244] S302: Collision detection of soft tissue with sutures and surgical instruments.

[0245] The soft tissue model is a mesh model consisting of multiple vertices and tetrahedrons defined by those vertices.

[0246] S303: Determine the temporary constraint function for the vertex based on the results of the collision detection and / or the stage flag.

[0247] If a collision is detected between the surface of soft tissue and the tip of a suture needle in the surgical instrument, a first collision response constraint is added to the puncture point where the collision occurred. This temporary constraint function is primarily applied during the puncture phase.

[0248] The second collision response constraint function mainly applies during the tensioning phase. For example... Figure 19 As shown, in one embodiment of this application, the specific process of adding a second collision response constraint function may include:

[0249] S331: If the stage is marked as the tightening stage, after calculating the model parameters of the suture in the current frame, calculate the second distance between the target particle point located inside the soft tissue and the nearest path point in the path point sequence.

[0250] The path point sequence is obtained by interpolating the movement path of the suture needle tail within the soft tissue using the particle point spacing of the suture line.

[0251] S332: If the second distance is greater than the distance threshold, then add a second collision response constraint to the collision tetrahedron containing the nearest path point.

[0252] The collision tetrahedron is moved a total distance threshold in the direction from the nearest path point to the target particle point.

[0253] S304: Use position dynamics-based calculations to obtain model parameters of soft tissue in the current frame.

[0254] The constraint functions for the vertices include temporary constraint functions. If the stage is marked as the tightening stage, the coordinates of each path point in the path point sequence can be updated based on the model parameters of the soft tissue in the current frame. Specifically, for each path point, the centroid coordinates of the path point in its tetrahedron before the addition of the second collision response constraint can be obtained. Then, combined with the coordinates of each vertex and the centroid coordinates of the tetrahedron in the soft tissue model parameters, the updated coordinates of the path point can be calculated.

[0255] S305: Determine whether to update the stage marker based at least on the results of the collision detection.

[0256] This step only needs to be executed after S302, and there are no restrictions on the execution order between it and S303 and subsequent steps.

[0257] If the stage marker is in the puncture stage and the needle tip of the suture needle is detected to collide with the surface of the soft tissue, then the stage marker is updated to the suturing stage.

[0258] Based on the collision detection results between the soft tissue surface and the needle, sub-stages within the puncture process can be distinguished. For details, please refer to... Figure 17 And corresponding implementation examples.

[0259] S306: Display soft tissue based on the model parameters of soft tissue in the current frame.

[0260] Please see Figure 20 As shown, the first embodiment of the soft tissue suturing simulation device provided in this application includes: an acquisition module 101, a detection module 102, an interpolation module 103, a calculation module 104, and a display module 105.

[0261] The acquisition module 101 is used to acquire models of soft tissue, sutures, and surgical instruments. The soft tissue model is a mesh model including multiple vertices and tetrahedrons defined by the vertices; the suture model includes multiple particle points arranged in a chain and rod nodes connecting adjacent particle points; the surgical instrument model includes multiple particles arranged according to their shape; and the surgical instrument includes a suture needle.

[0262] The detection module 102 is used to perform collision detection between the soft tissue, the suture, and the surgical instruments, and to determine the stage in the suturing process based at least on the result of the collision detection. The stage includes the puncture stage, the suturing stage, and the tightening stage.

[0263] The interpolation module 103 is used to interpolate the movement path of the needle tail of the suture needle within the soft tissue using the particle point spacing of the suture line during the suturing stage to obtain a path point sequence.

[0264] The calculation module 104 is used to calculate the model parameters of the soft tissue and suture frame by frame using position dynamics, and to calculate the model parameters of the surgical instrument frame by frame using position dynamics and / or kinematic equations, wherein the constraint function of the target particle point includes path point constraints based on the associated path points in the path point sequence corresponding to the target particle point, and the target particle point is the particle point of the suture located inside the soft tissue.

[0265] The display module 105 is used to display the soft tissue, sutures, and surgical instruments frame by frame according to the model parameters of the soft tissue, sutures, and surgical instruments.

[0266] Please see Figure 21 As shown, the second embodiment of the soft tissue suturing simulation device provided in this application includes: an acquisition module 201, a temporary constraint module 202, a calculation module 203, and a display module 204.

[0267] The acquisition module 201 is used to determine the target pose offset of the suture line in the current frame and acquire the stage flag. The suture line model includes multiple particle points arranged in a chain and rod nodes connecting adjacent particle points.

[0268] The temporary constraint module 202 is used to determine a temporary constraint function for the particle point based at least on a stage marker. The stage marker indicates the stage the current frame is in during soft tissue suturing, including a puncture stage, a suturing stage, and a tightening stage. If the stage marker is the suturing stage, the temporary constraint function for the target particle point located within the soft tissue includes path point constraints based on associated path points corresponding to the target particle point in a path point sequence. This path point sequence is obtained by interpolating the movement path of a first control point within the soft tissue using the particle point interval of the suture line. The first control point is a particle point directly connected to the needle tail of the suture needle.

[0269] The calculation module 203 is used to calculate the model parameters of the stitching line in the current frame based on the target pose offset using position dynamics. The constraint function of the particle points includes the temporary constraint function.

[0270] The display module 204 is used to display the suture line according to the model parameters of the suture line in the current frame.

[0271] Please see Figure 22 As shown, the third embodiment of the soft tissue suturing simulation device provided in this application includes: a calculation module 301, an acquisition module 302, an interpolation module 303, and a display module 304.

[0272] The calculation module 301 is used to calculate the model parameters of the suture needle in the current frame, wherein the model of the suture needle includes multiple mass points arranged according to their shape.

[0273] The acquisition module 302 is used to acquire a stage marker, which indicates the stage of the current frame in the soft tissue suturing process. The stage includes the puncture stage, the suturing stage, and the tightening stage.

[0274] The interpolation module 303 is used to interpolate the movement path of the suture needle tail in the soft tissue using the particle point interval of the suture line when the stage is marked as the suturing stage, in order to determine whether to update the path point sequence.

[0275] The display module 304 is used to display the suture needle according to the model parameters of the suture needle in the current frame.

[0276] Please see Figure 23 As shown, the fourth embodiment of the soft tissue suturing simulation device provided in this application includes: an acquisition module 401, a detection module 402, a temporary constraint module 403, a calculation module 404, an update module 405, and a display module 406.

[0277] The acquisition module 401 is used to acquire a stage flag in the current frame. The stage flag is used to indicate the stage in which the current frame is in the soft tissue suturing process. The stage includes the puncture stage, the suturing stage, and the tightening stage.

[0278] The detection module 402 is used to perform collision detection on the soft tissue and sutures and surgical instruments. The model of the soft tissue is a mesh model including multiple vertices and tetrahedrons defined by the vertices.

[0279] The temporary constraint module 403 is used to determine the temporary constraint function of the vertex based on the result of the collision detection and / or the stage flag.

[0280] The calculation module 404 is used to calculate the model parameters of the soft tissue in the current frame using position dynamics, wherein the constraint function of the vertex includes the temporary constraint function.

[0281] The update module 405 is used to determine whether to update the stage flag, at least based on the result of the collision detection.

[0282] The display module 406 is used to display the soft tissue according to the model parameters of the soft tissue in the current frame.

[0283] It should be noted that the soft tissue suturing simulation device provided in the above embodiments is only illustrated by the division of the above-described program modules in the process of implementing the soft tissue suturing simulation method. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the method steps described above. In addition, the soft tissue suturing simulation device and the soft tissue suturing simulation method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0284] See Figure 24 The diagram shown is a schematic representation of a soft tissue suturing simulation device provided according to an embodiment of this application. The object simulation device provided in this application includes a memory 502 and a processor 501. The memory 502 stores a computer program executable by the processor 501. When the computer program is executed by the processor 501, it implements the soft tissue suturing simulation method as described in any embodiment of this application.

[0285] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described soft tissue suturing simulation method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium includes, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0286] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for simulating soft tissue suturing, characterized in that, include: Calculate the model parameters of the suture needle in the current frame, wherein the model of the suture needle includes multiple particles arranged according to its shape; Obtain a stage marker, which is used to indicate the stage of the current frame in the soft tissue suturing process, the stage including the puncture stage, the suturing stage and the tightening stage; If the stage is marked as the suturing stage, the movement path of the needle tail in the soft tissue is interpolated using the particle point interval of the suture to determine whether to update the path point sequence. The model of the suture includes multiple particle points arranged in a chain and rod nodes connected to adjacent particle points. The suture needle is displayed according to the model parameters of the suture needle in the current frame; The step of interpolating the movement path of the suture needle tail within the soft tissue using the particle point spacing of the suture thread to determine whether to update the path point sequence includes: The position of the suture needle tail in the current frame is determined based on the model parameters of the suture needle in the current frame; Determine whether the position of the needle tail in the current frame is within the soft tissue; If so, determine whether the first distance between the position of the needle tail in the current frame and the current path point in the path point sequence is within the deviation range determined based on the particle point interval; If the first distance is within the deviation range, then the position of the needle tail in the current frame is added to the path point sequence as a new path point; If the first distance is greater than the upper limit of the deviation interval, then the particle point interval is used to interpolate between the current path point and the position of the needle tail in the current frame, and the interpolation result is added to the path point sequence as a new path point. If the first distance is less than the lower limit of the deviation interval, the path point sequence is not updated.

2. The soft tissue suturing simulation method as described in claim 1, characterized in that, The model parameters for calculating the suture needle in the current frame include: Determine whether the suture needle is in a clamped state in the current frame; If the suture needle is in a clamped state, obtain the pose of the first clamping body that clamps the suture needle, and calculate the model parameters of the suture needle in the current frame using kinematic equations based on the pose of the first clamping body. If the suture needle is not in a clamped state, the overall collision constraint of the suture needle is determined based on the collision detection result between the suture needle and the soft tissue, and the model parameters of the suture needle in the current frame are calculated based on the overall collision constraint using position dynamics.

3. The soft tissue suturing simulation method as described in claim 1, characterized in that, The model parameters for calculating the suture needle in the current frame include: The model parameters of the suture needle in the current frame are calculated using position dynamics.

4. The soft tissue suturing simulation method as described in claim 1, characterized in that, Also includes: Collision detection is performed between the soft tissue and the suture needle; At least based on the results of the collision detection, determine whether to update the stage flag.

5. The soft tissue suturing simulation method as described in claim 4, characterized in that, The step of determining whether to update the stage flag based at least on the result of the collision detection includes: If a collision is detected between the needle tip of the suture needle and the surface of the soft tissue, the cumulative frame count is incremented by 1, and the collision response displacement of the current frame is temporarily stored. The collision response displacement is the position of the puncture point that collides with the needle tip in the model parameters of the soft tissue in the current frame minus the position of the puncture point in the model parameters of the soft tissue in the previous frame. Otherwise, the cumulative frame count is set to 0, and all temporarily stored collision response displacements are cleared. If the cumulative number of frames is greater than the first threshold, then all temporarily stored collision response displacements are summed to obtain the cumulative displacement amount. The cumulative displacement is compared with a second threshold to determine whether the needle has pierced the surface of the soft tissue. If the needle has not yet pierced the surface of the soft tissue and the stage marker is empty, then the stage marker is updated to "pierced into the soft tissue" in the puncture stage. If the needle has pierced the surface of the soft tissue, and the stage marker is empty or the needle has penetrated into the soft tissue during the puncture stage, then the stage marker is updated to indicate that the needle has penetrated into the soft tissue during the puncture stage. If the needle has not yet pierced the surface of the soft tissue, and the stage marker is "pierced into the soft tissue" in the puncture stage, then the stage marker is updated to "pierced out of the soft tissue" in the puncture stage. If the needle has pierced the surface of the soft tissue, and the stage marker is either inserted into or exited the soft tissue during the puncture stage, then the stage marker is updated to "exited from the soft tissue" during the puncture stage.

6. The soft tissue suturing simulation method as described in claim 5, characterized in that, The step of comparing the cumulative displacement with a second threshold to determine whether the needle has pierced the surface of the soft tissue includes: If the cumulative displacement in the current frame is greater than the second threshold, it is determined that the needle has pierced the surface of the soft tissue; otherwise, it is determined that the needle has not yet pierced the surface of the soft tissue.

7. The soft tissue suturing simulation method as described in claim 4, characterized in that, The step of determining whether to update the stage flag based at least on the result of the collision detection includes: If the stage marker is the puncture stage where the needle has penetrated into the soft tissue and a collision between the needle tip and the surface of the soft tissue is detected, then the stage marker is updated to the suturing stage.

8. A soft tissue suturing simulation device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the soft tissue suturing simulation method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the soft tissue suturing simulation method as described in any one of claims 1 to 7.

Images