Soft tissue suture simulation method, device and medium

The soft tissue suturing simulation method addresses model penetration issues by using a particle point chain and rod nodes model with constraint functions, achieving real-time, accurate suturing simulations with reduced distortion.

CN119055356AActive Publication Date: 2024-12-03SHENZHEN JINGFENG MEDICAL TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202310648260.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-01
Publication Date
2024-12-03
Estimated Expiration
2043-06-01

AI Technical Summary

Technical Problem

Existing surgical simulation methods face complexity due to the use of different models for sutures, soft tissues, and surgical instruments, leading to model penetration and increased simulation complexity, especially when dealing with more morphological organs and tissues.

Method used

A soft tissue suturing simulation method using a particle point chain and rod nodes model, with temporary and regular constraint functions, interpolates the movement path of a suture line and adds path point constraints to reduce distortion during the suturing process.

Benefits of technology

The method simulates soft tissue suturing in real time with high accuracy by minimizing drastic length changes and reducing model distortion, enhancing the realism and efficiency of surgical simulations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119055356A_ABST
    Figure CN119055356A_ABST
Patent Text Reader

Abstract

The invention provides a soft tissue suture simulation method, which comprises the following steps: determining a target pose offset of a suture line in a current frame, and obtaining a stage mark, a model of the suture line comprising a plurality of particle points arranged in a chain shape and rod nodes connected between adjacent particle points; a temporary constraint function of the particle point is determined at least according to a stage mark, the stage mark is used for representing the stage of the current frame in the soft tissue suturing process, and the stage comprises a puncture stage, a suturing stage and a tensioning stage; the temporary constraint function of a target particle point located in the soft tissue comprises a waypoint constraint based on an associated waypoint corresponding to the target particle point in a waypoint sequence, and the waypoint sequence is obtained by interpolating a moving path of a first control point in the soft tissue by using a particle point interval of a suture line; the first control point is a particle point directly connected with the needle tail of the suture needle; according to the target pose offset, performing calculation based on position dynamics to obtain model parameters of a suture line in the current frame, wherein constraint functions of particle points comprise temporary constraint functions; and displaying the suture line according to the model parameters of the suture line in the current frame. The invention further provides soft tissue suturing simulation equipment and a storage medium.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of computer graphics, and in particular to a soft tissue suturing simulation method, device and computer-readable storage medium. Background Art

[0002] Suturing is an essential part of surgical simulation. The suturing process involves many aspects, such as simulation of sutures, simulation of soft tissues, simulation of surgical instruments such as suture needles, surgical forceps, tweezers, etc., and also includes the interaction between the above entities, such as the surgical forceps holding the suture needle to pierce the surface of the soft tissue and enter the soft tissue, the suture needle drives the suture thread through the soft tissue, the suture thread pulls the soft tissue together, and the suture thread is finally tied to fix the soft tissue.

[0003] Traditional methods mostly use the Follow The Leader (FTL) method to simulate sutures, the Mass-Spring Model (MSM) to simulate soft tissues, and the particle model to simulate surgical instruments. Since the three use different models, there is inevitably a certain degree of model penetration when dealing with the interaction between them, and this method is not convenient for extended simulation of more morphological organs and tissues, making the entire surgical simulation composed of multiple different frameworks, which increases the complexity of the simulation. Summary of the invention

[0004] In order to solve the existing technical problems, the present application provides a soft tissue suturing simulation method, device and computer-readable storage medium that can simulate the soft tissue suturing process in real time and accurately.

[0005] In a first aspect of an embodiment of the present application, a soft tissue suturing simulation method is provided, comprising: determining a target pose offset of a suture line in a current frame, and obtaining a stage mark, wherein a model of the suture line comprises a plurality of particle points arranged in a chain and rod nodes connected between adjacent particle points; determining a temporary constraint function of the particle point at least according to the stage mark, wherein the stage mark is used to indicate the stage in which the current frame is located in the soft tissue suturing process, wherein the stage comprises a puncture stage, a suturing stage, and a tensioning stage, wherein if the stage mark is a suturing stage, the temporary constraint function of the target particle point located inside the soft tissue comprises a path point constraint based on an associated path point in a path point sequence corresponding to the target particle point, wherein the path point sequence is obtained by interpolating a moving path of a first control point in the soft tissue using a particle point interval of the suture line, wherein the first control point is a particle point directly connected to the needle tail of the suture needle; obtaining model parameters of the suture line in the current frame using a position dynamics-based calculation according to the target pose offset, wherein the constraint function of the particle point comprises a temporary constraint function; and displaying the suture line according to the model parameters of the suture line in the current frame.

[0006] According to a second aspect of an embodiment of the present application, a soft tissue suturing simulation device is provided, including a memory and a processor. A computer program is stored in the memory. When the computer program is executed by the processor, a soft tissue suturing simulation method as in any embodiment of the present application is implemented.

[0007] According to a third aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, a soft tissue suturing simulation method as in any embodiment of the present application is implemented.

[0008] In the above embodiment, the soft tissue suturing simulation method uses the particle point interval of the suture line to interpolate the movement path of the first control point in the soft tissue to obtain a path point sequence during the suturing stage, and then adds path point constraints to the target particle points based on the correspondence between the path point sequence and the target particle points in the suture line located inside the soft tissue, so that the part of the suture line located inside the soft tissue will not produce drastic length changes under the action of the path point constraint, thereby reducing the distortion that may be caused by this.

[0009] In the above embodiments, the soft tissue suturing simulation device and the computer-readable storage medium respectively belong to the same concept as the corresponding soft tissue suturing simulation method embodiments, and thus have the same technical effects as the corresponding soft tissue suturing simulation method embodiments, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of a model of a suture line with an elastic rod introduced;

[0011] Figure 2 Flowchart for introducing position-based dynamics of elastic rods;

[0012] Figure 3 Schematic diagram of the various stages of the soft tissue suturing process;

[0013] Figure 4 This is a schematic diagram of the process of the first embodiment of the soft tissue suturing simulation method of the present application;

[0014] Figure 5 This is a schematic diagram of a process for determining whether a needle has pierced the surface of a soft tissue in an embodiment of the present application;

[0015] Figure 6 for Figure 4 The specific process diagram of S3 in the figure;

[0016] Figure 7 A schematic diagram of a process of adding a path point constraint to a target particle point in an embodiment of the present application;

[0017] Figure 8 for Figure 7The specific process diagram of S44;

[0018] Fig. 9 It is a schematic diagram of a process of constraining the collision response between a suture line and soft tissue in one embodiment of the present application;

[0019] Fig.10 for Fig. 9 The specific process diagram of S49;

[0020] Fig.11 This is a schematic diagram of the flow chart of the second embodiment of the soft tissue suturing simulation method of the present application;

[0021] Fig.12 A schematic diagram of a real-time process of obtaining a path point sequence in an embodiment of the present application;

[0022] Fig.13 for Fig.11 The specific process diagram of S102;

[0023] Fig.14 This is a schematic diagram of the flow chart of the third embodiment of the soft tissue suturing simulation method of the present application;

[0024] Fig.15 For the case where the suture needle is treated as a rigid body Fig.14 The specific process diagram of S201;

[0025] Fig.16 for Fig.14 The specific process diagram of S203;

[0026] Fig.17 This is a flow chart of determining the sub-stages of the puncture stage in one embodiment of the present application;

[0027] Fig.18 This is a schematic flow chart of a fourth embodiment of the soft tissue suturing simulation method of the present application;

[0028] Fig.19 A flowchart of adding a second collision response constraint function in an embodiment of the present application;

[0029] Fig. 20 This is a schematic structural diagram of the first embodiment of the soft tissue suturing simulation device of the present application;

[0030] Fig.21 This is a schematic structural diagram of a second embodiment of the soft tissue suturing simulation device of the present application;

[0031] Fig. 22 This is a schematic structural diagram of a third embodiment of the soft tissue suturing simulation device of the present application;

[0032] Fig.23This is a schematic structural diagram of a fourth embodiment of the soft tissue suturing simulation device of the present application;

[0033] Fig.24 This is a schematic structural diagram of an embodiment of a soft tissue suturing simulation device of the present application. DETAILED DESCRIPTION

[0034] The technical solution of the present application is further elaborated in detail below in conjunction with the accompanying drawings and specific embodiments of the specification.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the present application. The terms used herein in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the implementation of the present application. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0036] The various embodiments of the present application can be combined with each other without conflict. In the following description, the expression "some embodiments" is involved, which describes a subset of possible embodiments, but it should be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0037] The order of steps given in the embodiments of the present application is for illustration only. If the steps can be performed in other orders to achieve the same effect, these orders are also within the scope of protection of the present application.

[0038] In the field of computer graphics, the common physical method for real-time simulation of object deformation generally updates the acceleration through external forces first, and then integrates to obtain the velocity and position. However, PBD directly corrects the position through the constraint function, and then updates the velocity through the position. The system equations in the PBD algorithm are constructed by the constraint function equation group, which includes multiple constraint function equations (which can be referred to as constraint functions).

[0039] In PBD, the model of an object includes multiple particle points that make up the object, also called mass points. The parameters of the particle points include their position, velocity and mass. The parameters of the constraint function include the position of these particle points. The constraint function is used to describe the relationship between the particle points that make up the object.

[0040] The PBD algorithm mainly includes the initial stage and the real-time simulation stage. In the initial stage, the parameters of all particle points and rod nodes that make up the object are initialized according to the preset object parameters, such as the initial position, initial posture, number of particle points, etc. Then it enters the real-time simulation stage, which executes the process of simulating the deformation of the object according to the environment (including possible operation instructions from the user) in a loop in frames, which can specifically include three parts: prediction, constraint correction and variable update.

[0041] In the prediction part of the current frame, the external environment force of the current frame is converted into displacement as much as possible, and the external force that cannot be directly converted into displacement (such as gravity) is retained. Based on the position of each particle point in the previous frame, the position of each particle point in the current frame is predicted by combining the displacement obtained by conversion and the external force that cannot be directly converted.

[0042] The above prediction part often only uses integrals (such as explicit Euler integrals) to directly predict the position of each particle point in the current frame, without considering the mechanical properties of the object and the interaction with the environment. Therefore, it cannot be directly used as the simulation result of the current frame, but the position of the particle point must be corrected using the constraint function. Constraint functions are used to limit the relationship between particle points. There are two main types of constraint functions: one is a temporary constraint function, which is generally generated temporarily when the object is detected to interact with external objects in the environment; the other is a conventional constraint function, which is determined by the mechanical properties of the object. The process of calculating the corrected position that satisfies the constraint function as much as possible based on the constraint function and the current position is called constraint projection, which is the core process of PBD. Constraint correction is essentially an iterative optimization of the position of the particle point based on the constraint function equation group. The specific implementation is generally to iteratively execute the process of using the constraint functions in the constraint function equation group to perform constraint projection in turn until the set number of iterations is reached, so as to find the particle point position that satisfies all the constraint functions in the constraint function equation group as much as possible.

[0043] The result obtained after constraint correction is the position of each particle point in the current frame, based on which the speed of each particle point in the current frame can be calculated, and then the corresponding variables are updated for display and calculation of the next frame. The Δt used in PBD is generally the default frame spacing, that is, the duration between two adjacent frames under normal circumstances.

[0044] For soft tissue, its model is a mesh model including multiple vertices and tetrahedrons defined by the vertices. The vertices here are similar to particle points, and its constraint functions mainly include the distance constraint between adjacent vertices and the volume invariance constraint of each tetrahedron.

[0045] For surgical instruments, such as surgical forceps, needle holders, tweezers, suture needles, etc., they can be approximated as rigid bodies, and their models generally include multiple mass points / particle points determined according to their specific shapes. For example, the model of a suture needle may include multiple mass points arranged in a chain. Since surgical instruments other than suture needles are generally under the direct control of the user, the PBD process can be omitted and the kinematic equations can be directly used to calculate their posture and the position of each mass point in the model.

[0046] For suture lines, in order to more accurately describe their shape, rod nodes, also called elastic rods (rods), are further introduced. Figure 1 As shown, the model of the suture line includes a plurality of particle points arranged in a chain (e.g. Figure 1 p1,p2,…,p n ) and rod nodes connecting adjacent particle points (e.g. Figure 1 q1,q2,…,q n-1 ). The extension direction in the figure is only used to specify the numbering order of the particle points and the rod nodes, and does not represent the actual extension direction. The parameters of the rod nodes include their posture, angular velocity and moment of inertia. Correspondingly, the parameters of the constraint function also include the posture of these rod nodes. Among them, for the convenience of calculation, the posture is generally represented by a unit quaternion. Of course, in other embodiments, the posture can also be represented by a transformation / rotation matrix.

[0047] In practical applications, sutures are often connected to an external object that controls the movement of the sutures, and can also be called a control body. For example, if the suture is a suture, then the control body is a suture needle, surgical forceps, tweezers, etc. that directly drives the movement of the suture; if the suture is hair, the control body is the skin and hair accessories that fix the hair, and objects that directly drive the movement of the hair, such as fingers and combs. The particle point in the suture that is directly connected to the control body, that is, the particle point closest to the control body, can be called a control point.

[0048] The process of introducing PBD with elastic rod is as follows Figure 2 As shown, the PBD process is described in detail below with reference to the accompanying drawings.

[0049] Figure 2 Steps 1-4 in the figure are the initial stage, in which the parameters of all the particle points and rod nodes that make up the object are initialized according to the preset object parameters, such as the initial position, initial posture, number of particle points, etc. Then the real-time simulation stage is entered, and the process of simulating the deformation of the object according to the environment (including possible operation instructions from the user) is executed cyclically in frames, that is, it is executed once for each frame. Figure 2 Steps 5-21 in the above process may specifically include three parts: prediction, constraint modification and variable update.

[0050] In the prediction part of the current frame (corresponding to Figure 2 In steps 6-12), the effect of the force of the external environment on the object in the current frame is converted into displacement as much as possible, and the external force that cannot be directly converted into displacement (such as gravity) is retained; the effect of the torque of the external environment on the object in the current frame is converted into angular displacement as much as possible, and the external torque that cannot be directly converted into angular displacement is retained. Based on the position of each particle point in the previous frame, the position of each particle point in the current frame is predicted in combination with the displacement obtained by conversion and the external force that cannot be directly converted; based on the posture of each rod node in the previous frame, the posture of each rod node in the current frame is predicted in combination with the angular displacement obtained by conversion and the external torque that cannot be directly converted.

[0051] The above prediction part often only uses integrals (such as explicit Euler integrals) to directly predict the position of each particle point and the posture of each rod node in the current frame, without considering the mechanical properties of the object and the interaction with the environment. Therefore, it cannot be directly used as the simulation result of the current frame, but the position of the particle point and the posture of the rod node must be corrected using constraint functions. Constraint functions are used to limit the relationship between particle points, rod nodes, and particle points and rod nodes. There are two main types: one is a temporary constraint function, which is generally generated temporarily when the object is detected to interact with external objects in the environment; the other is a regular constraint function, which is determined by the mechanical properties of the object. For example, for sutures and similar linear objects, the corresponding regular constraint function can be given according to their mechanical properties that are not easy to stretch and shear, but easy to bend and twist. The process of calculating the corrected position / posture that satisfies the constraint function as much as possible based on the constraint function and the current position / posture is called constraint projection, which is the core process of PBD. Constraint correction is essentially an iterative optimization of the particle point position and the rod node posture based on the constraint function equation group. The specific implementation is generally to iteratively execute the process of using the constraint functions in the constraint function equation group to perform constraint projection in turn until the set number of iterations is reached, so as to find the particle point position and rod node posture that satisfies all the constraint functions in the constraint function equation group as much as possible. The constraint correction part corresponds to Figure 2 Steps 13-15 in which step 13 is used to perform collision detection to generate a temporary constraint function, and steps 14-15 are to iteratively optimize the positions of the particle points and the postures of the rod nodes using the constraint function equations.

[0052] Update some corresponding Figure 2 In steps 16-21 of the PBD, the result after constraint correction is the position of each particle point and the posture of each rod node in the current frame. Based on this, the velocity of each particle point and the angular velocity of each rod node in the current frame can be calculated, and then the corresponding variables are updated for display and calculation of the next frame. The Δt used in PBD is generally the default frame spacing, that is, the duration between two adjacent frames under normal circumstances.

[0053] After executing the simulated object deformation process of the current frame, the model parameters of the current frame can be obtained, including the parameters of the updated particle points and rod nodes, wherein the updated positions of each particle point and the postures of each rod node can be used to determine the shape of the stitching line of the current frame for display.

[0054] It should be noted that the vertices, particle points and mass points in this application are actually equivalent in the PBD framework. In order to simplify the description, different names are selected for the particle points in different models, that is, the particle points in soft tissue are called vertices, the particle points in surgical instruments are called mass points, and the particle points in sutures are called particle points.

[0055] Combined with the above preliminary introduction to the soft tissue suturing simulation method, Figures 3 to 19 The soft tissue suturing simulation method provided by the embodiment of the present application is further described in detail. Since object simulation is a real-time process, when the current frame is mentioned below, it means to describe the specific process of object simulation within a frame by taking the current frame as an example. In fact, the process must be executed for each frame.

[0056] See also Figure 3 ,The soft tissue suturing process mainly includes the following stages: puncture stage, suturing stage, tensioning stage and knotting stage.

[0057] From the perspective of user operation, during the puncture stage, the user uses a first clamp (generally surgical forceps, needle holder, etc.) to clamp the middle and rear part of the suture needle and insert it into the soft tissue. After the needle tip of the suture thread penetrates the soft tissue, it passes through the soft tissue until the needle tip of the suture thread pierces the soft tissue. The user then moves the first clamp to the front of the suture needle and continues to pull the suture needle until the needle tail of the suture needle (i.e., the particle in the suture needle that is directly connected to the suture thread) enters the soft tissue.

[0058] The suturing stage generally starts when the tail of the suture needle drives the suture into the soft tissue and ends when the tension mark appears. There are two main tension marks. One is that there is a pre-tied knot at the tail of the suture. In this case, the tension mark is that the pre-tied knot contacts the surface of the soft tissue; the other is that there is no pre-tied knot. In this case, the tension mark is that the user uses a second clamp (another surgical instrument different from the first clamp, such as forceps, etc.) to directly clamp the tail of the suture.

[0059] The tensioning stage generally starts from the appearance of the tensioning mark and ends when the suture thread collides with itself under the drive of the first clamping body and the second clamping body.

[0060] The knotting stage generally starts when the suture line collides with itself under the drive of the first clamping body and the second clamping body, and the specific end time can be determined according to needs, for example, by cutting the suture line with surgical scissors.

[0061] For ease of description, this application only takes one puncture as an example. In actual applications, multiple punctures may occur. In this case, each piece of soft tissue has its own puncture stage and suturing stage, which is not limited here.

[0062] See also Figure 4 The first embodiment of the soft tissue suturing simulation method provided in the present application includes the following steps.

[0063] S1: Obtain models of soft tissue, sutures, and surgical instruments.

[0064] The model of soft tissue is a mesh model including multiple vertices and tetrahedrons defined by the vertices, the model of suture includes multiple particle points arranged in a chain and rod nodes connected between adjacent particle points, and the model of surgical instrument includes multiple mass points arranged according to its shape, and the surgical instrument includes a suture needle. For detailed information about the model, please refer to the previous related description.

[0065] S2: performing collision detection on the soft tissue, the suture, and the surgical instrument, and determining the stage of the suturing process at least based on the result of the collision detection.

[0066] The stages include the puncture stage, the suturing stage, the tensioning stage, and the knotting stage. The previous stages are described from the perspective of user operation. In the actual simulation of soft tissue suturing, the stage of the current frame in the suturing process can be determined based on the collision detection results between the soft tissue, sutures, and surgical instruments.

[0067] Collision detection between soft tissue, sutures, and surgical instruments, in addition to collision detection between soft tissue and sutures, collision detection between soft tissue and surgical instruments, and collision detection between sutures and surgical instruments, can also include self-collision detection of sutures and / or collision detection between different surgical instruments.

[0068] In addition to the collision detection results, the stage of the current frame can also be determined by combining the historical records of the stages, the relative position relationship between the surgical instrument and the soft tissue, etc. The following example is used to illustrate.

[0069] The puncture stage mainly focuses on the collision detection between the suture needle and the soft tissue. According to whether the surface of the soft tissue is pierced and the direction of the collision between the suture needle and the soft tissue surface, the puncture stage can be further divided into multiple sub-stages, as follows:

[0070] a. If the needle tip of the suture needle collides with the surface of the soft tissue from the outside to the inside but does not pierce the surface of the soft tissue, the suturing stage is the piercing into the soft tissue during the puncture stage.

[0071] In this sub-stage, since the suture needle is generally clamped by the first clamping body, it is not necessary to perform a collision response on the suture needle, but only perform a collision response on the soft tissue.

[0072] b. If the suture needle has pierced the surface of the soft tissue and the needle head is located inside the soft tissue, the suturing stage is the puncture stage during which the soft tissue has been pierced.

[0073] c. If the needle collides with the surface of the soft tissue from the inside to the outside but does not pierce the surface of the soft tissue, the suturing stage is piercing the soft tissue during the puncture stage.

[0074] In this sub-stage, since the suture needle is generally clamped by the first clamping body, it is not necessary to perform a collision response on the suture needle, but only perform a collision response on the soft tissue.

[0075] d. If the suture needle has pierced the surface of the soft tissue and the needle tip is located outside the soft tissue, the suturing stage is the stage where the soft tissue has been pierced during the puncture stage.

[0076] The needle tip of a suture needle may be a small area of ​​the needle tip. The direction in which the suture needle collides with the surface of the soft tissue, i.e., "from the outside of the soft tissue to the inside" and "from the inside of the soft tissue to the outside" mentioned above, can be implemented in a variety of ways. One is to consult the historical records of stage changes. If the puncture stage of the soft tissue did not appear in the previous stage, the direction is from the outside to the inside, otherwise the direction is from the inside to the outside; another is to refer to the relative position relationship between the suture needle and the soft tissue. If the parts of the suture needle except the part that collides with the soft tissue are all located outside the soft tissue, the direction is from the outside to the inside, and if the middle part of the suture needle is located inside the soft tissue, the direction is from the inside to the outside; another is to refer to the relative relationship between the direction of the suture needle's speed and the soft tissue. If the speed of the suture needle points to the inside of the soft tissue, the direction is from the outside to the inside, and if the speed of the suture needle points to the outside of the soft tissue, the direction is from the inside to the outside.

[0077] like Figure 5 As shown, in one embodiment of the present application, determining whether the needle has pierced the surface of the soft tissue may specifically include:

[0078] S21: performing collision detection between the needle head of the suture needle and the surface of the soft tissue.

[0079] S22: If the number of frames in which it is detected that the needle tip collides with the surface of the soft tissue continuously is greater than a first threshold, the collision response displacement of the soft tissue is accumulated to obtain a cumulative displacement.

[0080] S23: If the accumulated displacement is greater than the second threshold, it is determined that the suture needle has pierced the surface of the soft tissue.

[0081] Since the suture needle is generally clamped by the first clamping body during the process of piercing the soft tissue surface, if the suture needle is regarded as a rigid body, the collision response may not be performed on the suture needle, but only on the soft tissue.

[0082] Collision response refers to the processing of colliding objects. This processing is generally as close to people's understanding of physical laws as possible to avoid obvious abnormal interactions between colliding objects, such as mold penetration. When the needle of the suture needle punctures the surface of the soft tissue, it generates the puncture force of the needle on the soft tissue and the resistance of the soft tissue to the needle. Since the soft tissue has a certain elasticity, the soft tissue will not be punctured immediately, but will produce a certain deformation at the puncture position along the movement direction of the suture needle. As the puncture force continues to increase and exceeds the maximum resistance that the soft tissue can generate, the soft tissue will be punctured.

[0083] In order to simplify the simulation of puncture process under the PBD framework, the simulation of puncture force and resistance of soft tissue is abandoned in this embodiment. Instead, the cumulative displacement of the collision response displacement of soft tissue is used to reflect the magnitude of puncture force, and the second threshold is used to reflect the maximum resistance that soft tissue can produce. The force comparison is converted into a comparison between the cumulative displacement and the second threshold, so that it is more suitable for the PBD framework. The second threshold can be set according to the characteristics of different organs and tissues. For the same piece of soft tissue, the second threshold used for piercing the soft tissue surface from the inside to the outside and from the outside to the inside can be the same or different.

[0084] In order to prevent the collision between the needle and the soft tissue surface from being mistakenly judged as puncture, in this embodiment, it is necessary to detect that the needle and the surface of the soft tissue collide continuously, and the number of frames of continuous collision is greater than the first threshold value before confirming that the needle has begun to puncture the soft tissue surface, and then judge whether the needle has punctured the soft tissue surface based on the comparison between the cumulative displacement and the second threshold value. After the needle starts to puncture the soft tissue surface, if the cumulative displacement in the current frame is less than the second threshold value, it is judged that the soft tissue surface has not been punctured, and the cumulative displacement is retained for the next frame. In order to prevent the jitter of the soft tissue surface from affecting the result of the puncture judgment, the collision response displacement is temporarily stored and accumulated in the format of a vector, that is, the cumulative displacement of the current frame is essentially the displacement between the position of the soft tissue surface in the current frame under the collision response of the soft tissue and the needle and the position before the collision with the needle.

[0085] The beginning of the suturing stage is marked by the detection of the tail of the suture needle entering the soft tissue, or the tail of the needle just colliding with the soft tissue. The detection of the tightening mark means the transition from the suturing stage to the tightening stage. If the tightening mark is that the pre-tied knot contacts the surface of the soft tissue, the corresponding collision detection result is that the pre-tied knot has just been detected to have collided with the surface of the soft tissue. If the tightening mark is that the user uses the second clamp to directly clamp the tail of the suture thread, the corresponding collision detection result is that the tail of the suture thread, that is, the part located outside the soft tissue and on the side where the suture needle penetrates the soft tissue, has just been detected to have collided with the second clamp.

[0086] The detection of the tension mark means the start of the tension phase, while the presence of at least two control points in the suture and the detection of a self-collision means the end of the tension phase.

[0087] In order to simulate the process of soft tissue suturing under the framework of PBD, at least one stage mark can be designed to distinguish different stages, and the stage mark can be updated as the stage changes. During the real-time simulation of different types of objects, the stage mark can be read to perform corresponding operations, such as adding corresponding temporary constraints (collision response constraints, path point constraints, etc.), updating path points, etc. Since the stages that need to be paid attention to in the simulation of different types of objects are not necessarily the same, you can choose to set different stage marks for different types of objects, or you can set a unified stage mark for all objects. The latter is used as an example in this application.

[0088] S3: In the suturing stage, the particle point interval of the suture line is used to interpolate the movement path of the needle tail of the suture needle in the soft tissue to obtain a path point sequence.

[0089] During the movement of the needle tail in the soft tissue, the position of the needle tail in each frame, which can also be called a moving point, is arranged in time order to form a moving path. During the suturing stage, the suture generally moves along the moving path when moving in the soft tissue, so the constraint of the moving path on the suture can be used as the collision response constraint between the suture and the soft tissue acting on the suture.

[0090] Since the distribution of moving points in the moving path is affected by the user's operation and is not necessarily uniform, if temporary constraints are added to the particle points located inside the soft tissue in the suture line, that is, the target particle points, directly according to the moving path, the spacing between adjacent target particle points after the update will be affected by the distribution of the moving points and will no longer be uniform. There may be a large error with the initially set particle point spacing, that is, the spacing between adjacent particle points in the suture line, thereby causing a significant change in the length of the suture line.

[0091] In order to avoid obvious length changes of the suture when it moves in the soft tissue and reduce the resulting distortion, in this embodiment, the particle point interval of the suture is used to interpolate the moving path to obtain a path point sequence, and then the path point sequence is used to add temporary constraints to the target particle points.

[0092] like Figure 6 As shown, in one embodiment of the present application, S3 specifically includes:

[0093] S31: In the stitching stage, the position of the needle tail in the current frame is obtained.

[0094] For the suture needle, if it is treated as a rigid body, then when the suture needle is clamped by other surgical instruments, the PBD process can be omitted, and the kinematic equations can be directly used in combination with the posture of the surgical instrument holding the suture needle to calculate its posture and the position of each particle in the model; when the suture needle is not clamped by other surgical instruments, in order to prevent penetration of the model, the PBD / extended position-based dynamics (XPBD) process can be used to calculate the position of each particle in the model. Specifically, the overall collision constraint of the suture needle can be determined according to the results of the collision detection between the suture needle and the soft tissue, and the position of each particle in the suture needle model can be calculated using position-based dynamics based on the overall collision constraint.

[0095] If the elasticity of the suture needle is taken into account, it cannot be treated as a rigid body. Regardless of whether it is clamped by other surgical instruments, the PBD / XPBD process must be used to calculate the position of each particle in the model.

[0096] According to the above-mentioned method of calculating the position of each particle in the suture needle, the position of each particle in the suture needle in the current frame can be obtained. Combined with the position of the needle tail in the suture needle, the position of the needle tail in the current frame can be obtained.

[0097] S32: If the position of the needle tail in the current frame is within the soft tissue, determine whether a first distance between the position of the needle tail in the current frame and the current path point in the path point sequence is within a deviation interval determined based on the particle point interval.

[0098] The current path point is the last path point added to the path point sequence, that is, the latest path point in the path point sequence. Generally speaking, the first path point added to the path point sequence is the position where the needle tail is first detected to collide with the soft tissue. The first distance is a scalar.

[0099] The deviation interval can be written in the form of [k1*rd, k2*rd], where rd is the particle point interval, 0 < k1 ≤ 1 ≤ k2. Generally, the differences between k1, k2 and 1 are all relatively small values, such as 0.9 and 1.1. The smaller the differences between k1, k2 and 1, the less likely the length of the suture line will change drastically during the suture stage, but the more likely it is to bring deviations between the path point sequence and the moving path. The values of k1 and k2 can be determined according to the setting of the elasticity of the suture line in stretching when establishing the suture line model. For example, if the suture line is set to be completely inextensible, then both k1 and k2 are 1, and the greater the elasticity in stretching, the greater k1 and k2 are.

[0100] If the first distance is within the deviation interval, jump to S33; if the first distance is greater than the upper limit of the deviation interval, jump to S34; if the first distance is less than the lower limit of the deviation interval, jump to S35.

[0101] S33: Take the position of the needle tail in the current frame as a new path point and add it to the path point sequence.

[0102] S34: Interpolate between the current path point and the position of the needle tail in the current frame using the particle point interval, and take the interpolation result as a new path point and add it to the path point sequence.

[0103] The interpolation result is on the line connecting the current path point and the position of the needle tail in the current frame, and the distance from the current path point is k*rd. The value range of k does not exceed [k1, k2]. The values of k in different frames can be the same, such as all being 1, or different, such as randomly selecting a value within the value range.

[0104] S35: Do not update the path point sequence.

[0105] If the position of the needle tail in the current frame is not within the soft tissue, it means that the needle tail has left the soft tissue, and the moving path of the needle tail within the soft tissue has ended, and the path point sequence no longer needs to be updated.

[0106] S4: Use the position-based dynamics to calculate the model parameters of the soft tissue and the suture line frame by frame, and use the position-based dynamics and / or kinematic equations to calculate the model parameters of the surgical instrument frame by frame.

[0107] During the suture stage, the main temporary constraint considered is the path point constraint of the target particle points. The target particle points are the particle points of the suture line located inside the soft tissue. Each target particle point has an associated path point in the path point sequence. The path point constraint of each target particle point refers to the constraint between the target particle point and its associated path point.

[0108] During the suture stage, in the part of generating temporary constraints in PBD, path point constraints can be added for the target particle points, such as Figure 7As shown, in one embodiment of the present application, adding a path point constraint to a target particle point may specifically include:

[0109] S41: In the suturing stage, determine whether the needle tail in the current frame is located inside the soft tissue.

[0110] If the needle tail is located inside the soft tissue, jump to S42; if the needle tail is located outside the soft tissue, jump to S43.

[0111] S42: taking a particle point in the suture line that is directly connected to the needle tail as a reference point, and determining a position change amount of the reference point according to a position change amount of the needle tail.

[0112] The position delta is the difference between the position in the current frame and the position in the previous frame.

[0113] The particle point directly connected to the needle tail in the suture is the control point directly connected to the needle tail, referred to as the first control point in this application. The position change of the reference point can be directly calculated based on the position change of the needle tail. For example, when the needle tail and the first control point are set to overlap in space, the position change of the needle tail can be directly used as the position change of the reference point.

[0114] Jump to S44.

[0115] S43: The particle point closest to the soft tissue in the suture line is taken as the reference point, and the position change of the reference point is calculated based on position dynamics.

[0116] The particle point that leaves the soft tissue the closest is the particle point outside the soft tissue, on the side where the suture needle passes through the soft tissue and closest to the surface of the soft tissue. When the needle tail just leaves the soft tissue, the reference point is still the first control point. In other cases, the reference point is not the first control point, and its position is not directly affected by the suture needle like the position of the first control point. PBD is needed to calculate the position change of the reference point.

[0117] S44: Determine the target particle point in the current frame and the corresponding relationship between the target particle point and the path point in the path point sequence according to the position change of the reference point and the particle point interval.

[0118] According to the position change of the reference point and the particle point interval, it can be determined whether a new particle point has entered the soft tissue to become a target particle point and / or a new target particle point has entered the outside of the soft tissue to become a non-target particle point.

[0119] The position of the first control point in the suture line is directly determined by the position of the needle tail and does not participate in the correction and update of the PBD. Even if the first control point is located inside the soft tissue, it cannot be processed as a target particle point.

[0120] When the needle tail is located inside the soft tissue, the update of the path point sequence and the target particle point can be synchronized with the path point in the path point sequence, that is, a path point and a target particle point are added in the same frame.

[0121] like Figure 8 As shown, in one embodiment of the present application, S44 may specifically include:

[0122] S441: Calculate the required position change of the reference point of the current frame according to the position change of the reference point in the current frame and the position of the remaining reference points in the previous frame.

[0123] The position change of the reference point in the current frame and the position of the remaining reference points in the previous frame are both vectors, and the vector sum of the two is generally calculated as the required position change.

[0124] S442: Determine whether the required position change of the reference point is greater than the particle point interval.

[0125] If so, it means that new particle points have entered the soft tissue, and the target particle points and their correspondence with the path point sequence need to be updated, and the process goes to S443; otherwise, it means that no new particle points have entered the soft tissue, and the target particle points and their correspondence with the path point sequence do not need to be updated, and the process goes to S444.

[0126] S443: Determine the number of particle points of the suture line moving into the soft tissue and the remaining reference point positions in the current frame based on the ratio of the required position change of the reference point to the particle point interval; update the target particle points in the previous frame and the corresponding relationship between them and the path points in the path point sequence based on the number of particle points of the suture line moving into the soft tissue to obtain the target particle points in the current frame and the corresponding relationship between them and the path points in the path point sequence.

[0127] Generally, the contrast value is rounded down to obtain the number of particle points n of the suture line moving into the soft tissue in the current frame. The remaining reference point position quantity has the same direction as the required position change quantity, and its size is the reference point position quantity minus n*rd.

[0128] The first n particle points that have not entered the soft tissue and are closest to the soft tissue surface in the previous frame become new target particle points. If the reference point in the previous frame is not the first control point, the reference point of the previous frame is moved n particle points away from the first control point as a new reference point for subsequent use. At the same time, the oldest n target particle points in the previous frame, that is, the earliest n target particle points that enter the soft tissue, are no longer target particle points. In other words, the oldest n target particle points are removed from the target particle points, and the total number of target particle points remains unchanged in combination with the previous n newly added target particle points. If the reference point in the previous frame is the first control point, the number m of other particle points that leave the soft tissue in the current frame except the first control point is counted in combination with the required position change of the reference point. If m=0, the reference point does not need to be updated and the target particle point does not need to be removed; if m>0, the reference point of the previous frame is moved m particle points away from the first control point as a new reference point for subsequent use, and the oldest m target particle points are removed from the target particle points.

[0129] After updating the target particle point, if there is no new path point added to the path point sequence in the current frame, generally when the needle tail has left the soft tissue, the sequence number of the target particle point corresponding to each path point in the path point sequence in the previous frame is moved by n in the direction away from the first control point, which is the sequence number of the target particle point corresponding to each path point in the path point sequence in the current frame.

[0130] For example, assuming that the target particle points M, M+1, …M+k-1 in the previous frame correspond to path points k, k-1, …1 respectively, and the reference point M-1 is not the first control point. After calculation, it is found that N particle points move in and out of the soft tissue in the current frame. Then, the reference point in the current frame is updated to M+N-1, and the target particle points M+N, M+N+1, …M+N+k-1 correspond to path points k, k-1, …1 respectively.

[0131] After updating the target particle points, if there are new path points added to the path point sequence in the current frame, generally when the needle tail is still inside the soft tissue, the serial numbers of the target particle points corresponding to each path point in the path point sequence in the previous frame are moved by n in the direction away from the first control point, and the target particle points that have no corresponding relationship with the path points in the path point sequence of the previous frame after the movement are arranged in the direction pointing to the first control point, and the newly added path points in the current frame are arranged in the order of joining the path point sequence, and the corresponding relationship between the two is established in turn, so as to obtain the serial numbers of the target particle points corresponding to each path point in the path point sequence in the current frame.

[0132] For example, assuming that the target particle points M, M+1, …M+k-1 in the previous frame correspond to path points k, k-1, …1 respectively, and the reference point M-1 is the first control point. After calculation, N particle points have moved into the soft tissue and N new path points have been added in the current frame, and the reference point M-1 is still the first control point. Then the target particle points M, M+1, …M+N+k-1 correspond to path points k+N, k+N-1, …1 respectively.

[0133] S444: The required position change of the reference point is used as the remaining reference point position, and the target particle point in the previous frame and the corresponding relationship between it and the path point in the path point sequence are used as the target particle point in the current frame and the corresponding relationship between it and the path point in the path point sequence.

[0134] S45: Adding path point constraints to the target particle points based on the corresponding relationship.

[0135] According to the corresponding relationship, the associated path point of each target particle point can be determined. The path point constraint generally constrains the position of the target particle point to the position of the associated path point. The path point constraint can be written in the form of an equation, that is, the distance between the target path point and the associated path point is 0; or written in the form of an inequality, that is, the distance between the target path point and the associated path point is less than a very small fixed value; or the distance between the target path point and the associated path point is compared with a fixed value. If it is less than the fixed value, the path point constraint is not added, otherwise the distance between the target path point and the associated path point is added to be 0.

[0136] In practical applications, the first control point may not be the endpoint of the suture line. In this case, both sides of the first control point can independently update their respective target particle points and their correspondence with the path point sequence, that is, the same path point may correspond to two different target particle points at the same time.

[0137] In order to ensure the real-time simulation effect of the suturing stage and reduce possible instability, in some embodiments, the collision response constraint between the suture line and the soft tissue acting on the soft tissue can be omitted during the suturing stage. The external manifestation is that the soft tissue will not be tightened by the suture line during the suturing stage, and the collision response constraint between the suture line and the soft tissue will be applied to the soft tissue in the subsequent tightening stage. Of course, in other embodiments, the collision response constraint between the suture line and the soft tissue can be applied to the soft tissue during the suturing stage.

[0138] like Fig. 9 As shown, in one embodiment of the present application, the collision response constraint between the suture line and the soft tissue may specifically include:

[0139] S46: In the tensioning stage, after calculating the model parameters of the stitching line in the current frame, a second distance between the target particle point and the nearest path point is calculated.

[0140] The position of the target particle point is calculated with the path point constraint added. The target particle point can be determined by updating the reference point frame by frame, or by performing collision detection on the suture and soft tissue. If it is the latter, it is necessary to exclude the particle points that only collide with the soft tissue surface to avoid mistakenly using the particle points that fall on the soft tissue surface as the target particle point.

[0141] S47: If the second distance is greater than the distance threshold, a collision response constraint is added to the collision tetrahedron where the nearest path point is located.

[0142] The second distance is greater than the distance threshold, that is, the distance between the target particle point and the nearest path point is still too far when the path point constraint is added, which means that the soft tissue has undergone obvious deformation under the pull of the suture line and a collision response is required.

[0143] The specific collision response constraint can be to move the collision tetrahedron as a whole in the direction from the nearest path point to the particle point by a distance threshold. Since the constraint function of the soft tissue includes a vertex-based constraint function, the movement of the collision tetrahedron is generally converted into the movement of each vertex of the collision tetrahedron. For example, assuming that the calculated displacement is Δp, its direction is from the nearest path point to the particle point, and its size is the distance threshold, the overall displacement of the collision tetrahedron can be to add a temporary constraint of p=p+Δp to all four vertices of the collision tetrahedron. The aforementioned constraint method actually also limits the collision tetrahedron to be as non-deformed as possible, which may cause the surface of the soft tissue to be uneven. To reduce this phenomenon, the temporary constraint can be modified to an inequality, such as |p-(p+Δp)|<0.1|Δp|, thereby expanding the movable range of each vertex of the collision tetrahedron.

[0144] S48: Calculate model parameters of the soft tissue in the current frame using position-based dynamics based on the collision response constraints.

[0145] S49: Update the coordinates of the path points according to the model parameters of the soft tissue.

[0146] The coordinates of the waypoints can be updated based on the barycentric coordinates, also called mass center coordinates or volume coordinates. The waypoints here can be all the waypoints in the waypoint sequence.

[0147] like Fig.10 As shown, in one embodiment of the present application, S49 specifically includes:

[0148] S491: Obtain the centroid coordinates of the path point in the tetrahedron before the collision response constraint is added.

[0149] Barycentric coordinates, also called mass coordinates or volume coordinates, are coordinates defined by the four vertices of a tetrahedron. The coordinates of any point in a tetrahedron can be written as the weighted average of the coordinates of the four vertices of the tetrahedron, and these four weights are the barycentric coordinates of the point.

[0150] S492: Calculate the coordinates of the updated path points by combining the coordinates of the vertices and the center of gravity of the tetrahedron in the model parameters of the soft tissue.

[0151] The centroid coordinates of the path point are used as weights to calculate the weighted average of the coordinates of the vertices of the tetrahedron in the model parameters of the soft tissue, and the result obtained is the coordinate of the updated path point.

[0152] S5: Soft tissues, sutures, and surgical instruments are displayed frame by frame according to their model parameters.

[0153] In each frame, after completing the calculation of the model parameters of the soft tissue, sutures and surgical instruments, the morphology of the soft tissue, sutures and surgical instruments can be determined, and on this basis the model can be rendered and then output to a display device for display.

[0154] S6: Calculate the feedback force according to the feedback force strategy corresponding to the stage.

[0155] If there is no feedback force function, this step can be omitted.

[0156] This step only needs to be executed after S2, and there is no restriction on the execution order between S3-S5.

[0157] If the stage is before the puncture stage, the feedback force is 0.

[0158] If the stage is the puncture stage of piercing into soft tissue, the feedback force is the product of the accumulated displacement and the first proportional coefficient.

[0159] If the stage is the soft tissue has been penetrated in the puncture stage, the feedback force is the first default value.

[0160] If the stage is the puncture phase in which the soft tissue is punctured, the feedback force is the sum of the product of the accumulated displacement and the second proportional coefficient and the first default value.

[0161] If the stage is the puncture stage in which the soft tissue has been punctured, the feedback force is the second default value.

[0162] If the stage is the suturing stage, the feedback force is the second default value.

[0163] The feedback force calculation strategy corresponding to the tensioning stage is to calculate the feedback force based on the second default value and the displacement of at least part of the tetrahedron in the soft tissue. For example, the displacement of the nearest path point before and after the update can be calculated based on the displacement of the collision tetrahedron, and then the product of the displacement and the third proportional coefficient and the sum of the second default value are calculated as the feedback force.

[0164] Through the implementation of this embodiment, a unified position dynamics-based framework is used to simulate soft tissue, sutures and suture needles, which can effectively reduce the penetration between different models. In addition, in the suturing stage, the particle point interval of the suture line is used to interpolate the movement path of the needle tail of the suture needle in the soft tissue to obtain a path point sequence, and then based on the correspondence between the path point sequence and the target particle point located inside the soft tissue in the suture line, a path point constraint is added to the target particle point, so that the part of the suture line located inside the soft tissue will not produce a drastic length change under the action of the path point constraint, reducing the distortion that may be caused by this.

[0165] This embodiment describes the simulation of the overall suturing process. The following describes the real-time simulation processes of the suture thread, suture needle and soft tissue respectively in conjunction with the accompanying drawings.

[0166] like Fig.11 As shown, the second embodiment of the soft tissue suturing simulation method provided by the present application includes the following steps.

[0167] S101: Determine the target pose offset of the stitching line in the current frame and obtain a stage flag.

[0168] This embodiment is a real-time simulation process of each suture line in the soft tissue suturing process, in which the parts that are the same or similar to the above content are not described in detail. The suture line model includes a plurality of particle points arranged in a chain and rod nodes connected between adjacent particle points.

[0169] The initial number of control points on the suture line and the specific position of each control point can be determined according to the actual application scenario. For example, when the suture line is not pre-fixed with the suture needle, refer to the threading operation before suturing, that is, first pass the suture line through the hole at the tail end of the suture needle and then clamp the threaded line at the front end of the needle holder. The simulated threaded suture line actually has 3 control points, 1 connected to the suture needle and 2 connected to the needle holder; when the suture needle and the suture line are pre-fixed together, the part directly connected to the suture needle is the control point of the suture line.

[0170] The number and / or position of control points may change according to the user's operation. For example, during the suturing process, the user operates the needle holder to insert the suture needle into the tissue and then releases the needle holder. At this time, the control point of the threaded suture thread is only one connected to the suture needle; during the knotting stage, the user will use tweezers to clamp the suture thread left outside the needle insertion side of the tissue, and a new control point for connecting the tweezers is added.

[0171] The target pose offset is the pose offset of the connection part (hereinafter referred to as the connection part) where the control body is directly connected to the suture line, including the target position offset and the target pose offset. According to the operation instructions given by the user, the pose of each connection part in the current frame can be calculated, and then the difference between the pose in the current frame and the pose in the previous frame is calculated as the command pose offset. If the pose is expressed in unit quaternion, the pose difference refers to the angle difference between the two quaternions. For example, if the pose in the previous frame is q n-1 , the pose of the current frame is q n , then the difference between the posture of the current frame and the previous frame is The superscript -1 indicates inversion. Without considering other factors, the command posture offset can be directly used as the target posture offset. Of course, the target posture offset can be obtained after processing the command posture offset, as described in the subsequent embodiments.

[0172] Each connection part has a corresponding control point, that is, a directly connected control point. The target position offset of each connection part is transferred from the control body to the suture line, and acts on the corresponding control point and the rod node corresponding to the control point, respectively. The target position offset acts on the control point, and the target attitude offset acts on the rod node corresponding to the control point. For ease of description, the following example is taken as an example when the number of control points is 1, and the case when the number of control points is greater than 1 can be obtained by analogy.

[0173] The rod node corresponding to the control point is the rod node directly connected to the control point. If the control point is the endpoint of the suture line, there is only one rod node directly connected to the control point as the rod node corresponding to the control point. Figure 1 For example, if Figure 1 If the particle point p1 is the control point, the corresponding rod node is q1. If the control point is not the end point of the suture line, there are two rod nodes directly connected to the control point. The rod node corresponding to the control point can be one of the two rod nodes, or both of them can be used as the rod nodes corresponding to the control point. Figure 1 For example, if Figure 1 If the particle point p2 is the control point, the corresponding rod node can be at least one of q1 and q2.

[0174] The position of the control point in the current frame and the posture of the rod node corresponding to the control point are determined by the target posture offset. Specifically, the sum of the position of the control point in the previous frame and the target position offset is calculated as the position of the control point in the current frame, and the product of the posture of the rod node corresponding to the control point in the previous frame and the target posture offset is calculated as the posture of the rod node corresponding to the control point in the current frame. During the entire constraint correction process, the position of the control point is fixed to the calculated position in the current frame, and the posture of the rod node corresponding to the control point is fixed to the calculated posture in the current frame, which will not change like other particle points and rod nodes. In other words, the position of the control point and the posture of the rod node corresponding to the control point will not be updated by the constraint function equation group.

[0175] S102: Determine a temporary constraint function of the particle point at least according to the stage flag.

[0176] The stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, which includes the puncture stage, the suturing stage and the tensioning stage.

[0177] If the stage mark is the suturing stage, the temporary constraint function of the target particle point located inside the soft tissue includes a path point constraint based on the associated path point corresponding to the target particle point in the path point sequence, and the path point sequence is obtained by interpolating the moving path of the first control point in the soft tissue using the particle point interval of the suture line, and the first control point is a particle point directly connected to the needle tail of the suture needle.

[0178] like Fig.12 As shown, in one embodiment of the present application, the real-time process of obtaining a path point sequence may specifically include:

[0179] S121: Obtain the position of the first control point in the current frame.

[0180] During the entire suturing process, the suture thread is connected to the suture needle and moves under the drive of the suture needle, so the target pose offset includes the target position offset of the needle tail. The position offset of the first control point in the current frame can be determined based on the target pose offset of the current frame. Then, the sum of the position of the first control point in the previous frame and the position offset of the first control point in the current frame is calculated as the position of the first control point in the current frame.

[0181] S122: If the position of the first control point in the current frame is within the soft tissue, determine whether a first distance between the position of the first control point in the current frame and the current path point in the path point sequence is within a deviation interval determined based on the particle point interval.

[0182] If the first distance is within the deviation interval, jump to S123; if the first distance is greater than the upper limit of the deviation interval, jump to S124; if the first distance is less than the lower limit of the deviation interval, jump to S125.

[0183] S123: Add the position of the first control point in the current frame as a new path point to the path point sequence.

[0184] S124: interpolating between the current path point and the position of the first control point in the current frame using the particle point interval, and adding the interpolation result as a new path point to the path point sequence.

[0185] S125: Do not update the waypoint sequence.

[0186] like Fig.13 As shown, in an embodiment of the present application, S102 may specifically include:

[0187] S131: If the stage mark is the suturing stage, determine whether the first control point in the current frame is located inside the soft tissue.

[0188] If the first control point is located inside the soft tissue, the process jumps to S132 ; if the first control point is located outside the soft tissue, the process jumps to S133 .

[0189] S132: Taking the first control point as a reference point, and determining a position change amount of the reference point according to a position change amount of the first control point.

[0190] Jump to S134.

[0191] S133: Taking the particle point in the suture line that is closest to leaving the soft tissue as a reference point, and calculating the position change of the reference point based on position dynamics.

[0192] S134: Determine the target particle point in the current frame and the corresponding relationship between the target particle point and the path point in the path point sequence according to the position change of the reference point and the particle point interval.

[0193] For the specific process, please refer to Figure 8 and description of corresponding embodiments.

[0194] S135: Adding path point constraints to the target particle points based on the corresponding relationship.

[0195] S103: Perform collision detection on the suture thread and soft tissue, and surgical instruments other than the suture needle.

[0196] S104: Determine whether to update the phase flag at least based on the result of the collision detection.

[0197] There is no restriction on the execution order of this step and S105 - S106 .

[0198] If the stage mark is the puncture stage and it is detected that the first control point collides with the soft tissue, the stage mark is updated to the suturing stage.

[0199] If the phase mark is the suturing phase and it is detected that the preset knot in the suture line collides with the surface of the soft tissue, the phase mark is updated to the tightening phase.

[0200] If the stage mark is the suturing stage and it is detected that the suture line collides with the second clamping body, the stage mark is updated to the tensioning stage, and the second clamping body is a surgical instrument different from the suture needle and the first clamping body that clamps the suture needle.

[0201] S105: Obtain model parameters of the stitching line in the current frame using position-based dynamics calculation according to the target posture offset.

[0202] The constraint functions of particle points include temporary constraint functions.

[0203] S106: Displaying the seam line according to the model parameters of the seam line in the current frame.

[0204] like Fig.14 As shown, the third embodiment of the soft tissue suturing simulation method provided by the present application includes the following steps.

[0205] S201: Calculate the model parameters of the suture needle in the current frame.

[0206] This embodiment is a real-time simulation process of each frame of a suture needle in the process of soft tissue suturing. The model of the suture needle includes a plurality of mass points arranged according to its shape.

[0207] like Fig.15 As shown, in one embodiment of the present application, the suture needle is treated as a rigid body, and S201 may specifically include:

[0208] S211: Determine whether the suture needle is in a clamped state in the current frame.

[0209] If the suture needle is in a clamped state, jump to S212; otherwise jump to S213.

[0210] S212: Acquire the posture of the first clamping body that clamps the suture needle, and calculate the model parameters of the suture needle in the current frame using the kinematic equation according to the posture of the first clamping body.

[0211] S213: Determine the overall collision constraint of the suture needle according to the result of the collision detection between the suture needle and the soft tissue, and calculate the model parameters of the suture needle in the current frame based on the overall collision constraint using position-based dynamics.

[0212] If the elasticity of the suture needle is taken into account, it cannot be treated as a rigid body. Regardless of whether it is clamped by other surgical instruments, the PBD / XPBD process must be used to calculate the position of each particle in the model.

[0213] S202: Get the stage flag.

[0214] The stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, which includes the puncture stage, the suturing stage and the tensioning stage.

[0215] S203: If the stage mark is the suturing stage, the moving path of the needle tail of the suture needle in the soft tissue is interpolated using the particle point interval of the suture line to determine whether to update the path point sequence.

[0216] like Fig.16 As shown, in an embodiment of the present application, S203 may specifically include:

[0217] S231: If the stage mark is the suturing stage, determine the position of the needle tail of the suturing needle in the current frame according to the model parameters of the suturing needle in the current frame.

[0218] S232: If the position of the needle tail in the current frame is within the soft tissue, determine whether a first distance between the position of the needle tail in the current frame and the current path point in the path point sequence is within a deviation interval determined based on the particle point interval.

[0219] If the first distance is within the deviation interval, jump to S233; if the first distance is greater than the upper limit of the deviation interval, jump to S234; if the first distance is less than the lower limit of the deviation interval, jump to S235.

[0220] S233: Add the position of the needle tail in the current frame as a new path point to the path point sequence.

[0221] S234: interpolating between the current path point and the position of the needle tail in the current frame using the particle point interval, and adding the interpolation result as a new path point to the path point sequence.

[0222] S235: Do not update the waypoint sequence.

[0223] S204: Displaying the suture needle according to the model parameters of the suture needle in the current frame.

[0224] S205: Determine whether to update the stage flag at least based on the result of the collision detection between the soft tissue and the suture needle.

[0225] If PBD / XPBD is used to calculate the model parameters of the suture needle, the collision detection between the soft tissue and the suture needle is generally performed before or during S201. This step only needs to be performed after the collision detection between the soft tissue and the suture needle, and there is no restriction on the order with other steps.

[0226] If the stage mark is that the soft tissue has been pierced in the puncture stage and it is detected that the needle tail of the suture needle collides with the surface of the soft tissue, the stage mark is updated to the suturing stage.

[0227] The puncture stage can be divided into multiple sub-stages according to the puncture direction of the needle and whether the surface of the soft tissue is punctured. There are multiple ways to determine the puncture direction. The following is a specific description of determining the puncture direction based on the historical records of the stage mark.

[0228] like Fig.17 As shown, in one embodiment of the present application, the sub-stages of determining the puncture stage may specifically include:

[0229] S261: Determine whether a collision between the needle tip of the suture needle and the surface of the soft tissue is detected in the current frame.

[0230] If so, jump to S262; otherwise jump to S269.

[0231] S262: Add 1 to the accumulated number of frames and temporarily store the collision response displacement of the current frame.

[0232] The collision response displacement is the position of the puncture point that collides with the needle in the model parameters of the soft tissue in the current frame minus the position of the puncture point in the model parameters of the soft tissue in the previous frame.

[0233] S263: If the accumulated number of frames is greater than the first threshold, all temporarily stored collision response displacements are accumulated to obtain an accumulated displacement.

[0234] S264: Compare the accumulated displacement with the second threshold value to determine whether the needle has pierced the surface of the soft tissue.

[0235] Specifically, if the accumulated displacement in the current frame is greater than the second threshold, it is determined that the needle has pierced the surface of the soft tissue; otherwise, it is determined that the needle has not pierced the surface of the soft tissue. Alternatively, if the accumulated displacement in the current frame is greater than or equal to the second threshold, it is determined that the needle has pierced the surface of the soft tissue; otherwise, it is determined that the needle has not pierced the surface of the soft tissue.

[0236] S265: If the needle has not pierced the surface of the soft tissue and the stage mark is empty, the stage mark is updated to piercing into the soft tissue in the puncture stage.

[0237] S266: If the needle has pierced the surface of the soft tissue, and the stage mark is empty or has penetrated the soft tissue during the puncture stage, the stage mark is updated to have penetrated the soft tissue during the puncture stage.

[0238] S267: If the needle has not pierced the surface of the soft tissue, and the stage mark is "entered into the soft tissue" in the puncture stage, then the stage mark is updated to "pierced out of the soft tissue" in the puncture stage.

[0239] S268: If the needle has pierced the surface of the soft tissue, and the stage mark is "entered into the soft tissue" or "exited from the soft tissue" in the puncture stage, the stage mark is updated to "exited from the soft tissue" in the puncture stage.

[0240] S269: Set the accumulated frame number to 0, and clear all temporarily stored collision response displacements.

[0241] like Fig.18 As shown, the fourth embodiment of the soft tissue suturing simulation method provided by the present application includes the following steps.

[0242] S301: Obtain a phase flag in the current frame.

[0243] This embodiment is a real-time simulation process of each frame of soft tissue during the soft tissue suturing process. The stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, and the stages include the puncture stage, the suturing stage and the tensioning stage.

[0244] S302: Perform collision detection on soft tissue, sutures, and surgical instruments.

[0245] The model of the soft tissue is a mesh model including a plurality of vertices and tetrahedrons defined by the vertices.

[0246] S303: Determine a temporary constraint function of the vertex according to the collision detection result and / or the stage flag.

[0247] If it is detected that the surface of the soft tissue collides with the needle of the suture needle in the surgical instrument, a first collision response constraint is added to the puncture point that collides with the needle. This temporary constraint function is mainly used in the puncture stage.

[0248] The second collision response constraint function mainly acts on the tension stage. Fig.19 As shown, in one embodiment of the present application, the specific process of adding the second collision response constraint function may include:

[0249] S331: If the stage mark is the tensioning stage, after calculating the model parameters of the suture line in the current frame, calculate the second distance between the target particle point located inside the soft tissue and the nearest path point in the path point sequence.

[0250] The path point sequence is obtained by interpolating the movement path of the needle tail of the suture needle in the soft tissue using the particle point interval of the suture line.

[0251] S332: If the second distance is greater than the distance threshold, a second collision response constraint is added to the collision tetrahedron where the nearest path point is located.

[0252] Moves the collision tetrahedron by the threshold distance in the direction from the nearest path point to the target particle point.

[0253] S304: Obtain model parameters of the soft tissue in the current frame using position-based dynamics calculation.

[0254] The vertex constraint function includes a temporary constraint function. If the stage flag is the tension stage, the coordinates of each path point in the path point sequence can be updated according to the model parameters of the soft tissue in the current frame. Specifically, for each path point, the centroid coordinates of the path point in the tetrahedron before the second collision response constraint is added can be obtained, and then the coordinates of each vertex of the tetrahedron in the model parameters of the soft tissue and the centroid coordinates are combined to calculate the coordinates of the updated path point.

[0255] S305: Determine whether to update the phase flag at least according to the result of the collision detection.

[0256] This step only needs to be executed after S302, and there is no restriction on the execution order between S303 and subsequent steps.

[0257] If the stage mark is that the soft tissue has been pierced in the puncture stage and it is detected that the needle tail of the suture needle collides with the surface of the soft tissue, the stage mark is updated to the suturing stage.

[0258] According to the collision detection results between the soft tissue surface and the needle, the sub-stages of the puncture stage can be distinguished. For details, please refer to Fig.17 and corresponding embodiments.

[0259] S306: Displaying the soft tissue according to the model parameters of the soft tissue in the current frame.

[0260] See also Fig. 20 As shown, the first embodiment of the soft tissue suturing simulation device provided by the present application includes: an acquisition module 101 , a detection module 102 , an interpolation module 103 , a calculation module 104 and a display module 105 .

[0261] The acquisition module 101 is used to acquire models of soft tissue, sutures and surgical instruments. The model of the soft tissue is a mesh model including multiple vertices and tetrahedrons defined by the vertices, the model of the suture includes multiple particle points arranged in a chain and rod nodes connected between adjacent particle points, the model of the surgical instrument includes multiple particle points arranged according to its shape, and the surgical instrument includes a suture needle.

[0262] The detection module 102 is used to perform collision detection on the soft tissue, the suture and the surgical instrument, and determine the stage in the suturing process at least based on the result of the collision detection, wherein the stage includes the puncture stage, the suturing stage and the tensioning stage.

[0263] The interpolation module 103 is used to interpolate the moving path of the needle tail of the suture needle in the soft tissue to obtain a path point sequence using the particle point interval of the suture line during the suturing stage.

[0264] The calculation module 104 is used to calculate the model parameters of the soft tissue and suture line frame by frame based on position dynamics, and to calculate the model parameters of the surgical instrument frame by frame based on position dynamics and / or kinematic equations, wherein the constraint function of the target particle point includes a path point constraint based on an associated path point in the path point sequence corresponding to the target particle point, and the target particle point is a particle point of the suture line located inside the soft tissue.

[0265] The display module 105 is used to display the soft tissue, suture line and surgical instrument frame by frame according to the model parameters of the soft tissue, suture line and surgical instrument.

[0266] See also Fig.21 As shown, the second embodiment of the soft tissue suturing simulation device provided by the present application includes: an acquisition module 201 , a temporary constraint module 202 , a calculation module 203 and a display module 204 .

[0267] The acquisition module 201 is used to determine the target position offset of the stitching line in the current frame and acquire the stage mark. The stitching line model includes a plurality of particle points arranged in a chain and rod nodes connected between adjacent particle points.

[0268] The temporary constraint module 202 is used to determine the temporary constraint function of the particle point at least according to the stage mark. The stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, and the stages include the puncture stage, the suturing stage and the tensioning stage. If the stage mark is the suturing stage, the temporary constraint function of the target particle point located inside the soft tissue includes a path point constraint based on an associated path point corresponding to the target particle point in a path point sequence, and the path point sequence is obtained by interpolating the moving path of the first control point in the soft tissue using the particle point interval of the suture line, and the first control point is a particle point directly connected to the needle tail of the suture needle.

[0269] The calculation module 203 is used to obtain the model parameters of the stitching line in the current frame according to the target posture offset using position dynamics-based calculation. The constraint function of the particle point includes the temporary constraint function.

[0270] The display module 204 is used to display the stitching line according to the model parameters of the stitching line in the current frame.

[0271] See also Fig. 22 As shown, the third embodiment of the soft tissue suturing simulation device provided by the present application includes: a calculation module 301 , an acquisition module 302 , an interpolation module 303 and a display module 304 .

[0272] The calculation module 301 is used to calculate the model parameters of the suture needle in the current frame, and the model of the suture needle includes a plurality of mass points arranged according to its shape.

[0273] The acquisition module 302 is used to acquire a stage mark, where the stage mark is used to indicate the stage where the current frame is located in the soft tissue suturing process, and the stage includes a puncture stage, a suturing stage, and a tensioning stage.

[0274] The interpolation module 303 is used to interpolate the moving path of the needle tail of the suture needle in the soft tissue using the particle point interval of the suture line to determine whether to update the path point sequence when the stage mark is the suture stage.

[0275] The display module 304 is used to display the suture needle according to the model parameters of the suture needle in the current frame.

[0276] See also Fig.23 As shown, the fourth embodiment of the soft tissue suturing simulation device provided by the present application includes: an acquisition module 401 , a detection module 402 , a temporary constraint module 403 , a calculation module 404 , an update module 405 and a display module 406 .

[0277] The acquisition module 401 is used to acquire a stage mark in the current frame, where the stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, where the stage includes a puncture stage, a suturing stage, and a tensioning stage;

[0278] The detection module 402 is used to perform collision detection on the soft tissue, sutures, and surgical instruments. The model of the soft tissue is a mesh model including a plurality of vertices and tetrahedrons defined by the vertices.

[0279] The temporary constraint module 403 is used to determine the temporary constraint function of the vertex according to the result of the collision detection and / or the stage flag.

[0280] The calculation module 404 is used to obtain the model parameters of the soft tissue in the current frame by using position-based dynamics calculation, wherein the constraint function of the vertex includes the temporary constraint function.

[0281] The updating module 405 is used to determine whether to update the phase flag at least according to the result of the collision detection.

[0282] The display module 406 is used to display the soft tissue according to the model parameters of the soft tissue in the current frame.

[0283] It should be noted that: the soft tissue suturing simulation device provided in the above embodiment only uses the division of the above program modules as an example to illustrate the process of implementing the soft tissue suturing simulation method. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the method steps described above. In addition, the soft tissue suturing simulation device provided in the above embodiment and the soft tissue suturing simulation method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0284] See also Fig.24 As shown, it is a schematic diagram of the structure of a soft tissue suturing simulation device provided according to an embodiment of the present application. The object simulation device provided in the embodiment of the present application includes a memory 502 and a processor 501, and the memory 502 stores a computer program executable by the processor 501. When the computer program is executed by the processor 501, a soft tissue suturing simulation method in any embodiment of the present application is implemented.

[0285] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned soft tissue suturing simulation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Computer-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0286] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A soft tissue suturing simulation method, characterized in that: include: Determine a target pose offset of a stitching line in a current frame and obtain a stage mark, wherein a model of the stitching line includes a plurality of particle points arranged in a chain and rod nodes connected between adjacent particle points; Determine the temporary constraint function of the particle point at least according to a stage mark, the stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, the stage including the puncture stage, the suturing stage and the tensioning stage, wherein if the stage mark is the suturing stage, the temporary constraint function of the target particle point located inside the soft tissue includes a path point constraint based on an associated path point corresponding to the target particle point in a path point sequence, the path point sequence is obtained by interpolating a moving path of a first control point in the soft tissue using a particle point interval of the suture line, the first control point being a particle point directly connected to a needle tail of a suture needle; According to the target posture offset, a model parameter of the suture line in the current frame is obtained by using a position dynamics-based calculation, wherein the constraint function of the particle point includes the temporary constraint function; The seam line is displayed according to the model parameters of the seam line in the current frame.

2. The soft tissue suturing simulation method according to claim 1, characterized in that: Also includes: Obtaining the position of the first control point in the current frame; If the position of the first control point in the current frame is within the soft tissue, determining whether a first distance between the position of the first control point in the current frame and the current path point in the path point sequence is within a deviation interval determined based on the particle point interval; If the first distance is within the deviation interval, adding the position of the first control point in the current frame as a new path point to the path point sequence; If the first distance is greater than the upper limit of the deviation interval, interpolating between the current path point and the position of the first control point in the current frame using the particle point interval, and adding the interpolation result as a new path point to the path point sequence; If the first distance is less than the lower limit of the deviation interval, the waypoint sequence is not updated.

3. The soft tissue suturing simulation method according to claim 2, characterized in that: The obtaining the position of the first control point in the current frame comprises: Determine the position offset of the first control point in the current frame according to the target posture offset; The sum of the position of the first control point in the previous frame and the position offset of the first control point in the current frame is calculated as the position of the first control point in the current frame.

4. The soft tissue suturing simulation method according to claim 1, characterized in that: The step of determining the temporary constraint function of the particle point at least according to the stage mark comprises: If the stage mark is the suturing stage, determining whether the first control point in the current frame is located inside the soft tissue; If the first control point is located inside the soft tissue, the first control point is used as a reference point, and the position change amount of the reference point is determined according to the position change amount of the first control point; if the first control point is located outside the soft tissue, the particle point in the suture line that is closest to leaving the soft tissue is used as a reference point, and the position change amount of the reference point is calculated based on position dynamics; Determine the target particle point in the current frame and the corresponding relationship between the target particle point and the path point in the path point sequence according to the position change of the reference point and the particle point interval; The path point constraint is added to the target particle point based on the corresponding relationship.

5. The soft tissue suturing simulation method according to claim 4, characterized in that: The step of determining the target particle point in the current frame and the corresponding relationship between the target particle point and the path point in the path point sequence according to the position change of the reference point and the particle point interval comprises: Calculate the required position change of the reference point in the current frame according to the position change of the reference point in the current frame and the position of the remaining reference points in the previous frame; Determine whether the required position change of the reference point is greater than the particle point interval; If not, the required position change of the reference point is used as the remaining reference point position, and the target particle point in the previous frame and the corresponding relationship between the target particle point and the path point in the path point sequence are used as the target particle point in the current frame and the corresponding relationship between the target particle point and the path point in the path point sequence; If so, the number of particle points of the suture line moving into the soft tissue and the remaining reference point positions in the current frame are determined according to the ratio of the required position change of the reference point to the particle point interval, and the target particle points in the previous frame and the corresponding relationship between them and the path points in the path point sequence are updated according to the number of particle points of the suture line moving into the soft tissue to obtain the target particle points in the current frame and the corresponding relationship between them and the path points in the path point sequence.

6. The soft tissue suturing simulation method according to claim 1, characterized in that: Also includes: Performing collision detection on the suture thread, the soft tissue, and surgical instruments other than the suture needle; Whether to update the phase flag is determined at least according to the result of the collision detection.

7. The soft tissue suturing simulation method according to claim 6, characterized in that: The step of determining whether to update the stage flag at least according to the result of the collision detection comprises: If the stage mark is the puncture stage and it is detected that the first control point collides with the soft tissue, the stage mark is updated to the suturing stage.

8. The soft tissue suturing simulation method according to claim 6, characterized in that: The step of determining whether to update the stage flag at least according to the result of the collision detection comprises: If the stage mark is the suturing stage and it is detected that the preset knot in the suture line collides with the surface of the soft tissue, the stage mark is updated to the tightening stage.

9. The soft tissue suturing simulation method according to claim 6, characterized in that: The step of determining whether to update the stage flag at least according to the result of the collision detection comprises: If the stage mark is the suturing stage and it is detected that the suture line collides with the second clamping body, the stage mark is updated to the tensioning stage, and the second clamping body is a surgical instrument different from the suture needle and the first clamping body that clamps the suture needle.

10. A soft tissue suturing simulation device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the soft tissue suturing simulation method according to any one of claims 1 to 9 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the soft tissue suturing simulation method according to any one of claims 1 to 9 is implemented.

Citation Information

Patent Citations

  • Tissue deformation method based on mass spring and hydromechanics

    CN105912859A

  • Virtual operation method of abdominal cavity minimally invasive surgery robot

    CN109009434A

  • Movable cable motion simulation method based on position dynamics and controller

    CN114237076A

  • Stitching interactive simulation method in virtual operation

    CN115439611A

  • Apparatus and method for minimally invasive suturing

    US20080255590A1