Soft tissue suturing simulation methods, apparatuses, and media

Abstract

The application provides a soft tissue suturing simulation method, device and medium, the method comprising: obtaining a model of soft tissue and a suture thread; determining a target particle point according to the model of the suture thread; determining a path point sequence according to the model of the soft tissue; determining whether it is in a suturing stage or a tensioning stage in a suturing process; if it is in the suturing stage or the tensioning stage in the suturing process, determining a foot of a perpendicular of the target particle point according to the target particle point and the path point sequence; determining a first constraint function of the target particle point based on the target particle point and the foot of the perpendicular, the first constraint function being used to constrain a distance between the target particle point and the foot of the perpendicular to satisfy a preset condition; obtaining a model parameter of the suture thread in a current frame by using position dynamics based on the first constraint function; and displaying the suture thread according to the model parameter of the suture thread in the current frame, so that a realistic suturing simulation experience can be achieved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer graphics, and particularly relates to a soft tissue suturing simulation method, device and medium. BACKGROUND

[0002] Suturing is an indispensable link in surgical simulation. The suturing process involves multiple aspects such as simulation of suturing thread, simulation of soft tissue, simulation of surgical instruments such as suturing needle, surgical forceps, forceps, etc., and interaction between the above entities, such as the surgical forceps clamping the suturing needle to pierce the surface of the soft tissue and enter the soft tissue inside, the suturing needle driving the suturing thread through the soft tissue, the suturing thread tightening the soft tissue to fit together, and the suturing thread finally knotting to fix the soft tissue.

[0003] Traditional methods mostly use the Follow The Leader (FTL) method to simulate the suturing thread, use the Mass-Spring Model (MSM) to simulate the soft tissue, and use the particle model to simulate the surgical instruments. Since the three use different models, there is inevitably a certain mode penetration when processing their interaction, and this way is not convenient for expanding simulation of more forms of organ tissues, so that the entire surgical simulation is composed of multiple different frameworks, increasing the complexity of simulation. SUMMARY

[0004] To solve the existing technical problems, the present application provides a soft tissue suturing simulation method, device and computer readable storage medium capable of simulating the soft tissue suturing process in real time and accurately.

[0005] In a first aspect, the embodiments of the present application provide a soft tissue suturing simulation method, which comprises:

[0006] obtaining a model of soft tissue and a model of suturing thread, the model of the suturing thread comprising a plurality of particle points arranged in a chain;

[0007] determining a target particle point according to the model of the suturing thread, the target particle point being a particle point of the suturing thread located inside the soft tissue;

[0008] determining a path point sequence according to the model of the soft tissue;

[0009] judging whether it is in a suturing stage or a tightening stage in the suturing process;

[0010] if it is in the suturing stage or the tightening stage in the suturing process, determining a foot point of the target particle point according to the target particle point and the path point sequence;

[0011] determine a first constraint function of the target particle point based on the target particle point and the foot point, the first constraint function being used to constrain a distance between the target particle point and the foot point to satisfy a preset condition;

[0012] calculate a model parameter of the suture in the current frame based on the first constraint function using position dynamics;

[0013] display the suture according to the model parameter of the suture in the current frame.

[0014] Optionally, the model of the soft tissue includes a plurality of vertices and tetrahedrons defined by the vertices, and the method further includes:

[0015] add constraints for the vertices of the tetrahedrons based on the first constraint function and calculate a model parameter of the soft tissue in the current frame using position dynamics;

[0016] display the soft tissue according to the model parameter of the soft tissue in the current frame.

[0017] Optionally, the model of the soft tissue includes a plurality of vertices and tetrahedrons defined by the vertices, and the determining a path point sequence according to the model of the soft tissue includes:

[0018] obtain a position of a needle tail and determine an initial path point sequence according to the position of the needle tail, the initial path point sequence including a plurality of path points;

[0019] determine a tetrahedron in which the path points are located according to the model of the soft tissue;

[0020] add constraints for the vertices of the tetrahedron in which the path points are located based on the first constraint function and calculate a model parameter of the soft tissue in the current frame using position dynamics;

[0021] update coordinates of the path points according to the model parameter of the soft tissue;

[0022] determine the path point sequence according to the updated coordinates of the path points.

[0023] Optionally, the updating the coordinates of the path points according to the model parameter of the soft tissue includes:

[0024] obtain initial coordinates of each vertex of the tetrahedron in which the initial path points are located at a time when the position of the needle tail is obtained;

[0025] determine a barycentric coordinate coefficient according to the initial path points and the initial coordinates;

[0026] obtain current coordinates of each vertex of the tetrahedron in which the initial path points are located in the current frame in the model parameter of the soft tissue;

[0027] determining an updated coordinate of the path point according to the barycentric coordinate coefficient and the current coordinate.

[0028] Optionally, the method further comprises:

[0029] determining a second constraint function, the second constraint function comprising at least one of a distance constraint between adjacent vertices, a surface area invariance constraint of the tetrahedron, and a volume invariance constraint of the tetrahedron;

[0030] the adding of the constraint for the vertex of the tetrahedron in which the path point is located and the calculation of the model parameter of the soft tissue in the current frame using position dynamics based on the first constraint function, comprising:

[0031] the adding of the constraint for the vertex of the tetrahedron and the calculation of the model parameter of the soft tissue in the current frame using position dynamics based on the first constraint function and the second constraint function.

[0032] Optionally, the method further comprises:

[0033] performing collision detection between a needle head of a suture needle and a surface of the soft tissue;

[0034] if a frame number of continuous collision between the needle head and the surface of the soft tissue is greater than a first threshold value, determining a third constraint function, and adding a constraint for the vertex of the tetrahedron based on the third constraint function, the third constraint function comprising that a collision response displacement of the soft tissue is the same as a collision response displacement of the needle head, the collision response displacement of the soft tissue being a position of a puncture point in the model parameter of the soft tissue in the current frame minus a position of the puncture point in the model parameter of the soft tissue in a previous frame.

[0035] Optionally,

[0036] the method further comprises:

[0037] if the collision between the needle head and the surface of the soft tissue is detected, adding 1 to a cumulative frame number and temporarily storing a collision response displacement of the soft tissue in the current frame, otherwise setting the cumulative frame number to 0 and clearing all temporarily stored collision response displacements;

[0038] if the cumulative frame number is greater than the first threshold value, adding all temporarily stored collision response displacements to obtain a cumulative displacement amount;

[0039] comparing the cumulative displacement amount with a second threshold value to determine whether the needle head pierces the surface of the soft tissue.

[0040] Optionally, the method further comprises:

[0041] an acquisition phase flag, the phase flag being used to indicate a phase in which the current frame is located in a soft tissue suturing process, the phase including a puncture phase, a suturing phase and a tensioning phase;

[0042] if the needle has not yet penetrated the surface of the soft tissue and the phase flag is empty, updating the phase flag to be in the puncture phase of penetrating the soft tissue;

[0043] if the needle has penetrated the surface of the soft tissue and the phase flag is empty or in the puncture phase of penetrating the soft tissue, updating the phase flag to be in the puncture phase of having penetrated the soft tissue;

[0044] if the needle has not yet penetrated the surface of the soft tissue and the phase flag is in the puncture phase of having penetrated the soft tissue, updating the phase flag to be in the puncture phase of penetrating out of the soft tissue;

[0045] if the needle has penetrated the surface of the soft tissue and the phase flag is in the puncture phase of having penetrated the soft tissue or penetrating out of the soft tissue, updating the phase flag to be in the puncture phase of having penetrated out of the soft tissue.

[0046] Optionally, the determining whether the soft tissue suturing process is in the suturing phase or the tensioning phase includes:

[0047] matching a constraint path point corresponding to the target particle point from the path point sequence;

[0048] calculating a distance between the target particle point and the constraint path point;

[0049] if the distance is greater than a preset distance threshold, determining that the soft tissue suturing process is in the suturing phase, otherwise determining that the soft tissue suturing process is in the tensioning phase.

[0050] Optionally, the calculating the distance between the target particle point and the constraint path point includes:

[0051] finding a target particle point with the largest deformation among the target particle points;

[0052] calculating a distance between the target particle point with the largest deformation and the constraint path point.

[0053] Optionally, the matching a constraint path point corresponding to the target particle point from the path point sequence includes:

[0054] judging a position of the target particle point among the target particle points;

[0055] if the target particle point is a first target particle point, taking a first path point as the constraint path point;

[0056] If the target particle point is a particle point between a target particle point and a last target particle point, a path point closest to the target particle point is found from the path point sequence as the constraint path point;

[0057] If the target particle point is a last target particle point, a last path point is taken as the constraint path point.

[0058] In a second aspect, the embodiments of the present application provide a soft tissue suturing simulation method, including:

[0059] Obtaining a model of soft tissue and a model of a suture line, the model of the suture line including a plurality of particle points arranged in a chain, and the model of the soft tissue including a plurality of vertices and tetrahedrons defined by the vertices;

[0060] According to the model of the suture line, determining a target particle point, the target particle point being a particle point of the suture line located inside the soft tissue;

[0061] According to the model of the soft tissue, determining a path point sequence;

[0062] Judging whether a suturing stage or a tensioning stage in a suturing process is reached;

[0063] If the suturing stage or the tensioning stage in the suturing process is reached, according to the target particle point and the path point sequence, determining a foot of the perpendicular of the target particle point;

[0064] Based on the target particle point and the foot of the perpendicular, determining a first constraint function of the target particle point, the first constraint function being used to constrain a distance between the target particle point and the foot of the perpendicular to satisfy a preset condition;

[0065] Based on the first constraint function, using position dynamics to calculate a model parameter of the soft tissue in a current frame;

[0066] Displaying the soft tissue according to the model parameter of the soft tissue in the current frame.

[0067] In a third aspect of the embodiments of the present application, a soft tissue suturing simulation device is provided, including a memory and a processor, the memory storing a computer program, and the computer program being executed by the processor to implement the soft tissue suturing simulation method of any of the embodiments of the present application.

[0068] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which can store a program, and the program is executed by a processor to implement part or all steps of the soft tissue suturing simulation method of the first aspect.

[0069] In a fifth aspect, the embodiments of the present application provide a computer program product, which, when running on a computer, causes some or all of the steps of the soft tissue suturing simulation method of any one of the first aspect to be performed.

[0070] It can be understood that the beneficial effects of the second aspect to the fifth aspect described above can be referred to the related description in the first aspect, which will not be repeated here.

[0071] Compared with the prior art, the embodiments of the present application have the beneficial effects that: the embodiments of the present application obtain the model of soft tissue and the model of suture line, the model of suture line includes a plurality of particle points arranged in a chain; according to the model of suture line, a target particle point is determined, the target particle point is a particle point of the suture line located inside the soft tissue; a path point sequence is determined according to the model of soft tissue; it is judged whether it is in a suturing stage or a tensioning stage in the suturing process; if it is in the suturing stage or the tensioning stage in the suturing process, a foot point of the target particle point is determined according to the target particle point and the path point sequence; based on the target particle point and the foot point, a first constraint function of the target particle point is determined, the first constraint function is used to constrain the distance between the target particle point and the foot point to satisfy a preset condition; based on the first constraint function, a model parameter of the suture line in a current frame is calculated using position dynamics; the suture line is displayed according to the model parameter of the suture line in the current frame, which can realize a realistic suturing simulation experience. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0073] Figure 1 The model diagram of the suture line of the elastic rod is introduced;

[0074] Figure 2 The flowchart of the position-based dynamics of the suture line of the elastic rod is introduced;

[0075] Figure 3 The flowchart of the soft tissue suturing simulation method in an embodiment of the present application is introduced;

[0076] Figure 4 The schematic diagram of each stage of the soft tissue suturing process is introduced;

[0077] Figure 5 The puncture schematic diagram in the soft tissue suturing simulation method in another embodiment of the present application is introduced;

[0078] Figure 6 Fig. 1 is a schematic diagram of a first constraint function in an embodiment of the present application;

[0079] Figure 7 Fig. 2 is a schematic diagram of an embodiment of a soft tissue suturing simulation device in another embodiment of the present application; DETAILED DESCRIPTION

[0080] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0081] It is to be understood that the terminology “includes”, “has”, “holds”, “contains” used in the present specification and in the accompanying claims are used inclusively and that phenomena also include additional, unrecited members.

[0082] It is also to be understood that the terminology “and / or” used in the present specification and in the accompanying claims are used to describe one or more of the associated listed items and all possible combinations of the items, and include the combinations.

[0083] In the field of computer graphics, a common physical method for simulating the deformation of an object in real time generally first updates the acceleration by an external force, and then integrates to obtain the velocity and position. PBD is directly modified by a constraint function to update the position, and then updates the velocity by the position. The system equation in the PBD algorithm is constructed by a constraint function equation set, and the constraint function equation set includes a plurality of constraint function equations (which can be referred to as constraint functions).

[0084] In PBD, the model of an object includes a plurality of particle points (also referred to as mass points) that constitute the object, and the parameters of the particle points include their positions, velocities and masses. The parameters of the constraint function include the positions of the particle points, and the constraint function is used to describe the relationship between the particle points that constitute the object.

[0085] In the prediction part of the current frame, the force of the external environment on the object in the current frame is converted into displacement as much as possible, and the external force (such as gravity) that cannot be directly converted into displacement is retained. Based on the positions of the particle points in the previous frame, the positions of the particle points in the current frame are predicted in combination with the converted displacement and the external force that cannot be directly converted.

[0086] The aforementioned prediction process often simply uses integration (e.g., explicit Euler integral) to directly predict the positions of each particle in the current frame, without considering the object's mechanical properties or its interaction with the environment. Therefore, it cannot be directly used as the simulation result for the current frame; instead, constraint functions are used to correct the particle positions. Constraint functions are used to restrict the relationships between particle points and are mainly divided into two types: temporary constraint functions, which are generally generated temporarily when an interaction between an object and an external object in the environment is detected; and regular constraint functions, which are determined by the object's mechanical properties. The process of calculating the corrected position that best satisfies the constraint functions based on the constraint functions and the current position is called constraint projection, which is the core process of PBD. Constraint correction essentially involves iteratively optimizing the particle positions based on the constraint function equations. Specifically, this is typically achieved by iteratively executing constraint projection using the constraint functions in the constraint function equations until a predetermined number of iterations is reached, in order to find the particle positions that best satisfy all constraint functions in the constraint function equations.

[0087] After constraint correction, the result is the position of each particle point in the current frame. Based on this, the velocity of each particle point in the current frame can be calculated, and then the corresponding variables are updated for display and calculation in the next frame. The Δt used in PBD is generally the default frame interval, that is, the duration between two adjacent frames under normal circumstances.

[0088] For soft tissue, the model is a mesh model consisting of multiple vertices and tetrahedrons defined by the vertices. Here, the vertices are similar to particle points. The constraint functions mainly include at least one of the following: distance constraints between adjacent vertices, surface area invariance constraints of tetrahedrons, and volume invariance constraints of each tetrahedron.

[0089] Surgical instruments, such as forceps, needle holders, tweezers, and suture needles, can be approximated as rigid bodies. Their models typically include multiple point masses / particles determined by their specific shapes. For example, a suture needle model may include multiple point masses arranged in a chain. Since surgical instruments other than suture needles are generally directly controlled by the user, the PBD (Programmable Analytical Design) process can be omitted, and their attitude and the positions of each point mass in the model can be directly calculated using kinematic equations.

[0090] For sutures, to more accurately describe their morphology, rod nodes, also known as elastic rods, are further introduced. For example... Figure 1 As shown, the suture model includes multiple particle points arranged in a chain (e.g. Figure 1 p1, p2, ..., p n ) and rod nodes connecting adjacent particle points (e.g. Figure 1 In q1, q2, ..., q n-1The extension directions shown in the diagram are only used to specify the numbering order of particle points and lever nodes and do not represent the actual extension directions. The parameters of the lever nodes include their attitude, angular velocity, and moment of inertia. Correspondingly, the parameters of the constraint function also include the attitude of these lever nodes. For ease of calculation, the attitude is generally represented using a unit quaternion; however, in other embodiments, a transformation / rotation matrix can also be used to represent the attitude.

[0091] In practical applications, sutures are often connected to an external object, which controls the movement of the suture and can be called the control body. For example, if the suture is the suture itself, then the control body is the suture needle, surgical forceps, tweezers, etc., that directly move the suture. If the suture is hair, the control body is the skin or hair ornament that holds the hair in place, and the object that directly moves the hair is a finger or comb. The particle point in the suture that is directly connected to the control body, that is, the particle point closest to the control body, can be called the control point.

[0092] The process of introducing a flexible rod in a PBD is as follows: Figure 2 As shown in the attached diagram, the PBD process will be explained in detail below.

[0093] Figure 2 Steps 1-4 constitute the initial stage. In this stage, based on preset object parameters such as initial position, initial posture, and number of particle points, the parameters of all particle points and rod nodes that make up the object are initialized. Then, the real-time simulation stage begins, simulating the object's deformation process frame-by-frame based on environmental influences (including potential user commands). Figure 2 Steps 5-21 can specifically include three parts: prediction, constraint correction, and variable update.

[0094] In the prediction part of the current frame (corresponding to) Figure 2 In steps 6-12), the forces exerted on the object by the external environment in the current frame are converted into displacements as much as possible; external forces that cannot be directly converted into displacements (such as gravity) are retained. Similarly, the torques exerted on the object by the external environment in the current frame are converted into angular displacements as much as possible; external torques that cannot be directly converted into angular displacements are retained. Based on the positions of each particle point in the previous frame, the positions of each particle point in the current frame are predicted by combining the converted displacements and external forces that cannot be directly converted. Similarly, based on the attitudes of each link node in the previous frame, the attitudes of each link node in the current frame are predicted by combining the converted angular displacements and external torques that cannot be directly converted.

[0095] The above prediction part is often only used to directly predict the positions of the particle points and the poses of the rod nodes in the current frame by using integral (for example, explicit Euler integral), without considering the mechanical properties of the object and the interaction with the environment, and thus cannot be directly used as the simulation result of the current frame, but needs to be corrected by using a constraint function. The constraint function is used to limit the positions of the particle points, the poses of the rod nodes, and the relationship between the particle points and the rod nodes, and mainly includes two types, one is a temporary constraint function, which is generally generated temporarily when it is detected that the object interacts with an external object in the environment, and the other is a regular constraint function, which is determined according to the mechanical properties of the object. For example, for a suture and similar linear object, a corresponding regular constraint function can be given according to the mechanical properties that it is not easy to stretch and shear, but easy to bend and twist. The process of calculating the corrected positions / poses that meet the constraint function as much as possible according to the constraint function and the current positions / poses is called constraint projection, which is the core process of PBD. The constraint correction is essentially an iterative optimization of the positions of the particle points and the poses of the rod nodes based on the constraint function equation set, and the specific implementation is generally to iteratively execute the constraint projection using the constraint functions in the constraint function equation set until a set number of iterations is reached, so as to obtain the particle point positions and the rod node poses that meet all the constraint functions in the constraint function equation set as much as possible. The constraint correction part corresponds to steps 13-15 in Figure 2 , wherein step 13 is used for collision detection to generate a temporary constraint function, and steps 14-15 are used for iterative optimization of the positions of the particle points and the poses of the rod nodes by using the constraint function equation set.

[0096] The update part corresponds to steps 16-21 in Figure 2 . After the constraint correction is completed, the obtained result is the positions of the particle points and the poses of the rod nodes in the current frame, based on which the velocities of the particle points and the angular velocities of the rod nodes in the current frame can be calculated, and then the corresponding variables are updated for display and calculation of the next frame. The Δt used in PBD is generally the default frame interval, that is, the time length between adjacent two frames under normal circumstances.

[0097] After the simulation object deformation process of the current frame is executed, the model parameters of the current frame can be obtained, including the updated parameters of the particle points and the rod nodes, wherein the updated positions of the particle points and the poses of the rod nodes can be used to determine the shape of the suture in the current frame for display.

[0098] It should be noted that the vertices, particle points and mass points in the present application are actually identical in the PBD framework, and only different names are selected for the particle points in different models for the sake of simplifying the description, that is, the particle points in soft tissue are called vertices, the particle points in surgical instruments are called mass points, and the particle points in sutures are called particle points.

[0099] In combination with the foregoing preliminary introduction of the soft tissue suturing simulation method, the following will be combined with the Figure 3 The soft tissue suturing simulation method provided by the embodiments of the present application will be further described in detail. Since object simulation is a real-time process, when referring to the current frame below, it means that the specific process of object simulation in a frame is described taking the current frame as an example. In fact, the process is executed for each frame.

[0100] For ease of description, only one puncture is taken as an example for description in the present application. In actual application, multiple punctures can occur. In this case, each piece of soft tissue has its own puncture stage and suturing stage, which is not limited herein.

[0101] As shown in Figure 3 The soft tissue suturing simulation method provided by the embodiments of the present application includes the following steps.

[0102] S1: Obtain models of soft tissue, suturing thread, and surgical instrument.

[0103] The model of the soft tissue is a mesh model including a plurality of vertices and tetrahedrons defined by the vertices. The model of the suturing thread includes a plurality of particle points arranged in a chain and a rod node connected between adjacent particle points. The model of the surgical instrument includes a plurality of mass points arranged according to the shape of the surgical instrument. The surgical instrument includes a suturing needle. The specific information of the model can be referred to the foregoing related description. The model is constructed according to the prior art, and the embodiments of the present application do not make any limitation thereto.

[0104] S2: Determine a target particle point according to the model of the suturing thread, the target particle point being a particle point of the suturing thread located inside the soft tissue.

[0105] S3: Determine a path point sequence according to the model of the soft tissue.

[0106] The position of the needle tail in each frame during the movement of the needle tail in the soft tissue, which can also be referred to as a moving point, is arranged in time sequence to form a moving path. In the suturing stage, the suturing thread generally moves along the moving path when moving in the soft tissue. Therefore, the constraint of the moving path on the suturing thread can be used as the collision response constraint between the suturing thread and the soft tissue acting on the suturing thread.

[0107] Since the distribution of the moving points in the moving path is affected by the operation of the user and is not necessarily uniform, if the temporary constraint is directly added to the particle points in the suturing thread, i.e., the target particle points, according to the moving path, the interval between adjacent target particle points after updating will be affected by the distribution of the moving points and will no longer be uniform and can have a large error from the initially set interval between the particle points, i.e., the interval between adjacent particle points in the suturing thread, thereby causing a significant change in the length of the suturing thread.

[0108] To avoid the length of the suture thread changing obviously during the movement of the suture thread in the soft tissue and reduce the distortion caused thereby, in this embodiment, the path point sequence is determined according to the model of the soft tissue, and specifically includes:

[0109] S31: Obtain the position of the needle tail, and determine an initial path point sequence according to the position of the needle tail, the initial path point sequence including a plurality of path points.

[0110] For the suture needle, if it is treated as a rigid body, in the case where the suture needle is clamped by other surgical instruments, the PBD process can be omitted, and the pose of the suture needle and the positions of the particles in the model are directly calculated by using the kinematics equation combined with the pose of the surgical instrument clamping the suture needle; in the case where the suture needle is not clamped by other surgical instruments, to prevent the suture needle from being penetrated, the positions of the particles in the model can be calculated by using the PBD / XPBD process, specifically, the overall collision constraint of the suture needle can be determined according to the result of the collision detection between the suture needle and the soft tissue, and the positions of the particles in the suture needle model are calculated based on the overall collision constraint by using the position dynamics. If the elasticity of the suture needle is considered, it cannot be treated as a rigid body, and whether it is clamped by other surgical instruments or not, the positions of the particles in the model need to be calculated by using the PBD / XPBD process. According to the above calculation method of the positions of the particles in the suture needle, the positions of the particles in the suture needle in the current frame can be obtained, and combined with the position of the needle tail in the suture needle, the position of the needle tail in the current frame can be obtained.

[0111] S32: Determine the tetrahedron in which the path point is located according to the model of the soft tissue.

[0112] S33: Add constraints for the vertices of the tetrahedron based on the first constraint function and calculate the model parameters of the soft tissue in the current frame by using the position dynamics. In the embodiment of the application, the model parameters can be referred to as the parameters of the model, that is, the parameters of the particle points in the model, for example, the aforementioned position, velocity and mass.

[0113] S34: Update the coordinates of the path point according to the model parameters of the soft tissue, and specifically includes:

[0114] S341: Obtain the initial coordinates of each vertex corresponding to the tetrahedron in which the initial path point is located at the time when the position of the needle tail is obtained.

[0115] S342: Determine the barycentric coordinate coefficient according to the initial path point and the initial coordinates. The coordinates of any point in the tetrahedron can be written as the weighted average of the coordinates of the four vertices of the tetrahedron, and the four weights are the barycentric coordinate coefficients of the point. In the entire suturing process, the barycentric coordinate coefficients of the path points can remain unchanged.

[0116] S343: Obtain current coordinates of each vertex corresponding to the tetrahedron in which the initial path point is located in the model parameters of the soft tissue in the current frame.

[0117] S344: Determine the coordinates of the updated path point according to the barycentric coordinate and the current coordinates.

[0118] S35: Determine the path point sequence according to the coordinates of the updated path point.

[0119] In the embodiment of the application, the path point sequence is first established by the position of the needle tail, and then continuously iteratively optimized. In the iterative optimization process, the barycentric coordinate of each path point in the tetrahedron is kept unchanged. The process of position dynamics calculation is a continuous iterative optimization process, so the process of determining the path point sequence is a process of continuously updating the previous iterative result using the new iterative result after updating the initial path point sequence.

[0120] Further, a second constraint function can be determined, the second constraint function including at least one of a distance constraint between adjacent vertices, a tetrahedron surface area invariable constraint, and a tetrahedron volume invariable constraint. Then, based on the first constraint function and the second constraint function, constraints are added to the vertices of the tetrahedron, and the model parameters of the soft tissue in the current frame are calculated using position dynamics.

[0121] Step S34 updates the coordinates of the path point according to the model parameters of the soft tissue, specifically including:

[0122] S341: Obtain the barycentric coordinates of the path point in the tetrahedron before adding the collision response constraint.

[0123] The barycentric coordinates, also known as centroid coordinates or volume coordinates, refer to the coordinates defined by the four vertices of the tetrahedron. The coordinates of any point in the tetrahedron can be written as a weighted average of the coordinates of the four vertices of the tetrahedron. The four weights are the barycentric coordinates of the point.

[0124] S342: Obtain the coordinates of the updated path point by combining the coordinates of each vertex of the tetrahedron in the model parameters of the soft tissue and the barycentric coordinates.

[0125] The barycentric coordinates of the path point are used as weights to calculate the weighted average of the coordinates of each vertex of the tetrahedron in the model parameters of the soft tissue. The result obtained is the coordinates of the updated path point.

[0126] Further, a second constraint function can be determined, the second constraint function including at least one of a distance constraint between adjacent vertices, a surface area invariance constraint of the tetrahedron, and a volume invariance constraint of the tetrahedron; then, based on the first constraint function and the second constraint function, adding constraints for vertices of the tetrahedron and calculating model parameters of the soft tissue in the current frame using position dynamics.

[0127] Then, coordinates of the path points are updated according to the model parameters of the soft tissue, and a path point sequence is determined according to the updated coordinates of the path points.

[0128] S4: determining whether it is in a suture stage or a tensioning stage in the suture process.

[0129] As shown in FIG. 1, the soft tissue suture process mainly includes the following stages: a puncture stage, a suture stage, a tensioning stage, and a knotting stage. In order to simulate the process of soft tissue suture in the framework of PBD, at least one stage flag can be designed to distinguish different stages, and the stage flag can be updated as the stage changes. In the real-time simulation process of different types of objects, the stage flag can be read to perform corresponding operations, such as adding corresponding temporary constraints (collision response constraints, first constraints, etc.), updating path points, etc. Since the stages that need to be concerned about in the simulation of different types of objects are not necessarily the same, different stage flags can be selected for different types of objects, or a unified stage flag can be set for all objects. In this application, the latter is taken as an example for illustration. Figure 4

[0130] From the perspective of user operation, in the puncture stage, the user uses a first clamping body (generally a surgical forceps, a needle holder, etc.) to clamp the middle and rear part of the suture needle to the soft tissue, and the needle head of the suture thread penetrates into the soft tissue and travels in the soft tissue until the needle head of the suture thread penetrates out of the soft tissue. Then the user moves the first clamping body to the front part of the suture needle, and then continues to pull the suture needle until the needle tail (i.e., the mass point of the suture needle directly connected to the suture thread) of the suture needle enters the soft tissue.

[0131] The suture stage generally starts from the needle tail of the suture needle driving the suture thread into the soft tissue, and ends when the tensioning flag appears. There are two kinds of tensioning flags. In one case, there is a pre-knotted knot at the tail of the suture thread, and the tensioning flag is the pre-knotted knot contacting the surface of the soft tissue. In another case, there is no pre-knotted knot, and the tensioning flag is that the user directly clamps the tail of the suture thread using a second clamping body (another surgical instrument different from the first clamping body, such as a forceps, etc.).

[0132] The tensioning stage generally starts from the appearance of the tensioning flag, and ends when the self-collision of the suture thread under the driving of the first clamping body and the second clamping body appears.

[0133] ​The knotting stage generally starts from the collision of the suture thread under the driving of the first clamping body and the second clamping body, and the end time can be determined according to requirements, for example, the suture thread is cut by surgical scissors.

[0134] Optionally, the stages of the suturing process can be different according to different knotting modes of the suture thread. For example, if multiple knots are needed in the suturing process, the tensioning stage and the knotting stage can be included after multiple suturing stages; or if only one knot is needed at the beginning and the end of the suturing, the tensioning stage and the knotting stage can be included after the first puncture stage and the suturing stage, and after the subsequent multiple puncture stages and suturing stages.

[0135] Optionally, the determination of whether the suturing process is in the suturing stage or the tensioning stage can be based on a stage flag. Specifically, the stage flag can be determined based on the collision detection result, the history record of the stage, and the relative position relationship between the surgical instrument and the soft tissue. In addition to the collision detection between the soft tissue and the suture thread, the collision detection between the soft tissue and the surgical instrument, and the collision detection between the suture thread and the surgical instrument, the stage flag can also include the self-collision detection of the suture thread and / or the collision detection between different surgical instruments.

[0136] Optionally, the determination of whether the suturing process is in the suturing stage or the tensioning stage can also include:

[0137] S41: matching a constraint path point corresponding to the target particle point from the path point sequence;

[0138] S42: calculating the distance between the target particle point and the constraint path point;

[0139] S43: if the distance is greater than a preset distance threshold, determining that the suturing process is in the suturing stage, otherwise determining that the suturing process is in the tensioning stage.

[0140] Specifically, step S41 can include:

[0141] S411: determining the position of the target particle point in the plurality of target particle points. Since the suture thread includes a plurality of particle points, the plurality of particle points can be assigned with serial numbers to determine the position relationship of the particle points in the plurality of particle points.

[0142] S412: if the target particle point is the first target particle point, the first path point is taken as the constraint path point;

[0143] If the target particle point is a particle point between one target particle point and the last target particle point, then the path point closest to the target particle point is found from the path point sequence as the constraint path point; optionally, the distance between the target particle point and each path point is calculated, and then the path point closest to the target particle point is selected from multiple distances as the constraint path point.

[0144] If the target particle point is the last target particle point, then the last path point will be used as the constraint path point.

[0145] Specifically, step S42 may include:

[0146] S421: Find the target particle point with the largest deformation among the target particle points.

[0147] During suturing, the target particle point closest to the surface of the soft tissue is the target particle point with the greatest deformation. For example... Figure 5 As shown, with the right side as the needle insertion side and the left side as the needle exit side, during the pulling process of the suture, the tension on each suture particle point, for example, when all particle points are on the right side, the target particle point on the right side that is closest to the surface of the soft tissue is the target particle point with the greatest deformation; when there are particle points on the left side, the target particle point on the left side that is closest to the surface of the soft tissue is the target particle point with the greatest deformation.

[0148] S422: Calculate the distance between the target particle point with the largest deformation and the constraint path point.

[0149] S5: If the process is in the suturing or tightening stage, determine the perpendicular foot of the target particle point based on the target particle point and the path point sequence, specifically including S51 to S53:

[0150] S51: Match the constraint path point corresponding to the target particle point from the path point sequence. For specific implementation, please refer to S41, which will not be repeated here.

[0151] S52: Determine two path points adjacent to the constrained path point;

[0152] S53: Determine the perpendicular foot based on the target particle point, the two adjacent path points, and the constraint path point.

[0153] like Figure 6 As shown, S is the target particle point, Q is the perpendicular foot, P is the constraint path point, and P′ and P″ are two adjacent path points. Points SPP′ and SPP″ form a triangle. If the angle between sides SP and PP′ is acute, then the perpendicular foot Q is inside triangle SPP′; if the angle between sides SP and PP″ is acute, then the perpendicular foot Q is inside triangle SPP″.

[0154] S6: determining a first constraint function of the target particle point based on the target particle point and the foot point, the first constraint function being used to constrain a distance between the target particle point and the foot point to satisfy a preset condition. Optionally, the first constraint function is used to constrain the distance between the target particle point and the foot point to be 0.

[0155] If the suture stage or the tensioning stage is in the suture process, the main temporary constraint is the constraint based on the first constraint function, which can be referred to as the first constraint. Since the first constraint function is a function of the distance, the first constraint is a distance constraint; at the same time, the first constraint is a constraint generated by the interaction of the suture line and the soft tissue, and thus is also a collision constraint. Each target particle point has a constraint path point corresponding thereto in the path point sequence, and the first constraint of each target particle point refers to the constraint between the target particle point and the foot point determined according to the constraint path point.

[0156] S7: determining model parameters of the suture line in the current frame based on the first constraint function by using position dynamics calculation.

[0157] The following formula can be used when the position dynamics calculation is used, so as to achieve the final iterative optimization and obtain the model parameters of the model of the suture line by calculating the position change amount △pi of the suture line particle.

[0158] △pi = -SWi▽piC(p1, p2, …, pn)

[0159] Wherein, S is a scaling factor, Wi is a mass inverse, △pi is a position update amount of the i-th point, ▽pi is a constraint gradient, C(p1, p2, …, pn) is a constraint function, and p1, p2, …, pn are a plurality of points, for example, a plurality of particle points of the suture line.

[0160] The first constraint function includes a plurality of parameters, for example, the mass inverse Wi of the target particle point. If it is determined that the tensioning stage is in the suture process, Wi in the first constraint function can be set to 0, and the position update amount of the particle point of the suture line corresponding thereto is 0, that is, the particle point of the suture line is basically stationary, mainly relying on the continuous updating of the foot point, or in other words, mainly relying on the continuous updating of the path point of the soft tissue, so that the distance between the particle point of the suture line and the foot point tends to 0.

[0161] S8: displaying the suture line according to the model parameters of the suture line in the current frame.

[0162] The model parameters of the current frame can be obtained after the simulation object deformation process of the current frame is performed, specifically including the parameters of the updated particle points and the rod nodes, wherein the positions of the updated particle points and the postures of the rod nodes can be used to determine the shape of the suture line in the current frame for display.

[0163] Optionally, the model of the soft tissue includes a plurality of vertices and tetrahedrons defined by the vertices, and the soft tissue suturing simulation method in the embodiments of the present application further includes:

[0164] adding constraints for the vertices of the tetrahedrons based on the first constraint function and calculating the model parameters of the model of the soft tissue in the current frame using position dynamics;

[0165] displaying the soft tissue according to the model parameters of the model of the soft tissue in the current frame.

[0166] Similarly, when using position dynamics calculation, the following formula can be used to calculate the position change △pi of the particles or the feet of the soft tissue, so as to achieve the final iterative optimization and obtain the model parameters of the model of the soft tissue.

[0167] Similarly, the change △'pi of the soft tissue or the feet

[0168] △'pi=-S'W'i▽'pi C'(p1',p2',...,pn')

[0169] Similarly, S' is a scaling factor, W'i is a mass inverse, △'pi is the position update of the i-th point, △'pi is a constraint gradient, C'(p1',p2',...,pn') is a constraint function, and p1', p2' to pn' are a plurality of points, such as a plurality of particle points or a plurality of feet of the soft tissue.

[0170] The embodiments of the present application adjust the change △pi of the suture and the change △'pi of the soft tissue through iterative optimization, so that the suture and the soft tissue both satisfy the constraint function, thereby achieving a good suturing simulation experience simulation, such as smooth sliding through the soft tissue under the driving of the needle, being able to perform suturing of the soft tissue, and the two tissues after suturing being able to better fit together without obvious penetration.

[0171] In one embodiment, the soft tissue suturing simulation method further includes:

[0172] performing collision detection on the needle head of the suturing needle and the surface of the soft tissue;

[0173] If the number of frames in which continuous collision of the needle head and the surface of the soft tissue is detected is greater than a first threshold value, a third constraint function is determined, and constraints are added for the vertices of the tetrahedrons based on the third constraint function, the third constraint function including that the collision response displacement of the soft tissue in the current frame is the same as the collision response displacement of the needle head in the current frame, the collision response displacement of the soft tissue in the current frame being the position of a puncture point in the model parameters of the soft tissue in the current frame minus the position of the puncture point in the model parameters of the soft tissue in the previous frame. The position of the puncture point is a kind of model parameter.

[0174] Further, the vertex of the tetrahedron can be added with constraints based on the first constraint function and the third constraint function, and the model parameters of the soft tissue in the current frame are calculated using position dynamics.

[0175] In the embodiments of the present application, simulation can be realized by constraining the tetrahedron of the soft tissue, and the constraint of the tetrahedron can be selected according to specific conditions. For example, the third constraint function and / or the second constraint function can be selected during puncture, and the first constraint function and / or the second constraint function can be selected during the suturing stage or the tensioning stage.

[0176] The puncture stage mainly focuses on the collision detection between the suture needle and the soft tissue. According to whether the surface of the soft tissue is pierced and the direction of the collision between the suture needle and the surface of the soft tissue, the puncture stage can be further divided into multiple sub-stages, which are as follows:

[0177] a. If the needle head of the suture needle collides with the surface of the soft tissue from the outside to the inside but does not pierce the surface of the soft tissue, the suturing stage is the penetration into the soft tissue in the puncture stage.

[0178] In this sub-stage, since the suture needle is generally clamped by the first clamping body, the collision response of the suture needle can not be performed, and only the collision response of the soft tissue is performed.

[0179] b. If the suture needle has pierced the surface of the soft tissue, and the needle head is located inside the soft tissue, the suturing stage is the penetration into the soft tissue in the puncture stage.

[0180] c. If the needle head collides with the surface of the soft tissue from the inside to the outside but does not pierce the surface of the soft tissue, the suturing stage is the penetration out of the soft tissue in the puncture stage.

[0181] In this sub-stage, since the suture needle is generally clamped by the first clamping body, the collision response of the suture needle can not be performed, and only the collision response of the soft tissue is performed.

[0182] d. If the suture needle has pierced the surface of the soft tissue, and the needle head is located outside the soft tissue, the suturing stage is the penetration out of the soft tissue in the puncture stage.

[0183] The needle head of the suture needle can refer to a small area of the needle tip portion. The direction in which the suture needle collides with the soft tissue surface, i.e., the above-mentioned "from the outside of the soft tissue to the inside" and "from the inside of the soft tissue to the outside", can be implemented in various ways. One way is to refer to the historical record of the phase change. If the previous stage does not appear the puncture stage of the soft tissue, the direction is from the outside to the inside, and vice versa. Another way is to refer to the relative position relationship between the suture needle and the soft tissue. If all parts of the suture needle except the part colliding with the soft tissue are located outside the soft tissue, the direction is from the outside to the inside. If the suture needle is located inside the soft tissue, the direction is from the inside to the outside. Another way is to refer to the relative relationship between the direction of the speed of the suture needle and the soft tissue. If the speed of the suture needle points to the inside of the soft tissue, the direction is from the outside to the inside. If the speed of the suture needle points to the outside of the soft tissue, the direction is from the inside to the outside.

[0184] Further, whether to update the phase flag can be determined according to the collision detection result between the soft tissue and the suture needle:

[0185] If the phase flag is in the puncture stage and the needle tail of the suture needle is detected to collide with the surface of the soft tissue, the phase flag is updated to the suture stage.

[0186] According to the puncture direction of the needle head and whether the needle head penetrates the surface of the soft tissue, the puncture stage can be divided into multiple sub-stages. The puncture direction can be determined in various ways. The following takes the history record of the phase flag as an example to illustrate the puncture direction.

[0187] Optionally, whether the needle head penetrates the surface of the soft tissue can specifically include:

[0188] S46: Collision detection is performed on the needle head of the suture needle and the surface of the soft tissue.

[0189] S47: If the number of frames in which the needle head collides with the surface of the soft tissue continuously is greater than a first threshold, the collision response displacement of the soft tissue is accumulated to obtain an accumulated displacement.

[0190] S48: If the accumulated displacement is greater than a second threshold, it is determined that the suture needle penetrates the surface of the soft tissue.

[0191] Since the suture needle is generally clamped by the first clamping body during the process of penetrating the surface of the soft tissue, if the suture needle is regarded as a rigid body, the collision response of the suture needle can be ignored, and only the collision response of the soft tissue is considered.

[0192] In an embodiment of the present application, determining the sub-stage of the puncture stage can specifically include:

[0193] S261: It is determined whether the needle head of the suture needle collides with the surface of the soft tissue in the current frame.

[0194] If yes, go to S262; otherwise go to S269.

[0195] S262: add 1 to the accumulated frame number, and temporarily store the collision response displacement of the current frame.

[0196] The collision response displacement is the position of the puncture point in the model parameters of the soft tissue in the current frame minus the position of the puncture point in the model parameters of the soft tissue in the previous frame.

[0197] S263: if the accumulated frame number is greater than the first threshold value, then accumulate all the temporarily stored collision response displacements to obtain an accumulated displacement amount.

[0198] S264: compare the accumulated displacement amount with a second threshold value to determine whether the needle has penetrated the surface of the soft tissue.

[0199] Specifically, if the accumulated displacement amount in the current frame is greater than the second threshold value, it is determined that the needle has penetrated the surface of the soft tissue; otherwise, it is determined that the needle has not penetrated the surface of the soft tissue. Alternatively, if the accumulated displacement amount in the current frame is greater than or equal to the second threshold value, it is determined that the needle has penetrated the surface of the soft tissue; otherwise, it is determined that the needle has not penetrated the surface of the soft tissue.

[0200] S265: if the needle has not penetrated the surface of the soft tissue, and the phase flag is empty, then update the phase flag to be in the puncture phase of penetrating into the soft tissue.

[0201] S266: if the needle has penetrated the surface of the soft tissue, and the phase flag is empty or in the puncture phase of penetrating into the soft tissue, then update the phase flag to be in the puncture phase of having penetrated into the soft tissue.

[0202] S267: if the needle has not penetrated the surface of the soft tissue, and the phase flag is in the puncture phase of having penetrated into the soft tissue, then update the phase flag to be in the puncture phase of penetrating out of the soft tissue.

[0203] S268: if the needle has penetrated the surface of the soft tissue, and the phase flag is in the puncture phase of having penetrated into the soft tissue or penetrating out of the soft tissue, then update the phase flag to be in the puncture phase of having penetrated out of the soft tissue.

[0204] S269: set the accumulated frame number to 0, and clear all the temporarily stored collision response displacements.

[0205] Steps S265 to S268 are used to update the phase flag, and therefore before step S265, it can further include obtaining a phase flag, which is used to indicate the phase in which the current frame is in during the soft tissue suturing process, and the phase includes a puncture phase, a suturing phase, and a tightening phase.

[0206] The collision response refers to processing the object that has collided, which generally tries to be close to human cognition of physical laws to avoid obviously abnormal interaction between the objects that have collided, such as penetration. When the needle head of the suture needle punctures the surface of the soft tissue, the puncture force of the needle head on the soft tissue and the resistance of the soft tissue to the needle head are generated. Since the soft tissue has a certain elasticity, the soft tissue will not be immediately pierced, but will be deformed in the puncture position along the movement direction of the suture needle. When the puncture force continues to increase and exceeds the maximum resistance that the soft tissue can generate, the soft tissue will be pierced.

[0207] In order to more simply simulate the process of piercing under the PBD framework, the simulation of the puncture force and the resistance of the soft tissue is abandoned in this embodiment, and the cumulative displacement amount of the collision response displacement of the soft tissue is selected to reflect the size of the puncture force, and the second threshold is used to reflect the maximum resistance that the soft tissue can generate, so that the force comparison is converted into the comparison between the cumulative displacement amount and the second threshold, and the PBD framework is more adapted. The second threshold can be set according to the characteristics of different organs and different tissues. For the same piece of soft tissue, the second threshold used for piercing the surface of the soft tissue from inside to outside and piercing the surface of the soft tissue from outside to inside can be the same or different.

[0208] In order to prevent the rubbing and collision of the needle head and the surface of the soft tissue from being incorrectly judged as puncture, in this embodiment, it is necessary to detect that the needle head and the surface of the soft tissue continuously collide, and the frame number of the continuous collision is greater than the first threshold, so as to confirm that the needle head starts to puncture the surface of the soft tissue, and starts to judge whether the needle head pierces the surface of the soft tissue according to the comparison between the cumulative displacement amount and the second threshold. After the needle head starts to puncture the surface of the soft tissue, if the cumulative displacement amount in the current frame is less than the second threshold, it is determined that the surface of the soft tissue has not been pierced, and the cumulative displacement amount is retained for use in the next frame. In order to prevent the jitter of the surface of the soft tissue from affecting the result of the piercing judgment, the collision response displacement is temporarily stored and accumulated in the form of a vector, that is, the cumulative displacement amount of the current frame is actually the displacement between the position of the surface of the soft tissue under the action of the collision response of the soft tissue and the needle head and the position before the surface of the soft tissue collides with the needle head in the current frame.

[0209] In one embodiment, the suture simulation method provided by the present application can further include:

[0210] obtaining a model of soft tissue and a model of a suture line, the model of the suture line including a plurality of particle points arranged in a chain, and the model of the soft tissue including a plurality of vertices and tetrahedrons defined by the vertices;

[0211] determining a target particle point according to the model of the suture line, the target particle point being a particle point of the suture line located inside the soft tissue;

[0212] determining a sequence of path points according to the model of the soft tissue;

[0213] Determine whether the process is in the suturing or tightening stage;

[0214] If the process is in the suturing or tightening stage, the foot of the target particle point is determined based on the target particle point and the path point sequence.

[0215] Based on the target particle point and the perpendicular foot, a first constraint function for the target particle point is determined. The first constraint function is used to constrain the distance between the target particle point and the perpendicular foot to meet a preset condition.

[0216] Based on the first constraint function, the model parameters of the soft tissue in the current frame are calculated using positional dynamics.

[0217] The soft tissue is displayed based on the model parameters of the soft tissue in the current frame.

[0218] Furthermore, it may also include:

[0219] Based on the first constraint function, the model parameters of the suture line in the current frame are calculated using positional dynamics.

[0220] The suture is displayed based on the model parameters of the suture in the current frame.

[0221] like Figure 7 The diagram shown is a schematic representation of a soft tissue suturing simulation device provided according to an embodiment of this application. The object simulation device provided in this application includes a memory 502 and a processor 501. The memory 502 stores a computer program executable by the processor 501. When the computer program is executed by the processor 501, it implements the soft tissue suturing simulation method as described in any embodiment of this application.

[0222] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described soft tissue suturing simulation method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium includes, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0223] It should be noted that other sorting schemes that can be easily conceived by those skilled in the art within the technical scope disclosed in this invention should also be within the protection scope of this invention, and will not be elaborated here.

[0224] For example, the computer program can be divided into one or more modules / units, one or more modules / units are stored in the memory and executed by the processor to complete the present application. One or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program.

[0225] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0226] The embodiments of the present application provide a computer program product, when the computer program product runs on the mobile terminal, so that the mobile terminal executes the steps to realize the above-mentioned various method embodiments.

[0227] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions. Each functional unit and module in the embodiments can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0228] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0229] As used in the specification and in the claims, the term “if’ can be interpreted as meaning “when” or “upon” or “in response to a determination” or “in response to a detection” depending on the context. Similarly, the phrase “if it is determined” or “if a described condition or event is detected” can be interpreted as meaning “upon a determination” or “in response to a determination” or “upon a detection of the described condition or event” or “in response to a detection of the described condition or event,” depending on the context.

[0230] In addition, the terms “first”, “second”, “third”, etc. are used in the description and in the claims of this application merely to differentiate descriptions, and cannot be understood as indicating or implying relative importance.

[0231] Reference in the specification to “one embodiment” or “some embodiments” etc. means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase “in one embodiment” or “in some embodiments” in various places in the specification are not necessarily all referring to the same embodiment, although it can be. The terms “comprising,” “including,” “having” and their variants, etc. mean “including but not limited to”, unless otherwise expressly specified or limited by the context.

[0232] Those skilled in the art can realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0233] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other ways. For example, the division of the above-described apparatus / terminal device embodiments is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0234] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0235] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0236] If the integrated module / unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electric carrier signal and telecommunication signal.

[0237] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of soft tissue suturing simulation, characterized in that, The method comprises: obtaining a model of soft tissue and a model of a suture line, the model of the suture line comprising a plurality of particle points arranged in a chain; determining a target particle point according to the model of the suture line, the target particle point being a particle point of the suture line located inside the soft tissue; determining a path point sequence according to the model of the soft tissue; determining whether a suture stage or a tensioning stage in a suture process is reached; if the suture stage or the tensioning stage in the suture process is reached, determining a foot point of the target particle point according to the target particle point and the path point sequence; determining a first constraint function of the target particle point based on the target particle point and the foot point, the first constraint function being used to constrain a distance between the target particle point and the foot point to satisfy a preset condition, wherein the preset condition is that the distance between the target particle point and the foot point is 0 or the distance between the target particle point and the foot point approaches 0; calculating a model parameter of the suture line in a current frame based on the first constraint function using position dynamics; displaying the suture line according to the model parameter of the suture line in the current frame.

2. The soft tissue suturing simulation method of claim 1, wherein, The model of the soft tissue comprises a plurality of vertices and tetrahedrons defined by the vertices, and the method further comprises: adding a constraint to a vertex of the tetrahedron based on the first constraint function and calculating a model parameter of the soft tissue in the current frame using position dynamics; displaying the soft tissue according to the model parameter of the soft tissue in the current frame.

3. The soft tissue suturing simulation method of claim 1, wherein, The model of the soft tissue comprises a plurality of vertices and tetrahedrons defined by the vertices, and the determining a path point sequence according to the model of the soft tissue comprises: obtaining a position of a needle tail and determining an initial path point sequence comprising a plurality of path points according to the position of the needle tail; determining a tetrahedron in which the path points are located according to the model of the soft tissue; adding a constraint to a vertex of the tetrahedron in which the path points are located based on the first constraint function and calculating a model parameter of the soft tissue in the current frame using position dynamics; updating coordinates of the path points according to the model parameter of the soft tissue; determining the path point sequence according to the updated coordinates of the path points.

4. The soft tissue suturing simulation method of claim 3, wherein, The updating coordinates of the path points according to the model parameter of the soft tissue comprises: obtaining initial coordinates of each vertex of the tetrahedron in which the initial path points are located at a time when the position of the needle tail is obtained; determining a barycentric coordinate coefficient according to the initial path points and the initial coordinates; obtaining current coordinates of each vertex of the tetrahedron in which the initial path points are located in the current frame in the model parameter of the soft tissue; determining updated coordinates of the path points according to the barycentric coordinate coefficient and the current coordinates.

5. The soft tissue suturing simulation method of claim 3, wherein, The method further comprises: determining a second constraint function, the second constraint function comprising at least one of a distance constraint between adjacent vertices, a surface area invariance constraint of a tetrahedron, and a volume invariance constraint of a tetrahedron; the adding a constraint to a vertex of the tetrahedron in which the path points are located based on the first constraint function and calculating a model parameter of the soft tissue in the current frame using position dynamics comprises: adding constraints for vertices of the tetrahedron based on the first and second constraint functions and calculating model parameters of the soft tissue in the current frame using position dynamics.

6. The soft tissue suturing simulation method of claim 3, wherein, The method further comprises: detecting collision between a needle head of a suture needle and a surface of the soft tissue; if a number of frames in which the collision between the needle head and the surface of the soft tissue is detected consecutively is greater than a first threshold, determining a third constraint function, and adding constraints for vertices of the tetrahedron based on the third constraint function, the third constraint function comprising that a collision response displacement of the soft tissue is equal to a collision response displacement of the needle head, the collision response displacement of the soft tissue being a position of a puncture point in the model parameters of the soft tissue in the current frame minus a position of the puncture point in the model parameters of the soft tissue in a previous frame.

7. The soft tissue suturing simulation method of claim 6, wherein The method further comprises: if the collision between the needle head and the surface of the soft tissue is detected, incrementing a cumulative frame number by 1 and temporarily storing a collision response displacement of the soft tissue in the current frame, otherwise setting the cumulative frame number to 0 and clearing all temporarily stored collision response displacements; if the cumulative frame number is greater than the first threshold, accumulating all temporarily stored collision response displacements to obtain a cumulative displacement amount; comparing the cumulative displacement amount with a second threshold to determine whether the needle head has penetrated the surface of the soft tissue.

8. The soft tissue suturing simulation method of claim 7, wherein, The method further comprises: obtaining a stage flag, the stage flag being used to represent a stage in which the current frame is located in a soft tissue suturing process, the stage comprising a puncture stage, a suturing stage and a tightening stage; if the needle head has not penetrated the surface of the soft tissue and the stage flag is empty, updating the stage flag to be in the puncture stage of penetrating into the soft tissue; if the needle head has penetrated the surface of the soft tissue and the stage flag is empty or in the puncture stage of penetrating into the soft tissue, updating the stage flag to be in the puncture stage of having penetrated into the soft tissue; if the needle head has not penetrated the surface of the soft tissue and the stage flag is in the puncture stage of having penetrated into the soft tissue, updating the stage flag to be in the puncture stage of penetrating out of the soft tissue; if the needle head has penetrated the surface of the soft tissue and the stage flag is in the puncture stage of having penetrated into the soft tissue or in the puncture stage of penetrating out of the soft tissue, updating the stage flag to be in the puncture stage of having penetrated out of the soft tissue.

9. The soft tissue suturing simulation method of claim 1, wherein, The determination of whether the soft tissue is in the suturing stage or the tightening stage in the suturing process comprises: matching a constraint path point corresponding to the target particle point from the sequence of path points; calculating a distance between the target particle point and the constraint path point; if the distance is greater than a preset distance threshold, determining that the soft tissue is in the suturing stage in the suturing process, otherwise determining that the soft tissue is in the tightening stage in the suturing process.

10. The soft tissue suturing simulation method of claim 9, wherein, The calculation of the distance between the target particle point and the constraint path point comprises: finding a target particle point with the largest deformation from the target particle points; calculating a distance between the target particle point with the largest deformation and the constraint path point.

11. The soft tissue suturing simulation method of claim 9, wherein, The matching of the constraint path point corresponding to the target particle point from the sequence of path points comprises: determining a position of the target particle point in a plurality of target particle points; if the target particle point is a first target particle point, taking a first path point as the constraint path point; if the target particle point is a particle point between a target particle point and a last target particle point, finding a path point closest to the target particle point from the path point sequence as the constraint path point; if the target particle point is a last target particle point, taking a last path point as the constraint path point.

12. A method of soft tissue suturing simulation, characterized in that, The method comprises: obtaining a model of soft tissue and a model of a suture line, the model of the suture line comprising a plurality of particle points arranged in a chain, and the model of the soft tissue comprising a plurality of vertices and tetrahedrons defined by the vertices; determining a target particle point according to the model of the suture line, the target particle point being a particle point of the suture line located inside the soft tissue; determining a path point sequence according to the model of the soft tissue; determining whether a suture stage or a tensioning stage in a suture process is reached; if the suture stage or the tensioning stage in the suture process is reached, determining a foot of the target particle point according to the target particle point and the path point sequence; determining a first constraint function of the target particle point based on the target particle point and the foot, the first constraint function being used to constrain a distance between the target particle point and the foot to satisfy a preset condition, wherein the preset condition is that the distance between the target particle point and the foot is 0 or the distance between the target particle point and the foot approaches 0; calculating a model parameter of the soft tissue in a current frame using position dynamics based on the first constraint function; displaying the soft tissue according to the model parameter of the soft tissue in the current frame.

13. A soft tissue suturing simulation device, characterized by, The soft tissue suture simulation method comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to implement the soft tissue suture simulation method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the soft tissue suture simulation method according to any one of claims 1 to 12.

Images