Soft tissue suture simulation method, device and medium

Abstract

The invention provides a soft tissue suture simulation method and device and a medium. The method comprises the steps that models of soft tissue and suture lines are obtained; determining a target particle point according to the model of the suture line; determining a path point sequence according to the soft tissue model; judging whether a sewing stage or a tensioning stage is in the sewing process or not; if in a sewing stage or a tensioning stage in the sewing process, determining a perpendicular foot of the target particle point according to the target particle point and the path point sequence; based on the target particle point and the perpendicular foot, determining a first constraint function of the target particle point, the first constraint function being used for constraining the distance between the target particle point and the perpendicular foot to meet a preset condition; based on the first constraint function, using position dynamics to calculate and obtain a model parameter of a suture line in the current frame; and the suture line is displayed according to the model parameters of the suture line in the current frame, so that vivid suture simulation experience can be realized.

Description

Technical Field

[0001] The present application belongs to the technical field of computer graphics, and in particular relates to a soft tissue suturing simulation method, device and medium. Background Art

[0002] Suturing is an essential part of surgical simulation. The suturing process involves many aspects, such as simulation of sutures, simulation of soft tissues, simulation of surgical instruments such as suture needles, surgical forceps, tweezers, etc., and also includes the interaction between the above entities, such as the surgical forceps holding the suture needle to pierce the surface of the soft tissue and enter the soft tissue, the suture needle drives the suture thread through the soft tissue, the suture thread pulls the soft tissue together, and the suture thread is finally tied to fix the soft tissue.

[0003] Traditional methods mostly use the Follow The Leader (FTL) method to simulate sutures, the Mass-Spring Model (MSM) to simulate soft tissues, and the particle model to simulate surgical instruments. Since the three use different models, there is inevitably a certain degree of model penetration when dealing with the interaction between them, and this method is not convenient for extended simulation of more morphological organs and tissues, making the entire surgical simulation composed of multiple different frameworks, which increases the complexity of the simulation. Summary of the invention

[0004] In order to solve the existing technical problems, the present application provides a soft tissue suturing simulation method, device and computer-readable storage medium that can simulate the soft tissue suturing process in real time and accurately.

[0005] In a first aspect, an embodiment of the present application provides a soft tissue suturing simulation method, the method comprising:

[0006] Acquire a model of soft tissue and suture line, wherein the model of suture line includes a plurality of particle points arranged in a chain shape;

[0007] Determining a target particle point according to the model of the suture line, wherein the target particle point is a particle point of the suture line located inside the soft tissue;

[0008] determining a sequence of waypoints according to the model of the soft tissue;

[0009] Determine whether it is in the suturing stage or the tensioning stage of the suturing process;

[0010] If it is in the stitching stage or the tensioning stage of the stitching process, determining the foot of the vertical of the target particle point according to the target particle point and the path point sequence;

[0011] Based on the target particle point and the foot of the perpendicular, determining a first constraint function of the target particle point, wherein the first constraint function is used to constrain the distance between the target particle point and the foot of the perpendicular to meet a preset condition;

[0012] Based on the first constraint function, using position dynamics calculation to obtain the model parameters of the suture line in the current frame;

[0013] The seam line is displayed according to the model parameters of the seam line in the current frame.

[0014] Optionally, the model of the soft tissue includes a plurality of vertices and a tetrahedron defined by the vertices, and the method further includes:

[0015] Based on the first constraint function, adding constraints to the vertices of the tetrahedron and using position dynamics to calculate the model parameters of the soft tissue in the current frame;

[0016] The soft tissue is displayed according to the model parameters of the soft tissue in the current frame.

[0017] Optionally, the model of the soft tissue includes a plurality of vertices and a tetrahedron defined by the vertices, and determining a path point sequence according to the model of the soft tissue includes:

[0018] Acquire the position of the needle tail, and determine an initial path point sequence according to the position of the needle tail, wherein the initial path point sequence includes a plurality of path points;

[0019] Determining the tetrahedron where the path point is located according to the soft tissue model;

[0020] Based on the first constraint function, adding constraints to the vertices of the tetrahedron where the path point is located and using position dynamics to calculate the model parameters of the soft tissue in the current frame;

[0021] updating the coordinates of the path points according to the model parameters of the soft tissue;

[0022] The waypoint sequence is determined according to the coordinates of the updated waypoints.

[0023] Optionally, updating the coordinates of the path points according to the model parameters of the soft tissue includes:

[0024] Obtaining the initial coordinates of each vertex of the tetrahedron where the initial path point is located corresponding to the moment of obtaining the position of the needle tail;

[0025] Determining the center of gravity coordinate coefficients according to the initial path point and the initial coordinates;

[0026] Obtaining the current coordinates of each vertex of the tetrahedron where the initial path point is located in the model parameters of the soft tissue corresponding to the current frame;

[0027] The updated coordinates of the path point are determined according to the barycenter coordinate coefficients and the current coordinates.

[0028] Optionally, the method further includes:

[0029] Determining a second constraint function, wherein the second constraint function includes at least one of a distance constraint between adjacent vertices, a surface area invariant constraint of the tetrahedron, and a volume invariant constraint of the tetrahedron;

[0030] The adding constraints to the vertices of the tetrahedron where the path point is located based on the first constraint function and using position dynamics to calculate the model parameters of the soft tissue in the current frame includes:

[0031] Based on the first constraint function and the second constraint function, constraints are added to the vertices of the tetrahedron and positional dynamics are used to calculate model parameters of the soft tissue in the current frame.

[0032] Optionally, the method further includes:

[0033] Performing collision detection between the needle head of the suture needle and the surface of the soft tissue;

[0034] If the number of frames in which it is detected that the needle collides continuously with the surface of the soft tissue is greater than a first threshold, a third constraint function is determined, and constraints are added to the vertices of the tetrahedron based on the third constraint function, wherein the third constraint function includes that the collision response displacement of the soft tissue and the needle is the same, and the collision response displacement of the soft tissue is the position of the puncture point in the model parameters of the soft tissue in the current frame minus the position of the puncture point in the model parameters of the soft tissue in the previous frame.

[0035] Optional,

[0036] The method further comprises:

[0037] If it is detected that the needle tip collides with the surface of the soft tissue, the accumulated frame number is increased by 1, and the collision response displacement of the soft tissue in the current frame is temporarily stored; otherwise, the accumulated frame number is set to 0, and all temporarily stored collision response displacements are cleared;

[0038] If the accumulated number of frames is greater than the first threshold, all temporarily stored collision response displacements are accumulated to obtain an accumulated displacement;

[0039] The accumulated displacement is compared with a second threshold value to determine whether the needle has pierced the surface of the soft tissue.

[0040] Optionally, the method further includes:

[0041] Acquire a stage mark, where the stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, where the stages include the puncture stage, the suturing stage, and the tensioning stage;

[0042] If the needle has not pierced the surface of the soft tissue and the stage mark is empty, the stage mark is updated to piercing into the soft tissue in the puncture stage;

[0043] If the needle has pierced the surface of the soft tissue, and the stage mark is empty or has penetrated the soft tissue during the puncture stage, then the stage mark is updated to have penetrated the soft tissue during the puncture stage;

[0044] If the needle has not pierced the surface of the soft tissue, and the stage mark is "has pierced into the soft tissue" in the puncture stage, then the stage mark is updated to "has pierced out of the soft tissue" in the puncture stage;

[0045] If the needle has pierced the surface of the soft tissue, and the stage mark is "pierced into the soft tissue" or "pierced out of the soft tissue" in the puncture stage, the stage mark is updated to "pierced out of the soft tissue" in the puncture stage.

[0046] Optionally, the determining whether the suturing process is in the suturing stage or the tensioning stage includes:

[0047] Matching the constraint path point corresponding to the target particle point from the path point sequence;

[0048] Calculating the distance between the target particle point and the constraint path point;

[0049] If the distance is greater than a preset distance threshold, the suturing stage in the suturing process is determined, otherwise the tensioning stage in the suturing process is determined.

[0050] Optionally, calculating the distance between the target particle point and the constraint path point includes:

[0051] Find the target particle point with the largest deformation among the target particle points;

[0052] The distance between the target particle point with the largest deformation and the constraint path point is calculated.

[0053] Optionally, matching the constraint path point corresponding to the target particle point from the path point sequence includes:

[0054] Determining the position of the target particle point among multiple target particle points;

[0055] If the target particle point is the first target particle point, the first path point is used as the constraint path point;

[0056] If the target particle point is a particle point between a target particle point and a target particle point at the end, then find the path point closest to the target particle point from the path point sequence as the constrained path point;

[0057] If the target particle point is the last target particle point, the last path point is used as the constraint path point.

[0058] In a second aspect, an embodiment of the present application provides a soft tissue suturing simulation method, comprising:

[0059] Acquire models of soft tissue and suture line, wherein the model of suture line includes a plurality of particle points arranged in a chain shape, and the model of soft tissue includes a plurality of vertices and a tetrahedron defined by the vertices;

[0060] Determining a target particle point according to the model of the suture line, wherein the target particle point is a particle point of the suture line located inside the soft tissue;

[0061] determining a sequence of waypoints according to the model of the soft tissue;

[0062] Determine whether it is in the suturing stage or the tensioning stage of the suturing process;

[0063] If it is in the stitching stage or the tensioning stage of the stitching process, determining the foot of the vertical of the target particle point according to the target particle point and the path point sequence;

[0064] Based on the target particle point and the foot of the perpendicular, determining a first constraint function of the target particle point, wherein the first constraint function is used to constrain the distance between the target particle point and the foot of the perpendicular to meet a preset condition;

[0065] Based on the first constraint function, using position dynamics calculation to obtain the model parameters of the soft tissue in the current frame;

[0066] The soft tissue is displayed according to the model parameters of the soft tissue in the current frame.

[0067] According to a third aspect of the embodiments of the present application, a soft tissue suturing simulation device is provided, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, a soft tissue suturing simulation method as in any embodiment of the present application is implemented.

[0068] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which may store a program, and when the program is executed by a processor, it implements part or all of the steps of the soft tissue suturing simulation method of the first aspect.

[0069] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when run on a computer, enables execution of part or all of the steps of the soft tissue suturing simulation method of any one of the first aspects above.

[0070] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0071] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application obtain models of soft tissue and sutures, wherein the model of the sutures includes a plurality of particle points arranged in a chain; according to the model of the sutures, a target particle point is determined, wherein the target particle point is a particle point of the suture located inside the soft tissue; a path point sequence is determined according to the model of the soft tissue; it is determined whether the suture is in the suturing stage or the tensioning stage of the suturing process; if the suture is in the suturing stage or the tensioning stage of the suturing process, the foot of the perpendicular to the target particle point is determined according to the target particle point and the path point sequence; based on the target particle point and the foot of the perpendicular, a first constraint function of the target particle point is determined, wherein the first constraint function is used to constrain the distance between the target particle point and the foot of the perpendicular to meet a preset condition; based on the first constraint function, the model parameters of the suture in the current frame are obtained by using position dynamics calculation; the suture is displayed according to the model parameters of the suture in the current frame, so as to achieve a realistic suture simulation experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0073] Figure 1 A schematic diagram of a model of a suture line with an elastic rod introduced;

[0074] Figure 2 Flowchart for introducing position-based dynamics of elastic rods;

[0075] Figure 3 This is a schematic diagram of a flow chart of a soft tissue suturing simulation method embodiment in an embodiment of the present application;

[0076] Figure 4 Schematic diagram of the various stages of the soft tissue suturing process;

[0077] Figure 5 This is a schematic diagram of puncture in a soft tissue suturing simulation method in another embodiment of the present application;

[0078] Figure 6 This is a schematic diagram of a first constraint function in an embodiment of the present application;

[0079] Figure 7 This is a structural schematic diagram of an embodiment of a soft tissue suturing simulation device according to another embodiment of the present application; DETAILED DESCRIPTION

[0080] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0081] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0082] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0083] In the field of computer graphics, the common physical method for real-time simulation of object deformation generally updates the acceleration through external forces first, and then integrates to obtain the velocity and position. However, PBD directly corrects the position through the constraint function, and then updates the velocity through the position. The system equations in the PBD algorithm are constructed by the constraint function equation group, which includes multiple constraint function equations (which can be referred to as constraint functions).

[0084] In PBD, the model of an object includes multiple particle points that make up the object, also called mass points. The parameters of the particle points include their position, velocity and mass. The parameters of the constraint function include the position of these particle points. The constraint function is used to describe the relationship between the particle points that make up the object.

[0085] In the prediction part of the current frame, the external environment force of the current frame is converted into displacement as much as possible, and the external force that cannot be directly converted into displacement (such as gravity) is retained. Based on the position of each particle point in the previous frame, the position of each particle point in the current frame is predicted by combining the displacement obtained by conversion and the external force that cannot be directly converted.

[0086] The above prediction part often only uses integrals (such as explicit Euler integrals) to directly predict the position of each particle point in the current frame, without considering the mechanical properties of the object and the interaction with the environment. Therefore, it cannot be directly used as the simulation result of the current frame, but the position of the particle point must be corrected using the constraint function. Constraint functions are used to limit the relationship between particle points. There are two main types of constraint functions: one is a temporary constraint function, which is generally generated temporarily when the object is detected to interact with external objects in the environment; the other is a conventional constraint function, which is determined by the mechanical properties of the object. The process of calculating the corrected position that satisfies the constraint function as much as possible based on the constraint function and the current position is called constraint projection, which is the core process of PBD. Constraint correction is essentially an iterative optimization of the position of the particle point based on the constraint function equation group. The specific implementation is generally to iteratively execute the process of using the constraint functions in the constraint function equation group to perform constraint projection in turn until the set number of iterations is reached, so as to find the particle point position that satisfies all the constraint functions in the constraint function equation group as much as possible.

[0087] The result obtained after constraint correction is the position of each particle point in the current frame, based on which the speed of each particle point in the current frame can be calculated, and then the corresponding variables are updated for display and calculation of the next frame. The Δt used in PBD is generally the default frame spacing, that is, the duration between two adjacent frames under normal circumstances.

[0088] For soft tissue, its model is a mesh model including multiple vertices and tetrahedrons defined by the vertices. The vertices here are similar to particle points, and its constraint function mainly includes at least one of the distance constraint between adjacent vertices, the surface area invariance constraint of the tetrahedron, and the volume invariance constraint of each tetrahedron.

[0089] For surgical instruments, such as surgical forceps, needle holders, tweezers, suture needles, etc., they can be approximated as rigid bodies, and their models generally include multiple mass points / particle points determined according to their specific shapes. For example, the model of a suture needle may include multiple mass points arranged in a chain. Since surgical instruments other than suture needles are generally under the direct control of the user, the PBD process can be omitted and the kinematic equations can be directly used to calculate their posture and the position of each mass point in the model.

[0090] For suture lines, in order to more accurately describe their shape, rod nodes, also called elastic rods (rods), are further introduced. Figure 1 As shown, the model of the suture line includes a plurality of particle points arranged in a chain (e.g. Figure 1 p1,p2,…,p n ) and rod nodes connecting adjacent particle points (e.g. Figure 1 q1,q2,…,q n-1). The extension direction in the figure is only used to specify the numbering order of the particle points and the rod nodes, and does not represent the actual extension direction. The parameters of the rod nodes include their posture, angular velocity and moment of inertia. Correspondingly, the parameters of the constraint function also include the posture of these rod nodes. Among them, for the convenience of calculation, the posture is generally represented by a unit quaternion. Of course, in other embodiments, the posture can also be represented by a transformation / rotation matrix.

[0091] In practical applications, sutures are often connected to an external object that controls the movement of the sutures, and can also be called a control body. For example, if the suture is a suture, then the control body is a suture needle, surgical forceps, tweezers, etc. that directly drives the movement of the suture; if the suture is hair, the control body is the skin and hair accessories that fix the hair, and objects that directly drive the movement of the hair, such as fingers and combs. The particle point in the suture that is directly connected to the control body, that is, the particle point closest to the control body, can be called a control point.

[0092] The process of introducing PBD with elastic rod is as follows Figure 2 As shown, the PBD process is described in detail below with reference to the accompanying drawings.

[0093] Figure 2 Steps 1-4 in the figure are the initial stage, in which the parameters of all the particle points and rod nodes that make up the object are initialized according to the preset object parameters, such as the initial position, initial posture, number of particle points, etc. Then the real-time simulation stage is entered, and the process of simulating the deformation of the object according to the environment (including possible operation instructions from the user) is executed cyclically in frames, that is, it is executed once for each frame. Figure 2 Steps 5-21 in the above process may specifically include three parts: prediction, constraint modification and variable update.

[0094] In the prediction part of the current frame (corresponding to Figure 2 In steps 6-12), the effect of the force of the external environment on the object in the current frame is converted into displacement as much as possible, and the external force that cannot be directly converted into displacement (such as gravity) is retained; the effect of the torque of the external environment on the object in the current frame is converted into angular displacement as much as possible, and the external torque that cannot be directly converted into angular displacement is retained. Based on the position of each particle point in the previous frame, the position of each particle point in the current frame is predicted in combination with the displacement obtained by conversion and the external force that cannot be directly converted; based on the posture of each rod node in the previous frame, the posture of each rod node in the current frame is predicted in combination with the angular displacement obtained by conversion and the external torque that cannot be directly converted.

[0095] The above prediction part often only uses integrals (such as explicit Euler integrals) to directly predict the position of each particle point and the posture of each rod node in the current frame, without considering the mechanical properties of the object and the interaction with the environment. Therefore, it cannot be directly used as the simulation result of the current frame, but the position of the particle point and the posture of the rod node must be corrected using constraint functions. Constraint functions are used to limit the relationship between particle points, rod nodes, and particle points and rod nodes. There are two main types: one is a temporary constraint function, which is generally generated temporarily when the object is detected to interact with external objects in the environment; the other is a regular constraint function, which is determined by the mechanical properties of the object. For example, for sutures and similar linear objects, the corresponding regular constraint function can be given according to their mechanical properties that are not easy to stretch and shear, but easy to bend and twist. The process of calculating the corrected position / posture that satisfies the constraint function as much as possible based on the constraint function and the current position / posture is called constraint projection, which is the core process of PBD. Constraint correction is essentially an iterative optimization of the particle point position and the rod node posture based on the constraint function equation group. The specific implementation is generally to iteratively execute the process of using the constraint functions in the constraint function equation group to perform constraint projection in turn until the set number of iterations is reached, so as to find the particle point position and rod node posture that satisfies all the constraint functions in the constraint function equation group as much as possible. The constraint correction part corresponds to Figure 2 Steps 13-15 in which step 13 is used to perform collision detection to generate a temporary constraint function, and steps 14-15 are to iteratively optimize the positions of the particle points and the postures of the rod nodes using the constraint function equations.

[0096] Update some corresponding Figure 2 In steps 16-21 of the PBD, the result after constraint correction is the position of each particle point and the posture of each rod node in the current frame. Based on this, the velocity of each particle point and the angular velocity of each rod node in the current frame can be calculated, and then the corresponding variables are updated for display and calculation of the next frame. The Δt used in PBD is generally the default frame spacing, that is, the duration between two adjacent frames under normal circumstances.

[0097] After executing the simulated object deformation process of the current frame, the model parameters of the current frame can be obtained, including the parameters of the updated particle points and rod nodes, wherein the updated positions of each particle point and the postures of each rod node can be used to determine the shape of the stitching line of the current frame for display.

[0098] It should be noted that the vertices, particle points and mass points in this application are actually equivalent in the PBD framework. In order to simplify the description, different names are selected for the particle points in different models, that is, the particle points in soft tissue are called vertices, the particle points in surgical instruments are called mass points, and the particle points in sutures are called particle points.

[0099] Combined with the above preliminary introduction to the soft tissue suturing simulation method, Figure 3 The soft tissue suturing simulation method provided by the embodiment of the present application is further described in detail. Since object simulation is a real-time process, when the current frame is mentioned below, it means to describe the specific process of object simulation within a frame by taking the current frame as an example. In fact, the process must be executed for each frame.

[0100] For ease of description, this application only takes one puncture as an example. In actual applications, multiple punctures may occur. In this case, each piece of soft tissue has its own puncture stage and suturing stage, which is not limited here.

[0101] like Figure 3 As shown, the soft tissue suturing simulation method embodiment provided by the present application includes the following steps.

[0102] S1: Obtain models of soft tissues and sutures.

[0103] Among them, the model of soft tissue is a mesh model including multiple vertices and tetrahedrons defined by the vertices, the model of suture includes multiple particle points arranged in a chain and rod nodes connected between adjacent particle points, and the model of surgical instruments includes multiple particle points arranged according to their shapes, and the surgical instruments include suture needles. For specific information of the model, please refer to the relevant description above. The model is constructed according to the prior art, and the embodiments of this application do not impose any limitation on this.

[0104] S2: Determine a target particle point according to the model of the suture line, wherein the target particle point is a particle point of the suture line located inside the soft tissue.

[0105] S3: Determine a path point sequence according to the soft tissue model.

[0106] During the movement of the needle tail in the soft tissue, the position of the needle tail in each frame, which can also be called a moving point, is arranged in time order to form a moving path. During the suturing stage, the suture generally moves along the moving path when moving in the soft tissue, so the constraint of the moving path on the suture can be used as the collision response constraint between the suture and the soft tissue acting on the suture.

[0107] Since the distribution of moving points in the moving path is affected by the user's operation and is not necessarily uniform, if temporary constraints are added to the particle points located inside the soft tissue in the suture line, that is, the target particle points, directly according to the moving path, the spacing between adjacent target particle points after the update will be affected by the distribution of the moving points and will no longer be uniform. There may be a large error with the initially set particle point spacing, that is, the spacing between adjacent particle points in the suture line, thereby causing a significant change in the length of the suture line.

[0108] In order to avoid obvious length changes of the suture during its movement in the soft tissue and reduce the resulting distortion, in this embodiment, a path point sequence is determined according to the model of the soft tissue, specifically including:

[0109] S31: Acquire the position of the needle tail, and determine an initial path point sequence according to the position of the needle tail, wherein the initial path point sequence includes a plurality of path points.

[0110] For the suture needle, if it is treated as a rigid body, then when the suture needle is clamped by other surgical instruments, the PBD process can be omitted, and the kinematic equation can be directly used in combination with the posture of the surgical instrument that clamps the suture needle to calculate its posture and the position of each particle in the model; when the suture needle is not clamped by other surgical instruments, in order to prevent penetration, the PBD / position-based extended dynamics (XPBD) process can be used to calculate the position of each particle in the model. Specifically, the overall collision constraint of the suture needle can be determined based on the result of the collision detection between the suture needle and the soft tissue, and the position of each particle in the suture needle model can be calculated using position dynamics based on the overall collision constraint. If the elasticity of the suture needle is considered, it cannot be treated as a rigid body. Regardless of whether it is clamped by other surgical instruments, the PBD / XPBD process must be used to calculate the position of each particle in the model. According to the above-mentioned calculation method of the position of each particle in the suture needle, the position of each particle in the suture needle in the current frame can be obtained, and combined with the position of the needle tail in the suture needle, the position of the needle tail in the current frame can be obtained.

[0111] S32: determining the tetrahedron where the path point is located according to the soft tissue model;

[0112] S33: Based on the first constraint function, add constraints to the vertices of the tetrahedron and use position dynamics to calculate the model parameters of the soft tissue in the current frame. In the embodiment of the present application, the model parameters can be referred to as the parameters of the model, that is, the parameters of the particle points in the model, such as the aforementioned position, velocity and mass.

[0113] S34: Updating the coordinates of the path points according to the model parameters of the soft tissue, specifically comprising:

[0114] S341: Obtaining the initial coordinates of each vertex of the tetrahedron where the initial path point is located corresponding to the moment of obtaining the position of the needle tail;

[0115] S342: Determine the barycentric coordinate coefficient according to the initial path point and the initial coordinate. The coordinate of any point in the tetrahedron can be written as the weighted average of the coordinates of the four vertices of the tetrahedron, and the four weights are the barycentric coordinate coefficient of the point. During the entire stitching process, the barycentric coordinate coefficient of the path point can remain unchanged.

[0116] S343: Obtain the current coordinates of each vertex of the tetrahedron where the initial path point is located in the model parameters of the soft tissue corresponding to the current frame.

[0117] S344: Determine the updated coordinates of the path point based on the center of gravity coordinate coefficient and the current coordinates.

[0118] S35: Determine a path point sequence according to the coordinates of the updated path points.

[0119] In the embodiment of the present application, the path point sequence is first established by the position of the needle tail, and then continuously iterated and optimized. During the iterative optimization process, the centroid coordinate coefficients of each path point in the tetrahedron are kept unchanged. Among them, the process based on position dynamics calculation is a process of continuous iterative optimization, so the process of determining the path point sequence is a process of continuously updating the previous iteration results with the new iteration results after updating the initial path point sequence.

[0120] Furthermore, a second constraint function can be determined, which includes at least one of a distance constraint between adjacent vertices, a surface area invariance constraint of the tetrahedron, and a volume invariance constraint of the tetrahedron; then, based on the first constraint function and the second constraint function, constraints are added to the vertices of the tetrahedron and position dynamics are used to calculate the model parameters of the soft tissue in the current frame.

[0121] Step S34 updates the coordinates of the path points according to the model parameters of the soft tissue, specifically including:

[0122] S341: Obtain the centroid coordinates of the path point in the tetrahedron before the collision response constraint is added.

[0123] Barycentric coordinates, also called mass coordinates or volume coordinates, are coordinates defined by the four vertices of a tetrahedron. The coordinates of any point in a tetrahedron can be written as the weighted average of the coordinates of the four vertices of the tetrahedron, and these four weights are the barycentric coordinates of the point.

[0124] S342: The coordinates of the updated path points are calculated by combining the coordinates of the vertices of the tetrahedron and the coordinates of the center of gravity in the model parameters of the soft tissue.

[0125] The centroid coordinates of the path point are used as weights to calculate the weighted average of the coordinates of the vertices of the tetrahedron in the model parameters of the soft tissue, and the result obtained is the coordinate of the updated path point.

[0126] Furthermore, a second constraint function can be determined, which includes at least one of a distance constraint between adjacent vertices, a surface area invariance constraint of the tetrahedron, and a volume invariance constraint of the tetrahedron; then, based on the first constraint function and the second constraint function, constraints are added to the vertices of the tetrahedron and position dynamics are used to calculate the model parameters of the soft tissue in the current frame.

[0127] The coordinates of the path points are then updated according to the model parameters of the soft tissue, and a path point sequence is determined according to the updated coordinates of the path points.

[0128] S4: Determine whether the process is in the suturing stage or the tensioning stage.

[0129] like Figure 4 As shown, the soft tissue suturing process mainly includes the following stages: puncture stage, suturing stage, tensioning stage and knotting stage. In order to simulate the soft tissue suturing process under the framework of PBD, at least one stage mark can be designed to distinguish different stages, and the stage mark can be updated as the stage changes. During the real-time simulation of different types of objects, the stage mark can be read to perform corresponding operations, such as adding corresponding temporary constraints (collision response constraints, first constraints, etc.), updating path points, etc. Since the stages that need to be paid attention to in the simulation of different types of objects are not necessarily the same, you can choose to set different stage marks for different types of objects, or you can set a unified stage mark for all objects. The latter is used as an example in this application.

[0130] From the perspective of user operation, during the puncture stage, the user uses a first clamp (generally surgical forceps, needle holder, etc.) to clamp the middle and rear part of the suture needle and insert it into the soft tissue. After the needle tip of the suture thread penetrates the soft tissue, it passes through the soft tissue until the needle tip of the suture thread pierces the soft tissue. The user then moves the first clamp to the front of the suture needle and continues to pull the suture needle until the needle tail of the suture needle (i.e., the particle in the suture needle that is directly connected to the suture thread) enters the soft tissue.

[0131] The suturing stage generally starts when the tail of the suture needle drives the suture into the soft tissue and ends when the tension mark appears. There are two main tension marks. One is that there is a pre-tied knot at the tail of the suture. In this case, the tension mark is that the pre-tied knot contacts the surface of the soft tissue; the other is that there is no pre-tied knot. In this case, the tension mark is that the user uses a second clamp (another surgical instrument different from the first clamp, such as forceps, etc.) to directly clamp the tail of the suture.

[0132] The tensioning stage generally starts from the appearance of the tensioning mark and ends when the suture thread collides with itself driven by the first clamping body and the second clamping body.

[0133] The knotting stage generally starts when the suture line collides with itself under the drive of the first clamping body and the second clamping body, and the specific end time can be determined according to needs, for example, by cutting the suture line with surgical scissors.

[0134] Optionally, the stages of the suturing process may be different depending on the way the suture is knotted. For example, if multiple knots need to be tied during the suturing process, a tensioning stage and a knotting stage may be included after multiple suturing stages; or if knots only need to be tied at the beginning and end of suturing, a tensioning stage and a knotting stage may be included after the first puncture stage and suturing stage is completed, and a tensioning stage and a knotting stage may be included after the subsequent multiple puncture stages and suturing stages are completed alternately.

[0135] Optionally, the determination of whether the suturing process is in the suturing stage or the tensioning stage can be made based on a stage mark. Specifically, the stage mark is determined based on the collision detection result, the historical record of the stage, the relative position relationship between the surgical instrument and the soft tissue, etc. Among them, the expansion detection includes multiple: collision detection between soft tissue and suture, collision detection between soft tissue and surgical instrument, collision detection between suture and surgical instrument, and can also include self-collision detection of suture and / or collision detection between different surgical instruments.

[0136] Optionally, the determining whether the suturing process is in the suturing stage or the tensioning stage may further include:

[0137] S41: matching the constraint path point corresponding to the target particle point from the path point sequence;

[0138] S42: Calculate the distance between the target particle point and the constraint path point;

[0139] S43: If the distance is greater than a preset distance threshold, determining the suturing stage in the suturing process, otherwise determining the tensioning stage in the suturing process.

[0140] Wherein, step S41 may specifically include:

[0141] S411: Determine the position of the target particle point among the multiple target particle points. Since the stitching line includes a plurality of particle points, serial numbers may be assigned to the multiple particle points, and the order of the positions of the particle points among the multiple particle points may be determined by the serial numbers.

[0142] S412: If the target particle point is the first target particle point, the first path point is used as the constraint path point;

[0143] If the target particle point is a particle point between a target particle point and the last target particle point, then find the path point closest to the target particle point from the path point sequence as the constrained path point; optionally, calculate the distance between the target particle point and each path point, and then select the path point with the closest distance from multiple distances as the constrained path point.

[0144] If the target particle point is the last target particle point, the last path point is used as the constraint path point.

[0145] Wherein, step S42 may specifically include:

[0146] S421: Find the target particle point with the largest deformation among the target particle points.

[0147] During the suturing process, the target particle point closest to the surface of the soft tissue is the target particle point with the largest deformation. Figure 5 As shown, with the right side as the needle entry side and the left side as the needle exit side, during the pulling process of the suture, the tension exerted on the particle points of each suture, for example, when all the particle points are on the right side, the target particle point on the right side closest to the surface of the soft tissue is the target particle point with the largest deformation; when there are particle points on the left side, the target particle point on the left side closest to the surface of the soft tissue is the target particle point with the largest deformation.

[0148] S422: Calculate the distance between the target particle point with the largest deformation and the constraint path point.

[0149] S5: If the stitching process is in the stitching stage or the tensioning stage, determine the foot of the target particle point according to the target particle point and the path point sequence, specifically including S51 to S53:

[0150] S51: Match the constraint path point corresponding to the target particle point from the path point sequence. For specific implementation, please refer to S41 and will not be described in detail here.

[0151] S52: Determine two path points adjacent to the constraint path point;

[0152] S53: Determine the foot of the perpendicular according to the target particle point, the two adjacent path points, and the constraint path point.

[0153] like Figure 6 As shown, S is the target particle point, Q is the foot of the perpendicular, P is the constraint path point, and P′ and P″ are two adjacent path points. Point SPP′ forms a triangle, and point SPP″ forms a triangle. If the angle between the side SP and the side PP′ is an acute angle, the foot of the perpendicular Q is within the triangle SPP′; if the angle between the side SP and the side PP″ is an acute angle, the foot of the perpendicular Q is within the triangle SPP″.

[0154] S6: Based on the target particle point and the foot of the perpendicular, determine a first constraint function of the target particle point, the first constraint function is used to constrain the distance between the target particle point and the foot of the perpendicular to meet a preset condition. The first constraint function is optionally used to constrain the distance between the target particle point and the foot of the perpendicular to be 0.

[0155] If the suturing process is in the suturing stage or the tensioning stage, the temporary constraint that is mainly considered is the constraint based on the first constraint function, which can be called the first constraint. Since the first constraint function is a function of distance, the first constraint is a distance constraint; at the same time, the first constraint is a constraint generated by the interaction between the suture line and the soft tissue, so it is also a collision constraint. Each target particle point has a corresponding constraint path point in the path point sequence, and the first constraint of each target particle point refers to the constraint between the target particle point and the foot of the perpendicular determined according to the constraint path point.

[0156] S7: Based on the first constraint function, obtain the model parameters of the stitching line in the current frame using position dynamics calculation.

[0157] When using position dynamics calculation, the following formula can be used to calculate the position change △pi of the suture particles, thereby achieving the final iterative optimization and obtaining the model parameters of the suture model.

[0158] △pi=-SWi▽piC(p1, p2, ﹒﹒﹒, pn)

[0159] Among them, S is the scaling factor, Wi is the inverse of the mass, △pi is the position update amount of the i-th point, ▽pi is the constraint gradient, C(p1, p2, ﹒﹒﹒, pn) is the constraint function, p1, p2~pn are multiple points, such as multiple particle points of a suture line.

[0160] The first constraint function includes multiple parameters, such as the inverse mass Wi of the target particle point. If it is determined that the suturing process is in the tensioning stage, the Wi in the first constraint function can be set to 0, and the position update amount of the particle point of the corresponding suture line is 0, that is, the particle point of the suture line is basically stationary, and mainly relies on the continuous update of the foot of the perpendicular, or mainly relies on the continuous update of the path points of the soft tissue, so that the distance between the particle point of the suture line and the foot of the perpendicular approaches 0.

[0161] S8: Displaying the stitching line according to the model parameters of the stitching line in the current frame.

[0162] After executing the simulated object deformation process of the current frame, the model parameters of the current frame can be obtained, including the parameters of the updated particle points and rod nodes, wherein the updated positions of each particle point and the postures of each rod node can be used to determine the shape of the stitching line of the current frame for display.

[0163] Optionally, the soft tissue model includes a plurality of vertices and a tetrahedron defined by the vertices, and the soft tissue suturing simulation method in the embodiment of the present application further includes:

[0164] Based on the first constraint function, adding constraints to the vertices of the tetrahedron and using position dynamics to calculate the model parameters of the soft tissue in the current frame;

[0165] The soft tissue is displayed according to the model parameters of the soft tissue in the current frame.

[0166] Similarly, when using position dynamics calculation, the following formula can be used to calculate the position change △pi of the particles or the vertical foot of the soft tissue, so as to achieve the final iterative optimization and obtain the model parameters of the soft tissue model.

[0167] Similarly, the change in soft tissue or vertical foot △′pi

[0168] △′pi=-S′W′i▽′pi C′(p1′, p2′, ﹒﹒﹒, pn′)

[0169] Similarly, S′ is the scaling factor, W′i is the inverse of the mass, △′pi is the position update amount of the i-th point, ▽′pi is the constraint gradient, C′(p1′, p2′, ﹒﹒﹒, pn′) is the constraint function, p1′, p2′~pn′ are multiple points, such as multiple particle points or multiple perpendicular feet of soft tissue.

[0170] The embodiment of the present application iteratively optimizes and adjusts the change amount △pi of the suture line and the change amount △′pi of the soft tissue, so that the suture line and the soft tissue both satisfy the constraint function, thereby achieving a good suture simulation experience. For example, the needle can smoothly slide over the soft tissue and suture the soft tissue. After suturing, the two pieces of tissue can fit together better without obvious penetration.

[0171] In one embodiment, the soft tissue suturing simulation method further comprises:

[0172] Performing collision detection between the needle head of the suture needle and the surface of the soft tissue;

[0173] If the number of frames in which the needle and the surface of the soft tissue are detected to collide continuously is greater than a first threshold, a third constraint function is determined, and constraints are added to the vertices of the tetrahedron based on the third constraint function, wherein the third constraint function includes that the collision response displacement of the soft tissue and the needle in the current frame is the same, and the collision response displacement of the soft tissue in the current frame is the position of the puncture point in the model parameters of the soft tissue in the current frame minus the position of the puncture point in the model parameters of the soft tissue in the previous frame. The position of the puncture point is a model parameter.

[0174] Furthermore, based on the first constraint function and the third constraint function, constraints may be added to the vertices of the tetrahedron and position dynamics may be used to calculate the model parameters of the soft tissue in the current frame.

[0175] In the embodiment of the present application, simulation can be achieved by constraining the tetrahedron of the soft tissue, wherein the constraints imposed by the tetrahedron can be selected according to the specific situation. For example, during puncture, the third constraint function and / or the second constraint function can be selected, and during the suturing stage or the tensioning stage, the first constraint function and / or the second constraint function can be selected.

[0176] The puncture stage mainly focuses on the collision detection between the suture needle and the soft tissue. According to whether the surface of the soft tissue is pierced and the direction of the collision between the suture needle and the soft tissue surface, the puncture stage can be further divided into multiple sub-stages, as follows:

[0177] a. If the needle tip of the suture needle collides with the surface of the soft tissue from the outside to the inside but does not pierce the surface of the soft tissue, the suturing stage is the piercing into the soft tissue during the puncture stage.

[0178] In this sub-stage, since the suture needle is generally clamped by the first clamping body, it is not necessary to perform a collision response on the suture needle, but only perform a collision response on the soft tissue.

[0179] b. If the suture needle has pierced the surface of the soft tissue and the needle head is located inside the soft tissue, the suturing stage is the puncture stage during which the soft tissue has been pierced.

[0180] c. If the needle collides with the surface of the soft tissue from the inside to the outside but does not pierce the surface of the soft tissue, the suturing stage is piercing the soft tissue during the puncture stage.

[0181] In this sub-stage, since the suture needle is generally clamped by the first clamping body, it is not necessary to perform a collision response on the suture needle, but only perform a collision response on the soft tissue.

[0182] d. If the suture needle has pierced the surface of the soft tissue and the needle tip is located outside the soft tissue, the suturing stage is the stage where the soft tissue has been pierced during the puncture stage.

[0183] The needle tip of a suture needle may be a small area of ​​the needle tip. The direction in which the suture needle collides with the surface of the soft tissue, i.e., "from the outside of the soft tissue to the inside" and "from the inside of the soft tissue to the outside" mentioned above, can be implemented in a variety of ways. One is to consult the historical records of stage changes. If the puncture stage of the soft tissue did not appear in the previous stage, the direction is from the outside to the inside, otherwise the direction is from the inside to the outside; another is to refer to the relative position relationship between the suture needle and the soft tissue. If the parts of the suture needle except the part that collides with the soft tissue are all located outside the soft tissue, the direction is from the outside to the inside, and if the middle part of the suture needle is located inside the soft tissue, the direction is from the inside to the outside; another is to refer to the relative relationship between the direction of the suture needle's speed and the soft tissue. If the speed of the suture needle points to the inside of the soft tissue, the direction is from the outside to the inside, and if the speed of the suture needle points to the outside of the soft tissue, the direction is from the inside to the outside.

[0184] Furthermore, it can be determined whether to update the stage flag according to the result of the collision detection between the soft tissue and the suture needle:

[0185] If the stage mark is that the soft tissue has been pierced in the puncture stage and it is detected that the needle tail of the suture needle collides with the surface of the soft tissue, the stage mark is updated to the suturing stage.

[0186] The puncture stage can be divided into multiple sub-stages according to the puncture direction of the needle and whether the needle punctures the surface of the soft tissue. There are multiple ways to determine the puncture direction. The following is a specific description of determining the puncture direction based on the historical records of the stage mark.

[0187] Optionally, determining whether the needle has pierced the surface of the soft tissue may specifically include:

[0188] S46: Perform collision detection between the needle head of the suture needle and the surface of the soft tissue.

[0189] S47: If the number of frames in which it is detected that the needle tip collides with the surface of the soft tissue continuously is greater than a first threshold, the collision response displacement of the soft tissue is accumulated to obtain a cumulative displacement.

[0190] S48: If the accumulated displacement is greater than the second threshold, it is determined that the suture needle has pierced the surface of the soft tissue.

[0191] Since the suture needle is generally clamped by the first clamping body during the process of piercing the soft tissue surface, if the suture needle is regarded as a rigid body, the collision response may not be performed on the suture needle, but only on the soft tissue.

[0192] In an embodiment of the present application, the sub-stages of determining the puncture stage may specifically include:

[0193] S261: Determine whether a collision between the needle tip of the suture needle and the surface of the soft tissue is detected in the current frame.

[0194] If so, jump to S262; otherwise jump to S269.

[0195] S262: Add 1 to the accumulated number of frames and temporarily store the collision response displacement of the current frame.

[0196] The collision response displacement is the position of the puncture point that collides with the needle in the model parameters of the soft tissue in the current frame minus the position of the puncture point in the model parameters of the soft tissue in the previous frame.

[0197] S263: If the accumulated number of frames is greater than the first threshold, all temporarily stored collision response displacements are accumulated to obtain an accumulated displacement.

[0198] S264: Compare the accumulated displacement with the second threshold value to determine whether the needle has pierced the surface of the soft tissue.

[0199] Specifically, if the accumulated displacement in the current frame is greater than the second threshold, it is determined that the needle has pierced the surface of the soft tissue; otherwise, it is determined that the needle has not pierced the surface of the soft tissue. Alternatively, if the accumulated displacement in the current frame is greater than or equal to the second threshold, it is determined that the needle has pierced the surface of the soft tissue; otherwise, it is determined that the needle has not pierced the surface of the soft tissue.

[0200] S265: If the needle has not pierced the surface of the soft tissue and the stage mark is empty, the stage mark is updated to piercing into the soft tissue in the puncture stage.

[0201] S266: If the needle has pierced the surface of the soft tissue, and the stage mark is empty or has penetrated the soft tissue during the puncture stage, the stage mark is updated to have penetrated the soft tissue during the puncture stage.

[0202] S267: If the needle has not pierced the surface of the soft tissue, and the stage mark is "entered into the soft tissue" in the puncture stage, then the stage mark is updated to "pierced out of the soft tissue" in the puncture stage.

[0203] S268: If the needle has pierced the surface of the soft tissue, and the stage mark is "entered into the soft tissue" or "exited from the soft tissue" in the puncture stage, then the stage mark is updated to "exited from the soft tissue" in the puncture stage.

[0204] S269: Set the accumulated frame number to 0, and clear all temporarily stored collision response displacements.

[0205] Steps S265 to S268 are used to update the stage flag, so before step S265, it can also include obtaining the stage flag, which is used to indicate the stage of the current frame in the soft tissue suturing process, and the stages include the puncture stage, the suturing stage and the tensioning stage.

[0206] Collision response refers to the processing of colliding objects. This processing is generally as close to people's understanding of physical laws as possible to avoid obvious abnormal interactions between colliding objects, such as mold penetration. When the needle of the suture needle punctures the surface of the soft tissue, it generates the puncture force of the needle on the soft tissue and the resistance of the soft tissue to the needle. Since the soft tissue has a certain elasticity, the soft tissue will not be punctured immediately, but will produce a certain deformation at the puncture position along the movement direction of the suture needle. As the puncture force continues to increase and exceeds the maximum resistance that the soft tissue can generate, the soft tissue will be punctured.

[0207] In order to simplify the simulation of puncture process under the PBD framework, the simulation of puncture force and resistance of soft tissue is abandoned in this embodiment. Instead, the cumulative displacement of the collision response displacement of soft tissue is used to reflect the magnitude of puncture force, and the second threshold is used to reflect the maximum resistance that soft tissue can produce. The force comparison is converted into a comparison between the cumulative displacement and the second threshold, so that it is more suitable for the PBD framework. The second threshold can be set according to the characteristics of different organs and tissues. For the same piece of soft tissue, the second threshold used for piercing the soft tissue surface from the inside to the outside and from the outside to the inside can be the same or different.

[0208] In order to prevent the collision between the needle and the soft tissue surface from being mistakenly judged as puncture, in this embodiment, it is necessary to detect that the needle and the surface of the soft tissue collide continuously, and the number of frames of continuous collision is greater than the first threshold value before confirming that the needle has begun to puncture the soft tissue surface, and then judge whether the needle has punctured the soft tissue surface based on the comparison between the cumulative displacement and the second threshold value. After the needle starts to puncture the soft tissue surface, if the cumulative displacement in the current frame is less than the second threshold value, it is judged that the soft tissue surface has not been punctured, and the cumulative displacement is retained for the next frame. In order to prevent the jitter of the soft tissue surface from affecting the result of the puncture judgment, the collision response displacement is temporarily stored and accumulated in the format of a vector, that is, the cumulative displacement of the current frame is essentially the displacement between the position of the soft tissue surface in the current frame under the collision response of the soft tissue and the needle and the position before the collision with the needle.

[0209] In one embodiment, the suturing simulation method provided by the present application may further include:

[0210] Acquire models of soft tissue and suture line, wherein the model of suture line includes a plurality of particle points arranged in a chain shape, and the model of soft tissue includes a plurality of vertices and a tetrahedron defined by the vertices;

[0211] Determining a target particle point according to the model of the suture line, wherein the target particle point is a particle point of the suture line located inside the soft tissue;

[0212] determining a sequence of waypoints according to the model of the soft tissue;

[0213] Determine whether it is in the suturing stage or the tensioning stage of the suturing process;

[0214] If it is in the stitching stage or the tensioning stage of the stitching process, determining the foot of the vertical of the target particle point according to the target particle point and the path point sequence;

[0215] Based on the target particle point and the foot of the perpendicular, determining a first constraint function of the target particle point, wherein the first constraint function is used to constrain the distance between the target particle point and the foot of the perpendicular to meet a preset condition;

[0216] Based on the first constraint function, using position dynamics calculation to obtain the model parameters of the soft tissue in the current frame;

[0217] The soft tissue is displayed according to the model parameters of the soft tissue in the current frame.

[0218] Furthermore, it may also include:

[0219] Based on the first constraint function, using position dynamics calculation to obtain the model parameters of the suture line in the current frame;

[0220] The seam line is displayed according to the model parameters of the seam line in the current frame.

[0221] like Figure 7 As shown, it is a schematic diagram of the structure of a soft tissue suturing simulation device provided according to an embodiment of the present application. The object simulation device provided in the embodiment of the present application includes a memory 502 and a processor 501, and the memory 502 stores a computer program executable by the processor 501. When the computer program is executed by the processor 501, a soft tissue suturing simulation method in any embodiment of the present application is implemented.

[0222] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned soft tissue suturing simulation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Computer-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0223] It should be noted that other sorting schemes that can be easily thought of by those skilled in the art within the technical scope disclosed in the present invention should also be within the protection scope of the present invention and will not be described in detail here.

[0224] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in a memory and executed by a processor to implement the present invention. One or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program.

[0225] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0226] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0227] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0228] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0229] As used in the specification of this application and the appended claims, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if the described condition or event is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce the described condition or event is detected" or "in response to detecting the described condition or event" depending on the context.

[0230] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0231] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0232] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0233] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0234] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0235] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0236] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0237] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A soft tissue suturing simulation method, characterized in that: The method comprises: Acquire a model of soft tissue and suture line, wherein the model of suture line includes a plurality of particle points arranged in a chain shape; Determining a target particle point according to the model of the suture line, wherein the target particle point is a particle point of the suture line located inside the soft tissue; determining a sequence of waypoints according to the model of the soft tissue; Determine whether it is in the suturing stage or the tensioning stage of the suturing process; If it is in the stitching stage or the tensioning stage of the stitching process, determining the foot of the vertical of the target particle point according to the target particle point and the path point sequence; Based on the target particle point and the foot of the perpendicular, determining a first constraint function of the target particle point, wherein the first constraint function is used to constrain the distance between the target particle point and the foot of the perpendicular to meet a preset condition; Based on the first constraint function, using position dynamics calculation to obtain the model parameters of the suture line in the current frame; The seam line is displayed according to the model parameters of the seam line in the current frame.

2. The soft tissue suturing simulation method according to claim 1, characterized in that: The model of the soft tissue includes a plurality of vertices and a tetrahedron defined by the vertices, and the method further includes: Based on the first constraint function, adding constraints to the vertices of the tetrahedron and using position dynamics to calculate the model parameters of the soft tissue in the current frame; The soft tissue is displayed according to the model parameters of the soft tissue in the current frame.

3. The soft tissue suturing simulation method according to claim 1, characterized in that: The model of the soft tissue includes a plurality of vertices and a tetrahedron defined by the vertices, and determining a path point sequence according to the model of the soft tissue includes: Acquire the position of the needle tail, and determine an initial path point sequence according to the position of the needle tail, wherein the initial path point sequence includes a plurality of path points; Determining the tetrahedron where the path point is located according to the soft tissue model; Based on the first constraint function, adding constraints to the vertices of the tetrahedron where the path point is located and using position dynamics to calculate the model parameters of the soft tissue in the current frame; updating the coordinates of the path points according to the model parameters of the soft tissue; The waypoint sequence is determined according to the coordinates of the updated waypoints.

4. The soft tissue suturing simulation method according to claim 3, characterized in that: The updating of the coordinates of the path points according to the model parameters of the soft tissue comprises: Obtaining the initial coordinates of each vertex of the tetrahedron where the initial path point is located corresponding to the moment of obtaining the position of the needle tail; Determining the center of gravity coordinate coefficients according to the initial path point and the initial coordinates; Obtaining the current coordinates of each vertex of the tetrahedron where the initial path point is located in the model parameters of the soft tissue corresponding to the current frame; The updated coordinates of the path point are determined according to the barycenter coordinate coefficients and the current coordinates.

5. The soft tissue suturing simulation method according to claim 3, characterized in that: The method further comprises: Determining a second constraint function, wherein the second constraint function includes at least one of a distance constraint between adjacent vertices, a surface area invariant constraint of the tetrahedron, and a volume invariant constraint of the tetrahedron; The adding constraints to the vertices of the tetrahedron where the path point is located based on the first constraint function and using position dynamics to calculate the model parameters of the soft tissue in the current frame includes: Based on the first constraint function and the second constraint function, constraints are added to the vertices of the tetrahedron and positional dynamics are used to calculate model parameters of the soft tissue in the current frame.

6. The soft tissue suturing simulation method according to claim 3, characterized in that: The method further comprises: Performing collision detection between the needle head of the suture needle and the surface of the soft tissue; If the number of frames in which it is detected that the needle collides continuously with the surface of the soft tissue is greater than a first threshold, a third constraint function is determined, and constraints are added to the vertices of the tetrahedron based on the third constraint function, wherein the third constraint function includes that the collision response displacement of the soft tissue and the needle is the same, and the collision response displacement of the soft tissue is the position of the puncture point in the model parameters of the soft tissue in the current frame minus the position of the puncture point in the model parameters of the soft tissue in the previous frame.

7. The soft tissue suturing simulation method according to claim 6, characterized in that: The method further comprises: If it is detected that the needle tip collides with the surface of the soft tissue, the accumulated frame number is increased by 1, and the collision response displacement of the soft tissue in the current frame is temporarily stored; otherwise, the accumulated frame number is set to 0, and all temporarily stored collision response displacements are cleared; If the accumulated number of frames is greater than the first threshold, all temporarily stored collision response displacements are accumulated to obtain an accumulated displacement; The accumulated displacement is compared with a second threshold value to determine whether the needle has pierced the surface of the soft tissue.

8. The soft tissue suturing simulation method according to claim 7, characterized in that: The method further comprises: Acquire a stage mark, where the stage mark is used to indicate the stage of the current frame in the soft tissue suturing process, where the stages include the puncture stage, the suturing stage, and the tensioning stage; If the needle has not pierced the surface of the soft tissue and the stage mark is empty, the stage mark is updated to piercing into the soft tissue in the puncture stage; If the needle has pierced the surface of the soft tissue, and the stage mark is empty or has penetrated the soft tissue during the puncture stage, then the stage mark is updated to have penetrated the soft tissue during the puncture stage; If the needle has not pierced the surface of the soft tissue, and the stage mark is "has pierced into the soft tissue" in the puncture stage, then the stage mark is updated to "has pierced out of the soft tissue" in the puncture stage; If the needle has pierced the surface of the soft tissue, and the stage mark is "pierced into the soft tissue" or "pierced out of the soft tissue" in the puncture stage, the stage mark is updated to "pierced out of the soft tissue" in the puncture stage.

9. The soft tissue suturing simulation method according to claim 1, characterized in that: The determining whether the suturing process is in the suturing stage or the tensioning stage includes: Matching the constraint path point corresponding to the target particle point from the path point sequence; Calculating the distance between the target particle point and the constraint path point; If the distance is greater than a preset distance threshold, the suturing stage in the suturing process is determined, otherwise the tensioning stage in the suturing process is determined.

10. The soft tissue suturing simulation method according to claim 8, characterized in that: The calculating the distance between the target particle point and the constraint path point comprises: Find the target particle point with the largest deformation among the target particle points; The distance between the target particle point with the largest deformation and the constraint path point is calculated.

11. The soft tissue suturing simulation method according to claim 9, characterized in that: The matching of the constraint path point corresponding to the target particle point from the path point sequence includes: Determining the position of the target particle point among multiple target particle points; If the target particle point is the first target particle point, the first path point is used as the constraint path point; If the target particle point is a particle point between a target particle point and a target particle point at the end, then find the path point closest to the target particle point from the path point sequence as the constrained path point; If the target particle point is the last target particle point, the last path point is used as the constraint path point.

12. A soft tissue suturing simulation method, characterized in that: The method comprises: Acquire models of soft tissue and suture line, wherein the model of suture line includes a plurality of particle points arranged in a chain shape, and the model of soft tissue includes a plurality of vertices and a tetrahedron defined by the vertices; Determining a target particle point according to the model of the suture line, wherein the target particle point is a particle point of the suture line located inside the soft tissue; determining a sequence of waypoints according to the model of the soft tissue; Determine whether it is in the suturing stage or the tensioning stage of the suturing process; If it is in the stitching stage or the tensioning stage of the stitching process, determining the foot of the vertical of the target particle point according to the target particle point and the path point sequence; Based on the target particle point and the foot of the perpendicular, determining a first constraint function of the target particle point, wherein the first constraint function is used to constrain the distance between the target particle point and the foot of the perpendicular to meet a preset condition; Based on the first constraint function, using position dynamics calculation to obtain the model parameters of the soft tissue in the current frame; The soft tissue is displayed according to the model parameters of the soft tissue in the current frame.

13. A soft tissue suturing simulation device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the soft tissue suturing simulation method according to any one of claims 1 to 12 is implemented.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the soft tissue suturing simulation method according to any one of claims 1 to 12 is implemented.

Images