A method for modeling and collision detection of soft tissue in virtual surgery

Abstract

The application discloses a kind of virtual surgery in stomach soft tissue modeling and collision detection method, belong to virtual surgery in soft tissue modeling field.The important index that virtual surgery system can reach the operation training effect includes the real-time, stability and accuracy of physical modeling and collision detection.The biomechanical characteristics of stomach soft tissue under external force are analyzed in the application, based on the soft tissue deformation method of particle-spring deformation model, an improved deformation model based on reset spring is proposed, which solves the problem of "super-elastic" and "collapse" of traditional particle-spring model, and improves the authenticity of soft tissue deformation.At the same time, for muscle layer, the application uses the combination of Axis Aligned Bounding Box (AABB) and Sphere Bounding Box (Sphere) collision detection algorithm for modeling;The mixed hierarchical algorithm of Axis Aligned Bounding Box (AABB) and Sphere Bounding Box (Sphere) is used to realize the modeling of mucous membrane layer and other soft tissues of stomach;Real-time and accuracy of collision detection are considered.

Description

Technical Field

[0001] This invention relates to the field of soft tissue modeling in virtual surgery, specifically to a method for modeling and collision detection of gastric soft tissue in virtual surgery. Background Technology

[0002] Currently, the main treatments for gastric cancer are partial gastrectomy and total gastrectomy. The choice of surgical method and technique has a crucial impact on the patient's survival and quality of life. Virtual surgery technology can accurately simulate and perform gastric surgery, which is essential for improving the success rate of surgery and the patient's recovery rate. Completing core tasks such as the intersection test of virtual surgical instruments with human soft tissue, deformation calculation, and tactile feedback force calculation within the limited time of the virtual surgery system is of great significance for promoting the development of virtual surgery systems. This can not only optimize surgical plans and improve surgical success rates, but also promote the development of medicine and education.

[0003] Soft tissue modeling is a core component of virtual surgery. The mass-spring model, first proposed in 1987, has become one of the most widely used methods for soft tissue deformation modeling, especially in the deformation simulation of surface models. The core idea of ​​the mass-spring model is to discretize the surface model of soft tissue into mass points with a certain mass, and then connect the geometrically connected mass points through virtual spring elements according to the geometric topological relationship of the model.

[0004] Existing studies on gastric soft tissue modeling use bending springs and structural springs to construct point-spring models to simulate soft tissue deformation. This can lead to the model failing to return to its initial state under specific spring forces, resulting in a "collapse" problem. Furthermore, when the external force applied to the gastric soft tissue model increases to a certain extent, it can also cause "hyperelasticity." However, real gastric soft tissue consists of the mucosa, muscularis propria, and serosa, which can effectively expand and contract during digestion, giving the stomach a certain degree of elasticity and resilience. Therefore, "hyperelasticity" and "collapse" are impossible in a real surgical environment. Moreover, soft tissue models typically have high complexity and detail; existing collision detection algorithms in virtual surgery often employ a single, high-precision algorithm, leading to unnecessary computational overhead.

[0005] Chinese patent application CN 110610039 A, published on December 24, 2019, describes a method for simulating soft tissue deformation using a point mass spring model. This method improves the parameter settings of the point mass spring model using the finite element method, including initial parameter settings, determination of spring parameters during soft tissue deformation, and parameterization of anisotropic characteristics of the soft tissue. This improves the accuracy of the point mass spring model. A hybrid bounding box algorithm combining axis-aligned bounding boxes (AABB) and directional bounding boxes (OBB) is used for collision detection, and the improved collision detection algorithm enhances the model's interactivity and real-time performance. Although this method improves accuracy through parameter adjustment and algorithm optimization, it still does not solve the problem of reduced simulation accuracy of the point mass spring model under extreme deformation or high-speed collision conditions. Furthermore, this invention uses a uniform hybrid bounding box algorithm for the stomach and does not further refine the collision detection algorithm to differentiate between different soft tissue characteristics such as the gastric muscle layer and mucosa layer. Summary of the Invention

[0006] 1. Technical problems to be solved

[0007] The technical problem this invention aims to solve is the need for physical modeling and collision detection algorithms in virtual surgery, tailored to the characteristics of gastric soft tissue. This addresses the issues of "collapse" and "hardening" during the deformation recovery process of soft tissue simulated by the mass-spring model, and also addresses the problem of maintaining accuracy in collision detection while reducing computational load. This invention provides an improved mass-spring model that introduces the concept of a return spring into the traditional soft tissue model, enabling more precise modeling of gastric soft tissue. This results in a more realistic deformation effect under external forces, thus more accurately simulating the mechanical behavior of biological tissue. Furthermore, it differentiates collision detection algorithms based on the characteristics of the gastric soft tissue basal layer, mucosa layer, and other soft tissue structures, employing different hybrid-layered collision detection algorithms to maintain collision detection accuracy while reducing computational overhead.

[0008] 2. Technical Solution

[0009] The objective of this invention is achieved through the following technical solutions.

[0010] In a first aspect, the present invention provides a method for modeling and collision detection of gastric soft tissue in virtual surgery, the method comprising the following steps:

[0011] Step S100: Obtain raw images of various soft tissues of the stomach from the medical image set and preprocess the raw images. Specifically, this includes image reading, image denoising, image resampling, adjusting the contrast and brightness of the resampled image, and image segmentation to obtain a preprocessed image with uniform resolution and size.

[0012] Step S200: In the preprocessed image, a threshold-based segmentation algorithm is used to process the image data to identify and segment different soft tissue regions of the stomach and save the data. Specifically, the soft tissue of the stomach is divided into three categories: muscle layer, mucosa layer and other soft tissues. The three categories of soft tissues are classified and labeled, and the shape and size data of the three categories of soft tissues are saved.

[0013] Preferably, in step S200, a threshold-based segmentation algorithm is used to process the image data to extract the three types of soft tissue regions of the stomach and save the relevant data.

[0014] In step S300, based on the shape and size data of the three types of soft tissues in the stomach segmented in step S200, and combined with the different nonlinear and viscoelastic mechanical properties of the three types of soft tissues, a mass-spring model of the stomach soft tissue is constructed, consisting of multiple mass points and springs. The mass-spring model includes structural springs, bending springs, and return springs. According to the biomechanical properties of the soft tissue, the elastic coefficient, damping coefficient, and other parameters of the springs are set, and all parameter combinations form internal and external constraints. The constructed mass-spring model is then visually rendered.

[0015] The specific process of step S300 includes the following sub-steps:

[0016] Sub-step S310: Based on the shape and size data of the three types of soft tissues in the stomach segmented in step S200, and combined with the different nonlinear and viscoelastic mechanical properties corresponding to the three types of soft tissues, a three-dimensional mesh model of the soft tissues is constructed using three-dimensional reconstruction technology.

[0017] Sub-step S320 involves using a mesh simplification algorithm to reduce the number of triangles on the 3D mesh model. Preferably, the number of triangles on the model surface after the reduction process is in the tens of thousands.

[0018] Sub-step S330 involves constructing a point-spring model for the triangular facet structure of the model's outer surface, using the vertices of the model surface as points of mass. The point-spring model includes structural springs and bending springs, and assigns a return spring to each point of mass. One end of the return spring is connected to the current position of the point of mass, and the other end is connected to the initial, undeformed position of the point of mass. The initial length of the return spring is set to 0. Based on the biomechanical properties of soft tissue, parameters such as the spring constant and damping coefficient are set, and the combination of all parameters forms the internal and external constraints.

[0019] Specifically, the detailed process of sub-step S330 is as follows:

[0020] First, we construct a basic framework for soft tissue based on the mass-spring model.

[0021] Suppose the surface of the soft deformation model has n vertices. These n vertices constitute the discrete particles in the mass-spring model, denoted by N. i (i = 0, 1, ..., n-1) represents the point. Based on the topological relationship between vertices on the surface of the soft deformation model, discrete particles are connected using spring elements. If particle i and particle j are discrete points in the particle spring model, then the spring element between particle i and particle j can be L. ij (i, j∈[0, n-1]) represents the motion of a particle i at time t. According to Newton's second law of motion, we first need to calculate the net force F acting on particle i. i F i It can be obtained by combining the internal forces (elastic force and damping force of the spring connected to it) and external forces (gravity and externally applied forces) acting on particle i:

[0022]

[0023] Among them, F i This represents the net force acting on particle i. This represents the internal force acting on particle i. This represents the external force acting on particle i;

[0024] Assuming the particle adjacent to i is j, then the resultant force of the elastic force and damping force exerted by j on i is:

[0025] F ij =K s (|X ij |-l ij )+K d (V j -V i )

[0026] Among them, F ij K represents the resultant force of the elastic and damping forces exerted on i by the mass j adjacent to i. s X represents the stiffness coefficient of the spring. ij l represents the displacement difference between particles i and j. ij K represents the original length of the spring. d V represents the damping coefficient of the spring. j V represents the velocity of particle j. i This represents the velocity of particle i.

[0027] We can obtain:

[0028]

[0029] Sub-step S340 applies the Delaunay algorithm to tetrahedronize the reduced-facet model to obtain a more refined model structure. Based on the tetrahedron subdivision results, a filling structure is constructed at the vertices of each tetrahedron.

[0030] Sub-step S350: Visually render the constructed point-spring model using OpenGL.

[0031] In step S400, based on the mass-spring model of the gastric soft tissue constructed in step S300, and according to the data of different tissues obtained in step S200 and the biological characteristics of different soft tissues in the stomach, a hybrid hierarchical bounding box collision detection algorithm is used to set different hybrid hierarchical bounding boxes for the soft tissues. A hybrid hierarchical bounding box tree is constructed from top to bottom, and different hybrid hierarchical bounding box collision detection algorithms are adopted for different soft tissues. For the muscle layer, a collision detection algorithm combining spherical bounding boxes and directed bounding boxes is used for modeling. For the mucosa layer and other soft tissues in the stomach, a hybrid hierarchical algorithm combining axis-aligned bounding boxes and spherical bounding boxes is used to model the mucosa layer and other soft tissues in the stomach, and real-time detection and collision response are performed.

[0032] Furthermore, the specific method for constructing a hybrid hierarchical bounding tree from top to bottom is as follows: divide the gastric soft tissue into several subsets, construct a specific bounding body for each subset, and then recursively perform similar operations on each subset until the current subset contains only one vertex.

[0033] Furthermore, based on the structural characteristics of different soft tissues in the stomach, different hybrid hierarchical bounding box collision detection algorithms are adopted. Specifically, for the muscle layer, a hybrid hierarchical bounding box collision detection algorithm combining spherical bounding boxes and directed bounding boxes is used. For the mucosa layer and other soft tissues, a hybrid hierarchical bounding box collision detection algorithm combining axis-aligned bounding boxes and spherical bounding boxes is used.

[0034] Step S500: Calculate the external force acting on the model based on the depth information between the surgical instrument and the soft tissue model. When a collision occurs, first determine the local bounding box where the collision point is located, and then update the bounding boxes of the parent nodes layer by layer upwards until the root node is updated. Specifically, update the positions of the model surface vertices corresponding to each bounding box to complete the deformation simulation of the soft tissue under stress.

[0035] After introducing a return spring, we use the implicit Euler algorithm to solve for the relationship between displacement, velocity, and time, as follows:

[0036]

[0037] Among them, V t This represents the velocity of the particle at time t. This represents the velocity of the particle at time t0. X represents the integral of the ratio of the net force acting on a particle to its mass over the time interval t0 and t. t This represents the displacement of the particle at time t. Indicates the particle at time t o Displacement at any moment It represents the integral of the velocity of the particle over the time interval t0 and t.

[0038] Using the finite difference method to approximate their derivatives, we can solve the above equations:

[0039]

[0040] in, V represents the first derivative of the velocity of a particle with respect to time. t+1 V represents the velocity of a particle at time t+1. t O(Δt) represents the velocity of the particle at time t, Δt represents the time step, and O(Δt) represents the velocity of the particle at time t. 2 () represents the remainder when the time step approaches 0. X represents the first derivative of the particle displacement with respect to time. t+1 X represents the displacement of the particle at time t+1. t This represents the displacement of the particle at time t;

[0041] The velocity and position of the particle in the next time step can be obtained from the above integral formula as follows:

[0042]

[0043] Among them, V t+1 V represents the velocity of a particle at time t+1. t Let F(t) represent the velocity of the particle at time t, Δt represent the time step, F(t) represent the net force acting on the particle at time t, m represent the mass of the particle, and X represent the velocity of the particle at time t. t+1 X represents the displacement of the particle at time t+1. t This represents the displacement of the particle at time t;

[0044] The improved deformation differential equation of the point mass spring system:

[0045]

[0046] in, This represents the first derivative of the particle's displacement with respect to time. Let V represent the first derivative of the particle's velocity with respect to time; let V represent the particle's velocity; let F represent the net force acting on the particle; and let M represent the particle's mass matrix. The expression for F can also be written in the following form:

[0047]

[0048] in, This represents the internal force acting on particle i. F represents the external force acting on particle i. s F represents the elastic force of a point mass spring. d Let X represent the damping force of the spring on the mass, K represent the stiffness matrix of the system, X represent the displacement of the mass, D represent the damping matrix of the system, and X represent the first derivative of the displacement of the mass with respect to time.

[0049] Therefore, we can conclude that:

[0050]

[0051] Where M represents the mass matrix of the particle. The second derivative of the particle displacement with respect to time is expressed as the following equation:

[0052]

[0053] Where, m i Let F represent the mass of particle i, a represent the acceleration of particle i during its motion, and F represent the acceleration of particle i during its motion. s F represents the elastic force of a point mass spring. d This represents the damping force of the spring on a point mass;

[0054] The particle differential equation of the surface particle-spring model of the soft tissue deformation model of a biological organism can be obtained from the above formula as follows:

[0055]

[0056] in, Let V represent the first derivative of the particle's displacement with respect to time, and let V represent the particle's velocity. denoted by , where m represents the first derivative of the velocity of the particle with respect to time;

[0057] The construction of springs between surface particles and filler models in a biological soft tissue simulation model establishes a mapping relationship between surface particles and filler models in a biological soft tissue deformation simulation model.

[0058] A return spring was added to the traditional spring model, so that all particles on the surface of the biological soft tissue deformation simulation model are connected to the center of the nearest filler model through independent springs. When the filler model moves or rotates, it will cause the particle spring models on the surface of the biological soft tissue deformation simulation model to follow it and make corresponding movements. The movements of all filler models in the biological soft tissue deformation simulation model satisfy the following dynamic differential equation:

[0059]

[0060] Where, m i This represents the mass of particle i. Let represent the second derivative of the displacement of particle i with respect to time. The formula for the external force acting on particle i, according to Newton's second law, is:

[0061]

[0062] in, V represents the first derivative of the displacement of particle i with respect to time. i This represents the velocity of particle i. This represents the first derivative of the velocity of particle i with respect to time. Let m represent the second derivative of the displacement of particle i with respect to time. i Let be the mass of particle i.

[0063] In step S600, after the deformation calculation is completed, the virtual surgical scene is graphically refreshed to display the new shape and position of the soft tissue model; the feedback force of the model is input into the tactile device.

[0064] Step S700: Validate and optimize the generated soft tissue model. The validation operation includes model accuracy assessment and comparison with actual soft tissue.

[0065] Preferably, the validation and optimization of the generated soft tissue model includes: comparing the consistency of the physical properties of the soft tissue predicted by the model with the actual measured values ​​of the soft tissue on key parameters; conducting mechanical performance tests on the actual soft tissue and comparing the test results with the model prediction results to evaluate the accuracy of the model in mechanical behavior; and adjusting the model parameters according to the evaluation results to improve the accuracy and stability of the model.

[0066] 3. Beneficial effects

[0067] Compared with the prior art, the advantages of this invention are:

[0068] 1. This invention improves the spring-mass model by incorporating a return spring, discretizing complex soft tissue structures into a series of mass points and the springs connecting them. This discretization method significantly simplifies the model's complexity, making the previously continuous parameterization calculations simpler and more direct. It also enhances the model's real-time performance and interactivity, providing crucial information for optimizing surgical models. Key contributions include: the introduction of the return spring concept, combining biological principles and repair mechanisms, which efficiently simulates extreme soft tissue deformations, improving the model's accuracy and precision.

[0069] 2. This invention conducts in-depth research on different soft tissues of the stomach, combines their different physiological characteristics, and adopts a hybrid hierarchical collision detection algorithm. This reduces the number of accurate detections and repeated updates, effectively reduces the amount of computation in the soft tissue modeling process during virtual surgery, improves the computational efficiency of the model, thereby improving the real-time performance and interactivity of the model, and provides an important basis for the optimization of the surgical model.

[0070] The muscle layer is composed of smooth muscle and has strong extensibility. For the muscle layer, this invention uses a collision detection algorithm combining sphere bounding boxes and directed bounding boxes (OBBs) for modeling: axis-aligned bounding boxes can provide fast collision detection; directed bounding boxes can follow the direction and shape changes of the muscle layer, providing more accurate collision judgment.

[0071] The mucosa and other soft tissues of the stomach are extremely soft and elastic, making them suitable for relatively tight bounding box modeling methods. This invention employs a hybrid hierarchical algorithm combining axis-aligned bounding boxes (AABB) and sphere bounding boxes to model the mucosa and other soft tissues of the stomach. The axis-aligned bounding box (AABB) can quickly determine the approximate collision area, while the sphere bounding box can more closely conform to the soft shapes of the mucosa and other soft tissues of the stomach, thus effectively improving the accuracy and real-time performance of the experiment.

[0072] 3. The implicit Euler integral method is used to solve the equation. It has strong stability in the solution process, and can reduce the number of iterations and avoid numerical instability by increasing the time step. Attached Figure Description

[0073] Figure 1 Flowchart of gastric soft tissue modeling and collision detection method in virtual surgery;

[0074] Figure 2 A schematic diagram of a top-down method for constructing a hybrid hierarchical bounding box tree;

[0075] Figure 3 A schematic diagram of the Sphere-AABB mixed-layer bounding box for the gastric mucosa and other soft tissues of the stomach;

[0076] Figure 4 A schematic diagram of the Sphere-OBB hybrid hierarchical bounding box tree of the gastric muscle layer;

[0077] Figure 5 This is a schematic diagram illustrating the position update process of the model surface vertices corresponding to each bounding box after a collision. Detailed Implementation

[0078] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0079] The following are explanations of some of the technical terms and proper nouns that appear in this article:

[0080] Mass-Spring Model: The mass-spring model (MSM) treats the vertices in the soft tissue geometry model as mass points in the deformation model. The topological relationships between the vertices are described by virtual spring elements, commonly structural springs and bending springs. The mass-spring model mainly achieves real-time simulation of soft tissue deformation by performing force analysis on each mass point and calculating the corresponding velocity and displacement properties according to Newton's laws.

[0081] Return spring: A return spring is a virtual spring between a particle and itself, with an initial length of 0. Specifically, when a particle deviates from its equilibrium position, the spring between the particle's current position and its initial position is the return spring.

[0082] AABB (Axis-aligned bounding box): The corresponding Chinese definition is an axis-aligned bounding box, the smallest hexahedron containing the object with edges parallel to the coordinate axes. Therefore, describing an AABB only requires six scalars. AABBs are relatively simple to construct and require little storage space, but their compactness is poor, especially for irregular geometric shapes, where there is a lot of redundant space, and it is impossible to rotate the object accordingly. They are suitable for handling rigid and convex objects, and are not suitable for complex virtual environments involving soft deformations. AABBs are also a relatively simple type of bounding box. However, their compactness is poor for elongated objects placed along diagonal directions. Due to the simplicity of AABB intersection testing and their relatively good compactness, they are widely used and can also be used for collision detection of soft objects.

[0083] OBB (Oriented Bounding Box): A bounding box is an arbitrary, minimal rectangular prism. The most significant characteristic of OBBs is their arbitrary orientation, allowing them to enclose objects as tightly as possible based on their shape. However, this also complicates intersection testing. OBBs approximate objects more closely than AABBs and bounding spheres, significantly reducing the number of bounding boxes and thus avoiding numerous intersection detections between them. However, intersection detection between OBBs is more time-consuming than between AABBs or bounding spheres.

[0084] Sphere Bounding Box: A spherical bounding box, which is the smallest sphere containing the object. To determine the spherical bounding box, first calculate the average x, y, and z coordinates of the vertices of all elements in the set of basic geometric elements that make up the object to determine the center of the bounding sphere. Then, determine the radius r by the distance between the center of the sphere and the point determined by the three maximum coordinates. Collision detection of the bounding sphere mainly involves comparing the radii between two spheres and the distances from their centers.

[0085] Physically based deformation modeling is a crucial component of virtual surgical systems, and one of the key focuses of soft tissue deformation modeling is accurately simulating the stress-strain relationship of soft tissues. Precise soft tissue deformation modeling methods often require significant computational time, leading to a decrease in the real-time performance of the virtual surgical system. Therefore, it is necessary to improve the accuracy of soft tissue deformation modeling methods as much as possible without compromising system real-time performance.

[0086] The mass-spring model has been widely used in soft tissue deformation modeling due to its advantages such as simple principle, ease of implementation, and low computational cost. However, this method also has certain limitations. One of its drawbacks is the poor stability of the mass-spring model, which often exhibits a "hardening" phenomenon when simulating large deformations of soft bodies.

[0087] The purpose of this invention is to address the problems of existing technologies by improving the accuracy of the model while ensuring real-time performance, and to simulate the collision behavior of different soft tissues more accurately and in real time. First, we introduce the concept of a return spring into the virtual surgical system. The return spring is a virtual spring between a mass and itself, with an initial length of 0. When a mass deviates from its equilibrium position, it is subjected not only to the forces of the structure and the bending spring, but also to the force of the return spring, which drives the displaced mass back to its initial position. This more realistically simulates the deformation of gastric soft tissue.

[0088] In addition, this invention takes into account the specific characteristics of gastric soft tissue and designs collision detection algorithms differently for the different characteristics of the gastric soft tissue basal layer, mucosa layer and other soft tissue structures, thereby reducing computational overhead while maintaining collision detection accuracy.

[0089] Example

[0090] In a first aspect, the present invention provides a method for modeling and collision detection of gastric soft tissue in virtual surgery, such as... Figure 1 As shown, the method steps are as follows:

[0091] Step S100, Preprocessing of gastric soft tissue images: Obtain original images of various soft tissues of the stomach from medical image sets (such as CT, MRI and other scan images) and preprocess the original images.

[0092] First, image reading is performed. In some embodiments, medical image data can be read from the medical image set of the specialized medical image processing library pydicom and loaded into the program to extract the required image data. The relevant data includes different ages, genders, and disease states (such as normal, gastritis, gastric ulcer, gastric cancer, etc.).

[0093] Next, image denoising and resampling operations are performed. Preferably, mean filtering can be used to denoise the image to improve image quality. Then, the denoised image is resampled to normalize voxels of different sizes to the same size, facilitating subsequent processing and analysis.

[0094] Furthermore, the contrast and brightness of the resampled image are adjusted to make the soft tissue structures in the image more prominent.

[0095] Finally, image segmentation is performed to obtain a preprocessed image with uniform resolution and size.

[0096] Step S200: Gastric soft tissue region identification. Using image target detection technology, the gastric soft tissue region is identified and segmented in the preprocessed image, and the data is saved. The gastric soft tissue is divided into three categories: muscle layer, mucosa layer, and other soft tissues. These three categories are then classified and labeled, and their shape and size data are saved. This lays the foundation for subsequent model construction and hybrid hierarchical bounding box collision detection. Preferably, a threshold-based segmentation algorithm is used to process the image data to extract the three types of gastric soft tissue regions and save the relevant data.

[0097] Step S300, Construction of the gastric soft tissue mass-spring model: Based on the shape and size data of the three types of gastric soft tissue segmented in step S200, and combined with the different nonlinear and viscoelastic mechanical properties corresponding to the three types of soft tissue, a mass-spring model of the gastric soft tissue composed of multiple mass points and springs is generated. The mass-spring model includes structural springs, bending springs, and return springs. According to the biomechanical characteristics of soft tissue, the elastic coefficient, damping coefficient, and other parameters of the springs are set. All parameter combinations form internal and external constraints. The constructed mass-spring model is then visually rendered.

[0098] The specific process of step S300 includes the following sub-steps:

[0099] Sub-step S310 involves constructing a three-dimensional mesh model of the stomach tissue based on the shape and size data of the three types of soft tissues segmented in step S200, combined with the different nonlinear and viscoelastic mechanical properties corresponding to the three types of soft tissues, using three-dimensional reconstruction technology.

[0100] Sub-step S320 involves using a mesh simplification algorithm to reduce the number of triangles on the 3D mesh model. This brings the number of triangles on the model's surface within a reasonable range, typically in the tens of thousands after the reduction process.

[0101] Sub-step S330 involves constructing a point-spring model for the triangular facet structure of the model's outer surface, using the vertices of the model surface as points of mass. The point-spring model includes structural springs, bending springs, and a return spring for each point of mass. One end of the return spring is connected to the current position of the point of mass, while the other end is connected to the initial, undeformed position. Therefore, when a point of mass deforms under external force, it experiences not only the forces of the connected structural and bending springs but also the tension of the return spring. The initial length of the return spring is set to 0; they always attempt to pull the connected point of mass back to its initial position, thus simulating the elastic recovery behavior of soft tissue under external force.

[0102] Based on the biomechanical properties of soft tissue, parameters such as the spring constant and damping constant are set, and all parameter combinations form internal and external constraint conditions.

[0103] Sub-step S340 applies the Delaunay algorithm to tetrahedronize the reduced-facet model to obtain a more refined model structure. Based on the tetrahedron subdivision results, a filling structure is constructed at the vertices of each tetrahedron.

[0104] Sub-step S350 uses OpenGL to perform visual rendering of the constructed mass-spring model. This ensures that the model meets both the realism and real-time requirements of biological soft tissue deformation simulation.

[0105] Then, OpenGL was used to perform visual rendering of the constructed point mass-spring model.

[0106] Specifically, in sub-step S330, for the triangular facet structure of the model's outer surface, a mass-spring model is constructed using the vertices of the model's surface as mass points. The detailed process of constructing the mass-spring on the model's surface is as follows:

[0107] First, we construct a basic framework for soft tissue based on the mass-spring model.

[0108] Suppose the surface of the soft deformation model has n vertices. These n vertices constitute the discrete particles in the mass-spring model, denoted by N. i (i = 0, 1, ..., n-1) represents the point. Based on the topological relationship between vertices on the surface of the soft deformation model, discrete particles are connected using spring elements. If particle i and particle j are discrete points in the particle spring model, then the spring element between particle i and particle j can be L. ij(i, j∈[0, n-1]) represents the motion of a particle i at time t. According to Newton's second law of motion, we first need to calculate the net force F acting on particle i. i F i It can be obtained by combining the internal forces (elastic force and damping force of the spring connected to it) and external forces (gravity and externally applied forces) acting on particle i:

[0109]

[0110] Among them, f i This represents the net force acting on particle i. This represents the internal force acting on particle i. This represents the external force acting on particle i;

[0111] Assuming the particle adjacent to i is j, then the resultant force of the elastic force and damping force exerted by j on i is:

[0112] F ij =K s (|X ij |-l ij )+K d (V j -V i )

[0113] Among them, F ij K represents the resultant force of the elastic and damping forces exerted on i by the mass j adjacent to i. s X is the stiffness coefficient of the spring. ij l represents the displacement difference between particles i and j. ij K represents the original length of the spring. d V represents the damping coefficient. j V represents the velocity of particle j. i This represents the velocity of particle i.

[0114] We can obtain:

[0115]

[0116] Step S400, Real-time collision detection of the gastric soft tissue mass-spring model: Based on the mass-spring model of the gastric soft tissue constructed in step S300, and according to the data of different tissues obtained in step S200 and the biological characteristics of different soft tissues of the stomach, a hybrid hierarchical bounding box tree is constructed from top to bottom, and different hybrid hierarchical bounding box collision detection algorithms are adopted to perform real-time detection and collision response.

[0117] like Figure 2As shown, the specific method for constructing a hybrid hierarchical bounding tree from top to bottom is as follows: divide the stomach soft tissue into several subsets, construct a specific bounding body for each subset, and then recursively perform similar operations on each subset until the current subset contains only one vertex.

[0118] like Figure 3 , Figure 4 As shown, based on the structural characteristics of different soft tissues in the stomach, different hybrid hierarchical bounding box collision detection algorithms are adopted. The specific method is as follows: For the muscle layer, this invention uses a hybrid hierarchical bounding box collision detection algorithm that combines spherical bounding boxes and directed bounding boxes. The spherical bounding box provides fast collision detection, while the directed bounding box can follow the direction and shape changes of the muscle layer to provide more accurate collision judgment. This can effectively improve the accuracy and real-time performance of the experiment.

[0119] An OBB bounding box can be understood as an AABB bounding box with orientation. Bounding boxes constructed using the OBB algorithm can adapt to 3D models with arbitrary rotation angles. The construction of OBB bounding boxes can be represented as follows:

[0120] R={O+br1e1+br2e2+br3e3|b∈(-1,1)}

[0121] Where R represents the radius of the spherical bounding box, O represents the coordinates of the center point of the bounding box; b is a parameter whose value varies between -1 and 1, used to generate the boundary or extended region of the OBB; r1, r2, and r3 represent the radii of the bounding box on the x-axis, y-axis, and z-axis in three-dimensional space; and e1, e2, and e3 represent the orthogonal vectors used to describe the directions of the bounding box on the x-axis, y-axis, and z-axis in three-dimensional space.

[0122] For the mucosa and other soft tissues of the stomach, due to their extreme softness and elasticity, they are suitable for relatively tight bounding boxes. Therefore, in this experiment, a hybrid hierarchical bounding box collision detection algorithm using axis-aligned bounding boxes and spherical bounding boxes was used for this layer. Spherical bounding boxes can quickly determine the approximate collision area, while axis-aligned bounding boxes can fit more closely to the soft shape of the mucosa and improve the accuracy of collision detection.

[0123] Assuming we have an AABB bounding box, the method for calculating the coordinates of the center S of the corresponding sphere is as follows:

[0124]

[0125] Among them, G min (x,y,z) represents the point where the bounding box reaches its minimum value in the three-dimensional coordinate system; G max (x,y,z) represents the point where the bounding box reaches its maximum value in the three-dimensional coordinate system;

[0126] The radius R of the Sphere is:

[0127]

[0128] Where R represents the radius of the spherical bounding box, and G... max (x) represents the maximum value achieved by the bounding box on the x-axis of the three-dimensional coordinate system, G min (x) represents the minimum value obtained by the bounding box on the x-axis of the three-dimensional coordinate system, G max (y) represents the maximum value achieved by the bounding box on the y-axis of the three-dimensional coordinate system, G min (y) represents the minimum value obtained by the bounding box on the y-axis of the three-dimensional coordinate system, G max (z) represents the maximum value achieved by the bounding box on the z-axis of the three-dimensional coordinate system, G min (z) represents the minimum value that the bounding box achieves on the z-axis of the three-dimensional coordinate system.

[0129] Using different hybrid hierarchical bounding box collision detection algorithms for different soft tissues of the stomach is far more efficient than using a single hybrid hierarchical bounding box collision detection algorithm for different soft tissues in traditional soft tissue modeling. This largely ensures the real-time performance and accuracy of the virtual surgery simulation training system, balancing the accuracy and real-time performance of collision detection in the virtual surgery system. Collision detection experiments based on the stomach show that the proposed hybrid hierarchical bounding box method can improve the efficiency of collision detection, and the higher the complexity of the soft tissue geometry model, the more significant the improvement in collision detection efficiency.

[0130] Step S500, Deformation calculation of the gastric soft tissue mass-spring model: (e.g.) Figure 5 As shown, the external force on the model is calculated based on the depth information between the surgical instrument and the soft tissue model. When a collision occurs, the local bounding box of the collision point is first determined, and the bounding boxes of the parent nodes are updated layer by layer upwards until the root node is updated. Specifically, the positions of the model surface vertices corresponding to each bounding box are updated to realize the deformation simulation of soft tissue under force.

[0131] By introducing a return spring, we can obtain the relationship between displacement, velocity, and time. Choosing a reasonable and efficient numerical integration method is crucial for the accuracy, stability, and speed of the numerical solution. The implicit Euler algorithm has the advantage of low computational cost, which can improve the real-time performance and accuracy of the model. Therefore, the implicit Euler algorithm can be used for solving the problem.

[0132]

[0133] Among them, V t This represents the velocity of the particle at time t. This represents the velocity of the particle at time t0. X represents the integral of the ratio of the net force acting on a particle to its mass over the time interval t0 and t. t X is the displacement of the particle at time t. 0 It is the displacement of the particle at time t0. It represents the integral of the velocity of the particle over the time interval t0 and t.

[0134] Using the finite difference method to approximate their derivatives, we can solve the above equations:

[0135]

[0136] in, V represents the first derivative of the velocity of a particle with respect to time. t+1 V represents the velocity of a particle at time t+1. t O(Δt) represents the velocity of the particle at time t, Δt represents the time step, and O(Δt) represents the velocity of the particle at time t. 2 () represents the remainder when the time step approaches 0. X represents the first derivative of the particle displacement with respect to time. t+1 X represents the displacement of the particle at time t+1. t This represents the displacement of the particle at time t;

[0137] The velocity and position of the particle in the next time step can be obtained from the above integral formula:

[0138]

[0139] Among them, V t+1 V represents the velocity of a particle at time t+1. t Let F(t) represent the velocity of the particle at time t, Δt represent the time step, F(t) represent the net force acting on the particle at time t, m represent the mass of the particle, and X represent the velocity of the particle at time t. t+1 X represents the displacement of the particle at time t+1. t This represents the displacement of the particle at time t;

[0140] The improved deformation differential equation of the point mass spring system:

[0141]

[0142] in, This represents the first derivative of the particle's displacement with respect to time. Let V represent the first derivative of the particle's velocity with respect to time; let V represent the particle's velocity; let F represent the net force acting on the particle; and let M represent the particle's mass matrix. The expression for F can also be written in the following form:

[0143]

[0144] in, This represents the internal force acting on particle i. F represents the external force acting on particle i. s F represents the elastic force of a point mass spring. d Let X represent the damping force of the spring on a point mass, K represent the stiffness matrix of the system, X represent the displacement of the point mass, and D represent the damping matrix of the system. This represents the first derivative of the particle's displacement with respect to time.

[0145] Therefore, we can conclude that:

[0146]

[0147] Where M represents the mass matrix of the particle. The second derivative of the particle displacement with respect to time is expressed as the following equation:

[0148]

[0149] Where, m i Let F represent the mass of particle i, a represent the acceleration of particle i during its motion, and F represent the acceleration of particle i during its motion. s F represents the elastic force of a point mass spring. d This represents the damping force of the spring on a point mass;

[0150] The particle differential equation of the surface particle-spring model of the soft tissue deformation model of a biological organism can be obtained from the above formula as follows:

[0151]

[0152] in, Let V represent the first derivative of the particle's displacement with respect to time, and let V represent the particle's velocity. denoted by , where m represents the first derivative of the velocity of the particle with respect to time;

[0153] The combination of traditional mass-spring and return spring can more accurately simulate the deformation of soft tissues under complex conditions, laying the foundation for the next step of establishing mapping constraints for biological soft tissue deformation simulation models.

[0154] The construction of springs between surface particles and filler models in a biological soft tissue simulation model establishes a mapping relationship between surface particles and filler models in a biological soft tissue deformation simulation model.

[0155] All particles on the surface of the biological soft tissue deformation simulation model are connected to the center of the nearest filler model via independent springs. When the filler model moves or rotates, it will cause the particle spring models on the surface of the biological soft tissue deformation simulation model to follow suit and make corresponding movements. The movements of all filler models in the biological soft tissue deformation simulation model satisfy the following dynamic differential equation:

[0156]

[0157] Where, m i This represents the mass of particle i. Let represent the second derivative of the displacement of particle i with respect to time. Let i be the external force acting on particle i. According to Newton's second law, we have the following formula:

[0158]

[0159] in, V represents the first derivative of the displacement of particle i with respect to time. i This represents the velocity of particle i. This represents the first derivative of the velocity of particle i with respect to time. Let m represent the second derivative of the displacement of particle i with respect to time. i Let be the mass of particle i.

[0160] Step S600, Visual Rendering: After deformation calculation is completed, the virtual surgical scene is graphically refreshed to display the new shape and position of the soft tissue model through visual rendering. The feedback force of the model is output to a haptic device so that the user can feel the interaction force between the surgical instruments and the soft tissue. This can be achieved using existing technologies.

[0161] Step S700, Model Optimization and Validation: The generated soft tissue model is validated and optimized. Validation includes model accuracy assessment and comparison with actual soft tissue. This ensures the model meets practical requirements.

[0162] Validating and optimizing the generated soft tissue model mainly involves the following aspects: comparing the model's predicted soft tissue physical properties (such as elastic modulus and Poisson's ratio) with actual measured soft tissue values ​​to ensure consistency in key parameters; conducting mechanical property tests (such as tensile, compressive, and shear tests) on the actual soft tissue and comparing the test results with the model's predictions to evaluate the model's accuracy in mechanical behavior; and then adjusting the model parameters based on the evaluation results to improve the model's accuracy and stability. This step is existing technology.

[0163] The present invention and its embodiments have been described above illustratively. This description is not restrictive, and the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. The accompanying drawings are only one embodiment of the present invention, and the actual structure is not limited thereto. No reference numerals in the claims should limit the scope of the claims. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the scope of protection of the present invention. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Multiple elements stated in the product claims may also be implemented by a single element through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for modeling and collision detection of gastric soft tissue in virtual surgery, characterized in that, The steps include the following: Step S100: Obtain original images of various soft tissues of the stomach from the medical image set and preprocess the original images; specifically, this includes image reading, image denoising, image resampling, adjusting the contrast and brightness of the resampled image, and image segmentation to obtain a preprocessed image with uniform resolution and size. Step S200: In the preprocessed image, a threshold-based segmentation algorithm is used to identify and segment different soft tissue regions of the stomach and save the data. Specifically, the soft tissue of the stomach is divided into three categories: muscle layer, mucosa layer and other soft tissues. The three categories of soft tissues are classified and labeled, and the shape and size data of the three categories of soft tissues are saved. In step S300, based on the shape and size data of the three types of soft tissues in the stomach segmented in step S200, and combined with the different nonlinear and viscoelastic mechanical properties of the three types of soft tissues, a mass-spring model of the stomach soft tissue composed of multiple mass points and springs is constructed. The mass-spring model includes structural springs, bending springs and return springs. The elastic coefficient and damping coefficient of the springs are set according to the biomechanical properties of the soft tissue. The combination of all parameters forms the internal and external constraints. Perform visual rendering on the constructed point mass-spring model; In step S400, based on the mass-spring model of the gastric soft tissue constructed in step S300, according to the data of different tissues obtained in step S200 and the biological characteristics of different soft tissues in the stomach, a hybrid hierarchical bounding box collision detection algorithm is used to set different hybrid hierarchical bounding boxes for the soft tissues, a hybrid hierarchical bounding box tree is constructed from top to bottom, and different hybrid hierarchical bounding box collision detection algorithms are used for different soft tissues to perform real-time detection and collision response. Step S500: Calculate the external force on the model based on the depth information between the surgical instrument and the soft tissue model. When a collision occurs, first determine the local bounding box where the collision point is located, and update the bounding box of the parent node layer by layer upwards until the root node is updated. Specifically, update the position of the model surface vertex corresponding to each bounding box to complete the deformation simulation of the soft tissue under force. Step S600: After the deformation calculation is completed, the virtual surgical scene is graphically refreshed to display the new shape and position of the soft tissue model; the feedback force of the model is input into the haptic device. Step S700: Validate and optimize the generated soft tissue model. The validation operation includes model accuracy assessment and comparison with actual soft tissue.

2. The method for modeling and collision detection of gastric soft tissue in virtual surgery according to claim 1, characterized in that, In step S200, a threshold-based segmentation algorithm is used to process the image data.

3. The method for modeling and collision detection of gastric soft tissue in virtual surgery according to claim 1, characterized in that, The specific process of step S300 includes the following sub-steps: Sub-step S310: Based on the shape and size data of the three types of soft tissues in the stomach segmented in step S200, and combined with the different nonlinear and viscoelastic mechanical properties of the three types of soft tissues, a three-dimensional mesh model of the soft tissues is constructed using three-dimensional reconstruction technology. Sub-step S320: Use a mesh simplification algorithm to reduce the surface area of ​​the 3D mesh model; Sub-step S330: For the triangular facet structure on the outer surface of the model, a mass-spring model is constructed with the vertices of the model surface as mass points. The mass-spring model includes structural springs and bending springs, and each mass point is assigned a reset spring. One end of the reset spring is connected to the current position of the mass point, and the other end is connected to the initial undeformed position of the mass point. The original length of the reset spring is set to 0. The elastic coefficient and damping coefficient of the spring are set according to the biomechanical characteristics of soft tissue. All the parameter combinations form the internal and external constraint conditions. Sub-step S340: Perform tetrahedral subdivision on the model after the reduction of the number of faces, and construct a filling structure at the vertex of each tetrahedron based on the result of the tetrahedral subdivision. Sub-step S350: Visually render the constructed mass-spring model.

4. The method for modeling and collision detection of gastric soft tissue in virtual surgery according to claim 3, characterized in that, After the face reduction process, the number of triangular faces on the model surface is in the tens of thousands.

5. The method for modeling and collision detection of gastric soft tissue in virtual surgery according to claim 1, characterized in that, In step S400, the specific method for constructing a hybrid hierarchical bounding tree from top to bottom is as follows: divide the gastric soft tissue into several subsets, construct a specific bounding body for each subset after division, and then recursively perform similar operations on each subset until the current subset contains only one vertex.

6. The method for modeling and collision detection of gastric soft tissue in virtual surgery according to claim 1, characterized in that, The specific method for using different hybrid hierarchical bounding box collision detection algorithms for different soft tissues in step S400 is as follows: for the muscle layer, a hybrid hierarchical bounding box collision detection algorithm combining spherical bounding boxes and oriented bounding boxes is used; for the mucosa layer and other soft tissues, a hybrid hierarchical bounding box collision detection algorithm combining axis-aligned bounding boxes and spherical bounding boxes is used.

7. The method for modeling and collision detection of gastric soft tissue in virtual surgery according to claim 1, characterized in that, In step S500, the implicit Euler algorithm is used to solve for the relationship between displacement, velocity, and time, as follows: in, Indicates that the particle is in Speed ​​of motion at any moment Indicates that the particle is in Speed ​​of motion at any moment This represents the ratio of the net force acting on a particle to its mass. and Points accumulated during this period Indicates that the particle is in Displacement at any moment Indicates that the particle is in Displacement at any moment The velocity of a particle is expressed as follows: and Points earned during this period; Using the finite difference method to approximate their derivatives, we can solve the above equations: in, This represents the first derivative of the velocity of a particle with respect to time. Indicates that the particle is in Speed ​​of motion at any moment Indicates that the particle is in Speed ​​of motion at any moment Indicates the time step. , This represents the first derivative of the particle's displacement with respect to time. Indicates that the particle is in Displacement at any moment Indicates that the particle is in Displacement at any given moment; The velocity and position of the particle in the next time step are: in, Indicates that the particle is in Speed ​​of motion at any moment Indicates that the particle is in Speed ​​of motion at any moment Indicates the time step. Indicates that the particle is in The combined force at every moment, Indicates the mass of a point mass. Indicates that the particle is in Displacement at any moment Indicates that the particle is in Displacement at any given moment.

8. A method for modeling and collision detection of gastric soft tissue in virtual surgery according to claims 1-7, characterized in that, The deformation differential equation of the improved point-spring model is: in, This represents the first derivative of the particle's displacement with respect to time. This represents the first derivative of the velocity of a particle with respect to time. Indicates the velocity of a particle. This represents the net force acting on a point mass. The mass matrix represents the mass of a point mass. The expression is as follows: in, Represents a point mass The internal force received Represents a point mass The external force received, This represents the elastic force of a spring on a point mass. This represents the damping force of the spring on a point mass. Represents the stiffness matrix of the system. Represents the displacement of a particle. Represents the damping matrix of the system. It represents the first derivative of the particle displacement with respect to time.

9. The method for modeling and collision detection of gastric soft tissue in virtual surgery according to claim 8, characterized in that, The particle differential equation of the surface particle-spring model in the soft tissue deformation model of a biological organism is shown below: in, Represents the velocity of a particle. This represents the first derivative of the velocity of a particle with respect to time. Represents a point mass The external force received, This represents the elastic force of a spring on a point mass. This represents the damping force of the spring on a point mass. It represents the mass of a point mass.

10. The method for modeling and collision detection of gastric soft tissue in virtual surgery according to claim 9, characterized in that, A return spring is added to the traditional spring model. All particles on the surface of the biological soft tissue deformation simulation model are connected to the center of the nearest filler model through independent springs. When the filler model moves or rotates, it will cause the particle spring models on the surface of the biological soft tissue deformation simulation model to follow it and make corresponding movements. The motion of all filler models in the biological soft tissue deformation simulation model satisfies the following dynamic differential equation: in, Represents a point mass quality Represents a point mass The second derivative of displacement with respect to time, Represents a point mass The external force acting on the body can be expressed by the formula derived from Newton's second law: in, Represents a point mass The first derivative of displacement with respect to time, Represents a point mass The speed of movement, Represents a point mass The first derivative of the velocity with respect to time Represents a point mass The second derivative of displacement with respect to time, Represents a point mass The external force received, Represents a point mass The quality.

Images