Soft tissue meshless deformation modeling method, device, equipment and medium

Abstract

The present invention provides a method, device, equipment and medium for meshless deformation modeling of soft tissue, which can be used in the field of virtual reality technology. The method includes: based on the multidimensional parameter tensor of volume data points, using an improved ray casting algorithm to construct an initial geometric model of the soft tissue structure; constructing a soft tissue continuous medium system equation; solving the system equation to obtain a deformation physical model of the soft tissue; based on the physical model, obtaining the displacement of the calculation point element; based on the initial geometric model and the displacement of the calculation point element, redistributing the volume data points to obtain a deformation model of the soft tissue, wherein the coefficients of the physical model are related to the multidimensional parameter tensor; the calculation point element is a volume data point uniformly selected in the initial geometric model. The inhomogeneous soft tissue model is constructed based on volume rendering and meshless method, which solves the problems of pathological grid and jaggedness existing in the grid method, and can describe the real internal structure and biomechanical property information of the soft tissue, with good visual and tactile effects.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to a method, device, equipment and medium for meshless deformation modeling of soft tissue. Background Art

[0002] In virtual surgery, a combination of a mesh-based physical model and a surface-based geometric model is usually used to model the deformation of soft tissues. This means that the organ surface and incision surface in the real surgical scene are extracted and attached to the physical structure constructed by finite element, spring-mass and other methods. Figure 1 As shown in the figure, when surgeons perform surgical operations on virtual soft tissues, such as cutting, stretching, and squeezing organs, when the tissue deformation is too large, the topological structure of the model will be destroyed, and then an incision will be generated. The position and shape of the incision are obtained through physical calculation, and the surface texture of the incision extracted in advance is attached to the incision. The method of constructing soft tissue models based on mesh method and surface rendering has high computational efficiency and is easy to implement. However, the number of pathological and deformed meshes will increase during the reorganization and regeneration of the mesh, resulting in abnormalities in the entire model; the surface texture of the incision is extracted in advance and does not match the incision generated in real time, and there is distortion at the connection part, and the incision cannot describe the real internal structural information; insufficient mesh and voxel density will result in jagged incisions and rough surfaces, and high density will greatly increase the amount of calculation, affecting the real-time performance of the simulation. At the same time, traditional grid-based algorithms find it difficult to establish a fitting relationship with discrete voxels in three-dimensional data fields. The physical model cannot introduce fine structural material coefficients. The interactions between the elements and the external force state calculated using constant parameters when solving the system equations cannot describe the differences between tissue structures. The feedback force changes in the same trend when operating on different parts. The constitutive equation established only by models such as non-uniform viscoelasticity and approximate incompressibility cannot characterize the heterogeneity of soft tissues. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] In order to solve at least one of the above-mentioned technical problems arising in the soft tissue deformation modeling in the prior art, the embodiments of the present invention provide a soft tissue meshless deformation modeling method, device, equipment and medium, which constructs a non-homogeneous soft tissue model based on volume rendering and meshless method, solves the problems of pathological grid and jaggedness existing in the grid method, and can describe the real internal structure and biomechanical property information of the soft tissue, and has better visual and tactile effects than the existing methods.

[0005] (II) Technical solution

[0006] In view of the above technical problems, embodiments of the present invention provide a method, device, equipment and medium for meshless deformation modeling of soft tissue.

[0007] According to a first aspect of the present invention, a method for meshless deformation modeling of soft tissue is provided, comprising: constructing an initial geometric model of the soft tissue structure based on a multidimensional parameter tensor of volume data points using an improved ray casting algorithm; constructing a soft tissue continuum system equation; solving the system equation using a non-homogeneous viscoelastic model based on a meshless radial basis point interpolation method to obtain a deformation physical model of the soft tissue; obtaining the displacement of calculation point elements based on the physical model; and redistributing the volume data points based on the initial geometric model and the displacement of the calculation point elements to obtain a deformation model of the soft tissue, wherein the coefficients of the physical model are related to the multidimensional parameter tensor; and the calculation point elements are volume data points uniformly selected in the initial geometric model.

[0008] In some exemplary embodiments, based on the multidimensional parameter tensor of volume data points, an improved ray casting algorithm is used to construct an initial geometric model of the soft tissue structure, including: acquiring volume data values ​​of volume data points of the soft tissue; preprocessing the volume data values ​​to obtain a multidimensional parameter tensor of the volume data points; constructing an object-image feature transfer function based on the multidimensional parameter tensor and a color index table to obtain color information of the volume data points; and based on the color information, using an improved ray casting algorithm to assign geometric attributes to each sampling point to construct an initial geometric model of the soft tissue structure, wherein the color information includes a color value and a transparency value; and the color index table includes an organ type of the soft tissue and a corresponding multidimensional parameter matrix.

[0009] In some exemplary embodiments, an object-image feature transfer function is constructed based on a multidimensional parameter tensor and a color index table to obtain color information of volume data points, including: determining the organ type corresponding to the soft tissue using the color index table based on the multidimensional parameter tensor; determining the corresponding multidimensional parameter matrix based on the organ type using the color index table; and calculating the color information of the volume data points based on the multidimensional parameter matrix and the multidimensional parameter tensor.

[0010] In some exemplary embodiments, based on color information, geometric properties are assigned to each sampling point using an improved ray casting algorithm to construct an initial geometric model of the soft tissue structure, including: using parallel projection to perform ray casting sampling to obtain sampling points; accumulating the color values ​​and transparency values ​​of the sampling points through alpha blending technology, and when the accumulated opacity is 1 or the light passes through the soft tissue, calculating the color synthesis value to obtain the final color value and the final transparency value; and based on the final color value and the final transparency value, constructing a three-dimensional soft tissue model containing internal fine structure information.

[0011] In some exemplary embodiments, based on the initial geometric model and the displacement of the calculation point elements, the volume data points are redistributed, including: for the volume data points selected as the calculation point elements, the corresponding position of the calculation point elements after deformation is the position of the volume data point after deformation; and for the volume data points not selected as the calculation point elements, the deformed position of the volume data point is calculated according to the deformed positions of the 8 calculation point elements closest to the volume data point.

[0012] In some exemplary embodiments, the modeling method also includes: calculating the stress of bleeding particles in soft tissue; obtaining the displacement of bleeding particles based on the stress of bleeding particles; obtaining a deformation model of bleeding particles based on the displacement; combining the deformation model of bleeding particles with the deformation model of soft tissue to obtain the final deformation model of soft tissue.

[0013] In some exemplary embodiments, the stress of the bleeding particle is related to the stress of the bleeding particle within a preset radius and the mass of the bleeding particle, wherein the mass of the bleeding particle is related to the distance of the bleeding particle from the bleeding surface.

[0014] According to a second aspect of the present invention, a soft tissue meshless deformation modeling device is provided, the device comprising: a first modeling module, used to construct an initial geometric model of the soft tissue structure based on a multidimensional parameter tensor of volume data points and using an improved ray casting algorithm; an equation construction module, used to construct a soft tissue continuous medium system equation; a first calculation module, used to solve the system equation based on a meshless radial base point interpolation method and using a non-homogeneous viscoelastic model to obtain a deformation physical model of the soft tissue; a second calculation module, used to obtain the displacement of the calculation point elements based on the physical model; and a second modeling module, used to redistribute the volume data points based on the initial geometric model and the displacement of the calculation point elements to obtain a deformation model of the soft tissue, wherein the coefficients of the physical model are related to the multidimensional parameter tensor; the calculation point elements are volume data points uniformly selected in the initial geometric model.

[0015] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is executed by the processor to implement the above method.

[0016] According to a fourth aspect of the present invention, a storage medium is provided, characterized in that: computer-readable instructions are stored in the storage medium, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the above method.

[0017] (III) Beneficial effects

[0018] It can be seen from the above technical solutions that the method, device, equipment and medium for meshless deformation modeling of soft tissue provided by the embodiments of the present invention have at least the following beneficial effects:

[0019] (1) Multidimensional parameters are obtained by preprocessing the volume data. The multidimensional parameters are introduced into the calculation of the physical model of soft tissue, and the correlation between the physical model coefficients and the volume data values ​​is established, which effectively improves the realism of tactile feedback.

[0020] (2) Since the volume data points cover the entire three-dimensional space data field, if all of them are used for point element calculation, it will be far from meeting the real-time requirements. Therefore, the present invention designs a volume data point redistribution algorithm to establish the geometric alignment of the calculation point elements and the volume data points. According to the shape of the volume data field, the calculation point elements are evenly distributed in the volume data field. The volume data points are redistributed according to the displacement field changes obtained by the deformation calculation results to achieve a smooth and realistic deformation visual effect.

[0021] (3) By constructing a non-homogeneous soft tissue deformation model, the problem that the homogeneous model in traditional surgical simulation cannot explore the internal structure information is solved, and the immersion, interactivity and training effect of the virtual surgical system on surgeons are improved.

[0022] (4) By modeling adaptive bleeding particles, the mass of the bleeding particles is set according to the distance between the bleeding particles and the bleeding surface during the modeling process, thereby improving the real-time performance of the cutting process in virtual surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0024] Figure 1 The schematic diagram of the incision obtained by combining the physical model based on the grid with the geometric model based on the surface rendering is shown schematically;

[0025] Figure 2 A schematic diagram of a process flow of a method for meshless deformation modeling of soft tissue according to an embodiment of the present invention is shown;

[0026] Figure 3 A schematic diagram of a design route for meshless deformation modeling of soft tissue according to an embodiment of the present invention is schematically shown;

[0027] Figure 4 A schematic diagram of a process of constructing an initial geometric model of a soft tissue structure according to an embodiment of the present invention is schematically shown;

[0028] Figure 5A schematic diagram of a process of constructing an object-image feature transfer function based on a multidimensional parameter tensor and a color index table to obtain color information of a volume data point according to an embodiment of the present invention is shown;

[0029] Figure 6 A schematic diagram of a design process of an object-image feature transfer function according to an embodiment of the present invention is shown;

[0030] Figure 7 The schematic diagram shows a process of assigning geometric attributes to each sampling point and constructing an initial geometric model of a soft tissue structure by using an improved ray casting algorithm according to an embodiment of the present invention;

[0031] Figure 8 A schematic diagram of the principle of an improved ray casting algorithm according to an embodiment of the present invention is shown schematically;

[0032] Fig. 9 A schematic diagram schematically shows the relationship between the calculated point element and the volume data point according to an embodiment of the present invention;

[0033] Fig.10 The figure schematically shows the visual effect of liver tissue cutting simulation at different cutting depths according to an embodiment of the present invention, wherein: Fig.10 (a) The cutting depth is 10 mm; Fig.10 (b) The cutting depth is 20 mm; Fig.10 (c) The cutting depth is 30 mm; and Fig.10 The cutting depth in (d) is 40 mm;

[0034] Fig.11 The figure schematically shows the volume data values ​​collected by liver tissue cutting simulation at different cutting depths according to an embodiment of the present invention, wherein: Fig.11 (a) The cutting depth is 10 mm; Fig.11 (b) The cutting depth is 20 mm; Fig.11 (c) The cutting depth is 30 mm; and Fig.11 The cutting depth in (d) is 40 mm;

[0035] Fig.12 A schematic diagram of a process of another method for meshless deformation modeling of soft tissue according to an embodiment of the present invention is shown;

[0036] Fig.13 A schematic diagram schematically showing the relationship between the mass of bleeding particles and the distance from the bleeding surface according to an embodiment of the present invention;

[0037] Fig.14 A schematic diagram schematically showing the relationship between the size and position of bleeding particles according to an embodiment of the present invention;

[0038] Fig.15 A structural block diagram of a soft tissue meshless deformation modeling device according to an embodiment of the present invention is schematically shown; and

[0039] Fig.16 A block diagram of an electronic device for a meshless deformation modeling method of soft tissue according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Figure 2 A schematic flowchart of a meshless deformation modeling method for soft tissue according to an embodiment of the present invention is shown schematically.

[0042] like Figure 2 As shown, a meshless deformation modeling method for soft tissue according to an embodiment of the present invention includes steps S110-S140, and the design route of the model is shown in FIG. Figure 3 .

[0043] In step S110, an initial geometric model of the soft tissue structure is constructed using an improved ray casting algorithm based on the multidimensional parameter tensor of the volume data points.

[0044] In some exemplary embodiments, step S110 includes steps S111-S114, see Figure 4 .

[0045] In step S111 , volume data values ​​of volume data points of soft tissue are acquired.

[0046] In the embodiment of the present invention, data is obtained by scanning the organ tomograms obtained by CT and MRI instruments, and a full set of CT and MRI grayscale images can be converted into volume data values, and concepts related to volume data are voxels and volume textures.

[0047] In step S112, the volume data values ​​are preprocessed to obtain a multi-dimensional parameter tensor of the volume data points.

[0048] In some exemplary embodiments, the multidimensional parameter tensor includes grayscale value, density value, gradient value and second-order derivative value, wherein the volume data value includes grayscale value; the density value is the average of the volume data values ​​of 27 volume data points adjacent to the coordinate position of the volume data point to be sought; the gradient value is calculated by performing central difference calculation on the volume data value; and the second-order derivative value is calculated by performing central difference calculation on the gradient value. Through the influence of parameters such as gradient value, density value and second-order derivative on the fine structure of soft tissue, a multidimensional parameter tensor that accurately maps the optical properties and physiological characteristics of soft tissue is established.

[0049] For example, the gradient value is calculated by the central difference method, and the calculation formula is as follows:

[0050] (1)

[0051] (2)

[0052] (3)

[0053] (4)

[0054] in, The coordinates are The volume data value of the volume data point, The coordinates are The gradient value of the volume data point along the x direction, The coordinates are The gradient value of the volume data point along the y direction, The coordinates are The gradient value of the volume data point along the z direction, The coordinates are The gradient value of the volume data point.

[0055] The density value is calculated as follows:

[0056] (5)

[0057] in, The coordinates are The density value of the volume data point.

[0058] The second-order derivative value is obtained by further calculating the central difference of the gradient value. The calculation formula is as follows:

[0059] (6)

[0060] (7)

[0061] (8)

[0062] in, The coordinates are The second-order derivative value of the volume data point along the x direction, The coordinates are The second-order derivative value of the volume data point along the y direction, The coordinates are The second-order derivative value of the volume data point along the z direction.

[0063] In step S113, based on the multidimensional parameter tensor and the color index table, an object-image feature transfer function is constructed to obtain color information of the volume data points, wherein the color information includes color values ​​and transparency values; and the color index table includes the organ type of the soft tissue and the corresponding multidimensional parameter matrix.

[0064] In some exemplary embodiments, step S113 includes steps S1131-S1133, see Figure 5 .

[0065] In step S1131, based on the multidimensional parameter tensor, the organ type corresponding to the soft tissue is determined using a color index table.

[0066] By introducing a multidimensional parameter tensor, establishing an object-image feature transfer function, and combining the true color characteristics of soft tissue, the volume data value is mapped to the color information of soft tissue. Figure 6 ,Depend on Figure 6 It can be seen that the object-image feature transfer function is composed of a multi-dimensional parameter tensor and a color index table, wherein the color information index table and the color index table are shown in Table 1.

[0067] Table 1 Color index table

[0068]

[0069] In step S1132, based on the organ type, the corresponding multidimensional parameter matrix is ​​determined using the color index table.

[0070] In step S1133, color information of the volume data points is calculated based on the multidimensional parameter matrix and the multidimensional parameter tensor.

[0071] For example, the organ type is determined by a multidimensional parameter tensor and an index table, and then the corresponding parameter matrix is ​​determined. The optical properties of the volume data points can be calculated by the following formula.

[0072] (R,G,B,A)=(p,g,d,s)∙Tab.A (9)

[0073] Among them, R is the value corresponding to the red color value, G is the value corresponding to the green color value, B is the value corresponding to the blue color value, A is the transparency value, p is the gray value, g is the gradient value, d is the density value, s is the second-order derivative value, and Tab.A is the four-dimensional parameter matrix corresponding to the index table.

[0074] In step S114, based on the color information, an improved ray casting algorithm is used to assign geometric attributes to each sampling point to construct an initial geometric model of the soft tissue structure.

[0075] In the embodiment of the present invention, the color information given to the soft tissue body model consists of red, green, blue and transparency (RGBA). The traditional ray casting algorithm directly converts the one-dimensional volume data into multi-dimensional color values ​​and opacity through the empirical transfer function in the classification stage, resulting in low resolution between different structures, blurred tissue boundaries and other problems. The present invention performs preprocessing calculations on the original volume data to obtain parameters of more than four dimensions to map the color information, so that the distinction between structures is more accurate.

[0076] In some exemplary embodiments, step S114 includes steps S1141-S1143, see Figure 7 .

[0077] In step S1141, ray casting sampling is performed by parallel projection to obtain sampling points.

[0078] When the sampling point is not located on a volume data point with a value, a plane interpolation algorithm is used to calculate the volume data value of the sampling point.

[0079] For example, to improve the calculation efficiency, the volume data value V0 of the light sampling point P0 (x, y) that is not on the volume data point with a value is obtained by plane interpolation calculation. The calculation formula is as follows:

[0080] (10)

[0081] Where V 01 、V 11 、V 00 、V 10 It is divided into the volume data values ​​of the four volume data points around point P0 (x, y). x1 and x2 are the horizontal coordinate values ​​of the four selected surrounding volume data points, and y1 and y2 are the horizontal coordinate values ​​of the four selected surrounding volume data points.

[0082] In step S1142, the color value and transparency value of the sampling point are accumulated by alpha blending technology. When the accumulated opacity is 1 or the light passes through the soft tissue, the color synthesis value is calculated to obtain the final color value and the final transparency value.

[0083] According to the principle of the ray casting algorithm, the light is emitted from the viewport and passes through the entire volume data field. Data resampling and calculation follow the alpha blending technology, and the calculation formula is as follows:

[0084] (11)

[0085] (12)

[0086] in, For the The color value of the sampling point, For the The transparency value of each sampling point, For the The accumulated color values ​​of all sampling points before the sampling point and For the The accumulated transparency value of all sampling points before the sampling point.

[0087] In step S1143, a three-dimensional soft tissue model including internal fine structure information is constructed based on the final color value and the final transparency value.

[0088] In the geometric modeling of soft tissue microstructure based on ray casting algorithm, the volume data comes from the patient's medical images. The volume data values ​​of different soft tissues may be the same or similar. When preprocessing, they are given the same optical properties and material coefficients, which makes it difficult to achieve the expected visual and tactile distinction standards. The embodiment of the present invention improves the ray casting algorithm to give realistic geometric properties to each volume data point, and establishes a geometric model of liver tissue microstructure that conforms to the principle of visual observation.

[0089] Figure 8 The schematic diagram of the principle of the improved ray casting algorithm according to an embodiment of the present invention is schematically shown.

[0090] like Figure 8 As shown, the improved ray casting algorithm according to the embodiment of the present invention comprises the following steps:

[0091] Data preprocessing: Preprocess the abdominal CT data, including noise reduction, removal of abnormal points, etc.

[0092] Data value classification: preliminary classification of data values ​​according to regions and boundaries.

[0093] Soft tissue extraction and classification: Calculate the multidimensional parameter tensor, classify and calculate the volume data values ​​of the soft tissue according to the color information index table, and assign corresponding color information and transparency.

[0094] Data resampling: Resampling is performed in the three-dimensional data field through the parallel projection method, and the volume data values ​​of the sampling points are calculated through the planar interpolation method.

[0095] Color information synthesis: The color value and transparency are synthesized through alpha blending technology. When the accumulated opacity is 1 or the light passes through the soft tissue, the color synthesis value is calculated to reconstruct a three-dimensional soft tissue model containing internal fine structure information.

[0096] The geometric modeling method of soft tissue structure provided by the embodiment of the present invention can clearly describe the internal structure of the tissue compared to the soft tissue model reconstructed by surface rendering. Compared with the traditional ray casting algorithm, the tissue boundary of the present invention is clear and has a higher visual resolution.

[0097] In step S120, a soft tissue continuum system equation is constructed.

[0098] For example, human soft tissue can be regarded as a special elastic tissue, and its motion state can be described by continuous elasticity theory. The constitutive equation is , where D is the elasticity matrix,

[0099] (13)

[0100] in, .

[0101] In step S130, based on the meshless radial basis point interpolation method, a non-homogeneous viscoelastic model is used to solve the system equations to obtain a deformation physical model of the soft tissue, wherein the coefficients of the physical model are related to the multidimensional parameter tensor.

[0102] For example, due to the influence of collagen and other components in soft tissue, biomechanical properties such as viscosity must be considered when calculating its motion state. Therefore, the relationship between stress and strain of soft tissue needs to introduce the influence of time t. The stress at time t is It is expressed as:

[0103] (14)

[0104] in, is the time constant, for The need to adapt at all times.

[0105] The meshless radial base point interpolation method is used to solve the system equations, and the Kelvin viscoelasticity is introduced to calculate the stress of the volume data point i at time t. get:

[0106] (15)

[0107] in, and are coefficients, obtained by calculating the multidimensional parameter tensor mapping obtained from real soft tissue CT data. is a function that takes into account the strain-dependent elastic nonlinearity, , is Young's modulus, t0 is the time constant, n is the number of points that have an effect on the volume data point, is the time constant associated with the volume data point i. From the right side of the equal sign, we can see that the state response of strain can be divided into steady-state strain response and transient strain response, which are respectively obtained by multiplying the strain with the steady-state elastic modulus and the transient modulus. The two different state responses of strain together constitute stress. The heterogeneity is to be realized by establishing a functional relationship between the voxel material coefficient and the physical model coefficient.

[0108] In step S140, based on the physical model, the displacement of the calculation point element is obtained, wherein the calculation point element is the volume data point uniformly selected in the initial geometric model.

[0109] For example, from the balanced equation: Calculate the displacement vector u of the volume data point, where σ is the stress, External force is the damping coefficient, is the first-order derivative of the displacement vector u, is the second-order derivative of the displacement vector u, L T is the displacement vector of the deformation.

[0110] In step S150, based on the initial geometric model and the displacement of the calculated point elements, the volume data points are redistributed to obtain a deformation model of the soft tissue.

[0111] In some exemplary embodiments, for a volume data point selected as a calculation point element, the corresponding position of the calculation point element after deformation is the position of the volume data point after deformation; and for a volume data point not selected as a calculation point element, the position of the volume data point after deformation is calculated according to the positions of the eight calculation point elements nearest to the volume data point.

[0112] For example, multi-dimensional parameters are introduced into the calculation of the physical model of soft tissue, and the correlation between the physical model coefficients and the volume data values ​​is established, which effectively improves the fidelity of tactile feedback. Since the volume data points cover the entire three-dimensional space data field, all of them are used for point element calculation, which is far from meeting the real-time requirements. Therefore, the embodiment of the present invention designs a voxel redistribution algorithm to establish the geometric registration of the calculation point element and the voxel, see Fig. 9 ,According to the shape of the volume data field, the calculation points are evenly distributed in the volume data field, and the voxels are redistributed according to the displacement field changes obtained by the deformation calculation results to achieve a smooth and realistic deformation visual effect.

[0113] In the stress calculation, the coefficient and The functional relationship between the voxel material coefficient and the physical properties of the study area is connected with the real properties of the soft tissue, and the heterogeneous deformation simulation of the microstructure is realized to calculate the point For example, the coefficients of the physical model and The relationship with the material coefficient is shown as follows:

[0114] (16)

[0115] Among them, a, b, c and d are equilibrium coefficients, which can be obtained by stress testing real soft tissues, and p i (i, j, k) is the physical model coefficient related to the gray value and density at the volume data point (i, j, k), τ i (i, j, k) is the coefficient of the physical model related to the gradient value at the volume data point (i, j, k), F i In the example, i is 1 or 2. When i is 1, F i (i, j, k) is the gray value at the volume data point (i, j, k); when i is 2, F i (i, j, k) is the density value at the volume data point (i, j, k); In , the value of i is 1 or 2. When i is 1, it is the modulus of the gradient value of the volume data point with coordinates (i, j, k); when i is 1, it is the modulus of the second-order derivative value.

[0116] Fig.10 The figure schematically shows the visual effect of liver tissue cutting simulation at different cutting depths according to an embodiment of the present invention, wherein: Fig.10 (a) The cutting depth is 10 mm; Fig.10 (b) The cutting depth is 20 mm; Fig.10 (c) The cutting depth is 30 mm; and Fig.10 The cutting depth in (d) is 40 mm.

[0117] like Fig.10 As shown in the figure, as the cutting depth increases, the internal structure observed from the incision changes. When the cutting depth is 10 mm, the incision depth is shallow, and the scalpel does not touch the internal vascular tissue. As the depth increases, the vascular tissue inside the liver is gradually exposed from the incision and destroyed.

[0118] Fig.11 The figure schematically shows the volume data values ​​collected by liver tissue cutting simulation at different cutting depths according to an embodiment of the present invention, wherein: Fig.11 (a) The cutting depth is 10 mm; Fig.11 (b) The cutting depth is 20 mm; Fig.11(c) The cutting depth is 30 mm; and Fig.11 The cutting depth in (d) is 40 mm.

[0119] like Fig.11 As shown in the figure, when the incision depth is 10 mm, the collected volume data values ​​are all greater than 0.5. As the depth increases, the volume data values ​​less than 0.5 increase. Among them, the volume data of liver tissue is greater than 0.5, and the volume data value of blood vessels is less than 0.5, corresponding to Fig.10 From the visual effects of the four simulations at different depths, we can see that since the volume data value inside the blood vessel is relatively low, in the simulation with an incision depth of 10 mm, the scalpel did not touch the blood vessel, and the volume data of the liver tissue was mainly collected, so the data values ​​were all between 0.75 and 1. The greater the depth, the more volume data inside the blood vessel is collected, indicating that the cutting operation gradually destroys the internal vascular structure.

[0120] Fig.12 A schematic diagram of a process of another soft tissue meshless deformation modeling method according to an embodiment of the present invention is schematically shown.

[0121] like Fig.12 As shown, another meshless deformation modeling method for soft tissue according to an embodiment of the present invention is Figure 2 The method shown also includes steps S160-S190.

[0122] In step S160 , the stress of bleeding particles in the soft tissue is calculated.

[0123] In some exemplary embodiments, the stress of the bleeding particle is related to the stress of the bleeding particle within a preset radius and the mass of the bleeding particle, wherein the mass of the bleeding particle is related to the distance of the bleeding particle from the bleeding surface.

[0124] For example, in order to improve the real-time performance of the cutting process in virtual surgery, an adaptive bleeding particle modeling method is used to improve the computational efficiency of the simulation. The simulation is based on the improved smooth particle hydrodynamics algorithm. For bleeding particle i, its force is calculated through the particle j within its action radius. The calculation formula is as follows:

[0125] (17)

[0126] Where N is the number of particles within the bleeding particle action radius, is the density of particle j, is the mass of particle j, is the weight function, which depends on the distance between particles ( ) and a scalar parameter h.

[0127] The traditional algorithm assumes that the mass of the particles is equal, which is m. In the present invention, the relationship between the mass of the bleeding particle and the distance from the bleeding surface is shown in Fig.13 , where the mass and radius of the bleeding particles change adaptively with the position of the particles. The particles on the bleeding surface have the greatest impact on the simulation visual effect, so it is necessary to reduce the particle radius to improve the resolution. The particles in the soft tissue incision have little impact on the visual effect, so the particle radius can be appropriately increased to improve the calculation efficiency. According to the particle radius size and the constant density value, the mass of the bleeding particle i is obtained. The calculation formula is as follows:

[0128] (18)

[0129] in, is the mass of the bottom bleed particle farthest from the bleed surface, is the shortest distance between bleeding particle i and the bleeding surface, is the distance between the bleeding surface and the soft tissue, is the mass of the bleeding surface particles The mass of the bottom bleed particles The size distribution of bleeding particles calculated by formula (18) can be found in Fig.14 , since the density of bleeding particles remains constant, therefore, Fig.14 The size distribution in the sample can reflect the mass distribution.

[0130] In step S170 , the displacement of the bleeding particle is obtained based on the stress of the bleeding particle.

[0131] In step S180, a deformation model of the bleeding particles is obtained based on the displacement.

[0132] In step S190, the deformation model of the bleeding particles is combined with the deformation model of the soft tissue to obtain a final deformation model of the soft tissue.

[0133] Fig.15 The structural block diagram of a soft tissue meshless deformation modeling device according to an embodiment of the present invention is schematically shown.

[0134] like Fig.15 As shown, a soft tissue meshless deformation modeling device 800 according to an embodiment of the present invention includes a first modeling module 810, a construction equation module 820, a first calculation module 830, a second calculation module 840 and a second modeling module 850.

[0135] The first modeling module 810 is used to construct an initial geometric model of the soft tissue structure based on the multi-dimensional parameter tensor of the volume data points by using an improved ray casting algorithm.

[0136] The equation building module 820 is used to build the soft tissue continuum system equation.

[0137] The first calculation module 830 is used to solve the system equations based on the gridless radial basis point interpolation method and the non-homogeneous viscoelastic model to obtain the deformation physical model of the soft tissue, wherein the coefficients of the physical model are related to the multi-dimensional parameter tensor.

[0138] The second calculation module 840 is used to obtain the displacement of the calculation point element based on the physical model, wherein the calculation point element is the volume data point uniformly selected in the initial geometric model.

[0139] The second modeling module 850 is used to redistribute the volume data points based on the initial geometric model and the displacement of the calculated point elements to obtain a deformation model of the soft tissue

[0140] In some specific embodiments, any multiple modules among the first modeling module 810, the equation building module 820, the first calculation module 830, the second calculation module 840, and the second modeling module 850 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0141] In some specific embodiments, at least one of the first modeling module 810, the equation building module 820, the first calculation module 830, the second calculation module 840, and the second modeling module 850 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the first modeling module 810, the equation building module 820, the first calculation module 830, the second calculation module 840, and the second modeling module 850 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0142] Fig.16 A block diagram of an electronic device for a meshless deformation modeling method of soft tissue according to an embodiment of the present invention is schematically shown.

[0143] like Fig.16As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an on-board memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0144] In RAM603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM602 and RAM603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to an embodiment of the present invention by executing the program in ROM602 and / or RAM603. It should be noted that the program can also be stored in one or more memories other than ROM602 and RAM603. Processor 601 can also perform various operations of the method flow according to an embodiment of the present invention by executing the program stored in one or more memories.

[0145] In some specific embodiments, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the I / O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0146] An embodiment of the present invention shows a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0147] In some specific embodiments, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM602 and / or RAM603 described above and / or one or more memories other than ROM602 and RAM603.

[0148] An embodiment of the present invention shows a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0149] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0150] In some specific embodiments, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0151] In some specific embodiments, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0152] In some specific embodiments, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages, specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, such as Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0153] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0154] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A meshless deformation modeling method for soft tissue, characterized in that: include: Based on the multidimensional parameter tensor of volume data points, an improved ray casting algorithm is used to construct the initial geometric model of the soft tissue structure; Constructing the soft tissue continuum system equations; Based on the meshless radial basis point interpolation method, the system equations are solved by using a non-homogeneous viscoelastic model to obtain a deformation physical model of soft tissue; Based on the physical model, the displacement of the calculation point element is obtained; as well as Based on the initial geometric model and the displacement of the calculation point element, the volume data points are redistributed to obtain a deformation model of the soft tissue, Wherein, the coefficients of the physical model are related to the multidimensional parameter tensor; The calculation point elements are the volume data points uniformly selected in the initial geometric model; The method of constructing an initial geometric model of the soft tissue structure based on the multidimensional parameter tensor of the volume data points and using an improved ray casting algorithm includes: Obtaining volume data values ​​of volume data points of soft tissue; Preprocessing the volume data values ​​to obtain a multidimensional parameter tensor of the volume data points; Based on the multidimensional parameter tensor and the color index table, construct an object-image feature transfer function to obtain color information of the volume data points; and Based on the color information, an improved ray casting algorithm is used to assign geometric attributes to each sampling point and construct an initial geometric model of the soft tissue structure. Wherein, the color information includes a color value and a transparency value; and The color index table includes organ types of soft tissues and corresponding multidimensional parameter matrices; The constructing of an object-image feature transfer function based on the multidimensional parameter tensor and the color index table to obtain the color information of the volume data point comprises: Based on the multidimensional parameter tensor, determining the organ type corresponding to the soft tissue using the color index table; Based on the organ type, determining a corresponding multidimensional parameter matrix using the color index table; and Calculate the color information of the volume data points based on the multidimensional parameter matrix and the multidimensional parameter tensor; The step of assigning geometric attributes to each sampling point based on the color information and constructing an initial geometric model of the soft tissue structure by using an improved ray casting algorithm includes: The sampling points are obtained by performing ray casting sampling in a parallel projection manner; Accumulating the color value and transparency value of the sampling point through the alpha blending technique, and when the accumulated opacity is 1 or the light passes through the soft tissue, calculating the color synthesis value to obtain the final color value and the final transparency value; and Based on the final color value and the final transparency value, a three-dimensional soft tissue model including internal fine structure information is constructed.

2. The method according to claim 1, characterized in that Based on the initial geometric model and the displacement of the calculation point element, redistributing the volume data points includes: For a volume data point selected as a calculation point element, the position of the corresponding calculation point element after deformation is the position of the volume data point after deformation; and For a volume data point that is not selected as a calculation point element, the deformed position of the volume data point is calculated according to the deformed positions of the eight calculation point elements that are the closest to the volume data point.

3. The method according to claim 1, characterized in that Also includes: Calculate stresses of bleeding particles in soft tissues; Based on the stress of the bleeding particle, obtaining the displacement of the bleeding particle; Based on the displacement, a deformation model of the bleeding particle is obtained; as well as The deformation model of the bleeding particles is combined with the deformation model of the soft tissue to obtain a final deformation model of the soft tissue.

4. The method according to claim 3, characterized in that The stress of the bleeding particles is related to the stress of the bleeding particles within a preset radius and the mass of the bleeding particles. The mass of the bleeding particles is related to the distance between the bleeding particles and the bleeding surface.

5. A modeling device for implementing the soft tissue meshless deformation modeling method according to any one of claims 1 to 4, characterized in that: The device comprises: The first modeling module is used to construct an initial geometric model of the soft tissue structure using an improved ray casting algorithm based on a multidimensional parameter tensor of volume data points; The equation building module is used to build the equations of the soft tissue continuum system; A first calculation module is used to solve the system equations based on a meshless radial base point interpolation method and a non-homogeneous viscoelastic model to obtain a deformation physical model of the soft tissue; A second calculation module, configured to obtain the displacement of a calculation point element based on the physical model; and The second modeling module is used to redistribute the volume data points based on the initial geometric model and the displacement of the calculation point elements to obtain a deformation model of the soft tissue. Wherein, the coefficients of the physical model are related to the multidimensional parameter tensor; The calculation point elements are the volume data points uniformly selected in the initial geometric model.

6. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the method according to any one of claims 1 to 4.

Images