An adaptive mesh refinement method for non-uniform soft tissue mesh models

Abstract

The invention discloses an adaptive mesh refinement method for a non-uniform soft tissue mesh model, which belongs to the technical field of computer graphics, and comprises: generating an eye movement heat map according to eye movement data of a user gazing at a primary tetrahedral mesh model; extracting a primary refinement area of ​​the primary tetrahedral mesh model according to the eye movement heat map, and obtaining a vertex density value of the primary tetrahedral mesh model; performing primary refinement on the primary refinement area based on the vertex density value to obtain a secondary tetrahedral mesh model; extracting a secondary refinement area from the secondary tetrahedral mesh model according to the conduction range of an external force; performing secondary refinement on the secondary refinement area based on the vertex density value to obtain a third tetrahedral mesh model adapted to the non-uniform soft tissue; wherein the primary tetrahedral mesh model is constructed and obtained through a CT image of the non-uniform soft tissue; and the authenticity and real-time performance of the simulation are improved.

Description

Technical Field

[0001] The invention relates to an adaptive grid refinement method for a non-uniform soft tissue grid model, and belongs to the technical field of computer graphics. Background Art

[0002] The continuous development of virtual reality technology has made a huge contribution to the smooth progress of surgery. Using virtual reality technology to simulate the surgical process can not only improve the surgical skills of medical staff, but also can combine medical equipment to select remote surgery for patients. For highly infectious diseases, it can greatly reduce the risk of cross-infection between doctors and patients. In virtual surgery, the simulated immersion and the authenticity of soft tissue deformation are both crucial.

[0003] Among the existing methods for improving soft tissue models, researchers have focused on studying how to better refine the grid into a finer grid while using various auxiliary means to maintain its efficiency, so as to improve the accuracy and real-time performance of the model. For example, Xie et al. (Xie Hujin, Song Jialu, Zhong Yongmin, Li Jiankun, Gu Chengfan, Choi Kup-Sze, Extended Kalman Filter Nonlinear Finite Element Method for Nonlinear Soft Tissue Deformation) proposed an extended Kalman filter nonlinear finite element method for nonlinear soft tissue deformation, and used the extended Kalman filter to assist the finite element model in calculation, so that the model can establish a higher-precision uniform grid model while meeting the real-time requirements. However, since human organs actually contain soft tissues of various densities, they may deform differently under the action of external forces. Therefore, the method of only pursuing the establishment of a higher-precision uniform grid model for soft tissue without affecting real-time performance is obviously insufficient. In addition, the calculation amount of establishing a high-precision model for the entire model is large, and the real-time performance of the simulation needs to be improved. Summary of the invention

[0004] The purpose of the present invention is to provide an adaptive mesh refinement method for a non-uniform soft tissue mesh model, which can improve the authenticity and real-time performance of non-uniform soft tissue simulation.

[0005] To achieve the above objectives, the present invention is implemented by adopting the following technical solutions:

[0006] The present invention provides an adaptive mesh refinement method for a non-uniform soft tissue mesh model, comprising:

[0007] Generate an eye movement heat map based on the eye movement data of the user looking at the first-level tetrahedral mesh model;

[0008] Extracting the initial refinement area of ​​the first-level tetrahedral mesh model according to the eye movement heat map, and obtaining the vertex density value of the first-level tetrahedral mesh model;

[0009] Performing primary refinement on the primary refinement area based on vertex density values ​​to obtain a secondary tetrahedral mesh model;

[0010] Extracting a further refined area from the secondary tetrahedral mesh model according to the conduction range of the external force;

[0011] The re-refined area is re-refined based on the vertex density value to obtain a three-level tetrahedral mesh model suitable for the non-uniform soft tissue;

[0012] The first-level tetrahedral mesh model is constructed and acquired through CT images of inhomogeneous soft tissue.

[0013] Furthermore, the initial refinement area of ​​the first-level tetrahedral mesh model is extracted according to the eye movement heat map, including:

[0014] The coordinates of the red area in the eye movement hotspot map are extracted, and the vertices on the surface of the first-level tetrahedral mesh model covered by the red area are marked according to the coordinates of the red area. The area formed by the adjacent vertices marked at the first level is used as a fixation area, and the fixation area with the longest total fixation time is screened out. The center point of the selected fixation area is obtained, and a circle is drawn with the center point of the area as the center of the circle and the maximum value from the center point of the area to the edge vertex of the selected fixation area as the radius. The tetrahedron composed of all vertices in the circle is the initial refinement area.

[0015] Furthermore, the vertex density value of the first-level tetrahedral mesh model is obtained by the following method:

[0016] ρ=b+kCt

[0017] Among them, ρ is the density value of the vertex, Ct is the CT value in the CT image data, and b and k are the conversion parameters between the CT value and the density value.

[0018] Furthermore, the primary refinement of the primary refinement area is performed based on the density value, including:

[0019] The tetrahedrons that have not been marked at the second level in the primary refinement area are uniformly refined one by one until the tetrahedron edge length is detected to meet the preset edge length requirement and then the tetrahedron is marked at the second level.

[0020] Furthermore, the preset side length range is set based on the density value:

[0021]

[0022] l min =0.3lmax

[0023] Among them, l max is the maximum value of the preset edge length, a is the parameter of the ratio of the tetrahedron edge length to density, is the tetrahedron density, V j is the density value of the jth vertex of the tetrahedron, l min It is the minimum value of the preset side length.

[0024] Furthermore, the transmission range of the external force is obtained by the following method:

[0025] The external force point starts to move under the action of the external force, and its adjacent vertices start to move under the action of the force applied by the external force point, and the external force begins to be transmitted. The target vertex is the vertex that currently produces displacement, and the moving distance of its adjacent vertices is obtained by the following method:

[0026]

[0027] Among them, h is the moving distance of the adjacent vertex of the target vertex, d is the distance between the target vertex and its adjacent vertex before displacement, and h t is the displacement of the target vertex, t is the displacement time, is the density mean of the target vertex and its adjacent vertices;

[0028] Until it is detected that the moving distance of the adjacent vertices is less than the preset vertex displacement threshold, all the displaced vertices constitute the transmission range of the external force.

[0029] Furthermore, during the vertex movement process, a threshold for the distance change between adjacent vertices is preset:

[0030]

[0031] where d s is the initial distance between the target vertex and its adjacent vertices, c is the parameter of the ratio of displacement to density value, d t is the distance between the target vertex and its adjacent vertex after displacement, is the mean density of the target vertex and its adjacent vertices.

[0032] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0033] The present invention provides an adaptive mesh refinement method for a non-uniform soft tissue mesh model. The method generates an eye movement heat map according to the eye movement data of a user gazing at a primary tetrahedral mesh model, senses the operation area in advance, reduces the amount of real-time calculation, and improves the real-time performance of the simulation. The method extracts the primary refinement area of ​​the primary tetrahedral mesh model according to the eye movement heat map, obtains the vertex density value of the primary tetrahedral mesh model, and performs primary refinement based on the density value on the primary refinement area. When performing the primary refinement, the non-uniform density inside the human organs is taken into consideration, and the soft tissues of different densities will produce different deformations when subjected to force. The primary tetrahedral mesh model is initially refined based on the vertex density value, which helps to improve the authenticity of the model simulation. The method extracts a secondary refinement area from the secondary tetrahedral mesh model according to the conduction range of the external force, and performs a secondary refinement on the secondary refinement area based on the density value. The adaptive mesh refinement of the non-uniform soft tissue mesh model is realized based on the primary refinement area and the secondary refinement area. Different from the refinement performed on the overall model in the prior art, the algorithm of the present invention simplifies the amount of calculation and improves the real-time performance of the simulation.

[0034] When calculating the displacement of the vertices of the deformed part, taking into account the characteristics of non-uniform soft tissue in real environment, a threshold of distance change between adjacent vertices is preset to prevent excessive expansion and contraction of soft tissue during simulation, making the simulation closer to the real deformation of soft tissue and further improving the authenticity of model simulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of an adaptive mesh refinement method for a non-uniform soft tissue mesh model provided by an embodiment of the present invention;

[0036] Figure 2 It is a detailed flow chart provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the protection scope of the present invention.

[0038] like Figure 1 As shown, an embodiment of the present invention provides an adaptive mesh refinement method for a non-uniform soft tissue mesh model, comprising:

[0039] A CT image is acquired, and a first-level tetrahedron mesh model is constructed according to the CT image. The first-level tetrahedron mesh model has low precision.

[0040] S1. Generate an eye movement heat map based on the eye movement data of the user looking at the first-level tetrahedral mesh model.

[0041] S2. extracting the primary refinement area of ​​the first-level tetrahedral mesh model according to the eye movement heat map, and obtaining the vertex density value of the first-level tetrahedral mesh model.

[0042] CT data in the CT image is obtained, and density values ​​of all vertices of the primary tetrahedral mesh model are converted according to the characteristics of the CT value in the CT data.

[0043] The vertex density value ρ is: ρ = b + kCt

[0044] Among them, Ct is the CT value in the CT image data, b and k are the conversion parameters between the CT value and the density value; the CT value is the corresponding value of each tissue in the CT image and the X-ray attenuation coefficient, and is a unit of measurement for determining the density of a local tissue or organ in the human body. Therefore, the vertex density value ρ is set to be proportional to the CT value.

[0045] According to the color distribution in the eye movement hotspot map, the coordinates of the red area are extracted, and the vertices in the first-level tetrahedral mesh model corresponding to the red area are marked according to the coordinates of the red area. The area surrounded by the adjacent vertices marked by the first level is regarded as a fixation area, and the total fixation time T of the fixation area i is i for:

[0046]

[0047] Where N is the number of vertices in the gaze area, t j is the fixation time of the jth vertex in the fixation area.

[0048] Select the fixation area with the longest total fixation time, and set the coordinates P of the center point of the selected fixation area to be:

[0049]

[0050] Where N is the number of vertices in the gaze area, P j is the coordinate of the jth vertex, t j is the fixation time of the jth vertex in the fixation area, T i is the total fixation time of fixation area i.

[0051] A circle is constructed with the center point of the selected attention area as the center and the maximum value from the center to the edge vertex of the selected attention area as the radius. The tetrahedron formed by all vertices within the range of the circle is the initial refinement area.

[0052] S3. Performing primary refinement on the primary refinement area based on vertex density values ​​to obtain a secondary tetrahedral mesh model.

[0053] like Figure 2As shown, in the primary refinement area, tetrahedrons that have not been marked for secondary refinement are screened out and refined one by one, and the tetrahedrons are judged to prevent the repeated refinement of the already refined tetrahedrons.

[0054] The tetrahedron density value is defined as the average of the density values ​​of its four vertices, that is, the tetrahedron density value V is:

[0055]

[0056] Among them, V j is the density value of the jth vertex of the tetrahedron.

[0057] For tetrahedrons that are not marked at the secondary level, the threshold of the tetrahedron edge length is calculated based on the density value:

[0058]

[0059] l min =0.3l max

[0060] Among them, l max is the maximum value of the tetrahedron edge length, a is the parameter of the ratio of the tetrahedron edge length to the density, is the tetrahedron density, V j is the density value of the jth vertex of the tetrahedron, l min is the minimum edge length of a tetrahedron.

[0061] The tetrahedrons that have not been marked at the secondary level are uniformly subdivided using traditional tetrahedral meshes, and the edge lengths of the refined tetrahedrons are then judged. If they meet the preset edge length range, the tetrahedrons are marked at the secondary level. If they do not meet the preset edge length range, the tetrahedrons are further refined until the edge lengths of the tetrahedrons meet the preset edge length range.

[0062] Repeat the above steps until all tetrahedrons in the initial refinement area have been marked at the second level to obtain a second-level tetrahedron mesh model.

[0063] S4. Extract the refined area from the secondary tetrahedral mesh model according to the conduction range of the external force.

[0064] Preset a vertex displacement threshold h min When the displacement of the top is less than the vertex displacement threshold, it is assumed that the force will not continue to be transmitted to particles farther away, and the boundary of the further refined area is determined by the transmission range of the external force. In this embodiment, the vertex displacement threshold is set to 0.01.

[0065] The target vertex is the vertex that currently generates displacement. Using the mass-spring model, it is assumed that there is a massless spring between the target vertex and its adjacent vertices. When the target vertex moves, it moves together with the adjacent vertices connected to the spring. The displacement of the adjacent vertices depends on the density and the distance between the target vertex. The calculation formula for the adjacent vertex movement distance h is:

[0066]

[0067] Among them, d is the distance between the target vertex and its adjacent vertex before displacement, h t is the displacement of the target vertex, t is the displacement time, is the mean density of the target vertex and its adjacent vertices.

[0068] Until it is detected that the moving distance of the adjacent vertex is less than the preset vertex displacement threshold h min , then all the vertices that produce displacement constitute the conduction range of the external force.

[0069] During the vertex movement process, a threshold for the distance change between adjacent vertices is preset:

[0070]

[0071] where d s is the initial distance between the target vertex and its adjacent vertices, c is the parameter of the ratio of displacement to density value, d t is the distance between the target vertex and its adjacent vertex after displacement, is the density mean of the target vertex and its adjacent vertices; when d t When the maximum or minimum value is exceeded, d t Reset to maximum or minimum value.

[0072] S5. Refine the re-refined area based on the vertex density value to obtain a three-level tetrahedral mesh model suitable for the non-uniform soft tissue.

[0073] The re-refined area obtained in S4 is further refined based on the density value. The re-refining step is equivalent to the initial refinement step in S3, and a three-level tetrahedral mesh model is obtained, thereby completing the adaptive mesh refinement of the non-uniform soft tissue mesh model.

[0074] The embodiment of the present invention proposes an adaptive mesh refinement method for a non-uniform soft tissue mesh model, which improves the accuracy of deformation simulation and the real-time performance of the simulation. In addition, the present invention also defines a density-based adjacent vertex distance threshold. Different from the traditional direct fixed stretching ratio, the algorithm takes into account the idea that the softer the soft tissue, the better the stretchability, that is, the smaller the density of the soft tissue, the better the stretchability, and the larger the density, the worse the stretchability, and a new definition of the vertex displacement threshold is given.

[0075] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0076] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0079] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An adaptive mesh refinement method for a non-uniform soft tissue mesh model, characterized in that: include: Generate an eye movement heat map based on the eye movement data of the user looking at the first-level tetrahedral mesh model; Extracting the initial refinement area of ​​the first-level tetrahedral mesh model according to the eye movement heat map, and obtaining the vertex density value of the first-level tetrahedral mesh model; Performing primary refinement on the primary refinement area based on vertex density values ​​to obtain a secondary tetrahedral mesh model; Extracting a further refined area from the secondary tetrahedral mesh model according to the conduction range of the external force; The re-refined area is re-refined based on the vertex density value to obtain a three-level tetrahedral mesh model suitable for the non-uniform soft tissue; Wherein, the first-level tetrahedral mesh model is constructed and acquired through CT images of inhomogeneous soft tissue; Perform primary refinement based on density value on the primary refinement area, including: The tetrahedrons that have not been marked at the secondary level in the primary refinement area are uniformly refined one by one until the tetrahedron side length meets the preset side length requirement and then the tetrahedron is marked at the secondary level; The preset side length range is set based on the density value: l min ＝0.3l max Among them, l max is the maximum value of the preset edge length, a is the parameter of the ratio of the tetrahedron edge length to density, is the tetrahedron density, V j is the density value of the jth vertex of the tetrahedron, l min It is the minimum value of the preset side length.

2. The method for adaptive mesh refinement of a non-uniform soft tissue mesh model according to claim 1, characterized in that: The first refinement area of ​​the first-level tetrahedral mesh model is extracted according to the eye movement heat map, including: The coordinates of the red area in the eye movement hotspot map are extracted, and the vertices on the surface of the first-level tetrahedral mesh model covered by the red area are marked according to the coordinates of the red area. The area formed by the adjacent vertices marked at the first level is used as a fixation area, and the fixation area with the longest total fixation time is screened out. The center point of the selected fixation area is obtained, and a circle is drawn with the center point of the area as the center of the circle and the maximum value from the center point of the area to the edge vertex of the selected fixation area as the radius. The tetrahedron composed of all vertices in the circle is the initial refinement area.

3. The method for adaptive mesh refinement of a non-uniform soft tissue mesh model according to claim 1, characterized in that: The vertex density value of the first-level tetrahedral mesh model is obtained by the following method: ρ=b+kCt Among them, ρ is the density value of the vertex, Ct is the CT value in the CT image data, and b and k are the conversion parameters between the CT value and the density value.

4. The method for adaptive mesh refinement of a non-uniform soft tissue mesh model according to claim 1, characterized in that: The transmission range of the external force is obtained by the following method: The external force point starts to move under the action of the external force, and its adjacent vertices start to move under the action of the force applied by the external force point, and the external force begins to be transmitted. The target vertex is the vertex that currently produces displacement, and the moving distance of its adjacent vertices is obtained by the following method: Among them, h is the moving distance of the adjacent vertex of the target vertex, d is the distance between the target vertex and its adjacent vertex before displacement, and h t is the displacement of the target vertex, t is the displacement time, is the density mean of the target vertex and its adjacent vertices; Until it is detected that the moving distance of the adjacent vertices is less than the preset vertex displacement threshold, all the displaced vertices constitute the transmission range of the external force.

5. The method for adaptive mesh refinement of a non-uniform soft tissue mesh model according to claim 4, characterized in that: During the vertex movement process, a threshold for the distance change between adjacent vertices is preset: Among them, d s is the initial distance between the target vertex and its adjacent vertices, c is the parameter of the ratio of displacement to density value, d t is the distance between the target vertex and its adjacent vertex after displacement, is the mean density of the target vertex and its adjacent vertices.

Images