Three-dimensional medical image cross-section rendering method, device, equipment and medium
By segmenting and simulating static 3D medical images, and combining this with graphics processor intersection testing, the accuracy and real-time performance issues of deformed images in traditional methods have been resolved, enabling precise visualization of deformed 3D medical images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- RUIJIN HOSPITAL AFFILIATED TO SHANGHAI JIAO TONG UNIV SCHOOL OF MEDICINE
- Filing Date
- 2026-03-11
- Publication Date
- 2026-06-12
AI Technical Summary
Traditional 3D medical image cross-section rendering methods cannot meet the accuracy and real-time requirements of deformed images. Especially when considering deformation and dynamic scenes, they lead to problems such as the inability to define deformation vectors within the domain when the deformation range is too large or irregular, loss of some content of the original image, and computational complexity.
By segmenting static 3D medical images, generating mask images, determining the positions and texture coordinates of mesh vertices, constructing original tetrahedral mesh data, performing deformation simulation, using a graphics processor to perform intersection tests, determining the intersection points of the target cross-section and the tetrahedral mesh data, and constructing cross-sectional triangular mesh data for rendering.
It achieves accurate and real-time visualization of three-dimensional medical images after deformation, avoids the problem of the reverse deformation field being within the domain when the deformation range is too large or irregular, and provides a true reflection of the dynamic deformation of organs and tissues and accurate cross-sectional capture.
Smart Images
Figure CN122199767A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for rendering cross-sections of three-dimensional medical images. Background Technology
[0002] Traditional 3D medical imaging, such as CT (Computed Tomography), MRI (Magnetic Resonance Imaging), and 3D ultrasound, consists of grayscale static images defined on a 3D regular grid. Their cross-sectional imaging typically employs multi-plane reconstruction (MPR) methods, which do not consider the needs of deformation and dynamic scenes. In medical scenarios such as simulated surgical training, doctor-patient communication, and intraoperative navigation, it is often necessary to reconstruct a personalized 3D human body model based on the patient's 3D medical images. This involves constructing a physical simulation data structure to simulate the dynamic deformation of tissues and organs caused by the action of surgical instruments or physiological activities such as breathing and heartbeat. However, this highly realistic deformation can cause the 3D structure of the human body to deviate from the original medical image, making it difficult to obtain cross-sectional data of the deformed image.
[0003] Generally, the deformed image is achieved through a two-dimensional or three-dimensional inverse deformation field, that is, the original three-dimensional medical image is defined as... The deformed image is The reverse deformation field is The deformed 3D image is J(x)=I(x+u(x)), but the construction of the reverse deformation field needs to be carried out in the preset J image domain (usually an axis-aligned bounding box space). It is necessary to calculate the coordinates of the regular grid points in the domain relative to the corresponding points in the original image domain and calculate the difference. When the deformation range is too large or irregular, the deformation vector in the domain cannot be defined and some content of the original image is lost. At the same time, because the data structure (such as tetrahedral element) used by physical simulations such as the Finite Element Method (FEM) is inconsistent with the regular grid points of the 3D image, the calculation of the deformation field is more complicated and cannot meet the accuracy and real-time requirements of the cross-sectional imaging of the deformed image in relevant medical scenarios.
[0004] In summary, improving the accuracy and real-time performance of 3D medical image cross-sectional rendering is a problem that needs to be solved in this field. Summary of the Invention
[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for rendering cross-sections of three-dimensional medical images, thereby improving the accuracy and real-time performance of rendering cross-sections of three-dimensional medical images. The specific solution is as follows: In a first aspect, this application discloses a method for rendering cross-sectional views of three-dimensional medical images, including: Static 3D medical images are segmented to obtain mask images for each segmentation unit; The position coordinates of each mesh vertex are determined based on the mask image. Corresponding texture coordinates are added based on the relative position of each mesh vertex in the static three-dimensional medical image. The vertex index of each mesh vertex is determined. The original tetrahedral mesh data is constructed based on the original array composed of the position coordinates, texture coordinates and vertex index of each mesh vertex. Deformation simulation is performed on the original tetrahedral mesh data to obtain the current tetrahedral mesh data; The target cross section to be rendered is subjected to an intersection test with the current tetrahedral mesh data to determine the intersection points in the target tetrahedron that intersect with the target cross section; wherein, the target tetrahedron is the tetrahedron that intersects with the target cross section in the intersection test results; Based on each of the intersection points, construct the cross-sectional triangle mesh data of the target cross-section, and render the cross-sectional triangle mesh data to obtain a visual image of the target.
[0006] Optionally, determining the position coordinates of each grid vertex based on the mask image includes: Based on the mask image and the tetrahedral mesh generation algorithm, each segmentation unit is divided into multiple first tetrahedrons, and the position coordinates of each mesh vertex in each first tetrahedron are determined.
[0007] Optionally, determining the position coordinates of each grid vertex based on the mask image includes: The contour information of the segmentation unit in the mask image is extracted using a segmentation contour extraction algorithm, and a triangular patch model of the segmentation unit is generated based on the contour information. The closed space enclosed by the triangular patch model is defined as the target construction range. Tetrahedrals are divided within the target construction range according to the contour boundary and mesh density requirements of the triangular patch model to obtain multiple second tetrahedrals. Determine the position coordinates of each grid vertex in each of the second tetrahedrons.
[0008] Optionally, the step of performing an intersection test between the target section to be rendered and the current tetrahedral mesh data includes: Set the target cross section to be rendered based on the target's midpoint coordinates and target normal vector; The current tetrahedral mesh data is transferred to the cache space of the graphics processor so that the graphics processor can perform parallel intersection tests on multiple tetrahedra in the current tetrahedral mesh data.
[0009] Optionally, the step of performing an intersection test between the target section to be rendered and the current tetrahedral mesh data includes: Determine the directed distance between the target cross section to be rendered and each of the mesh vertices in the current tetrahedral mesh data, so as to obtain the distance result of each tetrahedron in the current tetrahedral mesh data; Count the number of directed distances less than 0, the number of directed distances equal to 0, and the number of directed distances greater than 0 in the current tetrahedron distance results; If the distance result of the current tetrahedron meets the preset tetrahedron intersection condition, then the first intersection test result of the current tetrahedron is generated, and the array position corresponding to the current tetrahedron is marked as the first preset flag in the intersection flag array; the first intersection test result indicates that the current tetrahedron intersects with the target cross section; If the distance result of the current tetrahedron does not meet the preset tetrahedron intersection condition, a second intersection test result of the current tetrahedron is generated, and the array position corresponding to the current tetrahedron is marked as the second preset flag in the intersection flag array; the second intersection test result indicates that the current tetrahedron does not intersect with the target cross section; The preset intersecting condition is that both the first quantity and the third quantity are greater than 0, or the second quantity is a preset quantity threshold.
[0010] Optionally, determining the intersection points of the target tetrahedron with the target cross-section includes: The intersection flag array is compressed using a prefix sum algorithm to obtain a compressed array; wherein the tetrahedrons in the compressed array are the target tetrahedrons; Determine the current edge in the target tetrahedron and determine whether the current edge satisfies the preset edge intersection condition; If the current edge satisfies the preset edge intersection condition, the position coordinates of the intersection point between the current edge and the target cross section are determined according to the ratio of the directed distances of the mesh vertices in the current edge; If the current edge does not meet the preset edge intersection condition, then a new current edge is determined in the target tetrahedron, and the process jumps back to the step of determining whether the current edge meets the preset edge intersection condition. The preset edge intersection condition is that the directed distances between the grid vertices in the current edge are greater than 0 and less than 0, respectively.
[0011] Optionally, constructing the cross-sectional triangle mesh data of the target cross-section based on each of the intersection points includes: The texture coordinates of each intersection point are determined based on the ratio of the directed distances between the vertices of each mesh. Count the number of intersections of the current target tetrahedron; If the number of intersections is a preset number of intersections, then a single triangular mesh is constructed using the intersections of the current target tetrahedron; If the number of intersection points is greater than the preset number of intersection points, then two triangular meshes are constructed using the cross product of the vectors between the intersection points of the current target tetrahedron. Construct cross-sectional triangular mesh data of the target cross section, which includes the triangular mesh formed by the intersection points of each of the target tetrahedrons.
[0012] Secondly, this application discloses a three-dimensional medical image cross-sectional rendering device, comprising: The segmentation module is used to segment static 3D medical images to obtain mask images for each segmentation unit; The mesh construction module is used to determine the position coordinates of each mesh vertex according to the mask image, add corresponding texture coordinates according to the relative position of each mesh vertex in the static three-dimensional medical image, and determine the vertex index of each mesh vertex, so as to construct the original tetrahedral mesh data based on the original array composed of the position coordinates, texture coordinates and vertex index of each mesh vertex; The deformation module is used to perform deformation simulation on the original tetrahedral mesh data to obtain the current tetrahedral mesh data; The intersection point determination module is used to perform an intersection test between the target cross section to be rendered and the current tetrahedral mesh data to determine the intersection points in the target tetrahedron that intersect with the target cross section; wherein, the target tetrahedron is the tetrahedron that intersects with the target cross section in the intersection test results; The cross-section rendering module is used to construct cross-sectional triangle mesh data of the target cross-section based on each of the intersection points, and to render the cross-sectional triangle mesh data to obtain a target visualization image.
[0013] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the aforementioned disclosed three-dimensional medical image cross-section rendering method.
[0014] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed three-dimensional medical image cross-section rendering method.
[0015] The beneficial effects of this application are as follows: This application segments a static three-dimensional medical image to obtain a mask image for each segmentation unit; determines the position coordinates of each mesh vertex based on the mask image, adds corresponding texture coordinates based on the relative position of each mesh vertex in the static three-dimensional medical image, and determines the vertex index of each mesh vertex, thereby constructing original tetrahedral mesh data based on the original array composed of the position coordinates, texture coordinates, and vertex index of each mesh vertex; performs deformation simulation on the original tetrahedral mesh data to obtain current tetrahedral mesh data; performs an intersection test between the target cross section to be rendered and the current tetrahedral mesh data to determine the intersection points of the target tetrahedron with the target cross section; wherein, the target tetrahedron is the tetrahedron representing the intersection with the target cross section in the intersection test results; constructs cross-sectional triangle mesh data of the target cross section based on each intersection point, and renders the cross-sectional triangle mesh data to obtain a target visualization image. Therefore, this application obtains a mask image by segmenting a static 3D medical image. Based on this mask image, the position coordinates of the mesh vertices are accurately determined, corresponding texture coordinates are added, and vertex indices are specified. The constructed original tetrahedral mesh data can accurately match the spatial contour of the target organ tissue with the original image information, providing a reliable data foundation for subsequent processing. The current tetrahedral mesh data obtained by deforming the original tetrahedral mesh data can realistically reflect the dynamic deformation state of the organ tissue, solving the problem that traditional static image cross-section imaging cannot match deformation. By performing an intersection test between the target cross-section and the current tetrahedral mesh data and determining the intersection point, the intersection area between the deformed organ tissue and the target cross-section can be accurately captured. Based on the intersection point, cross-sectional triangular mesh data is constructed and rendered, which can realize real-time visualization of any cross-section of the 3D medical image after deformation. There is no need to define and calculate the inverse deformation field, avoiding the limitations of the inverse deformation field when the deformation range is too large or irregular, such as the inability to define the deformation vector in the domain, the loss of some content of the original image, and the complexity of calculation. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0017] Figure 1 This is a flowchart of a three-dimensional medical image cross-section rendering method disclosed in this application; Figure 2 This is a rendering of a specific tetrahedral mesh construction process disclosed in this application; Figure 3This is a schematic diagram of a specific tetrahedral mesh deformation disclosed in this application; Figure 4 This is a specific diagram illustrating the compression of an intersection result disclosed in this application; Figure 5 This is a specific quadrilateral segmentation diagram disclosed in this application; Figure 6 This is a schematic diagram of a specific cross-sectional triangular mesh data disclosed in this application; Figure 7 This is a schematic diagram of the structure of a three-dimensional medical image cross-section rendering device disclosed in this application; Figure 8 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0019] Traditional 3D medical images such as CT, MRI, and 3D ultrasound are grayscale static images defined on a 3D regular grid. Their cross-sectional imaging usually adopts multi-plane reconstruction methods, which do not take into account the needs of deformation and dynamic scenes. In medical scenarios such as simulated surgical training, doctor-patient communication, and intraoperative navigation, it is often necessary to reconstruct a personalized 3D human body model based on the patient's 3D medical images. By constructing a physical simulation data structure, the dynamic deformation of tissues and organs caused by the action of surgical instruments or physiological activities such as breathing and heartbeat can be simulated. However, this high degree of realism in deformation can cause the 3D structure of the human body to deviate from the original medical image, making it difficult to obtain the cross-sectional data of the deformed image.
[0020] Generally, the image after deformation is achieved through a two-dimensional or three-dimensional inverse deformation field. That is, the original three-dimensional medical image is defined as I:R³→R, the deformed image is J:R³→R³, the inverse deformation field is u:R³→R³, and the deformed three-dimensional image is J(x)=I(x+u(x)). However, the construction of the inverse deformation field needs to be carried out in the preset J image domain (usually an axis-aligned bounding box space). It is necessary to calculate the coordinates of the regular grid points in the domain relative to the corresponding points in the original image domain and calculate the difference. When the deformation range is too large or irregular, problems may occur such as the deformation vector in the domain being unable to be defined and the loss of some content of the original image. At the same time, since the data structure used by physical simulations such as the finite element method (FEM) (such as tetrahedral elements) is inconsistent with the regular grid points of the three-dimensional image, the calculation of the deformation field is more complicated and cannot meet the accuracy and real-time requirements of the cross-sectional imaging of the deformed image in relevant medical scenarios.
[0021] Therefore, this application provides a three-dimensional medical image cross-section rendering scheme to improve the accuracy and real-time performance of three-dimensional medical image cross-section rendering.
[0022] See Figure 1 As shown in the figure, this application discloses a method for rendering three-dimensional medical image cross-sections, including: Step S11: Segment the static three-dimensional medical image to obtain the mask image of each segmentation unit.
[0023] Using static 3D medical images such as CT and MRI as the processing objects, these static 3D medical images are essentially grayscale images defined on a 3D regular grid, containing the spatial distribution and grayscale information of human organs and tissues. The static 3D medical images are processed by automatic segmentation algorithms (such as threshold segmentation, deep learning segmentation models, etc.). Based on the differences in grayscale values, density features, or contour boundary information of different organs and tissues in the image, the target human organs and tissues (such as lungs, liver, tumors, etc.) are accurately separated from the background and other irrelevant tissues, resulting in multiple segmentation units corresponding to different organs and tissues. At the same time, a unique 3D mask image is generated for each segmentation unit. The mask image only retains the spatial contour and region information of the corresponding segmentation unit. That is, the region within the mask image represents the target segmentation unit, and the outer region represents the non-target region. This provides a precise basis for the tissue contour and spatial range for the subsequent construction of a tetrahedral mesh adapted to physical simulation based on the mask image.
[0024] Step S12: Determine the position coordinates of each mesh vertex according to the mask image, add corresponding texture coordinates according to the relative position of each mesh vertex in the static three-dimensional medical image, and determine the vertex index of each mesh vertex, so as to construct the original tetrahedral mesh data based on the original array composed of the position coordinates, texture coordinates and vertex index of each mesh vertex.
[0025] In a first specific embodiment, determining the position coordinates of each grid vertex based on the mask image includes: dividing each segmentation unit into multiple first tetrahedrons based on the mask image and the tetrahedron mesh generation algorithm, and determining the position coordinates of each grid vertex in each first tetrahedron.
[0026] Based on the 3D mask image corresponding to each segmentation unit, which accurately defines the closed 3D spatial range of each segmentation unit (target organ / tissue), the algorithm directly adopts tetrahedral mesh generation algorithms adapted to the direct generation requirements, such as isosurface stuffing. Taking the 3D entity space of the segmentation unit defined by the mask image as the partitioning object, the complete closed space of each segmentation unit is discretized into multiple first tetrahedrons with reasonable topological structure and uniform distribution. During the discretization process, the algorithm accurately generates the mesh vertices of each first tetrahedron according to the spatial contour boundary of the mask image and the preset mesh density requirements. The algorithm also determines the specific 3D spatial coordinates of each mesh vertex in the 3D coordinate system through spatial coordinate calculation. These coordinate values accurately match the spatial position of the segmentation unit represented by the mask image, and finally form the vertices array that records the position information of all mesh vertices, thus completing the determination of the position coordinates of the mesh vertices in each first tetrahedron.
[0027] In a second specific embodiment, determining the position coordinates of each grid vertex based on the mask image includes: extracting the contour information of the segmentation unit in the mask image using a segmentation contour extraction algorithm, and generating a triangular patch model of the segmentation unit based on the contour information; determining the closed space enclosed by the triangular patch model as the target construction range, performing tetrahedral division within the target construction range according to the contour boundary and grid density requirements of the triangular patch model to obtain multiple second tetrahedrons; and determining the position coordinates of each grid vertex in each second tetrahedron.
[0028] Based on the 3D mask image corresponding to each segmentation unit, the spatial contour information of the segmentation unit in the mask image is extracted using a segmentation contour extraction algorithm. Based on this contour information, a triangular patch model that can represent the surface morphology of the segmentation unit is generated. The closed space enclosed by the triangular patch model is determined as the target construction range of the tetrahedral mesh. Then, according to the contour boundary of the triangular patch model and the preset mesh density requirements, a tetrahedral mesh generation algorithm such as isosurface filling is used to divide the target construction range into tetrahedrals, thereby obtaining multiple second tetrahedrals with reasonable topological structures that meet the physical simulation requirements. During the division process, the algorithm combines the contour constraints and spatial position parameters of the triangular patch model to accurately calculate and determine the 3D spatial coordinates of each mesh vertex in each second tetrahedral, and finally forms the vertices array that records the position information of all vertices.
[0029] For example Figure 2 As shown, through static 3D medical images, segmentation algorithms are used to separate tissues and organs, generating masks for different tissues and organs; 3D human body models are constructed using segmentation information such as masks, generating triangular patch models; physical simulation data structures are constructed according to different physical simulation requirements, and tetrahedral meshes are constructed based on the bounding volume of the triangular patch models, using algorithms such as Isosurface Stuffing. Based on the 3D mask image corresponding to each segmentation unit, if a direct tetrahedral generation method is used, tetrahedral mesh generation algorithms such as isosurface stuffing are directly employed to discretize the closed 3D space of the segmentation unit defined by the mask image into multiple first tetrahedrals. During this process, the contour boundaries of the mask image and the preset mesh density are combined to generate each mesh vertex, and the 3D spatial coordinates of each mesh vertex are accurately calculated and determined. If a method using a triangular patch model is adopted, the contour information of the segmentation unit in the mask image is first extracted through a segmentation contour extraction algorithm to generate a triangular patch model representing its surface morphology. The closed space enclosed by this model is used as the target construction range, and tetrahedrals are divided within the range according to the contour boundaries and mesh density requirements to obtain multiple second tetrahedrals. The three-dimensional spatial coordinates of the mesh vertices in each second tetrahedron are determined simultaneously. After determining the position coordinates of all mesh vertices, the static three-dimensional medical image is treated as three-dimensional texture data. Based on the relative spatial position of each mesh vertex in the static three-dimensional medical image, the corresponding texture coordinates are added to each vertex through coordinate mapping. Then, based on the topology of the generated first or second tetrahedron, it is determined which four mesh vertices each tetrahedron is composed of, and then the vertex index corresponding to each mesh vertex is determined. Finally, the position coordinates, texture coordinates and vertex indexes of all mesh vertices are organized into the vertices array, texCoords array and tetraIndices array respectively. These three original arrays work together to form the original tetrahedral mesh data adapted to the subsequent physical deformation simulation.
[0030] Step S13: Perform deformation simulation on the original tetrahedral mesh data to obtain the current tetrahedral mesh data.
[0031] For example Figure 3As shown, based on the original tetrahedral mesh data (including the vertices array, texCoords array, and tetraIndices array), physical simulation algorithms such as the finite element method (FEM) are used. Combined with the biomechanical parameters of the target organ tissue (such as elastic modulus and Poisson's ratio) and preset deformation driving conditions (such as respiratory motion, surgical instrument force, and tissue traction displacement), mechanical calculations and deformation iterations are performed on the original tetrahedral mesh. During the calculation process, only the three-dimensional coordinate values of each vertex in the vertices array, which represents the spatial position of the mesh vertices, are updated, while the texCoords array, which represents the mapping relationship between the vertices and the original three-dimensional medical image texture, and the tetraIndices array, which represents the tetrahedral topology, remain unchanged. Through multiple iterative calculations, the spatial position of the mesh vertices gradually approaches the actual deformation state, and finally, the current tetrahedral mesh data that adapts to the dynamic scene requirements and completes the deformation update is obtained.
[0032] Step S14: Perform an intersection test between the target cross section to be rendered and the current tetrahedral mesh data to determine the intersection points in the target tetrahedron that intersect with the target cross section; wherein, the target tetrahedron is the tetrahedron that intersects with the target cross section in the intersection test results.
[0033] In this embodiment, the step of performing an intersection test between the target cross section to be rendered and the current tetrahedral mesh data includes: setting the target cross section to be rendered according to the target midpoint coordinates and the target normal vector; and transferring the current tetrahedral mesh data to the cache space of the graphics processor so that the graphics processor can perform parallel intersection tests on multiple tetrahedra in the current tetrahedral mesh data.
[0034] First, based on clinical visualization requirements, a target cross section (p, n) to be rendered is defined. This target cross section is jointly defined by the midpoint coordinates p and the target normal vector n. Then, the current tetrahedral mesh data after deformation (including the vertices array recording the 3D spatial coordinates of the vertices after deformation, the texCoords array recording the original 3D medical image texture sampling coordinates corresponding to the vertices, and the tetraIndices array recording the topological relationships of each tetrahedral vertex) is completely transferred to a pre-allocated, directly accessible cache space (such as unified memory or video memory) of the graphics processing unit (GPU). Leveraging the massively parallel computing architecture of the GPU, each tetrahedron in the current tetrahedral mesh is allocated as an independent computing unit to different stream processor cores of the GPU, achieving parallelized intersection testing of all tetrahedrons. Compared to the central processing unit (CPU), this approach is more efficient. The CPU's serial computing mode and the GPU's parallel processing capabilities can significantly improve the overall efficiency of intersection testing. Furthermore, the high-speed data read and write characteristics of the GPU cache can effectively reduce data transmission latency, ensuring that all tetrahedral intersection tests with the target cross-section can still be completed quickly when processing large-scale tetrahedral mesh data, providing efficient data support for subsequent intersection point calculation and cross-section mesh construction.
[0035] In this embodiment, the step of performing an intersection test between the target cross section to be rendered and the current tetrahedral mesh data includes: determining the directed distance between the target cross section to be rendered and each of the mesh vertices in the current tetrahedral mesh data to obtain the distance result of each tetrahedron in the current tetrahedral mesh data; counting the first number of directed distances less than 0, the second number of directed distances equal to 0, and the third number of directed distances greater than 0 in the current tetrahedral distance results; if the distance result of the current tetrahedron satisfies a preset tetrahedral intersection condition, then generating the first intersection test result of the current tetrahedron and marking it on the intersection marker. The array position corresponding to the current tetrahedron is marked as a first preset flag in the array; the first intersection test result indicates that the current tetrahedron intersects with the target cross section; if the distance result of the current tetrahedron does not meet the preset tetrahedron intersection condition, a second intersection test result of the current tetrahedron is generated, and the array position corresponding to the current tetrahedron is marked as a second preset flag in the intersection flag array; the second intersection test result indicates that the current tetrahedron does not intersect with the target cross section; wherein, the preset tetrahedron intersection condition is that both the first quantity and the third quantity are greater than 0, or the second quantity is a preset quantity threshold.
[0036] When performing an intersection test between the target cross-section to be rendered and the current tetrahedral mesh data, the first step is to use the formula based on the midpoint coordinates p and normal vector n of the target cross-section. Calculate the directed distance D between the target cross section to be rendered and each vertex of the current tetrahedral mesh data, where Given the coordinates of a single vertex of the current tetrahedron, calculate the directed distances to the four vertices of each tetrahedron one by one to obtain the distance result of the tetrahedron.
[0037] Next, the distance results for each tetrahedron are statistically analyzed to obtain the number of vertices with a directed distance less than 0 (first count), the number of vertices with a directed distance equal to 0 (second count), and the number of vertices with a directed distance greater than 0 (third count).
[0038] Then, it is determined whether the distance result of the current tetrahedron meets the preset tetrahedron intersection condition. The preset tetrahedron intersection condition is that the first and third quantities are both greater than 0 (indicating that the tetrahedron vertices are distributed across the target section), or the second quantity reaches the preset quantity threshold (usually the threshold is 3, that is, all 3 vertices fall on the target section). If the preset tetrahedron intersection condition is met, a first intersection test result representing the intersection of the current tetrahedron and the target section is generated, and the array position corresponding to the current tetrahedron in the pre-constructed intersection flag array is marked as the first preset flag (such as the value 1). If the preset tetrahedron intersection condition is not met, a second intersection test result representing the non-intersection of the current tetrahedron and the target section is generated, and the array position corresponding to the current tetrahedron in the intersection flag array is marked as the second preset flag (such as the value 0). This completes the intersection test determination for each tetrahedron in the current tetrahedron mesh data.
[0039] In this embodiment, determining the intersection point of the target tetrahedron with the target cross section includes: compressing the intersection flag array based on a prefix sum algorithm to obtain a compressed array; wherein, the tetrahedron in the compressed array is the target tetrahedron; determining the current edge in the target tetrahedron and judging whether the current edge satisfies a preset edge intersection condition; if the current edge satisfies the preset edge intersection condition, then determining the position coordinates of the intersection point of the current edge and the target cross section according to the ratio of the directed distances of the mesh vertices in the current edge; if the current edge does not satisfy the preset edge intersection condition, then determining a new current edge in the target tetrahedron and jumping back to the step of judging whether the current edge satisfies the preset edge intersection condition; wherein, the preset edge intersection condition is that the directed distances of the mesh vertices in the current edge are greater than 0 and less than 0, respectively.
[0040] When there are a large number of tetrahedrons, the intersection test output is a very long array of 0s and 1s. If subsequent tasks are directly distributed based on the length of this array, a considerable number of GPU computing threads will be called. When the flag bit is 0, the test returns directly. Most of the tasks are meaningless empty tasks, which wastes resources on the setting of computing tasks, the number of calls, and execution efficiency.
[0041] The prefix sum algorithm can compress the results of intersection tests in parallel, generating a small array where each array element stores the index ID of an intersecting tetrahedron, achieving the effect of... Figure 4 As shown, the intersection flag array composed of the first preset flag and the second preset flag is scanned and calculated based on the prefix sum algorithm. The prefix sum array is obtained by accumulating the values of adjacent array elements. After generating the prefix sum, the value of the last element of the prefix sum array is the value of the compressed array. Then, it is checked whether each element is different from the previous element. If there is a difference, it means that the tetrahedron represented by the index number of the element is an intersecting tetrahedron. This index is recorded in the compressed array. That is, the position marked as the first preset flag in the intersection flag array is located by using the difference between adjacent elements in the prefix sum array. The tetrahedron index at the corresponding position is extracted and stored, thus obtaining a compressed array containing only the index of the target tetrahedron.
[0042] Traverse the six edges of each target tetrahedron in the compressed array and determine each edge as the current edge. Determine whether the directed distances between the two endpoints of the current edge satisfy the preset edge intersection condition, i.e., one of the directed distances between the two endpoints is greater than 0 and the other is less than 0. If the current edge satisfies the preset edge intersection condition, perform linear interpolation on the current edge based on the ratio of the absolute values of the directed distances between the two endpoints to calculate the three-dimensional position coordinates of the intersection point between the current edge and the target cross section. If the current edge does not satisfy the preset edge intersection condition, skip the current edge and determine a new current edge in the target tetrahedron, and jump back to the step of determining whether the current edge satisfies the preset edge intersection condition, until all edges of the target tetrahedron have been traversed.
[0043] Step S15: Construct cross-sectional triangle mesh data of the target cross-section based on each of the intersection points, and render the cross-sectional triangle mesh data to obtain a target visualization image.
[0044] In this embodiment, constructing the cross-sectional triangle mesh data of the target cross section based on each of the intersection points includes: determining the texture coordinates of each intersection point according to the ratio of the directed distances between the vertices of each mesh; counting the number of intersection points of the current target tetrahedron; if the number of intersection points is a preset number, then using the intersection points of the current target tetrahedron to construct a single triangle mesh; if the number of intersection points is greater than the preset number, then using the cross product of the vectors between the intersection points of the current target tetrahedron to construct two triangle meshes using the intersection points of the current target tetrahedron; and constructing the cross-sectional triangle mesh data of the target cross section including the triangle meshes constructed by the intersection points of each target tetrahedron.
[0045] The texture coordinates corresponding to the intersection point are generated by linear interpolation of the texture coordinates corresponding to the two endpoints of the current edge (i.e., the texture sampling coordinates of the endpoints in the static 3D medical image) based on the ratio of the absolute values of the directed distances between the two endpoints.
[0046] Determine the triangular mesh for each target tetrahedron. Taking the current target tetrahedron as an example, count the number of intersections of the current target tetrahedron. It can be understood that the number of intersections between the current target tetrahedron and the target section is 3 or 4.
[0047] When the number of intersection points equals the preset number of intersection points (3), these 3 intersection points are directly used to construct a single triangular mesh based on topological relationships. This means that if the distance between these 3 vertices and the plane is 0, the triangle directly coincides with the plane, and the vertex information of the new triangle is the original information. Otherwise, since the intersection points have already been calculated in previous steps, the vertex information of a triangle is directly stored in the output data structure, and the function ends.
[0048] When the number of intersection points exceeds the preset number (i.e., 4), the resulting shape is a quadrilateral, which needs to be divided into two triangles. The two triangles share a side. A simple division might result in the two triangles not completely covering the quadrilateral. Figure 5 As shown, black lines represent shared edges. Correctly identifying shared edges effectively divides the quadrilateral into two triangles, while incorrectly selecting shared edges will not. Therefore, the cross product of vectors between each intersection point is calculated. Where E represents the vector between the intersection points, for example, "E 01 " represents the vector pointing from the 0th intersection point to the 1st intersection point, "E 02 "" represents the vector pointing from the 0th intersection point to the 2nd intersection point, and so on. Next, the dot product of the vector cross product is performed, i.e. The sign (positive / negative) of the dot product result is used to determine whether the directions of the two cross product vectors are consistent: if the dot product result is positive, it means that the directions of the two cross product vectors are the same, and the topological order of the corresponding intersection points conforms to the spatial logic of triangle construction; if it is negative, it means that the directions are opposite, and the order of the intersection points needs to be adjusted. Finally, this judgment is used to select the ideal shared edge that can split the 4 intersection points into two valid triangles; after determining the ideal shared edge, the quadrilateral can be divided into 2 triangles.
[0049] In this way, we obtain the triangular mesh of all target tetrahedrons, for example... Figure 6 As shown, Figure 6 The black border in the image represents a tetrahedral mesh, the red border represents tetrahedrons that intersect with the plane, and the green border represents triangular meshes constructed on the intersecting plane.
[0050] In other words, the output of the algorithm in this embodiment is the intersection area formed by the intersection of the cross section and the tetrahedral mesh, represented by a triangular planar mesh. Similarly, there are arrays representing vertex coordinates, texture coordinates, and vertex indices.
[0051] Algorithm: Generation of a tetrahedral solid mesh and a triangular mesh with cross-section of a 3D plane; Input: -vertices[vertex count] (vec3) / / Location information; -texCoords[vertex count](vec3) / / Texture coordinates; -tetraIndices[tetrahydro count][4](uint indices) / / Vertex index; The output is: -triPositions[triCount 3](vec3); -triTexCoords[triCount 3](vec3); -triIndices[triCount][3](uint); The rendering output of cross-sections is achieved using a triangular mesh. Each triangle is an independent facet with texture data, which can be grouped into a draw batch and directly input into the GPU. In a single draw call, all intersecting triangles on the entire plane are rendered. Specifically, this can be achieved by utilizing relevant elements from the DrawProceduralIndirect and Render CommandBuffer components in the Unity engine and inserting them into the rendering workflow.
[0052] The beneficial effects of this application are as follows: This application segments a static three-dimensional medical image to obtain a mask image for each segmentation unit; determines the position coordinates of each mesh vertex based on the mask image, adds corresponding texture coordinates based on the relative position of each mesh vertex in the static three-dimensional medical image, and determines the vertex index of each mesh vertex, thereby constructing original tetrahedral mesh data based on the original array composed of the position coordinates, texture coordinates, and vertex index of each mesh vertex; performs deformation simulation on the original tetrahedral mesh data to obtain current tetrahedral mesh data; performs an intersection test between the target cross section to be rendered and the current tetrahedral mesh data to determine the intersection points of the target tetrahedron with the target cross section; wherein, the target tetrahedron is the tetrahedron representing the intersection with the target cross section in the intersection test results; constructs cross-sectional triangle mesh data of the target cross section based on each intersection point, and renders the cross-sectional triangle mesh data to obtain a target visualization image. Therefore, this application obtains a mask image by segmenting a static 3D medical image. Based on this mask image, the position coordinates of the mesh vertices are accurately determined, corresponding texture coordinates are added, and vertex indices are specified. The constructed original tetrahedral mesh data can accurately match the spatial contour of the target organ tissue with the original image information, providing a reliable data foundation for subsequent processing. The current tetrahedral mesh data obtained by deforming the original tetrahedral mesh data can realistically reflect the dynamic deformation state of the organ tissue, solving the problem that traditional static image cross-section imaging cannot match deformation. By performing an intersection test between the target cross-section and the current tetrahedral mesh data and determining the intersection point, the intersection area between the deformed organ tissue and the target cross-section can be accurately captured. Based on the intersection point, cross-sectional triangular mesh data is constructed and rendered, which can realize real-time visualization of any cross-section of the 3D medical image after deformation. There is no need to define and calculate the inverse deformation field, avoiding the limitations of the inverse deformation field when the deformation range is too large or irregular, such as the inability to define the deformation vector in the domain, the loss of some content of the original image, and the complexity of calculation.
[0053] See Figure 7 As shown in the figure, this application discloses a three-dimensional medical image cross-sectional rendering device, including: The segmentation module 11 is used to segment static three-dimensional medical images to obtain mask images of each segmentation unit; Mesh construction module 12 is used to determine the position coordinates of each mesh vertex according to the mask image, add corresponding texture coordinates according to the relative position of each mesh vertex in the static three-dimensional medical image, and determine the vertex index of each mesh vertex, so as to construct the original tetrahedral mesh data based on the original array composed of the position coordinates, texture coordinates and vertex index of each mesh vertex; Deformation module 13 is used to perform deformation simulation on the original tetrahedral mesh data to obtain the current tetrahedral mesh data; The intersection point determination module 14 is used to perform an intersection test between the target cross section to be rendered and the current tetrahedral mesh data to determine the intersection points in the target tetrahedron that intersect with the target cross section; wherein, the target tetrahedron is the tetrahedron that intersects with the target cross section in the intersection test results; The cross-section rendering module 15 is used to construct cross-sectional triangle mesh data of the target cross-section based on each of the intersection points, and to render the cross-sectional triangle mesh data to obtain a target visualization image.
[0054] Furthermore, embodiments of this application also provide an electronic device. Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.
[0055] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the three-dimensional medical image cross-sectional rendering method performed by the electronic device disclosed in any of the foregoing embodiments.
[0056] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0057] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.
[0058] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.
[0059] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the three-dimensional medical image cross-sectional rendering method executed by the electronic device as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.
[0060] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed method for rendering three-dimensional medical image cross-sections. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0061] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0062] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software module may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, removable disk, CD-ROM (Compact Disc Read-Only Memory), or any other form of storage medium known in the art.
[0063] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0064] The above provides a detailed description of a three-dimensional medical image cross-section rendering method, apparatus, device, and medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only intended to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for rendering cross-sections of three-dimensional medical images, characterized in that, include: Static 3D medical images are segmented to obtain mask images for each segmentation unit; The position coordinates of each mesh vertex are determined based on the mask image. Corresponding texture coordinates are added based on the relative position of each mesh vertex in the static three-dimensional medical image. The vertex index of each mesh vertex is determined. The original tetrahedral mesh data is constructed based on the original array composed of the position coordinates, texture coordinates and vertex index of each mesh vertex. Deformation simulation is performed on the original tetrahedral mesh data to obtain the current tetrahedral mesh data; The target cross section to be rendered is subjected to an intersection test with the current tetrahedral mesh data to determine the intersection points in the target tetrahedron that intersect with the target cross section; wherein, the target tetrahedron is the tetrahedron that intersects with the target cross section in the intersection test results; Based on each of the intersection points, construct the cross-sectional triangle mesh data of the target cross-section, and render the cross-sectional triangle mesh data to obtain a visual image of the target.
2. The three-dimensional medical image cross-section rendering method according to claim 1, characterized in that, Determining the position coordinates of each grid vertex based on the mask image includes: Based on the mask image and the tetrahedral mesh generation algorithm, each segmentation unit is divided into multiple first tetrahedrons, and the position coordinates of each mesh vertex in each first tetrahedron are determined.
3. The three-dimensional medical image cross-section rendering method according to claim 1, characterized in that, Determining the position coordinates of each grid vertex based on the mask image includes: The contour information of the segmentation unit in the mask image is extracted using a segmentation contour extraction algorithm, and a triangular patch model of the segmentation unit is generated based on the contour information. The closed space enclosed by the triangular patch model is defined as the target construction range. Tetrahedrals are divided within the target construction range according to the contour boundary and mesh density requirements of the triangular patch model to obtain multiple second tetrahedrals. Determine the position coordinates of each grid vertex in each of the second tetrahedrons.
4. The three-dimensional medical image cross-section rendering method according to claim 1, characterized in that, The step of performing an intersection test between the target section to be rendered and the current tetrahedral mesh data includes: Set the target cross section to be rendered based on the target's midpoint coordinates and target normal vector; The current tetrahedral mesh data is transferred to the cache space of the graphics processor so that the graphics processor can perform parallel intersection tests on multiple tetrahedra in the current tetrahedral mesh data.
5. The three-dimensional medical image cross-sectional rendering method according to any one of claims 1 to 4, characterized in that, The step of performing an intersection test between the target section to be rendered and the current tetrahedral mesh data includes: Determine the directed distance between the target cross section to be rendered and each of the mesh vertices in the current tetrahedral mesh data, so as to obtain the distance result of each tetrahedron in the current tetrahedral mesh data; Count the number of directed distances less than 0, the number of directed distances equal to 0, and the number of directed distances greater than 0 in the current tetrahedron distance results; If the distance result of the current tetrahedron meets the preset tetrahedron intersection condition, then the first intersection test result of the current tetrahedron is generated, and the array position corresponding to the current tetrahedron is marked as the first preset flag in the intersection flag array; the first intersection test result indicates that the current tetrahedron intersects with the target cross section; If the distance result of the current tetrahedron does not meet the preset tetrahedron intersection condition, a second intersection test result of the current tetrahedron is generated, and the array position corresponding to the current tetrahedron is marked as the second preset flag in the intersection flag array; the second intersection test result indicates that the current tetrahedron does not intersect with the target cross section; The preset intersecting condition is that both the first quantity and the third quantity are greater than 0, or the second quantity is a preset quantity threshold.
6. The three-dimensional medical image cross-section rendering method according to claim 5, characterized in that, The determination of the intersection points of the target tetrahedron with the target cross section includes: The intersection flag array is compressed using a prefix sum algorithm to obtain a compressed array; wherein the tetrahedrons in the compressed array are the target tetrahedrons; Determine the current edge in the target tetrahedron and determine whether the current edge satisfies the preset edge intersection condition; If the current edge satisfies the preset edge intersection condition, the position coordinates of the intersection point between the current edge and the target cross section are determined according to the ratio of the directed distances of the mesh vertices in the current edge; If the current edge does not meet the preset edge intersection condition, then a new current edge is determined in the target tetrahedron, and the process jumps back to the step of determining whether the current edge meets the preset edge intersection condition. The preset edge intersection condition is that the directed distances between the grid vertices in the current edge are greater than 0 and less than 0, respectively.
7. The three-dimensional medical image cross-section rendering method according to claim 5, characterized in that, The construction of the cross-sectional triangle mesh data of the target cross-section based on each of the intersection points includes: The texture coordinates of each intersection point are determined based on the ratio of the directed distances between the vertices of each mesh. Count the number of intersections of the current target tetrahedron; If the number of intersections is a preset number of intersections, then a single triangular mesh is constructed using the intersections of the current target tetrahedron; If the number of intersection points is greater than the preset number of intersection points, then two triangular meshes are constructed using the cross product of the vectors between the intersection points of the current target tetrahedron. Construct cross-sectional triangular mesh data of the target cross section, which includes the triangular mesh formed by the intersection points of each of the target tetrahedrons.
8. A three-dimensional medical image cross-sectional rendering device, characterized in that, include: The segmentation module is used to segment static 3D medical images to obtain mask images of each segmentation unit; The mesh construction module is used to determine the position coordinates of each mesh vertex according to the mask image, add corresponding texture coordinates according to the relative position of each mesh vertex in the static three-dimensional medical image, and determine the vertex index of each mesh vertex, so as to construct the original tetrahedral mesh data based on the original array composed of the position coordinates, texture coordinates and vertex index of each mesh vertex; The deformation module is used to perform deformation simulation on the original tetrahedral mesh data to obtain the current tetrahedral mesh data; The intersection point determination module is used to perform an intersection test between the target cross section to be rendered and the current tetrahedral mesh data to determine the intersection points in the target tetrahedron that intersect with the target cross section; wherein, the target tetrahedron is the tetrahedron that intersects with the target cross section in the intersection test results; The cross-section rendering module is used to construct cross-sectional triangle mesh data of the target cross-section based on each of the intersection points, and to render the cross-sectional triangle mesh data to obtain a target visualization image.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the three-dimensional medical image cross-section rendering method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the three-dimensional medical image cross-section rendering method as described in any one of claims 1 to 7.