Intestinal tract three-dimensional model reconstruction method and system

Abstract

The present application relates to the field of three-dimensional modeling, and relates to a method and system for reconstructing a three-dimensional model of an intestine. A three-dimensional voxel model of the intestine is constructed. For each voxel edge of the three-dimensional voxel model of the intestine that intersects an isosurface, an intersection point of the isosurface and the voxel edge is obtained, and a vertex of each voxel sharing the voxel edge is calculated according to a preset method. If the vertices corresponding to the voxels sharing the voxel edge are all in a plane, the distances of the intersection points to the plane are calculated, and the area of a region formed by the vertices is calculated. An isosurface fitting mode is determined according to a ratio of the distances to the area. The isosurface is fitted according to the determined isosurface fitting mode, and the isosurface extraction of the intestine is completed. The generated three-dimensional surface of the intestine is smoothed and then rendered and displayed on a display device. The present application makes the extraction of the isosurface of the intestine more accurate, and the three-dimensional model of the intestine more realistic.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of three-dimensional model reconstruction, in particular to an intestinal three-dimensional model reconstruction method and system. BACKGROUND

[0002] Three-dimensional reconstruction is a process of using patient CT or MRI original data to perform three-dimensional modeling and form a three-dimensional visual model. Three-dimensional model reconstruction of the digestive tract provides doctors with more abundant and more intuitive lesion information, helping doctors to diagnose more accurately. It not only improves the accuracy of disease diagnosis and the efficiency of surgical planning, but also promotes the development of personalized treatment plans. In addition, through the intestinal three-dimensional modeling technology, doctors are provided with an unprecedented perspective and tool to better understand, diagnose and treat various medical conditions, and it is also helpful for students to improve their surgical skills through simulated surgery to prepare for actual operation. However, the existing three-dimensional reconstruction method based on CT images is not accurate enough for the fitting of the actual intestine, and can only depict the general shape of the intestine, and the details are not accurate enough. SUMMARY

[0003] In order to solve the above problems, in a first aspect, the present application provides an intestinal three-dimensional model reconstruction method, the method comprising the following steps:

[0004] S1, obtaining the photographed intestinal CT image, performing image segmentation on the preprocessed CT image to obtain the intestinal region image corresponding to each CT image; stacking the segmented intestinal region images in the order of the CT images to construct an intestinal three-dimensional voxel model;

[0005] S2, for each voxel edge intersecting with an isosurface in the intestinal three-dimensional voxel model, obtaining the intersection point of the isosurface and the voxel edge, and calculating the vertex of each voxel sharing the voxel edge according to a predetermined method; if the vertices corresponding to all voxels sharing the voxel edge are in a plane, calculating the distance from the intersection point to the plane and the area of the region composed of all the vertices, and determining the isosurface fitting mode according to the ratio of the distance to the area;

[0006] S3, fitting the isosurface according to the determined isosurface fitting mode to complete the isosurface extraction of the intestine; and rendering and displaying the generated intestinal three-dimensional surface on a display device after smoothing processing.

[0007] Preferably, the isosurface fitting mode is determined according to the ratio of the distance to the area, specifically:

[0008] If the ratio is less than a threshold value, the region composed of the vertices corresponding to all voxels sharing the voxel edge is divided into two triangular facets;

[0009] Otherwise, the intersection point is taken as the vertex of the pyramid, and the side of the pyramid is taken as the triangular facet.

[0010] Preferably, the smoothing processing, in particular:

[0011] The included angle of the normal vectors of all the triangular facets sharing the same edge is calculated, and if the included angle is greater than a preset value, the two end points of the edge are obtained, and each of the two end points is subjected to smoothing processing.

[0012] Preferably, the smoothing processing of each of the two end points, in particular:

[0013] After the two end points are subjected to the first smoothing processing, the included angle between the straight line where the two end points are located and the straight line where the two end points are located before the smoothing processing is taken as the first included angle.

[0014] After the two end points are subjected to the second smoothing processing, the included angle between the straight line where the two end points are located and the straight line where the two end points are located before the smoothing processing is taken as the second included angle.

[0015] The smoothing processing mode corresponding to the minimum value of the first included angle and the second included angle is taken as the smoothing processing mode of each of the two end points.

[0016] Preferably, the preset method is a Dual Contouring isosurface extraction method, the first smoothing processing mode is Laplacian-based smoothing, and the second smoothing processing mode is curvature-based smoothing.

[0017] In another aspect, the present application provides an intestinal three-dimensional model reconstruction system, which comprises the following modules:

[0018] A voxel model construction module is configured to obtain a photographed intestinal CT image, perform image segmentation on the preprocessed CT image to obtain an intestinal region image corresponding to each CT image, stack the segmented intestinal region images in the order of the CT images, and construct an intestinal three-dimensional voxel model.

[0019] An isosurface fitting mode determination module is configured to obtain the intersection point of an isosurface and a voxel edge intersecting with the isosurface in the intestinal three-dimensional voxel model, and calculate the vertex of each voxel sharing the voxel edge according to a preset method; if the vertices corresponding to all the voxels sharing the voxel edge are in a plane, calculate the distance from the intersection point to the plane and the area of the region composed of all the vertices, and determine the isosurface fitting mode according to the ratio of the distance to the area.

[0020] The isosurface extraction and rendering module is configured to fit the isosurface according to the determined isosurface fitting mode, complete the isosurface extraction of the intestinal tract, and render and display the generated three-dimensional surface of the intestinal tract on a display device after smoothing processing.

[0021] Preferably, the isosurface fitting mode is determined according to the ratio of the distance to the area, and specifically is:

[0022] If the ratio is less than a threshold value, an area formed by vertices corresponding to each voxel sharing the voxel edge is divided into two triangular facets.

[0023] Otherwise, the intersection point is taken as a pyramid vertex, and a side of the pyramid is taken as a triangular facet.

[0024] Preferably, the smoothing processing is specifically:

[0025] An included angle of normal vectors of the triangular facets sharing an edge is calculated, and if the included angle is greater than a preset value, two end points of the edge are obtained, and each of the two end points is smoothed.

[0026] Preferably, the smoothing processing of each of the two end points is specifically:

[0027] After the first smoothing mode is used to smooth the two end points, an included angle between a straight line where the two end points are located and a straight line where the two end points are located before the smoothing processing is taken as a first included angle.

[0028] After the second smoothing mode is used to smooth the two end points, an included angle between a straight line where the two end points are located and a straight line where the two end points are located before the smoothing processing is taken as a second included angle.

[0029] The smoothing mode corresponding to the minimum value of the first included angle and the second included angle is taken as a smoothing mode of the two end points, and each of the two end points is smoothed.

[0030] Preferably, the preset method is a Dual Contouring isosurface extraction method, the first smoothing mode is Laplacian-based smoothing, and the second smoothing mode is curvature-based smoothing.

[0031] In addition, the present application also provides a computer program product, which, when executed by a processor, implements the method as described above.

[0032] The intestine has many bends and folds on its surface. Current techniques do not smoothly render these folds, resulting in jagged edges when a 3D model of the intestine is magnified. This invention improves the extraction of isosurfaces. Specifically, for each voxel edge in the 3D voxel model of the intestine that intersects with an isosurface, the intersection point of the isosurface and the voxel edge is obtained. The vertices of each voxel sharing the voxel edge are calculated using a preset method. If the vertices corresponding to all voxels sharing the voxel edge lie in a plane, the distance from the intersection point to the plane and the area of ​​the region formed by all the vertices are calculated. The isosurface fitting method is determined based on the ratio of the distance to the area. This method enables better surface rendering of the intestine, resulting in more realistic details and a smaller difference from the real intestine. Attached Figure Description

[0033] Figure 1 This is a flowchart of Example 1;

[0034] Figure 2 A schematic diagram of a three-dimensional voxel model of the intestine;

[0035] Figure 3 A schematic diagram of voxels, intersections, and vertices;

[0036] Figure 4 This is a schematic diagram of the smoothing process. Detailed Implementation

[0037] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this invention, if the collection of personal privacy data is involved, such as facial recognition or mobile phone usage information, prior consent will be obtained, including but not limited to verbal reminders, posters, and mobile phone reminders; if there are conflicts with laws and regulations, production or use will be within the scope permitted by laws and regulations.

[0038] With reference to the drawings and embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of the present application.

[0039] In an embodiment, the present application provides an intestinal three-dimensional model reconstruction method, which comprises the following steps: Figure 1

[0040] S1, obtaining a photographed intestinal CT image, performing image segmentation on the preprocessed CT image to obtain an intestinal region image corresponding to each CT image; stacking the segmented intestinal region images in the order of the CT images to construct an intestinal three-dimensional voxel model;

[0041] The CT image photographed by the CT device is an image set, and each section is a CT image. The CT image includes not only the intestine but also non-intestinal organs such as the kidney and the spine. The intestinal region image of the CT image is retained by performing image segmentation on the CT image. Since there is a layer thickness in CT scanning, the segmented CT image is stacked in sequence to obtain the photographed intestinal space of the user. The voxel is similar to the pixel, but the voxel is a three-dimensional space with a certain volume, generally in the form of a cube. Figure 2 A voxel composed of an upper CT image and a lower CT image is shown. The cube has eight vertices, and the value of the vertex is the pixel value of the corresponding position of the upper and lower CT images. When all the segmented consumption system region images are stacked, the consumption system three-dimensional voxel model can be obtained according to the voxel generation method.

[0042] S2, for each voxel edge intersecting with an isosurface in the intestinal three-dimensional voxel model, obtaining the intersection point of the isosurface and the voxel edge, and calculating the vertex of each voxel sharing the voxel edge according to a preset method; if the vertices corresponding to all the voxel edges are in a plane, calculating the distance from the intersection point to the plane and the area of the region composed of all the vertices, and determining the isosurface fitting mode according to the ratio of the distance to the area;

[0043] ​The isosurface extraction is an important step of surface rendering, and is directly related to the effect of the final surface rendering or the rendering effect of the intestinal tract. The edge of a voxel corresponding to an intersection point of an isosurface is calculated to obtain a vertex of the voxel. In the Marching Cubes algorithm, the intersection point is directly taken as the vertex, but there are problems of model ambiguity and inaccurate features. The preset algorithm is adopted to calculate the vertex of each voxel sharing the edge of the voxel. It should be noted that the vertex herein does not represent the vertex of a cube of the voxel, but a vertex of a triangular patch. The vertex of each voxel sharing the edge of the voxel is calculated according to the preset method, wherein the vertex refers to a vertex of a triangular patch in the voxel. Of course, it is not limited to a triangular patch (Triangle Mesh), and if a quadrangular mesh is adopted, the vertex refers to a vertex of the quadrangular mesh. In a more specific embodiment, the preset method is a Dual Contouring isosurface extraction method.

[0044] There are four voxels around the edge of the voxel, and each voxel has a vertex, so there are finally four vertices, as shown in the following figure. Figure 3 The four vertices may be coplanar or may not be coplanar. When the four vertices are coplanar, the distance from the intersection point to the plane is further calculated, and the area of the region composed of all the vertices is calculated, and the isosurface fitting mode is determined according to the ratio of the distance to the area.

[0045] In a specific embodiment, the isosurface fitting mode determined according to the ratio of the distance to the area is specifically:

[0046] If the ratio is less than a threshold value, the region composed of the vertices corresponding to each voxel sharing the edge of the voxel is divided into two triangular patches.

[0047] Otherwise, the intersection point is taken as a vertex of a pyramid, and the side of the pyramid is taken as a triangular patch.

[0048] If the ratio of the height to the area is less than a threshold value, at this time, the four coplanar vertices and the intersection point form a relatively flat pyramid, and the region composed of the vertices corresponding to each voxel is also divided into two triangular patches. If the ratio is not less than the threshold value, it indicates that the difference between the quadrilateral and the isosurface is large at this time, and a re-division method is adopted to obtain a plurality of triangular patches, that is, the intersection point is taken as a vertex of a pyramid, and the side of the pyramid is taken as a triangular patch, at this time, the triangular patches are divided more finely, and the isosurface is more consistent.

[0049] In another embodiment, if all the shared voxel edges do not correspond to vertices on a plane, the Marching Cubes algorithm is still used to extract the triangular facets.

[0050] S3, fitting the isosurface according to the determined fitting mode of the isosurface, completing the extraction of the isosurface of the intestinal tract; and rendering and displaying the generated three-dimensional surface of the intestinal tract on a display device after smoothing processing.

[0051] After the triangular facets of the isosurface of the three-dimensional voxel model of the intestinal tract are obtained, the extraction of the isosurface of the intestinal tract is completed, and further smoothing processing is performed. The smoothing processing mode can be various, including but not limited to Laplacian-based smoothing and curvature-based smoothing. Then, the three-dimensional model of the intestinal tract is displayed on a display device such as a display, and medical personnel can view the three-dimensional model of the intestinal tract.

[0052] In another embodiment, the smoothing processing is specifically as follows.

[0053] The included angle of the normal vectors of the triangular facets sharing an edge is calculated, and if the included angle is greater than a preset value, the two end points of the shared edge are obtained, and each of the two end points is subjected to smoothing processing.

[0054] For all the triangular facets, the included angle of the normal vectors of the triangular facets sharing an edge is determined, and if the included angle is greater than a preset value, it indicates that the angle between the two triangular facets is smaller, Figure 4 The included angle of the normal vectors of the two triangular facets is shown. When the included angle is small, the area formed by the two triangular facets sharing an edge is poor in smoothness, that is, the corners are more prominent. Based on this, when the included angle is greater than a preset value, the two end points of the shared edge are obtained, and the two end points are subjected to smoothing processing. The smoothing processing mode can be various, including but not limited to Laplacian-based smoothing and curvature-based smoothing.

[0055] In order to make the line segment formed by the two end points after smoothing processing closer to the line segment formed by the two end points before smoothing processing, in a specific embodiment, the smoothing processing of each of the two end points is specifically as follows.

[0056] After the two end points are subjected to smoothing processing by using the first smoothing mode, the included angle of the straight line on which the two end points are located and the straight line on which the two end points are located before smoothing processing is taken as a first included angle.

[0057] After the two end points are subjected to smoothing processing by using the second smoothing mode, the included angle of the straight line on which the two end points are located and the straight line on which the two end points are located before smoothing processing is taken as a second included angle.

[0058] The smoothing processing mode corresponding to the minimum value of the first included angle and the second included angle is taken as the smoothing processing mode of the two end points, and each of the two end points is subjected to smoothing processing.

[0059] Preferably, the first smoothing processing mode is Laplacian-based smoothing, and the second smoothing processing mode is curvature-based smoothing.

[0060] In another embodiment, the first endpoint is processed by the first smoothing processing mode and the second smoothing processing mode respectively to obtain two coordinates P11 and P12 corresponding to the first endpoint;

[0061] The second endpoint is processed by the first smoothing processing mode and the second smoothing processing mode respectively to obtain two coordinates P21 and P22 corresponding to the second endpoint.

[0062] The two coordinates P11 and P12 corresponding to the first endpoint and the two coordinates P21 and P22 corresponding to the second endpoint are combined into four straight lines, and the straight line corresponding to the smallest angle between the straight line and the straight line before smoothing processing of the two endpoints is determined, and the coordinates corresponding to the straight line are taken as the smoothing processing result of the first endpoint and the second endpoint.

[0063] Specifically, P11 and P21 form a straight line, P11 and P22 form a straight line, P12 and P21 form a straight line, and P12 and P22 form a straight line. The angles between the four straight lines and the straight line before smoothing processing of the two endpoints are calculated respectively, and the coordinates of the straight line corresponding to the smallest angle are taken as the smoothing processing result. For example, the angle between the straight line formed by P12 and P22 and the straight line before smoothing processing of the two endpoints is the smallest, then P12 is taken as the smoothing processing result of the first endpoint, and P22 is taken as the smoothing processing result of the second endpoint.

[0064] In embodiment two, the present application provides an intestinal three-dimensional model reconstruction system, the system comprises the following modules:

[0065] A voxel model construction module is configured to acquire a photographed intestinal CT image, perform image segmentation on the preprocessed CT image to obtain an intestinal region image corresponding to each CT image, stack the segmented intestinal region images in the order of the CT images, and construct an intestinal three-dimensional voxel model.

[0066] An isosurface fitting mode determination module is configured to acquire intersection points of an isosurface and a voxel edge intersecting with the isosurface in the intestinal three-dimensional voxel model, and calculate vertices of each voxel sharing the voxel edge according to a preset method; if the vertices corresponding to all voxels sharing the voxel edge are in a plane, calculate distances from the intersection points to the plane and areas of a region composed of the vertices, and determine an isosurface fitting mode according to a ratio of the distances to the areas.

[0067] The isosurface extraction and rendering module is configured to fit the isosurface according to the determined isosurface fitting mode, complete the isosurface extraction of the intestinal tract, and render and display the generated three-dimensional surface of the intestinal tract on a display device after smoothing processing.

[0068] Preferably, the isosurface fitting mode is determined according to the ratio of the distance to the area, and specifically is:

[0069] If the ratio is less than a threshold value, an area composed of vertices corresponding to each voxel sharing the voxel edge is divided into two triangular facets.

[0070] Otherwise, the intersection point is taken as a pyramid vertex, and a side of the pyramid is taken as a triangular facet.

[0071] Preferably, the smoothing processing is specifically:

[0072] The included angle of normal vectors of the triangular facets sharing an edge is calculated, and if the included angle is greater than a preset value, two end points of the edge are obtained, and each of the two end points is smoothed.

[0073] Preferably, the smoothing processing of each of the two end points is specifically:

[0074] After the first smoothing mode is used to smooth the two end points, the included angle of a straight line where the two end points are located and a straight line where the two end points are located before the smoothing processing is taken as a first included angle.

[0075] After the second smoothing mode is used to smooth the two end points, the included angle of a straight line where the two end points are located and a straight line where the two end points are located before the smoothing processing is taken as a second included angle.

[0076] The smoothing mode corresponding to the minimum value of the first included angle and the second included angle is taken as the smoothing mode of the two end points, and each of the two end points is smoothed.

[0077] Preferably, the preset method is a Dual Contouring isosurface extraction method, the first smoothing mode is Laplacian-based smoothing, and the second smoothing mode is curvature-based smoothing.

[0078] Finally, the application further provides a computer readable storage medium, and a computer program is stored on the readable storage medium, and the computer program is used to implement the method as described above when executed by a processor.

[0079] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of the necessary universal hardware platform, and of course can also be implemented by means of the combination of hardware and software. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art, can be embodied in the form of a computer product. The present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0080] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and are not limited thereto, and other embodiments can also be used; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of reconstruction of a three-dimensional model of the intestinal tract, characterized in that, The method comprises the following steps: S1, obtaining a photographed intestinal CT image, performing image segmentation on the preprocessed CT image to obtain an intestinal region image corresponding to each CT image, stacking the segmented intestinal region images in the order of the CT images, and constructing an intestinal three-dimensional voxel model; S2, for each voxel edge intersecting with an isosurface in the intestinal three-dimensional voxel model, obtaining an intersection point of the isosurface and the voxel edge, and calculating a vertex of each voxel sharing the voxel edge according to a preset method; if the vertices corresponding to all voxels sharing the voxel edge are in a plane, calculating a distance from the intersection point to the plane and an area formed by all the vertices, and determining an isosurface fitting mode according to a ratio of the distance to the area; S3, fitting the isosurface according to the determined isosurface fitting mode to complete isosurface extraction of the intestine, and rendering and displaying the generated intestinal three-dimensional surface after smoothing processing on a display device. The determination of the isosurface fitting mode according to the ratio of the distance to the area is specifically: if the ratio is less than a threshold value, dividing a region formed by the vertices corresponding to all voxels sharing the voxel edge into two triangular facets; otherwise, taking the intersection point as a pyramid vertex and taking a side of the pyramid as a triangular facet.

2. The method of claim 1, wherein, The smoothing processing is specifically: calculating an included angle of normal vectors of all triangular facets sharing an edge, and if the included angle is greater than a preset value, obtaining two end points of the edge and performing smoothing processing on each of the two end points.

3. The method of claim 2, wherein, The smoothing processing on each of the two end points is specifically: taking an included angle between a straight line where the two end points are located after the smoothing processing in a first smoothing mode and a straight line where the two end points are located before the smoothing processing as a first included angle; taking an included angle between a straight line where the two end points are located after the smoothing processing in a second smoothing mode and a straight line where the two end points are located before the smoothing processing as a second included angle; taking a smoothing processing mode corresponding to a minimum value of the first included angle and the second included angle as a smoothing processing mode of the two end points, and performing smoothing processing on each of the two end points.

4. The method of claim 3, wherein, The preset method is a Dual Contouring isosurface extraction method, the first smoothing mode is Laplacian-based smoothing, and the second smoothing mode is curvature-based smoothing.

5. An intestinal tract three-dimensional model reconstruction system characterized by comprising: The system comprises the following modules: a voxel model construction module, configured to obtain a photographed intestinal CT image, perform image segmentation on the preprocessed CT image to obtain an intestinal region image corresponding to each CT image, stack the segmented intestinal region images in the order of the CT images, and construct an intestinal three-dimensional voxel model; an isosurface fitting mode determination module, configured to, for each voxel edge intersecting with an isosurface in the intestinal three-dimensional voxel model, obtain an intersection point of the isosurface and the voxel edge, and calculate a vertex of each voxel sharing the voxel edge according to a preset method; if the vertices corresponding to all voxels sharing the voxel edge are in a plane, calculate a distance from the intersection point to the plane and an area formed by all the vertices, and determine an isosurface fitting mode according to a ratio of the distance to the area; The isosurface extraction and rendering module is configured to fit the isosurface according to the determined isosurface fitting mode, complete the isosurface extraction of the intestinal tract, and render and display the generated three-dimensional surface of the intestinal tract on a display device after smoothing processing. The isosurface fitting mode is determined according to the ratio of the distance to the area, and specifically, if the ratio is less than a threshold value, an area formed by vertices corresponding to each voxel sharing the voxel edge is divided into two triangular facets; otherwise, the intersection point is taken as a pyramid vertex, and a side of the pyramid is taken as a triangular facet.

6. The system of claim 5, wherein, The smoothing processing is specifically as follows: An included angle of normal vectors of all triangular facets sharing an edge is calculated, and if the included angle is greater than a preset value, two end points of the edge are obtained, and each of the two end points is smoothed.

7. The system of claim 6, wherein, The smoothing processing of each of the two end points is specifically as follows: After the two end points are smoothed by using the first smoothing mode, an included angle between a straight line where the two end points are located and a straight line where the two end points are located before the smoothing processing is taken as a first included angle. After the two end points are smoothed by using the second smoothing mode, an included angle between a straight line where the two end points are located and a straight line where the two end points are located before the smoothing processing is taken as a second included angle. The smoothing processing mode corresponding to a minimum value of the first included angle and the second included angle is taken as a smoothing processing mode of the two end points, and each of the two end points is smoothed by using the smoothing processing mode.

8. A computer program product, characterised in that, The computer program product, when executed by the processor, implements the method of any one of claims 1-4.

Images