A method and system for filling three-dimensional organ with real texture

Abstract

The present invention discloses a method and system for filling three-dimensional organ with real texture, the method comprising: obtaining a three-dimensional model of a target organ, segmenting the surface of the three-dimensional model to form a plurality of surface slices; grouping the plurality of surface slices according to the normal direction, and establishing an ID index for each group; using the current visual direction as the standard direction, judging whether the difference between the normal direction of each group of surface slices and the standard direction meets a threshold, retaining the surface slices that meet the threshold, and hiding the surface slices that do not meet the threshold; performing texture filling on the retained surface slices according to the established ID index; adjusting the three-dimensional model to switch to the next visual direction until all surface slices are filled. The present application improves the fit of texture filling, and displays the folded surfaces and similar surfaces in the texture map separately when filling to avoid mutual interference.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method and system for filling three-dimensional organ real textures. Background Art

[0002] Virtual surgery technology, which combines virtual reality and digital simulation, has been highly recognized by industry personnel for its advantages of repeatability, low risk, and precise modeling. It provides a new technical approach for doctors' surgical skills training, surgical rehearsal, academic seminars, pathological research, etc. The existing methods of drawing human organs can be divided into two aspects: interactive real-time drawing based on volume data and real-time drawing based on surface mesh as part of virtual surgical drawing. Hurrnusiadis et al. used a semi-artificial method to construct virtual organs for cardiology education. This study focused on the performance of cardiac structure and functionality. Whether using three-dimensional organ models for academic education or surgical rehearsal, the surface texture of the constructed three-dimensional organ model needs to be recreated. However, since the organs are irregular structures, the models are relatively complex, and the positions are closely arranged, when filling textures, due to the interference of the surfaces of nearby objects, it often has a great impact on the drawing process, reduces the drawing efficiency, and even produces erroneous drawing results. In addition, existing texture filling methods directly perform texture mapping based on the acquired image; two similar surfaces cannot be distinguished, resulting in mapping failure, or the three-dimensional model formed is inconsistent with the actual situation due to distortion of the mapping angle. Summary of the invention

[0003] Therefore, the purpose of the present invention is to provide a method and system for filling three-dimensional organ real textures, which improves the fit of texture filling and displays folded surfaces and similar surfaces in texture maps separately when filling to avoid mutual interference.

[0004] In order to achieve the above object, the present invention provides a method for filling three-dimensional organ real texture, comprising the following steps:

[0005] S1, obtaining a three-dimensional model of a target organ, and segmenting the surface of the three-dimensional model to form a plurality of surface slices;

[0006] S2, grouping the multiple surface slices according to the normal direction, and establishing an ID index for each group;

[0007] S3, taking the current visual direction as the standard direction, determining whether the difference between the normal direction of each group of surface slices and the standard direction meets a threshold, retaining the surface slices that meet the threshold, and hiding the surface slices that do not meet the threshold;

[0008] S4, performing texture filling on the retained surface slices according to the established ID index;

[0009] S5. Adjust the 3D model to switch to the next visual direction, and repeat steps S3-S4 until all surface slices are filled.

[0010] Further preferably, in S1, the surface of the three-dimensional model is segmented to form a plurality of surface slices, including:

[0011] S101, using a three-dimensional grid to segment the surface of the three-dimensional model;

[0012] S102, in the current visual direction, select a surface slice, take the selected surface slice as the center, select surrounding surface slices according to a preset radius, and calculate the average of the angles between the normal vectors of other surrounding surface slices and the normal vector of the selected surface slice;

[0013] S103. When the mean value of the angle does not exceed the set threshold value, the selected surface slice and the surrounding surface slices are divided into a normally connected domain; according to the set step size, the selected surface slice is switched until all the surface slices can find their unique normally connected domain.

[0014] Further preferably, in S2, the plurality of surface slices are grouped according to the normal direction, and an ID index is established for each group, including:

[0015] Under the current visual direction, the first grouping is performed according to the change gradient of the mean angle of each normally connected domain;

[0016] In the first grouping result, for the normally connected domains with the same change gradient, secondary grouping is performed based on the distance between the filling base point selected by the user and each normally connected domain;

[0017] The step of establishing an ID index for each group includes establishing an ID index based on the initial grouping, and performing a secondary ID index on the normally connected domains with the same change gradient.

[0018] Further preferably, the mean angle of each normally connected domain is ; n is the total number of surface slices in the current normally connected domain;

[0019] in: For the surrounding surface slices, is the central surface slice; j is the serial number of the normally connected domain;

[0020] The gradient of the mean angle ; S j is the area of ​​the jth normally connected domain, and the area of ​​the normally connected domain is calculated according to the number of surface slices included.

[0021] Further preferably, in S3, when judging whether the difference between the normal direction of each group of surface slices and the standard direction meets the threshold, the following method is used:

[0022] Get the normal direction of the normally connected domain where any surface slice picked by the user is located. If the angle between the normal direction of the current normally connected domain and the standard direction is 0-90°, then according to the primary ID identification, retain the current normally connected domain and other normally connected domains in its group;

[0023] If the angle between the normal direction of the current normally connected domain and the standard direction is 90°-180°, the current normally connected domain and other normally connected domains in its group are hidden;

[0024] For the retained normally connected domains, the filling base point selected by the user is used as the origin, and the normally connected domains are sorted from near to far according to the distance between them and the origin in the secondary ID identification, and are displayed at intervals in the order of arrangement.

[0025] Further preferably, when acquiring texture data of the target organ, texture data in each direction is acquired according to the viewing directions of front view, rear view, left view, right view, top view, top view and 45° orthogonal view, and the acquired texture data is marked according to the viewing direction.

[0026] Further preferably, in S4, the step of performing texture filling on the retained surface slices includes:

[0027] The user's operation angle in the current visual direction is obtained, and the texture map in the closest viewing direction is selected according to the operation angle. The selected texture map is projected in the retained normal connected domain in the display order.

[0028] Further preferably, in S4, the step of performing texture filling on the retained surface slices includes:

[0029] The edge points of each group of normally connected domains are identified, and the feature points of the edges of two adjacent groups of normally connected domains are identified, and the identified feature points are marked as stitching points;

[0030] Subtract half of the current pixel value from the pixel value at the identified stitch point.

[0031] The present invention also provides a three-dimensional organ real texture filling system, which is used to implement the three-dimensional organ real texture filling method, comprising:

[0032] A data acquisition module acquires a three-dimensional model of a target organ and slices the surface of the three-dimensional model to form a plurality of surface slices;

[0033] The data processing module groups multiple surface slices according to the normal direction and establishes an ID index for each group; taking the current visual direction as the standard direction, it is determined whether the difference between the normal direction of each group of surface slices and the standard direction meets the threshold, retaining the surface slices that meet the threshold, and hiding the surface slices that do not meet the threshold;

[0034] The image filling module performs texture filling on the retained surface slices according to the established ID index, adjusts the 3D model to switch to the next visual direction, and repeats the above process until all surface slices are filled.

[0035] Compared with the prior art, the three-dimensional organ real texture filling method and system disclosed in this application have at least the following advantages:

[0036] 1. The real texture filling method provided by the present application generates surface slices using a three-dimensional model, and divides the normal connected domain according to the normal angle of each surface slice, thereby reducing the amount of calculation. At the same time, the grouping of the connected domain is used to realize the division of wrinkle surfaces, folded surfaces and adjacent surfaces in the model, thereby avoiding interference from other surfaces when filling texture on a certain surface;

[0037] 2. In the present application, feature points of the edge of the normally connected domain are identified and pixels are halved when filling, so as to avoid the phenomenon that when adjacent normally connected domains are filled, the edge is enhanced, resulting in distorted filling effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the method for filling three-dimensional organ real texture according to the present invention;

[0039] Figure 2 A flow chart for segmenting the surface of a three-dimensional model provided by the present invention. DETAILED DESCRIPTION

[0040] The present invention is further described in detail below through the accompanying drawings and specific embodiments.

[0041] like Figure 1 As shown, a method for filling three-dimensional organ real textures provided by an embodiment of one aspect of the present invention comprises the following steps:

[0042] S1, obtaining a three-dimensional model of a target organ, and dividing the surface of the three-dimensional model to form a plurality of surface slices; further, in S1, dividing the surface of the three-dimensional model to form a plurality of surface slices such as Figure 2 As shown, including:

[0043] S101, using a three-dimensional grid to segment the surface of the three-dimensional model;

[0044] S102, in the current visual direction, select a surface slice, take the selected surface slice as the center, select surrounding surface slices according to a preset radius, and calculate the average of the angles between the normal vectors of other surrounding surface slices and the normal vector of the selected surface slice;

[0045] S103. When the mean value of the angle does not exceed the set threshold value, the selected surface slice and the surrounding surface slices are divided into a normally connected domain; according to the set step size, the selected surface slice is switched until all the surface slices can find their unique normally connected domain.

[0046] It should be noted that the preset radius is not a circle radius in a strict sense, but rather extends outward in a grid form with the selected surface slice as the center, and the number of surface slices included in each normally connected domain is an integer.

[0047] It should be noted that when the present application determines the normal and visual direction, it is established based on the world coordinates of the three-dimensional model. When the visual direction changes, the normal direction of the selected surface slice changes accordingly. This transformation process can be obtained based on the affine matrix calculation. The affine matrix calculation belongs to the prior art and will not be repeated here.

[0048] S2, grouping the multiple surface slices according to the normal direction, and establishing an ID index for each group;

[0049] Further preferably, in S2, the step of grouping the plurality of surface slices according to the normal direction and establishing an ID index for each group includes:

[0050] In the current visual direction, the first grouping is performed according to the change gradient of the mean angle of each normally connected domain; the establishment of an ID index for each group includes establishing an ID identification according to the first grouping, and performing a second ID identification on the normally connected domains with the same change gradient.

[0051] Further preferably, the mean angle of each normally connected domain is ;

[0052] in: For the surrounding surface slices, is the central surface slice; j is the serial number of the normally connected domain; n is the total number of surface slices in the current normally connected domain.

[0053] The gradient of the mean angle ; S j is the area of ​​the jth normally connected domain, and the area of ​​the normally connected domain is calculated according to the number of surface slices included.

[0054] In the first grouping results, for the normally connected domains with the same change gradient, secondary grouping is performed based on the distance between the filling base point selected by the user and each normally connected domain. In the first grouping, the connected domains with a normal direction close to the selected area are retained. When there are two or more parallel faces with the same normal direction, re-grouping is required. When re-grouping, the domains are arranged according to the Euler distance between the filling base point selected by the user and the central surface slice of each normally connected domain, that is, the closer the distance to the filling base point, the higher the ranking in the sorting list, and the farther the distance, the lower the ranking.

[0055] S3, taking the current visual direction as the standard direction, determining whether the difference between the normal direction of each group of surface slices and the standard direction meets a threshold, retaining the surface slices that meet the threshold, and hiding the surface slices that do not meet the threshold;

[0056] Further preferably, in S3, when judging whether the difference between the normal direction of each group of surface slices and the standard direction meets the threshold, the following method is used:

[0057] Get the normal direction of the normally connected domain where any surface slice picked by the user is located. If the angle between the normal direction of the current normally connected domain and the standard direction is 0-90°, then according to the primary ID identification, retain the current normally connected domain and other normally connected domains in its group;

[0058] If the angle between the normal direction of the current normally connected domain and the standard direction is 90°-180°, the current normally connected domain and other normally connected domains in its group are hidden;

[0059] For the retained normally connected domains, the filling base point selected by the user is used as the origin, and the normally connected domains are sorted from near to far according to the distance between them and the origin in the secondary ID identification, and are displayed at intervals in the order of arrangement.

[0060] In order to reduce the amount of calculation, the number of normally connected domains to be displayed can be set each time the visual direction is changed; after the first grouping, the normally connected domains with similar change gradients are retained in one group; the normally connected domains with smaller angles with the visual direction are more likely to be retained; when filling display is required, the normally connected domains are displayed at intervals according to the ranking order. For example, the first normally connected domain that is closer to the filling base point is displayed first, and then the third or fourth normally connected domain is displayed after 2 or 3 positions. This operation is to avoid interference between normally connected domains that are too close when filling, resulting in unrealistic filling colors.

[0061] S4, performing texture filling on the retained surface slices according to the established ID index;

[0062] Further preferably, when acquiring texture data of the target organ, texture data in each direction is acquired according to the viewing directions of front view, rear view, left view, right view, top view, top view and 45° orthogonal view, and the acquired texture data is marked according to the viewing direction.

[0063] Further preferably, in S4, the step of performing texture filling on the retained surface slices includes:

[0064] The user's operation angle in the current visual direction is obtained, and the texture map in the closest viewing direction is selected according to the operation angle. The selected texture map is projected in the retained normal connected domain in the display order.

[0065] Further preferably, in S4, the step of performing texture filling on the retained surface slices includes:

[0066] The edge points of each group of normally connected domains are identified, and the feature points of the edges of two adjacent groups of normally connected domains are identified, and the identified feature points are marked as stitching points;

[0067] Subtract half of the current pixel value from the pixel value at the identified stitching point. By identifying the feature points of the edge of the normally connected domain and halving the pixels when filling, it is possible to avoid the edge enhancement when filling adjacent normally connected domains, resulting in distorted filling effects.

[0068] S5. Adjust the 3D model to switch to the next visual direction, and repeat steps S3-S4 until all surface slices are filled.

[0069] It should be noted that the normal direction angles between the surface slices established using the original three-dimensional model are relatively fixed and will not change with the visual angle; therefore, when the visual direction selected by the user is close to the orthographic angle when the three-dimensional model is modeled, the normal direction of the surface slice where the filling base point is located is subjected to the same affine transformation based on the filling base point selected by the user, and the normal direction of each surface slice under the visual direction can be obtained, and the normal direction of the normally connected domain can also be determined accordingly.

[0070] When filling texture images of the intestines, esophagus or other organs, since there are many wrinkles and multiple tortuous surfaces in the three-dimensional model, the method provided in the present application can be used to fill the texture of part of the surface according to the visual direction, and then the visual direction is changed to fill the texture of other surfaces. For example, when the organ is the intestine, the normally connected domain with a similar filling base point selected in the visual direction is first filled. Due to the wrinkles of the intestine, there must be an area that is folded over, and at this time, the area is hidden because the angle of the normally connected domain exceeds 90 degrees. When rotating to the opposite direction of the current viewing angle, this area is displayed and the previously filled area is hidden. In this way, when filling, the grouping of connected domains is used to realize the division of wrinkled surfaces, folded surfaces and adjacent surfaces in the model.

[0071] The present invention also provides a three-dimensional organ real texture filling system, which is used to implement the three-dimensional organ real texture filling method, comprising:

[0072] A data acquisition module acquires a three-dimensional model of a target organ and slices the surface of the three-dimensional model to form a plurality of surface slices;

[0073] The data processing module groups multiple surface slices according to the normal direction and establishes an ID index for each group; taking the current visual direction as the standard direction, it is determined whether the difference between the normal direction of each group of surface slices and the standard direction meets the threshold, retaining the surface slices that meet the threshold, and hiding the surface slices that do not meet the threshold;

[0074] The image filling module performs texture filling on the retained surface slices according to the established ID index, adjusts the 3D model to switch to the next visual direction, and repeats the above process until all surface slices are filled.

[0075] Obviously, the above embodiments are only examples for clear explanation, and are not intended to limit the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.

Claims

1. A method for filling three-dimensional organ real texture, characterized in that: The following steps are involved: S1, obtaining a three-dimensional model of a target organ, and segmenting the surface of the three-dimensional model to form a plurality of surface slices; S2. Grouping multiple surface slices according to the normal direction and establishing an ID index for each group; including: In the current visual direction, the first grouping is performed according to the change gradient of the mean angle of each normally connected domain; the mean angle of each normally connected domain ; in: For the surrounding surface slices, is the central surface slice; j is the serial number of the normally connected domain; n is the total number of surface slices in the current normally connected domain; The gradient of the mean angle ; S j is the area of ​​the jth normally connected domain, where the area of ​​the normally connected domain is calculated based on the number of surface slices included; In the first grouping result, for the normally connected domains with the same change gradient, secondary grouping is performed based on the distance between the filling base point selected by the user and each normally connected domain; The step of establishing an ID index for each group includes establishing an ID index based on the first grouping, and performing a secondary ID index on the normally connected domains with the same change gradient; S3, taking the current visual direction as the standard direction, determining whether the difference between the normal direction of each group of surface slices and the standard direction meets a threshold, retaining the surface slices that meet the threshold, and hiding the surface slices that do not meet the threshold; S4, performing texture filling on the retained surface slices according to the established ID index; S5. Adjust the 3D model to switch to the next visual direction, and repeat steps S3-S4 until all surface slices are filled.

2. The three-dimensional organ real texture filling method according to claim 1, characterized in that: In S1, the surface of the three-dimensional model is segmented to form multiple surface slices, including: S101, using a three-dimensional grid to segment the surface of the three-dimensional model; S102, in the current visual direction, select a surface slice, take the selected surface slice as the center, select surrounding surface slices according to a preset radius, and calculate the average of the angles between the normal vectors of other surrounding surface slices and the normal vector of the selected surface slice; S103. When the mean value of the angle does not exceed the set threshold value, the selected surface slice and the surrounding surface slices are divided into a normally connected domain; according to the set step size, the selected surface slice is switched until all the surface slices can find their unique normally connected domain.

3. The method for filling three-dimensional organ real texture according to claim 1, characterized in that: In S3, when judging whether the difference between the normal direction of each group of surface slices and the standard direction meets the threshold, the following method is used: Get the normal direction of the normally connected domain where any surface slice picked by the user is located. If the angle between the normal direction of the current normally connected domain and the standard direction is 0-90°, then according to the primary ID identification, retain the current normally connected domain and other normally connected domains in its group; If the angle between the normal direction of the current normally connected domain and the standard direction is 90°-180°, the current normally connected domain and other normally connected domains in its group are hidden; For the retained normally connected domains, the filling base point selected by the user is used as the origin, and the normally connected domains are sorted from near to far according to the distance between them and the origin in the secondary ID identification, and are displayed at intervals in the order of arrangement.

4. The method for filling three-dimensional organ real texture according to claim 1, characterized in that: When acquiring texture data of the target organ, texture data in each direction is acquired according to the viewing directions of the front view, rear view, left view, right view, top view, top view and 45° orthogonal view, and the acquired texture data is marked according to the viewing direction.

5. The three-dimensional organ real texture filling method according to claim 1, characterized in that: In S4, the texture filling of the retained surface slices includes: The user's operation angle in the current visual direction is obtained, and the texture map in the closest viewing direction is selected according to the operation angle. The selected texture map is projected in the retained normal connected domain in the display order.

6. The method for filling three-dimensional organ real texture according to claim 5, characterized in that: In S4, the texture filling of the retained surface slices includes: The edge points of each group of normally connected domains are identified, and the feature points of the edges of two adjacent groups of normally connected domains are identified, and the identified feature points are marked as stitching points; Subtract half of the current pixel value from the pixel value at the identified stitch point.

7. A three-dimensional organ real texture filling system, characterized in that: The method for implementing the three-dimensional organ real texture filling method according to any one of claims 1 to 6 comprises: A data acquisition module acquires a three-dimensional model of a target organ and slices the surface of the three-dimensional model to form a plurality of surface slices; The data processing module groups multiple surface slices according to the normal direction and establishes an ID index for each group; taking the current visual direction as the standard direction, it is determined whether the difference between the normal direction of each group of surface slices and the standard direction meets the threshold, retaining the surface slices that meet the threshold, and hiding the surface slices that do not meet the threshold; The image filling module performs texture filling on the retained surface slices according to the established ID index, adjusts the 3D model to switch to the next visual direction, and repeats the above process until all surface slices are filled.

Images