Image processing methods and electronic devices

Abstract

This application discloses an image processing method and an electronic device. The method includes: acquiring an original image of a target object, the original image including a main body and branches of the target object; segmenting the main body and branches of the target object based on iterative image segmentation to form multiple corresponding point elements; extracting the multiple point elements to form a set of point elements corresponding to the main body and a set of point elements corresponding to the branches, the set of point elements representing the skeleton structure of the main body and branches; performing three-dimensional reconstruction on the set of point elements to form a tubular topology and an ellipsoidal topology, the tubular topology corresponding to the main body and the ellipsoidal topology corresponding to the branches; and merging the tubular topology and the ellipsoidal topology to form a three-dimensional model of the target object.

Description

Technical Field

[0001] This application relates to the field of medical image processing, and in particular to an image processing method and electronic device. Background Technology

[0002] In lung tuberculosis surgery, it is often necessary to assess the patient's bronchial and vascular physiological structure to determine the surgical approach and entry point. This is typically achieved using lung CT images, combined with medical imaging tools to segment the bronchi and vessels layer by layer, thereby reconstructing their three-dimensional structure. However, due to the influence of patient respiration, CT equipment, and CT reconstruction, CT images often only show single or a few layers of bronchial and vascular structures, resulting in inconsistent bronchial and vascular morphologies that do not accurately reflect the patient's actual physiological structure. Therefore, effectively reconstructing the structures in medical images, especially the three-dimensional tubular structures of the lungs, is a pressing issue. Current techniques typically use the Marching Cube algorithm for three-dimensional reconstruction of medical images, generating a 3D model based on layer-by-layer annotations of CT images. To obtain a smooth surface, surface smoothing methods such as Laplacian and Taubin are often used. However, this method heavily relies on the CT image segmentation results. When the layers are thick or the segmentation results are poor, the reconstruction results are often suboptimal, exhibiting jagged, flat, or wavy patterns that do not conform to physiological structures, making it unsuitable for direct medical evaluation. Summary of the Invention

[0003] To address the aforementioned technical problems, this application provides an image processing method and an electronic device. This method improves upon the design of the three-dimensional reconstruction method, enabling the acquisition of high-precision reconstructions of tubular structures, bronchial tracts, and pulmonary arteries and veins in lung CT images. It employs a dense point cloud approach to form a complete mesh structure, thereby generating an initial tubular three-dimensional model.

[0004] An embodiment of this application provides an image processing method, the method comprising:

[0005] Obtain the original image of the target object, wherein the original image includes the main body and branches of the target object;

[0006] Based on iterative image segmentation, the main body and branch parts of the target object are segmented to form multiple corresponding point elements.

[0007] Extract the plurality of point elements to form a set of point elements corresponding to the main trunk and a set of point elements corresponding to the branch, wherein the set of point elements represents the skeleton structure of the main trunk and the branch;

[0008] The set of point elements is reconstructed in three dimensions to form a tubular topology and an ellipsoidal topology, wherein the tubular topology corresponds to the trunk and the ellipsoidal topology corresponds to the branch.

[0009] The tubular topology and the ellipsoidal topology are merged to form a three-dimensional model of the target object.

[0010] Preferably, the method further includes:

[0011] After obtaining the set of point elements, the set of point elements is refined so that the trunk and the branch parts represented by the set of point elements have a clear skeletal structure.

[0012] Preferably, the refinement process of the point element set includes:

[0013] The set of point elements is transformed into a directed graph structure with a tree structure, wherein the directed graph structure includes root node elements, transition node elements, and leaf node elements.

[0014] Preferably, the refinement process for the set of point elements further includes:

[0015] The positions of the set of point elements are corrected;

[0016] Based on the normal vector direction of the point element, obtain the cross section of the point element;

[0017] The position of the point element is corrected based on the cross-section of the point element;

[0018] Based on the roundness of the point element, the radius of the point element is obtained, wherein the radius of the point element characterizes the thickness of the point element.

[0019] Preferably, the refinement process for the set of point elements further includes:

[0020] If it is determined that the set of point elements includes abnormal skeleton structures, the set of point elements is pruned to remove the abnormal skeleton structures, wherein the abnormal skeleton structures are characterized as protrusions, burrs, and small forks of the target object.

[0021] Preferably, the step of performing three-dimensional reconstruction of the set of point elements to form a tubular topological structure and an ellipsoidal topological structure includes:

[0022] Calculate the normal vector of the point element based on the set of point elements corresponding to the main part;

[0023] Based on the normal vectors of the point elements, generate the corresponding planar circular points;

[0024] Connect the planar circular points corresponding to each main section to form the tubular topology.

[0025] Preferably, the step of performing three-dimensional reconstruction of the point element set to form a tubular topological structure and an ellipsoidal topological structure further includes:

[0026] Based on the set of point elements corresponding to the branch, obtain the normal vector of the point element;

[0027] Based on the normal vectors of the point elements, the corresponding ellipsoid points are generated;

[0028] Connect the ellipsoidal points corresponding to each branch to form the ellipsoidal topology.

[0029] Preferably, the method further includes:

[0030] Based on the Loop subdivision method, a web-like structure is formed between the main body and the branch parts of the 3D model to refine the 3D model. The web-like structure is a smooth transition connection structure.

[0031] The purpose of this application is to provide an electronic device, including:

[0032] The acquisition module is configured to acquire the original image of the target object, wherein the original image includes the main body and branches of the target object;

[0033] The segmentation module is configured to segment the main body and branches of the target object based on an iterative image segmentation method, forming multiple corresponding point elements;

[0034] The extraction module is configured to extract the plurality of point elements to form a set of point elements corresponding to the main trunk and a set of point elements corresponding to the branch, wherein the set of point elements represents the skeleton structure of the main trunk and the branch.

[0035] The reconstruction module is configured to perform three-dimensional reconstruction on the set of point elements to form a tubular topology and an ellipsoidal topology, wherein the tubular topology corresponds to the trunk portion and the ellipsoidal topology corresponds to the branch portion.

[0036] The merging module is configured to merge the tubular topology and the ellipsoidal topology to form a three-dimensional model of the target object.

[0037] The purpose of this application is to provide an electronic device, including a memory and a processor, wherein the memory stores an executable program, and the processor executes the executable program to implement the steps of the above method.

[0038] Compared with existing technologies, the beneficial effects of the embodiments of this application are as follows: With the assistance of CT images, this application can achieve high-precision reconstruction of tubular structures, bronchioles, and pulmonary arteries and veins in lung CT images, restoring the true physiological structure and providing sufficient and useful information for patient diagnosis and surgery. Based on traditional medical 3D reconstruction, this application designs and improves the 3D reconstruction method, integrating a skeleton representation method and using roundness detection to correct the skeleton structure, thereby ensuring the skeleton is distributed at the center of the original annotation results, enhancing its ability to abstract segmented data, which is beneficial for subsequent tubularization. To obtain high-precision reconstruction results, a dense point cloud approach is adopted, using smaller skeleton points to generate circular position coordinates point by point, and using cylindrical topology to construct the topological structure between point clouds, thereby forming a complete mesh structure and thus forming an initial tubular 3D model. Attached Figure Description

[0039] Figure 1 This is a flowchart of an image processing method according to an embodiment of this application;

[0040] Figure 2 This is a flowchart of one embodiment of step S400 of this application;

[0041] Figure 3 This is a flowchart of another embodiment of step S400 of this application;

[0042] Figure 4 A schematic diagram comparing traditional 3D reconstruction with Mesh-optimized tubular 3D reconstruction according to embodiments of this application;

[0043] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application;

[0044] Figure 6 This is a structural block diagram of another electronic device according to an embodiment of this application. Detailed Implementation

[0045] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0046] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0047] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0048] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0049] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.

[0050] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0051] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0052] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0053] like Figure 1 As shown in the figure, this application provides an image processing method, the method comprising:

[0054] S100, Obtain the original image of the target object;

[0055] In this embodiment, firstly, the original image of the target object is acquired. The original image includes the main body and branches of the target object. Specifically, the target object can be a diseased organ of a patient, such as the lungs; the original image can be a medical image of the diseased organ, such as a CT image; the main body of the target object included in the original image can be the main parts of the lung, such as the bronchi, pulmonary veins, pulmonary arteries, and blood vessels, and the branches can be extensions of the main parts of the lung, such as the bifurcations of blood vessels. This embodiment utilizes CT imaging technology to acquire the original image of the target object. The original image skeleton is extracted from the layer-by-layer segmentation results on the original CT image. For the input layer-by-layer segmentation data of the CT image, its dimensions are generally [D, H, W], where D represents the image depth (i.e., the number of CT images), H represents the height of a single layer, and W represents the width of a single layer. This embodiment uses a skeleton module to abstractly represent the segmentation data of the original image.

[0056] S200, based on the iterative image segmentation method, the main body and branch parts of the target object are segmented to form multiple corresponding point elements;

[0057] In this embodiment, after acquiring the original image including the main body and branches of the target object, the main body and branches of the target object are segmented using an iterative image segmentation method, such as an iterative algorithm, to form multiple corresponding point elements. These point elements represent the pixels in the segmented original image of the target object. Specifically, this embodiment can use a 3D central axis thinning method, continuously eroding and thinning the segmented original image of the target object through multiple iterations, ultimately ensuring that the original image of the target object retains only a single pixel connected component, thereby completing the extraction of the initial skeleton from the original image of the target object.

[0058] S300, extract the plurality of point elements to form a set of point elements corresponding to the main trunk and a set of point elements corresponding to the branch;

[0059] In this embodiment, after segmenting the main body and branches of the original image of the target object to form multiple point elements, these point elements are extracted to form a set of point elements corresponding to the main body of the target object and a set of point elements corresponding to the branches of the target object. The set of point elements represents the skeletal structure of the main body and the branches. Specifically, by extracting pixels from the original image of the target object, a set of pixels corresponding to the main body of the target object and a set of pixels corresponding to the branches of the target object can be formed.

[0060] S400, Perform three-dimensional reconstruction on the set of point elements to form tubular and ellipsoidal topological structures;

[0061] In this embodiment, after forming the point element sets corresponding to the main trunk and the branch trunk, three-dimensional reconstruction can be performed on these sets to form tubular and ellipsoidal topologies. The tubular topology corresponds to the main trunk, and the ellipsoidal topology corresponds to the branch trunk. Specifically, since the point element sets represent the skeleton structure of the main trunk and the branch trunk, the tubularization module can generate a three-dimensional model from one-dimensional skeleton points based on the skeleton structure. This model contains a complete point cloud and corresponding topological structures. Considering the physiological structure of the lung bronchi and blood vessels, smooth tubes and terminal ellipsoids are needed for simulation, and finally, they are combined. Therefore, the tubularization module can perform three-dimensional reconstruction on the point element sets to form tubular and ellipsoidal topologies. For example, the tubular topology can be a tubular mesh structure, and the ellipsoidal topology can be an ellipsoidal mesh structure. The tubular mesh structure corresponds to the main trunk, and the ellipsoidal mesh structure corresponds to the branch trunk.

[0062] S500, the tubular topology and the ellipsoidal topology are merged to form a three-dimensional model of the target object.

[0063] In this embodiment, after forming the tubular and ellipsoidal topologies, they are merged to form a 3D model of the target object. Specifically, since the tubular and ellipsoidal topologies are generated separately, no actual topological structure is formed at the bifurcation and end connections of the target object's skeleton. Therefore, Boolean operations are used to merge the tubular and ellipsoidal topologies, while removing some intersecting 3D point and surface structures, ultimately forming a single 3D model with a complete topological structure. This completes the initial 3D model generation.

[0064] This application, with the aid of CT imaging, enables high-precision reconstruction of tubular structures, bronchioles, and pulmonary arteries and veins in lung CT images, restoring the true physiological structure and providing sufficient and useful information for patient diagnosis and surgery. Based on traditional medical 3D reconstruction, this application designs and improves the 3D reconstruction method, integrating skeleton representation methods and combining the reconstruction results from the Marching Cube method. Roundness detection is used to correct the skeleton structure, ensuring that the skeleton is centered in the original annotation results, enhancing its ability to abstract segmented data and facilitating subsequent tubularization. To obtain high-precision reconstruction results, a dense point cloud approach is adopted. Small skeleton points are used to generate circular position coordinates point by point, and a cylindrical topology is used to construct the topology between point clouds. Boolean operations are then used to merge the pipes, forming a complete mesh structure, thus creating the initial tubular 3D model.

[0065] In one embodiment of this application, the method further includes:

[0066] After obtaining the set of point elements, the set of point elements is refined so that the trunk and the branch parts represented by the set of point elements have a clear skeletal structure.

[0067] In this embodiment, after forming the point element set corresponding to the trunk part of the target object and the point element set corresponding to the branch part of the target object, the point element set is refined to ensure that the trunk part and the branch part represented by the point element set have a clear skeleton structure. Specifically, the refinement of the point element set can be combined with skeleton representation methods, using roundness detection to correct the skeleton, thereby distributing the skeleton in the center of the original annotation result, improving its ability to abstract the segmented data, and facilitating subsequent tubularization. To better manage and identify the skeleton, it is also necessary to convert the skeleton into a directed graph structure with parent-child relationships, and perform fine-tuning methods such as outlier detection, correction, and pruning.

[0068] In one embodiment of this application, the refinement of the point element set includes:

[0069] The set of point elements is transformed into a directed graph structure with a tree structure, wherein the directed graph structure includes root node elements, transition node elements, and leaf node elements.

[0070] In this embodiment, the main body and branches of the original image of the target object are segmented to form multiple point elements. These point elements are then extracted to form a set of point elements corresponding to the main body and a set of point elements corresponding to the branches of the target object. In other words, after extracting the initial skeleton from the original image of the target object, the set of point elements is transformed into a directed graph structure with a tree structure. This directed graph structure includes root node elements, transition node elements, and leaf node elements. Specifically, after extracting the initial skeleton of the target object, it needs to be transformed into a directed graph structure with parent-child node relationships. Points along the skeleton path are distinguished and encoded, and root nodes, transition nodes, and leaf nodes are defined for optimization in subsequent modules.

[0071] In one embodiment of this application, the refinement of the point element set further includes:

[0072] The positions of the set of point elements are corrected;

[0073] Based on the normal vector direction of the point element, obtain the cross section of the point element;

[0074] The position of the point element is corrected based on the cross-section of the point element;

[0075] Based on the roundness of the point element, the radius of the point element is obtained, wherein the radius of the point element characterizes the thickness of the point element.

[0076] In this embodiment, the point element set is refined to precisely adjust the skeleton structure of the main and branch parts, ensuring that the skeleton structure of the main and branch parts can represent the segmentation information of the original image to the greatest extent. Specifically, for the initial skeleton structure, due to the limitations of layer thickness and orientation of the layer-by-layer segmentation data, the skeleton structure distribution is often not at the exact center of the 3D model. To better restore the realistic 3D reconstruction results, the position of the skeleton structure needs to be corrected. This embodiment first corrects the position of the point element set; based on the normal vector direction of the point element, the cross-section of the point element is obtained; then the position of the point element is corrected; based on the roundness of the point element, the radius of the point element is obtained, and the radius of the point element represents the thickness of the point element. Specifically, this embodiment combines traditional reconstruction methods to obtain the initial model, then obtains its cross-section along the normal vector direction of the skeleton points, and then uses the cross-section to correct the position of the skeleton points and obtains their radius based on their roundness. Thus, a better and more uniform skeleton point representation can be obtained.

[0077] In one embodiment of this application, the refinement of the point element set further includes:

[0078] If it is determined that the set of point elements includes abnormal skeleton structures, the set of point elements is pruned to remove the abnormal skeleton structures, wherein the abnormal skeleton structures are characterized as protrusions, burrs, and small forks of the target object.

[0079] In this embodiment, the refinement of the point element set also includes pruning the point element set. Since the skeleton input source is CT layer-by-layer segmentation data, it is easily affected by errors such as input protrusions and small forks, thus requiring pruning. When it is determined that the point element set includes abnormal skeleton structures, the point element set is pruned to remove these abnormal skeleton structures. Abnormal skeleton structures are characterized by protrusions, small burrs, and small forks in the target object. For example, in this embodiment, by identifying the skeleton structure of the bronchi and blood vessels in lung CT scans, shorter burrs and abnormal skeleton distributions are removed, thereby completing the pruning of the skeleton structure and avoiding redundant and invalid branches generated during subsequent tubularization.

[0080] In one embodiment of this application, such as Figure 2 As shown, the three-dimensional reconstruction of the point element set to form tubular and ellipsoidal topological structures includes:

[0081] S410, Calculate the normal vector of the point element based on the set of point elements corresponding to the main part;

[0082] S420, Based on the normal vector of the point element, generate the corresponding planar circular point;

[0083] S430, connect the planar circular points corresponding to each main part to form the tubular topology.

[0084] In this embodiment, after obtaining the precisely adjusted set of point elements, i.e., the precisely adjusted skeleton structure of the main body and branches, the skeleton structure of the main body and branches is reconstructed in three dimensions using a tubularization module to form a tubular network mesh structure. First, the normal vector of the point element is calculated based on the set of point elements corresponding to the main body; then, based on the normal vector of the point element, the corresponding planar circular points are generated; then, the planar circular points corresponding to each main body are connected to form a tubular topology. Specifically, based on the finely adjusted skeleton structure, a three-dimensional model is generated from the one-dimensional skeleton points, i.e., a three-dimensional model containing a complete point cloud and corresponding topology. Combining the physiological structure of the lung bronchi and blood vessels, smooth tubes and terminal ellipsoids are needed for simulation. Finally, they are merged together. First, a tubular mesh structure needs to be formed. For example, based on a given skeleton structure, the normal vectors of the skeleton points are first calculated. Planar ring points are generated based on the direction. It should be noted that, considering memory size, the resolution of the ring points is set to 20, that is, a point is taken along the ring at 13° as the point cloud of the ring. Along the trend of the skeleton points, surface information is generated between the two ring surfaces of adjacent skeleton points. The corresponding ring points are connected to form a tubular topology. After traversing all the skeleton points, a basic tubular mesh structure can be formed.

[0085] In one embodiment of this application, such as Figure 3 As shown, the step of performing three-dimensional reconstruction of the point element set to form tubular and ellipsoidal topological structures further includes:

[0086] S440, Based on the set of point elements corresponding to the branch portion, obtain the normal vector of the point element;

[0087] S450, Based on the normal vector of the point element, generate the corresponding ellipsoid point;

[0088] S460, connect the ellipsoidal points corresponding to each branch to form the ellipsoidal topology.

[0089] In this embodiment, after obtaining the precisely adjusted set of point elements, i.e., the precisely adjusted skeleton structure of the main body and branches, the skeleton structure of the branches is reconstructed in three dimensions using a tubularization module to form an ellipsoidal network mesh structure. First, the normal vectors of the point elements are obtained based on the set of point elements corresponding to the branches; then, based on the normal vectors of the point elements, corresponding ellipsoidal points are generated; next, the ellipsoidal points corresponding to each branch are connected to form an ellipsoidal topology. Specifically, combined with the leaf nodes of the skeleton structure, the normal vectors are retained at the ends of the tubules. To better restore the growth characteristics and smoothness of the lung tubular structure, ellipsoids are used at the ends along the direction of the tubes to simulate the end growth state of the bronchi and blood vessels. It should be noted that, considering the memory size, the ellipsoid resolution is also set to 20, and semi-circular point clouds are generated every 13° along the surface of the ellipsoid from the top and bottom vertices. Surface element information is generated between two adjacent semi-circular point clouds, and the corresponding ellipsoidal points are connected to form an ellipsoidal topology.

[0090] In one embodiment of this application, the method further includes:

[0091] Based on the Loop subdivision method, a web-like structure is formed between the main body and the branch parts of the 3D model to refine the 3D model. The web-like structure is a smooth transition connection structure.

[0092] In this embodiment, after forming a tubular topology in the main body and an ellipsoidal topology in the branch parts of the target object, a web-like structure is formed between the main body and branch parts of the 3D model using a Loop subdivision method to refine the 3D model. The web-like structure is a smooth transition connection structure. Specifically, in this embodiment, to better simulate the morphological growth of bronchi and blood vessels, a web-like structure needs to be formed at the bifurcation points of the tubes, i.e., a smooth transition connection. This embodiment uses the Loop subdivision method to smooth the mesh structure after merging the main bodies. By adjusting appropriate parameters, a more natural web-like transition structure can be formed at the connection points, thereby completing the final optimized 3D reconstruction of the tubular mesh.

[0093] In some embodiments, after obtaining the initial tubular 3D model, to ensure a natural transition at the pipe connections and a high-precision simulation of the main trunk of the tubular structure, the main trunk can be merged using a tubularization module. For tubular structures in lung CT scans, the main body often exhibits non-tubular features, such as the heart connected by blood vessels, or the main trachea connected by bronchi. Therefore, to accurately recreate their physiological structure, the main trunk needs to be preserved separately and merged into the tubular model. This embodiment uses a traditional reconstruction method for the main trunk, followed by mesh merging to connect with the tubular model, preserving the original main trunk morphology.

[0094] In some embodiments, Boolean operations are used to merge the mesh structures formed at the bifurcation points of the skeleton to remove self-intersections of the mesh structures and form the topological structure at the connection points. After the initial tubularization is formed, the Loop subdivision algorithm is applied to smooth the mesh, making its shape smoother and more natural, conforming to the growth law of physiological structure. The three-dimensional reconstruction result formed by the above embodiments has tubular characteristics and can simulate the physiological structure of the three-dimensional tubular structure of lung CT images, such as the segmentation results of bronchi, arteries and veins, to perform tubular three-dimensional reconstruction.

[0095] This application extracts the skeleton structure from the layer-by-layer segmentation results on the original CT scan, and then refines the skeleton through skeleton center correction, roundness detection, radius extraction, and pruning to maximize its representation of the original segmentation information. The root nodes of the skeleton are then extracted and transformed into a tree structure. Finally, a tubular network mesh structure is generated based on the skeleton's center point. Boolean operations are used to merge the meshes formed at the skeleton's bifurcation points, removing self-intersections and creating connection topologies. After this initial tubularization, a Loop subdivision algorithm is applied to smooth the mesh, making its shape smoother and more natural, conforming to the growth patterns of physiological structures. Through the above embodiments, the resulting 3D reconstruction has tubular characteristics and can simulate the physiological structure of 3D tubular structures in lung CT images, such as the segmentation results of bronchi, pulmonary arteries, and pulmonary veins. Figure 4 As shown, a comparison is made between traditional 3D reconstruction and the Mesh-optimized tubular 3D reconstruction of this application.

[0096] Based on the same inventive concept, embodiments of this application also provide an electronic device, such as... Figure 5 As shown, it includes:

[0097] The acquisition module is configured to acquire the original image of the target object, wherein the original image includes the main body and branches of the target object;

[0098] The segmentation module is configured to segment the main body and branches of the target object based on an iterative image segmentation method, forming multiple corresponding point elements;

[0099] The extraction module is configured to extract the plurality of point elements to form a set of point elements corresponding to the main trunk and a set of point elements corresponding to the branch, wherein the set of point elements represents the skeleton structure of the main trunk and the branch.

[0100] The reconstruction module is configured to perform three-dimensional reconstruction on the set of point elements to form a tubular topology and an ellipsoidal topology, wherein the tubular topology corresponds to the trunk portion and the ellipsoidal topology corresponds to the branch portion.

[0101] The merging module is configured to merge the tubular topology and the ellipsoidal topology to form a three-dimensional model of the target object.

[0102] In one embodiment of this application, the electronic device further includes:

[0103] The detailed module is configured as follows:

[0104] After obtaining the set of point elements, the set of point elements is refined so that the trunk and the branch parts represented by the set of point elements have a clear skeletal structure.

[0105] In one embodiment of this application, the refinement module is further configured to:

[0106] The set of point elements is transformed into a directed graph structure with a tree structure, wherein the directed graph structure includes root node elements, transition node elements, and leaf node elements.

[0107] In one embodiment of this application, the refinement module is further configured to:

[0108] The positions of the set of point elements are corrected;

[0109] Based on the normal vector direction of the point element, obtain the cross section of the point element;

[0110] The position of the point element is corrected based on the cross-section of the point element;

[0111] Based on the roundness of the point element, the radius of the point element is obtained, wherein the radius of the point element characterizes the thickness of the point element.

[0112] In one embodiment of this application, the refinement module is further configured to:

[0113] If it is determined that the set of point elements includes abnormal skeleton structures, the set of point elements is pruned to remove the abnormal skeleton structures, wherein the abnormal skeleton structures are characterized as protrusions, burrs, and small forks of the target object.

[0114] In one embodiment of this application, the reconstruction module is further configured to:

[0115] Calculate the normal vector of the point element based on the set of point elements corresponding to the main part;

[0116] Based on the normal vectors of the point elements, generate the corresponding planar circular points;

[0117] Connect the planar circular points corresponding to each main section to form the tubular topology.

[0118] In one embodiment of this application, the reconstruction module is further configured to:

[0119] Based on the set of point elements corresponding to the branch, obtain the normal vector of the point element;

[0120] Based on the normal vectors of the point elements, the corresponding ellipsoid points are generated;

[0121] Connect the ellipsoidal points corresponding to each branch to form the ellipsoidal topology.

[0122] In one embodiment of this application, the refinement module is further configured to:

[0123] Based on the Loop subdivision method, a web-like structure is formed between the main body and the branch parts of the 3D model to refine the 3D model. The web-like structure is a smooth transition connection structure.

[0124] Based on the same inventive concept, such as Figure 6 As shown, this embodiment also includes an electronic device, comprising:

[0125] Memory, used to store executable programs;

[0126] A processor for executing the executable program to implement the above method.

[0127] Since the principle of the electronic device in this application to solve the problem is similar to that of the method described above, the implementation of the electronic device can be found in the implementation of the method, and the repeated parts will not be described again.

[0128] This application provides a storage medium, which is a computer-readable medium, storing a computer program. When the computer program is executed by a processor, it implements the method provided in any embodiment of this application, including the method steps described above.

[0129] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk. Optionally, in this embodiment, the processor executes the method steps described in the above embodiments according to the program code stored in the storage medium. Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, which will not be repeated here. Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed on a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be executed in a different order than those described here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any specific hardware and software combination.

[0130] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0131] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An image processing method, characterized by, The method comprises: obtaining an original image of a target object, wherein the original image comprises a trunk portion and a branch portion of the target object; segmenting the trunk portion and the branch portion of the target object based on an iterative image segmentation manner to form a plurality of point elements corresponding to the trunk portion and the branch portion; extracting the plurality of point elements to form a point element set corresponding to the trunk portion and a point element set corresponding to the branch portion, wherein the point element sets represent the skeleton structures of the trunk portion and the branch portion; performing three-dimensional reconstruction on the point element sets to form a tubular topology structure corresponding to the trunk portion and an ellipsoidal topology structure corresponding to the branch portion; merging the tubular topology structure and the ellipsoidal topology structure to form a three-dimensional model of the target object; after obtaining the point element sets, performing refinement processing on the point element sets to make the trunk portion and the branch portion represented by the point element sets have clear skeleton structures; the refinement processing on the point element sets further comprises: correcting the positions of the point element sets; obtaining the cross sections of the point elements based on the normal vector directions of the point elements; correcting the positions of the point elements based on the cross sections of the point elements; obtaining the radii of the point elements based on the roundness of the point elements, wherein the radii of the point elements represent the thickness of the point elements.

2. The method of claim 1, wherein, the refinement processing on the point element sets comprises: converting the point element sets into a directed graph structure with a tree structure, wherein the directed graph structure comprises root node elements, transition node elements and leaf node elements.

3. The method of claim 1, wherein, the refinement processing on the point element sets further comprises: in a case where it is determined that the point element sets include abnormal skeleton structures, performing pruning processing on the point element sets to remove the abnormal skeleton structures, wherein the abnormal skeleton structures represent protrusions, small burrs and small bifurcations of the target object.

4. The method of claim 1, wherein, the three-dimensional reconstruction on the point element sets to form the tubular topology structure and the ellipsoidal topology structure comprises: calculating the normal vectors of the point elements according to the point element sets corresponding to the trunk portion; generating corresponding planar annular points based on the normal vectors of the point elements; connecting the planar annular points corresponding to each trunk portion to form the tubular topology structure.

5. The method of claim 1, wherein, the three-dimensional reconstruction on the point element sets to form the tubular topology structure and the ellipsoidal topology structure further comprises: obtaining the normal vectors of the point elements according to the point element sets corresponding to the branch portion; generating corresponding ellipsoidal points based on the normal vectors of the point elements; connecting the ellipsoidal points corresponding to each branch portion to form the ellipsoidal topology structure.

6. The method of claim 1, wherein, the method further comprises: forming a web structure between the trunk portion and the branch portion of the three-dimensional model based on a Loop subdivision manner to perform refinement processing on the three-dimensional model, wherein the web structure is a smooth transition connection structure.

7. An electronic device, comprising: ​ An acquisition module configured to acquire an original image of a target object, wherein the original image comprises a trunk portion and a branch portion of the target object; A segmentation module configured to perform segmentation processing on the trunk portion and the branch portion of the target object based on an iterative manner of segmenting an image, to form a plurality of point elements corresponding thereto; An extraction module configured to extract the plurality of point elements to form a point element set corresponding to the trunk portion and a point element set corresponding to the branch portion, wherein the point element sets represent skeleton structures of the trunk portion and the branch portion; A reconstruction module configured to perform three-dimensional reconstruction on the point element sets to form a tubular topology structure corresponding to the trunk portion and an ellipsoidal topology structure corresponding to the branch portion; A merging module configured to merge the tubular topology structure and the ellipsoidal topology structure to form a three-dimensional model of the target object; A refinement module configured to, after obtaining the point element sets, perform refinement processing on the point element sets, so that the trunk portion and the branch portion represented by the point element sets have clear skeleton structures; The refinement module is further configured to: correct positions of the point element sets; acquire cross sections of the point elements based on normal vector directions of the point elements; correct the positions of the point elements based on the cross sections of the point elements; acquire radii of the point elements based on roundnesses of the point elements, wherein the radii of the point elements represent thicknesses of the point elements.

8. An electronic device, comprising: A device comprising a memory and a processor, wherein the memory stores an executable program, and the processor executes the executable program to implement steps of the method according to any one of claims 1 to 6.

Images