Systems, methods, and media for automatically generating three-dimensional polygonal models from volume renderings

Abstract

The present application relates to systems and methods and media for automatically generating a three-dimensional polygonal model with color mapping from a volume rendering. The method includes generating a volume rendering from volume data. The method includes receiving a user selection to initiate model and color generation. The method includes automatically generating a 3D mask from the volume rendering in response to the user selection by segmenting at least one object in the volume rendering. The method includes automatically generating a 3D mesh of the at least one object based on the 3D mask in response to the user selection. The method includes automatically computing mesh colors based on the volume rendering in response to the user selection. The mesh colors are applied to the 3D mesh to generate a multi-colored 3D polygonal model. The method includes automatically outputting the multi-colored 3D polygonal model in response to the user selection.

Description

Technical Field

[0001] Some implementations relate to medical imaging. More specifically, some implementations relate to methods and systems for automatically generating three-dimensional (3D) polygonal models with color mapping from volumetric rendering. Background Technology

[0002] Various medical imaging techniques are used to image organs and soft tissues in the human body, such as ultrasound imaging, computed tomography (CT) scans, and magnetic resonance imaging (MRI). Typically, during medical imaging, an imaging dataset (including volumetric imaging datasets during 3D / 4D imaging) is acquired, and corresponding images are generated and rendered in real time (e.g., via a display) using the imaging dataset. However, in some cases, it may be desirable to generate 3D objects corresponding to structures depicted in medical image data; this is often referred to as 3D printing. 3D printing of physical models can provide anatomical structures for surgical planning, research, medical product development, souvenirs, etc. To perform 3D printing, a 3D printer is set up with a digital model of the structure to be generated. However, digital model generation is a time-consuming process, often involving significant user input. For example, a user may interact with image processing software to perform image segmentation to identify and depict structures such as bone, tissue, and blood vessels. The user may further interact with modeling software to generate 3D models, such as 3D polygon models, from the segmented image data. Additionally, the user may interact with modeling software to map colors onto the 3D polygon model. The 3D model can then be fed into 3D printing software for formatting and printing by a 3D printer. Therefore, the typical process for 3D printing a physical model from a medical imaging dataset can be a complex, time-consuming, and challenging task.

[0003] By comparing such systems with some aspects set forth with reference to the accompanying drawings in the remainder of this application, the additional limitations and disadvantages of conventional and traditional methods will become apparent to those skilled in the art. Summary of the Invention

[0004] A system and / or method are provided for automatically generating three-dimensional (3D) polygonal models with color mapping from volumetric rendering, as set forth more fully in the claims.

[0005] These and other advantages, aspects and novel features of the invention, as well as details of its illustrative embodiments, will be more fully understood from the following description and accompanying drawings. Attached Figure Description

[0006] Figure 1A This is a block diagram illustrating an exemplary medical imaging apparatus according to an exemplary embodiment for automatically generating a three-dimensional (3D) polygonal model with color mapping from a volumetric rendering for 3D printing.

[0007] Figure 1B This is a block diagram illustrating an exemplary medical imaging apparatus supporting three-dimensional (3D) printing according to an exemplary embodiment, wherein the offloading process is used to automatically generate a 3D polygonal model with color mapping from volumetric rendering.

[0008] Figure 2 This is a block diagram illustrating an exemplary combined model and color generation processor according to various embodiments, the exemplary combined model and color generation processor being operable to automatically generate a three-dimensional (3D) polygonal model with color mapping from a volumetric rendering for 3D printing.

[0009] Figure 3 An exemplary process for generating a three-dimensional (3D) printed model of anatomy from volumetric rendering is shown according to various implementation schemes.

[0010] Figure 4 The flowchart illustrates exemplary steps according to various implementations that can be performed to automatically generate a three-dimensional (3D) polygonal model with color mapping from a volumetric render for 3D printing. Detailed Implementation

[0011] Certain implementations can be found in methods and systems for automatically generating three-dimensional (3D) polygonal models with color mappings from volumetric rendering. Aspects of this disclosure can provide a 3D polygonal model with a color mapping corresponding to the volumetric rendering in response to a single user instruction (e.g., button press, touchscreen selection, etc.), thereby reducing 3D color model creation time, simplifying user workflows, and providing a realistic 3D printing color model that substantially matches the volumetric rendering (i.e., WYSIWYG). More specifically, various implementations facilitate 3D printing during medical imaging by generating multicolor 3D mesh data (also referred to herein as multicolor 3D polygonal models) based on volumetric rendering images. In this regard, during medical imaging, a volumetric dataset can be generated, and a volumetric rendering image can be generated and / or displayed based on the volumetric dataset. Multicolor 3D mesh data can then be generated based on the volumetric rendering image, the multicolor 3D mesh data being configured to enable the generation of a physical volumetric representation of one or more objects and / or structures in the volumetric rendering image. The multicolor 3D mesh data can then be used for 3D printing. For example, multicolor 3D mesh data can be used to generate 3D printing data, which can then be used to enable 3D printing with a corresponding 3D printer. 3D printing data can be configured and formatted based on predefined 3D printing standards or file formats supported by the 3D printer.

[0012] The foregoing summary of the invention and the following detailed description of certain embodiments will be better understood when read in conjunction with the accompanying drawings. For the purposes of the figures illustrating the functional blocks of various embodiments, these functional blocks do not necessarily represent a division between hardware circuits. Thus, for example, one or more functional blocks (e.g., a processor or memory) may be implemented in a single piece of hardware (e.g., a general-purpose signal processor or a block of random access memory, a hard disk, etc.) or in multiple pieces of hardware. Similarly, a program may be a standalone program, may be included as a subroutine in an operating system, may be a function in an installed software package, etc. It should be understood that the various embodiments are not limited to the arrangements and tools shown in the drawings. It should also be understood that embodiments may be combined, or other embodiments may be utilized, and structural, logical, and electrical changes may be made without departing from the scope of the various embodiments. Therefore, the following detailed description should not be considered limiting, and the scope of this disclosure is defined by the appended claims and their equivalents.

[0013] As used herein, elements or steps listed in the singular and beginning with the words “a” or “an” should be understood to not exclude multiple said elements or steps unless such exclusion is explicitly stated. Furthermore, references to “exemplary embodiments,” “various embodiments,” “certain embodiments,” “representative embodiments,” etc., are not intended to be construed as excluding the existence of additional embodiments that also incorporate the described features. Moreover, unless explicitly stated to the contrary, embodiments that “comprise,” “include,” or “have” one or more elements having a particular property may include additional elements that do not have that property.

[0014] Furthermore, as used herein, the term "image" broadly refers to both a visual image and the data representing that visual image. However, many implementations generate (or are configured to generate) at least one visual image. For example, as used herein, the term "image" is used to refer to an ultrasound image, a magnetic resonance imaging (MRI) image, a computed tomography (CT) image, and / or any suitable medical image. Additionally, in relation to ultrasound imaging, for example, the term "image" may refer to an ultrasound mode such as B-mode (2D mode), M-mode, three-dimensional (3D) mode, CF mode, PW Doppler, CW Doppler, MGD, and / or submodes of B-mode and / or CF such as shear wave elastography (SWEI), TVI, Angio, B-flow, BMI, BMI_Angio, and in some cases also MM, CM, TVD, where "image" and / or "plane" includes a single beam or multiple beams.

[0015] Furthermore, as used herein, the term processor or processing unit refers to any type of processing unit capable of performing the required computations required for various implementation schemes, such as single-core or multi-core: CPU, Accelerated Processing Unit (APU), graphics board, DSP, FPGA, ASIC, or combinations thereof.

[0016] In various implementations, imaging processing, including visualization enhancement, can be performed, for example, in software, firmware, hardware, or a combination thereof, to form an image.

[0017] Figure 1A This is a block diagram illustrating an example medical imaging apparatus 100 according to an exemplary embodiment, used to automatically generate a three-dimensional (3D) polygonal model with color mapping from a volumetric rendering for 3D printing. Reference Figure 1A The medical imaging device 100 includes a medical imaging system 110 and a 3D printer 120.

[0018] Medical imaging system 110 may include suitable logic, circuitry, interfaces, and / or code operable to acquire medical image data, process the medical image data to provide volumetric rendering, and process the volumetric rendering to provide a multicolor 3D mask suitable for 3D printing. In various embodiments, medical imaging system 110 may be an ultrasound system, an MRI imaging system, a CT imaging system, or any suitable imaging system operable to generate and render medical image data. Medical imaging system 110 may include imaging device 112, display / control unit 114, display screen 116, and user controls 118. Imaging device 112 may be an ultrasound probe, an MRI scanner, a CT scanner, or any suitable imaging device. Imaging device may include suitable logic, circuitry, interfaces, and / or code operable to capture and / or generate specific types of imaging signals (or corresponding data), for example, by movement over a patient's body (or parts thereof).

[0019] The display / control unit 114 may be one or more central processing units, microprocessors, microcontrollers, etc. The display / control unit 114 may be an integrated component or may be distributed in various locations. For example, the display / control unit 114 may support medical image acquisition and medical image visualization. Alternatively, medical image acquisition and medical image visualization may be distributed across various systems.

[0020] Display / control unit 114 may include suitable logic, circuitry, interfaces, and / or code operable to (e.g., via display screen 116) process image data and display images. For example, display / control unit 114 may include suitable logic, circuitry, interfaces, and / or code operable to acquire volumetric image data and perform volumetric rendering on 3D and / or 4D volumes. Display / control unit 114 can be used to generate and render volumetric rendering (e.g., 2D projection) of volumetric (e.g., 3D and / or 4D) datasets. In this regard, rendering a 2D projection of a 3D and / or 4D dataset may include setting or defining a perceptual angle in space relative to the object being displayed, and then defining or calculating the necessary information (e.g., opacity and color) for each voxel in the dataset. This may be accomplished, for example, using a suitable transfer function to define RGBA (red, green, blue, and alpha) values ​​for each voxel. The resulting volumetric rendering may include a depth map that associates depth values ​​with each pixel in the 2D projection. The display / control unit 114 can be used to present volumetric renderings at the display screen 116 and / or store the generated volumetric renderings at any suitable data storage medium.

[0021] Display / control unit 114 may also support user input (e.g., via user control 118) to allow, for example, control of medical imaging. For instance, display / control unit 114 may include suitable logic, circuitry, interfaces, and / or code operable in response to a user selection via user control 118 to generate a multicolor 3D polygon model (i.e., a multicolor 3D surface mesh) based on volumetric rendering. As an example, a user viewing a volumetric rendering at display screen 116 may wish to print a 3D model of the anatomical object depicted in the volumetric rendering. Therefore, the user can select 3D model and color generation options to receive a multicolor 3D polygon model, which can be provided to the 3D printing software of 3D printer 120 to print a 3D model of the object in multiple colors. The multicolor 3D polygon model can appear substantially as shown in the volumetric rendering, thus providing the user with a WYSIWYG one-click workflow from volumetric rendering to a multicolor 3D polygon model. See below for reference. Figure 2 A detailed description of generating multi-colored 3D polygonal models from volumetric rendering.

[0022] User control 118 can be used to input patient data, imaging parameters, settings, select protocols and / or templates, select examination types, select acquisition and / or display processing parameters, initiate volumetric rendering, initiate multicolor 3D mesh generation, etc. In an exemplary embodiment, user control 118 is operable to configure, manage, and / or control the operation of one or more components and / or modules in the medical imaging system 110. User control 118 may include one or more buttons, one or more rotary encoders, a touchscreen, motion tracking, voice recognition, a mouse device, a keyboard, a camera, and / or any other device capable of receiving user commands. In some embodiments, for example, one or more of user controls 118 may be integrated into other components such as display screen 116. For example, user control 118 may include a touchscreen display.

[0023] Display screen 116 can be any device capable of transmitting visual information to a user. For example, display screen 116 may include a liquid crystal display, a light-emitting diode display, and / or any suitable one or more displays. Display screen 116 may be operable to present medical images and / or any suitable information. For example, medical images presented on the display screen may include ultrasound images, CT images, MRI images, volumetric renderings, multicolor 3D meshes (also known as multicolor 3D polygon models), and / or any suitable information.

[0024] The 3D printer 120 may include suitable logic, circuitry, interfaces, and / or code operable to perform 3D printing. In this regard, the 3D printer 120 may be configured to generate (e.g., synthesize) a three-dimensional physical representation, for example, based on 3D printing data corresponding to and / or based on a multicolor 3D polygonal model of the object to be printed. The 3D printer 120 may be any commercially available product that can be coupled to the medical imaging system 110 via suitable connections, wired (e.g., rope) and / or wireless (e.g., WiFi, Bluetooth, etc.) communication. The 3D printer 120 may also be part of the medical imaging system 110 itself and may even be directly integrated into it.

[0025] In operation, the medical imaging system 110 can be used to generate and render volumetric renderings. Volumetric renderings can be used to generate multicolor 3D polygonal models suitable for 3D printing. The medical imaging system 110 is operable to support 3D printing, for example, via a 3D printer 120. The 3D printer 120 is operable to generate a physical volumetric representation of an object and / or structure in the volumetric rendering. For example, a prospective parent may want a 3D printed output having an ultrasound image displayed during an obstetric (OB) imaging scan, such as a fetus and / or specific features (e.g., a face). The corresponding 3D printed output or data can also be used as a reference for medical services, such as aiding in the generation of models for surgical planning. 3D physical objects can be synthesized using the 3D printer 120. The 3D printer 120 is operable to use an additive process to lay down continuous layers of material. The synthesized volumetric object can have virtually any shape and / or geometry. The 3D printer 120 and / or 3D printing operation can be configured and / or controlled based on 3D printing data 130, which may include information corresponding to and / or representing the object (or structure) to be printed. 3D printing data 170 can be generated based on a multi-color 3D polygon model and can be formatted according to one or more defined formats used in 3D printing, such as data based on the 3MF file format. In this respect, 3D printing data 130 can be generated and / or configured based on 3D modeling of objects and / or structures in volume rendering and can be formatted based on printing data formats supported in 3D printer 120.

[0026] like Figure 1A As shown, the generation of 3D printing data 130 is illustrated as being performed directly within the medical imaging system 110 (e.g., within the display / control unit 114, using a suitable processing circuitry system therein). However, this disclosure is not limited thereto. Rather, in some cases, at least some of the processing performed to generate 3D printing data based on imaging-related information can be offloaded to a dedicated system, which may be located near or away from the imaging setup.

[0027] Figure 1B This is a block diagram illustrating an exemplary medical imaging device 150 supporting three-dimensional (3D) printing according to an exemplary embodiment, wherein an offloading process is used to automatically generate a 3D polygonal model with color mapping from volumetric rendering. Reference Figure 1B The medical imaging device 150 may include a medical imaging system 110, a 3D printer 120, and a computing system 160.

[0028] The computing system 160 may include suitable logic, circuitry, interfaces, and / or code operable to process, store, and / or transmit data. In this regard, the computing system 160 is operable to receive 3D imaging data 170, such as a volumetric medical imaging dataset and / or a volumetric rendering corresponding to the volumetric medical imaging dataset. The computing system 160 is operable to generate a multi-color 3D surface mesh from the volumetric rendering. The computing system 160 is operable to form the multi-color 3D surface mesh to generate 3D printing data 130 that can be transmitted to the 3D printer 120. The computing system 160 may be a dedicated device specifically configured for use in conjunction with medical imaging, including support for 3D printing; or it may be a general-purpose computing system (e.g., a personal computer, server, etc.) set up and / or configured to perform the operations described with respect to the computing system 160. Communication between different components in the medical imaging apparatus 150 may be performed using available wired and / or wireless connections and / or according to any suitable communication (and / or networking) standards or protocols.

[0029] In an exemplary embodiment, 3D printing data 130 can be generated via medical imaging system 110 or computing system 160 based on a multicolor 3D surface mesh representation, which can be generated based on volume rendering of a volume dataset acquired via medical imaging system 110. Providing 3D printing in this manner ensures that the 3D print is substantially identical to the rendering on display 116. Moreover, this method enables a fully automated workflow from volume data to 3D printing, allowing for efficient and / or easy-to-use operation. Furthermore, rendering operations can enhance the quality of the 3D print. For example, rendering algorithms can act as nonlinear filters and produce highly reliable depth information compared to other segmentation methods. Rendered images can also be used to texture 3D imprints to enhance the quality of the printed object. This method can also allow users to control 3D printing, for example, based on user input (provided via user control 118). For example, 3D printing can be controlled by the user based on user input related to volume rendering (e.g., viewpoint selection, zoom, thresholding, etc.). Additionally, 3D printing can reflect the use of techniques available for volumetric rendering, such as removing unwanted volumetric portions (e.g., masking with MagiCut, Vocal, thresholding, etc.). In other words, 3D printing can include only the desired portions of an object.

[0030] Figure 2 This is a block diagram illustrating an exemplary combined model and color generation processor 220 according to various embodiments, operable to automatically generate a three-dimensional (3D) polygonal model with a color map 250 from a volumetric rendering 202 for 3D printing. Reference Figure 2The apparatus 200 includes a volumetric rendering input 202 provided to a combined model and color generation processor 220, which provides a multi-colored 3D polygonal model based on the volumetric rendering input 202. The volumetric rendering input 202 may include object information 204, including a series of volumetric data images 206, rendering options 208 (e.g., cutting planes), segmentation information 210 (e.g., if any segmentation is performed in conjunction with volumetric rendering), and volumetric rendering display information 212. The volumetric rendering display information 212 may include an opacity transfer function 214 and a color transfer function 216.

[0031] The combined model and color generation processor 220 may include suitable logic, circuitry, interfaces, and / or code operable to: (1) perform segmentation processing 222 to generate a 3D mask 228 from volumetric rendering input 202, (2) perform mesh generation and processing from the 3D mask 228 to create a 3D mesh 238, and (3) apply color 240 to the mesh 238 based on the volumetric rendering input 202 to generate a multi-colored 3D polygonal model 250. The combined model and color generation processor 220 may perform segmentation processing 222 to generate a 3D mask based on a volumetric data image series 206, rendering options 208, segmentation information (if any), and an opacity transfer function 214. Segmentation processing 222 may include 3D mask generation 224 and post-processing 226. For example, the combined model and color generation processor 220 can perform 3D mask generation 224 by applying a threshold to the opacity transfer function 214 to create a 3D mask by eliminating image data from an image series 206 that has an opacity below the threshold in the opacity transfer function 214. Volumetric rendering options 208 (such as cutting planes applied to generate volumetric rendering) can also be applied by the combined model and color generation processor 220 performing 3D mask generation 224 to eliminate image data from the image series 206 during mask formation. If the volumetric rendering 202 includes segmented volumes present in the view, the combined model and color generation processor 220 performing 3D mask generation 224 can apply segmentation information 210 from the object information 204 of the volumetric rendering 202 to eliminate image data from the image series 206 to form a mask. The generated mask can be post-processed 226 to improve the quality of the 3D printed model before performing mesh generation and 3D mask-based processing 230. For example, post-processing 226 may include 3D hole filling, widget removal, and / or any suitable post-processing 226 mechanism to improve the quality of the generated mask.

[0032] The combined model and color generation processor 220 performs mesh generation and processing 230 to generate a 3D mesh 238 based on a 3D mask 228. Mesh generation and processing 230 may include moving cube processing 232, smoothing filtering 234, and post-processing 236. For example, the combined model and color generation processor 220 may perform moving cube processing 232 by extracting a polygonal mesh of isosurfaces from 3D voxels. Moving cube processing 232 continues through voxels of the 3D mask 228, taking eight neighboring locations at a time to form an imaginary cube, and then determining the polygons needed to represent the portions of isosurfaces passing through that cube. The individual polygons are then blended onto the desired surface to form a 3D mesh. Before color is added to the 3D mesh 240, the 3D mesh created from the 3D mask 228 may undergo surface smoothing filtering 234 and additional post-processing 236, such as cross-point removal, surface extraction, and / or any suitable post-processing.

[0033] The combined model and color generation processor 220 can perform mesh shading 240 based on volumetric rendering input 202. Mesh shading 240 may include color calculation 242 based on opacity transfer function 214, color transfer function 216, and any segmentation information 210 from volumetric rendering input 202. For example, the combined model and color generation processor 220 can perform color calculation 242 by retrieving color map information 216 from a selected volumetric rendering view 202. Colors are then calculated using image characteristic values, which may be based on opacity information 214 and any segmentation information 210, and color map information 216 for each surface point in the 3D mesh 238. As an example, for each point on the surface of the 3D mesh 238, a normal vector with characteristic values ​​214, 210 along the normal is provided. Characteristic values ​​for each point on the surface of the 3D mesh 238 are calculated using the characteristic values ​​of the image at the points where they intersect as the normal vector is moved. The color map information 216 is applied to determine the appropriate color corresponding to each characteristic value for each point on the surface of the 3D mesh 238. The resulting 3D mesh with assigned color values ​​is the output 250 from the combined model and color generation processor 220 as a multi-color 3D polygon model. For example, the multi-color 3D polygon model 250 may be displayed on a display screen 116, stored on any suitable data storage medium, and / or formatted by 3D printing software and provided to the 3D printer 120 as 3D printing data 130 to generate a physical model.

[0034] Figure 3 Exemplary processes 300 for generating a three-dimensional (3D) printed model 340 of an anatomical structure from a volumetric render 310 are illustrated according to various embodiments. References Figure 3The process 300 includes generating a volumetric render 310, generating a multi-colored 3D polygon model 320 from the volumetric render 310, formatting the 3D polygon model 320 into 3D printing data 330, and printing the 3D printing data 330 to create a physical model 340 of the object depicted in the volumetric render 310. The volumetric render 310 may include object information 202, such as a series of volumetric data images 206, rendering options 208 (e.g., cutting planes), segmentation information 210 (e.g., if any segmentation is performed in conjunction with the volumetric render), and volumetric render display information 212. The volumetric render display information 212 may include an opacity transfer function 214 and a color transfer function 216. The volumetric render 310 may be rendered on a display screen 116 of a medical imaging system 110. The volumetric render 310 may be used, for example, in response to user input selections from a user control 118, to generate a multi-colored 3D polygon model 320. The 3D polygon model 320 may be substantially identical to the volumetric render 310. A 3D polygonal model 320 can be generated based on volumetric rendering 310 by a combined model and color generation processor 220 of the display / control unit 114 of the medical imaging system 110 and / or computing system 160. For example, the combined model and color generation processor 220 can perform 3D mask generation 224 and post-processing 226 using volumetric data image series 206, rendering options 208, segmentation information 210, and / or opacity transfer function 214. The combined model and color generation processor 220 can use 3D mask 228 to generate a mesh using moving cube processing 232, smoothing filtering 234, and post-processing 236. The combined model and color generation processor 220 can apply color to the 3D mesh 238 based on color transfer function 216 and characteristic values ​​from volumetric rendering 310 (such as opacity transfer function 214 and any segmentation information 210). The colored mesh can produce a multi-colored 3D polygonal model 320, which can be provided to 3D printing software to be formatted as 3D printing data 330. For example, the 3D printing software used to format the multi-color 3D polygon model 320 into 3D printing data 330 could be STRATASYS's GRABCAD, such as... Figure 3 As shown. 3D printing data 330 can be provided to 3D printer 120 to print a physical model 340 of the anatomical object.

[0035] Figure 4 Flowchart 400 is an exemplary flowchart of steps 402 to 416 according to various implementation schemes, which can be performed to automatically generate three-dimensional (3D) polygonal models 250, 320 with color mapping from volumetric renders 202, 310 for 3D printing. Reference Figure 4The diagram illustrates flowchart 400, which includes exemplary steps 402 to 416. Some embodiments may omit one or more steps, and / or perform the steps in a different order than listed, and / or combine certain steps discussed below. For example, some steps may not be performed in some embodiments. Also, some steps may be performed in a different chronological order than listed below, including simultaneous execution.

[0036] At step 402, volume data is acquired by the scanner 112 of the medical imaging system 110. For example, the volume data may be ultrasound image data acquired with an ultrasound probe, CT image data acquired with a CT scanner, MRI image data acquired with an MRI scanner, and / or any suitable medical volume imaging data acquired from a medical imaging device scanner.

[0037] At step 404, volumetric renderings 202 and 310 of the volume data can be generated and displayed. For example, medical imaging system 110 or computer system 160 can generate volumetric renderings 202 and 310 based on the volume data acquired at step 402. Volumetric renderings 202 and 310 may include object information 204, which includes a series of volumetric data images 206, rendering options 208 (e.g., cutting planes), segmentation information 210, and volumetric rendering display information 212. Volumetric rendering display information 212 may include an opacity transfer function 214 and a color transfer function 216. Volumetric renderings 202 and 310 may be presented on the display screen 116 of medical imaging system 110 and / or on any suitable display system.

[0038] At step 406, a selection for initiating model and color generation is received. For example, medical imaging system 110 or computer system 160 may receive a user selection instructing systems 110, 160 to create multi-colored 3D polygonal models 250, 320. In various embodiments, the user selection may be a single input, for example, received via user control 118. As an example, a single user input may be a button press, a touchscreen selection, etc., to initiate an automated process for creating multi-colored 3D polygonal models 250, 320 from volumetric rendering 202, 310.

[0039] In step 408, a 3D mask 228 is generated from volumetric rendering views 202, 310 by segmenting objects in views 222, 224, 226. For example, the combined model and color generation processor 220 of medical imaging system 110 or computer system 160 may perform segmentation processing 222 to generate a 3D mask 228 from volumetric rendering input 202. The combined model and color generation processor 220 of medical imaging system 110 or computer system 160 may perform segmentation processing 222 to generate a 3D mask based on volumetric data image series 206, rendering options 208, segmentation information (if any), and opacity transfer function 214. Segmentation processing 222 may include 3D mask generation 224 and post-processing 226, such as 3D hole filling, widget removal, and / or any suitable post-processing 226 mechanism to improve the quality of the generated mask.

[0040] At step 410, a mesh 238 may be generated and processed for each object based on the 3D mask 228. For example, a combined model and color generation processor 220 of a medical imaging system 110 or a computer system 160 may perform mesh generation and processing from the 3D mask 228 to create a 3D mesh 238. Mesh generation and processing 230 may include moving cube processing 232, smoothing filtering 234, and post-processing 236, such as self-intersection removal, surface extraction, and / or any suitable 3D mesh post-processing.

[0041] In step 412, mesh colors 240 and 242 can be calculated for each object from volumetric rendering views 202 and 310. For example, the combined model and color generation processor 220 can perform mesh shading 240 based on volumetric rendering inputs 202 and 310 to generate multi-colored 3D polygon models 250 and 320. Mesh shading 240 may include color calculations 242 based on an opacity transfer function 214, a color transfer function 216, and any segmentation information 210 from the volumetric rendering input 202. The 3D mesh with assigned color values ​​can be the output 250 from the combined model and color generation processor 220 as a multi-colored 3D polygon model 320.

[0042] At step 414, the color meshes 250 and 320 can be input into the 3D printing software solution. For example, a multi-color 3D polygon model 250 can be provided to the 3D printing software to format the color meshes 250 and 320 for use by the 3D printer 120.

[0043] In step 416, 3D printing can be performed based on the color grids 250 and 320. For example, the color grids 250 and 320 formatted by 3D printing software 330 can be used by 3D printer 120 to generate a physical model 340.

[0044] This disclosure provides a method 400 and a system 110, 160 for automatically generating three-dimensional (3D) polygonal models 250, 320 with color mapping from volumetric renders 202, 310. According to various embodiments, method 400 may include generating volumetric renders 202, 310 404 from volumetric data by at least one processor 114, 160. Method 400 may include receiving a user selection 406 for initiating model and color generation by at least one processor 114, 160. Method 400 may include automatically generating a three-dimensional (3D) mask 228 from volumetric renders 202, 310 408 by segmenting at least one object in the volumetric renders 202, 310 in response to a user selection by at least one processor 114, 160, 220, 222, 224 in response to a user selection. Method 400 may include automatically generating a 3D mesh 238 of at least one object 410 based on the 3D mask 228 by at least one processor 114, 160, 220, 230, 232 in response to a user selection. Method 400 may include automatically performing volumetric rendering 202, 310 to calculate a mesh color 412 by at least one processor 114, 160, 220, 240, 242 in response to a user selection. The mesh color may be applied to a 3D mesh 238 to generate a multi-colored 3D polygon model 250, 320. Method 400 may include automatically performing output multi-colored 3D polygon models 250, 320 by at least one processor 114, 160, 220 in response to a user selection.

[0045] In a representative implementation, generating 408 3D mask 228 includes applying a threshold to an opacity transfer function 214 to eliminate image data with opacity below the threshold in the opacity transfer function 214. In an exemplary implementation, generating 408 3D mask 228 includes applying a cutting plane 208 of volumetric rendering 202, 310. In some implementations, method 400 may include post-processing 408, 226 of the 3D mask 228. Post-processing 408, 226 may include one or both of 3D hole filling and widget removal. In various implementations, generating 410 3D mesh 238 includes performing a moving cube processing 232. In a representative implementation, method 400 may include performing 410 surface smoothing filtering 234 on the 3D mesh 238 and post-processing 236 on the 3D mesh 238. Post-processing 236 may include one or both of cross-point removal and surface extraction 236. In an exemplary embodiment, calculating the color of the 410 mesh may be based at least in part on the opacity transfer function 214 and the color transfer function 216 of the volume rendering 202, 310. In some embodiments, method 400 may include providing 414 multicolor 3D polygon models 250, 320 to 3D printing software to format the multicolor 3D polygon model 330 for use by the 3D printer 120.

[0046] Various embodiments provide a system 110, 160 for automatically generating three-dimensional (3D) polygonal models 250, 320 with color mapping from volumetric renders 202, 310. The system may include electronic devices 110, 160, including at least one processor 114, 160, 220. At least one processor 114, 160 is operable to generate volumetric renders 202, 310 from volumetric data. At least one processor 114, 160 is operable to receive a selection to initiate model and color generation. In response to a user selection, at least one processor 114, 160, 220, 222, 224 is operable to automatically generate a three-dimensional (3D) mask 228 from volumetric renders 202, 310 by segmenting at least one object in the volumetric renders 202, 310. In response to a user selection, at least one processor 114, 160, 220, 230, 232 is operable to automatically generate a 3D mesh 238 of at least one object based on the 3D mask 228. In response to user selection, at least one processor 114, 160, 220, 240, 242 is operable to automatically calculate mesh colors based on volumetric rendering 202, 310. The mesh colors can be applied to a 3D mesh 238 to generate multi-colored 3D polygonal models 250, 320. In response to user selection, at least one processor 114, 160, 220 is operable to automatically output multi-colored 3D polygonal models 250, 320.

[0047] In exemplary embodiments, electronic devices 110, 160 are operable to receive volumetric data from medical imaging device 110, which is operable to generate volumetric data based on a specific imaging technique. In some embodiments, electronic devices 110, 160 may include medical imaging device 110, which is operable to generate volumetric data based on a specific imaging technique. In various embodiments, at least one processor 114, 160, 220, 222, 224 is operable to generate a 3D mask 228 by applying a threshold to an opacity transfer function 214 to eliminate image data with opacity below a threshold in the opacity transfer function 214 and / or applying a cutting plane 208 of volumetric rendering 202, 310. In representative embodiments, at least one processor 114, 160, 220, 222, 226 is operable to perform one or both of 3D hole filling and widget removal on the 3D mask 228. In some embodiments, at least one processor 114, 160, 220, 230, 234, 236 is operable to perform surface smoothing filtering on the 3D mesh 238 and to post-process the 3D mesh 238 by performing one or both of self-crosspoint removal and surface extraction. In an exemplary embodiment, at least one processor 114, 160, 220 is operable to provide multicolor 3D polygon models 250, 320 to 3D printing software to format the multicolor 3D polygon model 330 for use by the 3D printer 120.

[0048] Some embodiments provide a non-transitory computer-readable medium on which a computer program is stored, the computer program having at least one code segment. The at least one code segment is executable by a machine to cause the machine to perform step 400. Step 400 may include generating volumetric renders 202, 310 from volumetric data 404. Step 400 may include receiving a user selection 406 for initiating model and color generation. Step 400 may include generating a 3D mask 228 from volumetric renders 202, 310 by segmenting at least one object in the volumetric renders 202, 310 in response to the user selection. Step 400 may include automatically generating a 3D mesh 238 of at least one object based on the 3D mask 228 in response to the user selection. Step 400 may include automatically calculating a mesh color 412 based on the volumetric renders 202, 310 in response to the user selection. The mesh color may be applied to the 3D mesh 238 to generate multi-colored 3D polygonal models 250, 320. Step 400 may include automatically executing the output of multi-color 3D polygon models 250 and 320 in response to a user selection.

[0049] In various embodiments, generating a 3D mask 408 may include applying a threshold to an opacity transfer function 214 to eliminate image data with opacity below the threshold in the opacity transfer function 214 and / or applying cutting planes 208 of volumetric rendering 202, 310. In a representative embodiment, step 400 may include performing a surface smoothing filter 234 410 on a 3D mesh 238 and post-processing 236 on the 3D mesh 238. Post-processing 236 may include one or both of cross-point removal and surface extraction. In an exemplary embodiment, calculating the mesh color 412 may be at least partially based on the opacity transfer function 214 and the color transfer function 216 of the volumetric rendering 202, 310. In some embodiments, step 400 may include providing a multi-color 3D polygon model 250, 320 414 to 3D printing software to format the multi-color 3D polygon model 330 for use by the 3D printer 120.

[0050] As used herein, the term "circuit" refers to physical electronic components (i.e., hardware) and any software and / or firmware ("code") that is configurable hardware, executed by the hardware, and / or otherwise associated with the hardware. For example, as used herein, a particular processor and memory may include a first "circuit" when executing one or more lines of first code, and a particular processor and memory may include a second "circuit" when executing one or more lines of second code. As used herein, "and / or" means any one or more items in a list linked by "and / or". For example, "x and / or y" means any element in the three-element set {(x),(y),(x,y)}. As another example, "x, y and / or z" means any element in the seven-element set {(x),(y),(z),(x,y),(x,z),(y,z),(x,y,z)}. As used herein, the term "exemplary" means used as a non-limiting example, instance, or illustration. As used herein, the term "for example (eg)" refers to a list of one or more non-limiting examples, instances, or illustrations. As used herein, a circuit is "operable to" perform a function whenever the circuit includes the necessary hardware and code to perform the function (if required), regardless of whether the execution of the function is disabled or not enabled by some user-configurable settings.

[0051] Other embodiments may provide a computer-readable device and / or a non-transitory computer-readable medium, and / or a machine-readable device and / or a non-transitory machine-readable medium, wherein the computer-readable device and / or the non-transitory computer-readable medium and / or the machine-readable device and / or the non-transitory machine-readable medium stores machine code and / or a computer program having at least one code segment executable by a machine and / or a computer, thereby enabling the machine and / or the computer to perform the steps as described herein for automatically generating a three-dimensional (3D) polygonal model with color mapping from volumetric rendering.

[0052] Therefore, this disclosure can be implemented in hardware, software, or a combination of hardware and software. This disclosure may be implemented centrally in at least one computer system or distributed, with different elements distributed across several interconnected computer systems. Any kind of computer system or other apparatus suitable for performing the methods described herein is appropriate. A typical combination of hardware and software may be a general-purpose computer system having a computer program that, when loaded and executed, controls the computer system to perform the methods described herein.

[0053] This disclosure can also be embedded in a computer program product that includes all the features of the methods described herein and is capable of executing those methods when loaded into a computer system. As used herein, a computer program means any expression of a set of instructions represented in any language, code, or notation, which is intended to cause a system with information processing capabilities to directly perform a particular function or to perform a particular function after being: a) translated into another language, code, or notation; or b) reproduced in a different material form.

[0054] While the invention has been described with reference to certain embodiments, those skilled in the art will understand that various changes and substitutions can be made without departing from the scope of the invention. Furthermore, many modifications can be made to adapt particular situations or materials to the teachings of the invention without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the specific embodiments disclosed, but rather to include all embodiments falling within the scope of the appended claims.

Claims

1. A method for automatically generating multi-color 3D polygon models from volumetric rendering, the method comprising: The volume rendering is generated from the volume data by at least one processor; The at least one processor receives user selections for initiating model and color generation; as well as In response to the user selection, the at least one processor automatically executes: A 3D mask is generated from the volumetric rendering by segmenting at least one object in the volumetric rendering. Generate a 3D mesh of the at least one object based on the 3D mask; Based on the volume rendering calculation of the mesh color, the mesh color is applied to the 3D mesh to generate the multi-color 3D polygon model; as well as Output the multi-color 3D polygon model. The calculation of mesh color based on the volume rendering includes calculating the mesh color based on the opacity transfer function, the color transfer function, and the segmentation information from the volume rendering.

2. The method of claim 1, wherein generating the 3D mask includes applying the volumetric rendering to the cutting plane.

3. The method of claim 1, wherein the method includes post-processing the 3D mask, wherein the post-processing includes one or both of 3D hole filling and part removal.

4. The method of claim 1, wherein generating the 3D mesh includes performing a moving cube process.

5. The method according to claim 1, wherein the method comprises: Perform surface smoothing filtering on the 3D mesh, and The 3D mesh is post-processed, wherein the post-processing includes one or both of cross-point removal and surface extraction.

6. The method of claim 1, wherein generating the 3D mask comprises applying a threshold to an opacity transfer function to eliminate image data having an opacity below the threshold in the opacity transfer function.

7. The method of claim 1, wherein the method comprises providing the multicolor 3D polygon model to 3D printing software to format the multicolor 3D polygon model for use by a 3D printer.

8. A system for automatically generating multi-color 3D polygon models from volumetric rendering, the system comprising: An electronic device, the electronic device including at least one processor, the processor being operable to: The volume rendering is generated from the volume data; Receive user selections for initiating model and color generation; In response to the user's selection, at least the following operations will be performed automatically: A 3D mask is generated from the volumetric rendering by segmenting at least one object in the volumetric rendering. Generate a 3D mesh of the at least one object based on the 3D mask; Based on the volume rendering calculation of the mesh color, the mesh color is applied to the 3D mesh to generate the multi-color 3D polygon model; as well as Output the multi-color 3D polygon model. The calculation of mesh color based on the volume rendering includes calculating the mesh color based on the opacity transfer function, the color transfer function, and the segmentation information from the volume rendering.

9. The system of claim 8, wherein the electronic device is operable to receive the volume data from a medical imaging device, the medical imaging device being operable to generate the volume data based on a specific imaging technique.

10. The system of claim 8, wherein the electronic device includes a medical imaging device operable to generate the volume data based on a specific imaging technique.

11. The system of claim 8, wherein the at least one processor is operable to generate the 3D mask by: The cutting plane is applied to the volume rendering.

12. The system of claim 8, wherein the at least one processor is operable to perform one or both of 3D hole filling on the 3D mask and widget removal on the 3D mask.

13. The system of claim 8, wherein the at least one processor is operable to: Perform surface smoothing filtering on the 3D mesh, and The 3D mesh is post-processed by performing one or both of self-intersection removal and surface extraction.

14. The system of claim 8, wherein the at least one processor is operable to provide the multicolor 3D polygon model to 3D printing software to format the multicolor 3D polygon model for use by a 3D printer.

15. A non-transitory computer-readable medium storing a computer program having at least one code segment that is executable by a machine to cause the machine to perform the method according to any one of claims 1 to 7.

Images