Method and apparatus for converting data of medical image for virtual procedure simulation
The conversion of medical images into a virtual 3D body model with predefined event detection addresses the challenge of accurate surgical simulation, enhancing procedural accuracy and safety.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- KOREA UNIV RES & BUSINESS FOUND
- Filing Date
- 2025-09-30
- Publication Date
- 2026-05-21
AI Technical Summary
Existing medical simulation technologies lack the ability to accurately simulate procedural paths and potential risks in three-dimensional virtual environments, which can lead to inaccuracies and increased risks during surgeries.
A method and device for converting medical images into a virtual 3D body model, generating position coordinate data, and determining predefined procedure effect events based on user input and position datasets, using Hounsfield units and isosurface extraction to classify and visualize anatomical structures.
Enhances the accuracy of procedural simulations by providing real-time visualization of potential risks and procedural effects, improving surgical planning and minimizing risks during surgeries.
Smart Images
Figure KR2025015518_21052026_PF_FP_ABST
Abstract
Description
Method and apparatus for converting medical image data for virtual procedure simulation
[0001] The following embodiments relate to a data conversion technique for medical images for virtual procedure simulations.
[0002] With the advancement of medical technology, simulation technology utilizing CT imaging can create a virtual body structure in three dimensions using multiple high-resolution cross-sectional images of the body's internal structure, allowing users to simulate procedure and surgical paths within this 3D virtual structure. Such virtual simulations can serve as an important tool for users to plan procedure paths in advance and identify potential risks that may actually occur during procedures and surgeries. CT-based 3D model generation and simulation technology can contribute to increasing the accuracy of procedures, minimizing risks during surgery, and enhancing the proficiency of medical staff.
[0003] A method for converting medical images for a virtual procedure simulation according to one embodiment may include: receiving medical images of body parts of a subject for a procedure; generating a virtual 3D body model including virtual body parts corresponding to each of the body parts based on the medical images; generating position coordinate data corresponding to the position of each pixel included in each of the medical images; generating a position dataset related to the positions of the virtual body parts by mapping the position coordinate data to the virtual 3D body model; and determining whether to generate a predefined procedure effect event based on the position of a virtual procedure tool moving according to user input and the position dataset related to the positions of the virtual body parts.
[0004] The operation of determining whether to generate the above-mentioned predefined procedure effect event may include the operation of determining whether to generate the predefined procedure effect event based on the movement of a virtual procedure tool moving into the interior of a virtual 3D body model after contacting the virtual 3D body model in the procedure simulation.
[0005] The operation of generating the above-mentioned virtual three-dimensional body model may include the operation of classifying pixels corresponding to each of the body parts of the subject of the procedure among all the pixels of the medical images, and the operation of determining the location and shape of virtual body parts corresponding to each of the body parts of the subject of the procedure based on the classified pixels.
[0006] The operation of classifying pixels corresponding to each of the body parts of the subject of the procedure may include classifying pixels corresponding to each of the body parts of the subject among all the pixels of the medical images based on Hounsfield units (HU), which indicate the degree of radiation absorption according to the type of medical images, or signal intensity, which indicates the intensity of the emitted magnetic resonance signal.
[0007] The operation of determining the location and shape of the virtual body parts may include the operation of generating the contours of the virtual body parts by isosurface extraction of pixels corresponding to each of the body parts of the subject of the procedure.
[0008] The above body parts may include at least one of the subject's bone, skin, spinal cord, nerve, blood vessel, and organ.
[0009] The above user input may include a user input corresponding to an action of piercing at least one of a virtual bone, virtual skin, virtual spinal cord, virtual nerve, virtual blood vessel, and virtual organ of a subject using a virtual surgical tool.
[0010] The above-defined predefined procedure effect event may include at least one of the following: an event in which the virtual procedure tool is not inserted into the interior of a virtual 3D body model when the position data of the virtual procedure tool matches part of a position dataset corresponding to a virtual bone; an event in which the virtual procedure tool is fixed to the virtual skin when the position data of the virtual procedure tool matches part of a position dataset corresponding to the virtual skin; an event in which virtual cerebrospinal fluid is ejected from the virtual spinal cord when the position data of the virtual procedure tool matches part of a position dataset corresponding to the virtual spinal cord; and an event in which virtual blood is ejected from the virtual blood vessel when the position data of the virtual procedure tool matches part of a position dataset corresponding to a virtual blood vessel.
[0011] In one embodiment, a method for converting medical image data for a virtual procedure simulation may further include the operation of extracting a location dataset corresponding to virtual body parts where a predefined procedure effect event occurs when a predefined procedure effect event occurs.
[0012] A data conversion device for medical images for a virtual procedure simulation according to one embodiment includes a memory containing instructions and a processor connected to the memory and for executing the instructions. When the instructions are executed by the processor, the data conversion device may receive medical images of body parts of a subject for a procedure, generate a virtual 3D body model including virtual body parts corresponding to each of the body parts based on the medical images, generate position coordinate data corresponding to the position of each pixel included in each of the medical images, generate a position dataset related to the positions of the virtual body parts by mapping the position coordinate data to the virtual 3D body model, and determine whether to generate a predefined procedure effect event based on the position of a virtual procedure tool that moves according to user input and the position dataset related to the positions of the virtual body parts.
[0013] When the above instructions are executed by a processor, the data conversion device may determine whether to generate a predefined surgical effect event based on the movement of a virtual surgical tool moving into the interior of a virtual 3D body model after contacting the virtual 3D body model in a surgical simulation.
[0014] When the above instructions are executed by a processor, the data conversion device may classify pixels corresponding to each of the body parts of the subject of the procedure among all the pixels of the medical images, and determine the location and shape of virtual body parts corresponding to each of the body parts of the subject of the procedure based on the classified pixels.
[0015] When the above instructions are executed by a processor, the data conversion device may classify pixels corresponding to each body part of the subject of the procedure among all pixels of the medical images based on Hounsfield units (HU), which indicate the degree of radiation absorption according to the type of medical images, or signal intensity, which indicates the intensity of the emitted magnetic resonance signal.
[0016] When the above instructions are executed by a processor, the data conversion device may generate the contours of virtual body parts by extracting equivalent surfaces from pixels corresponding to each of the body parts of the subject of the procedure.
[0017] When the above instructions are executed by a processor, the data conversion device may be configured to further extract a location dataset corresponding to virtual body parts where a predefined treatment effect event occurs when a predefined treatment effect event occurs.
[0018] FIG. 1 is a diagram illustrating an overview of a medical image data conversion system according to one embodiment.
[0019] FIG. 2 is a flowchart illustrating a method for converting medical image data according to one embodiment.
[0020] FIG. 3 is a drawing for illustrating an interface that displays a virtual three-dimensional body model according to one embodiment.
[0021] FIG. 4 is a drawing illustrating an interface for displaying a predefined treatment effect event on a virtual three-dimensional body model according to one embodiment.
[0022] FIG. 5 is a block diagram illustrating the configurations of a medical image data conversion device according to one embodiment.
[0023] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.
[0024] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.
[0025] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or joined to that other component, or that there may be other components in between.
[0026] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.
[0027] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.
[0028] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.
[0029]
[0030] FIG. 1 is a diagram illustrating an overview of a medical image data conversion system according to one embodiment.
[0031] Referring to FIG. 1, a data conversion device (120) can collect a medical image (110) including body parts of a subject for a procedure. The data conversion device (120) can generate a virtual three-dimensional body model based on the collected medical image (110). The data conversion device (120) can generate location coordinate data corresponding to the location of each pixel included in the medical image (110) based on the medical image (110). The data conversion device (120) can generate a location dataset related to the locations of virtual body parts by mapping the generated location coordinate data to the generated virtual three-dimensional body model. The data conversion device (120) can provide an interface (130) for a procedure simulation in which a virtual three-dimensional body model and a virtual procedure tool appear, and a procedure effect event that may occur when the virtual procedure tool comes into contact with the virtual three-dimensional body model appears.
[0032] The data conversion device (120) generates a virtual three-dimensional body model based on a medical image (110) of a subject of a procedure and maps position coordinate data generated corresponding to the location of each pixel included in the medical image (110) to the virtual three-dimensional body model, thereby determining whether to generate a predefined procedure effect event that may occur depending on the degree to which a virtual procedure tool, which moves according to user input, comes into contact with the virtual three-dimensional body model. The user (or medical staff, or the practitioner performing the procedure) can use the data conversion device (120) to generate a virtual three-dimensional body model from a medical image and separate virtual body parts from the generated virtual three-dimensional body model, and can also check in real time the procedure effect event occurring immediately in the virtual body parts by manipulating a virtual procedure tool on the virtual three-dimensional body model to which position coordinate data is mapped. The procedure described in this specification includes or may be replaced with the meanings of procedure, surgery, and other treatment methods.
[0033]
[0034] FIG. 2 is a flowchart illustrating a method for converting medical image data according to one embodiment. In one embodiment, the operations of the data conversion method may be performed by the data conversion device of FIG. 1 (e.g., the data conversion device (120) of FIG. 1). In one embodiment, at least one of the operations shown in FIG. 2 may be performed simultaneously or in parallel with other operations, and the order between the operations may be changed. Additionally, at least one of the operations may be omitted, and other operations may be performed additionally.
[0035] In operation (210), the data conversion device (120) may receive medical images of body parts of the subject of the procedure. The medical images may include, for example, multiple two-dimensional CT images obtained by irradiating X-rays onto the body parts of the subject of the procedure or multiple two-dimensional MRI images obtained by using resonance of cell atoms by a magnetic field, but are not limited thereto. The body parts may include at least one of the bone, skin, spinal cord, nerve, blood vessel, and various organs including the liver, pancreas, kidney, heart, etc. of the subject of the procedure. Medical images such as CT or MRI may be transmitted in the file format of DICOM (Digital Imaging and Communications in Medicine).
[0036] In operation (220), the data conversion device (120) can generate a virtual three-dimensional body model including virtual body parts corresponding to each of the body parts based on medical images. The virtual body parts may include virtual bones, virtual skin, virtual spinal cord, virtual nerves, virtual blood vessels, and virtual organs that represent the anatomical structures of the bone, skin, spinal cord, nerves, blood vessels, and organs of the subject of the procedure in three dimensions, and the virtual three-dimensional body model may mean a virtual three-dimensional model including virtual body parts.
[0037] In operation (220), a data conversion device (120) according to one embodiment may classify pixels corresponding to each of the body parts of the subject among the total pixels of the medical images in order to generate a virtual three-dimensional body model including virtual body parts from the received medical images. The data conversion device (120) may classify pixels corresponding to each of the body parts of the subject among the total pixels of the medical images based on Hounsfield units (HU), which indicate the degree of radiation absorption, or signal intensity, which indicates the intensity of the emitted magnetic resonance signal. For example, in the case of CT images, pixels corresponding to each of the body parts of the subject may be classified based on Hounsfield units, and in the case of MRI images, pixels corresponding to each of the body parts of the subject may be classified based on signal intensity. The total pixels of the medical images may be separated by body part based on different feature values of the body parts.
[0038] Next, the data conversion device (120) can determine the location and shape of virtual body parts corresponding to each of the body parts of the subject of the procedure based on the classified pixels. The data conversion device (120) can obtain the contours of the virtual body parts by extracting equivalent surfaces from the pixels corresponding to each of the body parts of the subject of the procedure. For example, the data conversion device (120) can extract equivalent surfaces from the pixels corresponding to each of the body parts using a mesh generation algorithm such as the Marching Cubes or Flying Edges algorithm. After the equivalent surface extraction process, the data conversion device (120) can leave only mesh data of a certain size or larger by using algorithms such as Depth-First Search (DFS), Breadth-First Search (BFS), or Union-Find (Disjoint-set Union) to remove mesh data such as noise in the medical image data or very small tissues unnecessary for the procedure simulation, and can perform mesh simplification using algorithms such as Fast Quadric Error Metric (QEM) to reduce the number of polygons while maintaining the original shape. That is, a data conversion device (120) according to one embodiment can perform 3D visualization by sequentially connecting 2D medical images, and can perform a process of simplifying the data so that only essential mesh data among numerous mesh data generated by extracting equivalent surfaces of pixels corresponding to each body part can be obtained and compressed so that it can be easily processed by an external device of the data conversion device (120) (e.g., smartphone, tablet PC, HMD, etc.). Through this, the data conversion device (120) can generate a virtual 3D body model that includes simplified virtual body parts processed to be distinguishable.Alternatively, the data conversion device (120) may separate the appropriate organ required for the procedure simulation by using algorithms such as U-Net and V-Net, which analyze medical images to automatically identify and classify the boundaries and structures of the organs.
[0039] In operation (220), a data conversion device (120) according to one embodiment can generate voxel data of medical images based on medical images and can generate a virtual three-dimensional body model based on the voxel data. A voxel is a volumetric element and may be a concept that reflects volume in a pixel.
[0040] In operation (230), the data conversion device (120) can generate position coordinate data corresponding to the position of each pixel included in each of the medical images. The data conversion device (120) can generate position coordinate data by arranging the positions of each pixel sequentially. For example, when 200 medical images having a size of 512 x 512 pixels are received, the data conversion device (120) can represent the position of each pixel included in each of the medical images as position coordinate data from (0, 0, 0) to (511, 511, 199).
[0041] In operation (240), the data conversion device (120) can generate a location dataset related to the locations of virtual body parts by mapping location coordinate data to a virtual three-dimensional body model. The data conversion device (120) can label the generated three-dimensional location coordinate data one-to-one with the virtual three-dimensional body model. Accordingly, the virtual three-dimensional body model can be in a state where coordinates corresponding to all locations in three dimensions are labeled. The location dataset may mean a set of location coordinate data mapped one-to-one to each of the virtual body parts or the virtual three-dimensional body model containing the virtual body parts to represent their locations.
[0042] In operation (250), the data conversion device (120) can determine whether to generate a predefined surgical effect event based on a position dataset related to the position of a virtual surgical tool and the positions of virtual body parts that move according to user input. The user input may include user input corresponding to a motion of piercing at least one of the virtual bone, virtual skin, virtual spinal cord, and virtual blood vessel of the subject of the procedure using the virtual surgical tool. The data conversion device (120) can detect whether the virtual surgical tool is located on a virtual 3D body model, that is, whether the position coordinates of the virtual body parts included in the position dataset match the position coordinates of the virtual surgical tool. The data conversion device (120) can generate different surgical effect events depending on the degree to which the position coordinates of the virtual body parts match the position coordinates of the virtual surgical tool. The data conversion device (120) can detect the movement of the virtual surgical tool in the surgical simulation.
[0043] In operation (250), the data conversion device (120) may determine whether to generate a predefined treatment effect event based on the movement of a virtual treatment tool moving into the interior of a virtual three-dimensional body model after contacting the virtual three-dimensional body model in the treatment simulation. To visualize the predefined treatment effect event that may occur depending on the movement of the virtual treatment tool contacting the virtual three-dimensional body model or moving into the interior of the virtual three-dimensional body model, the data conversion device (120) may include information regarding the treatment effect event that may occur when the virtual treatment tool contacts each virtual body part included in the virtual three-dimensional body model. The treatment effect event may be a virtual representation of the effects, reactions, or side effects of the treatment that may actually occur on the body parts of the subject when the actual treatment is performed, in a virtual form for the virtual body parts in the virtual treatment simulation. Based on data regarding the effects, reactions, or side effects of the treatment on the body parts, a predefined treatment effect event that may occur on the virtual body parts can be defined. For example, a predefined procedure effect event may include at least one of the following: an event in which the virtual procedure tool is not inserted into the interior of a virtual 3D body model when the position data of the virtual procedure tool matches some of the position datasets corresponding to a virtual bone; an event in which the virtual procedure tool is fixed to the virtual skin when the position data of the virtual procedure tool matches some of the position datasets corresponding to the virtual skin; an event in which virtual cerebrospinal fluid is ejected from the virtual spinal cord when the position data of the virtual procedure tool matches some of the position datasets corresponding to a virtual spinal cord; and an event in which virtual blood is ejected from the virtual blood vessel when the position data of the virtual procedure tool matches some of the position datasets corresponding to a virtual blood vessel.Additionally, for example, a predefined procedure effect event may include an event in which the degree of tearing of the virtual skin or virtual blood vessel or the speed and degree of blood spurting differ depending on the speed and extent to which the virtual procedure tool moves inside the virtual skin or virtual blood vessel.
[0044] In operation (250), the data conversion device (120) may provide a notification or warning message to the user regarding a procedure effect event occurring on virtual body parts that are not to be touched when a virtual procedure tool comes into contact with virtual body parts that are not to be touched during the procedure simulation process. For example, the data conversion device (120) may provide a warning message to the user when a virtual procedure tool comes into contact with a nerve or various essential organs that are not to be touched during the procedure.
[0045] After operation (250), the data conversion device (120) can extract a location dataset corresponding to virtual body parts where a predefined treatment effect event occurs when a predefined treatment effect event occurs. By extracting the virtual body parts themselves where the treatment effect event occurs or the location dataset corresponding thereto, the data conversion device (120) can separately store or utilize only the information of the virtual body parts that change due to the treatment effect event or the location dataset corresponding thereto.
[0046]
[0047] FIG. 3 is a drawing for illustrating an interface that displays a virtual three-dimensional body model according to one embodiment.
[0048] Referring to FIG. 3, a virtual three-dimensional body model (310) may be generated by sequentially connecting CT image cross-sections (320) included in medical images received by a data conversion device (e.g., the data conversion device (120) of FIG. 1). Based on the sequentially connected medical images, a virtual three-dimensional body model (310) including virtual body parts corresponding to each of the body parts of the subject of the procedure may be generated.
[0049] The user can enlarge or reduce the size of the CT image screen (330) corresponding to the currently selected CT image cross-section (320), and can display or edit specific areas. Additionally, the user can rotate the generated virtual 3D body model (310) and enlarge or reduce the size of the virtual 3D body model (310). The user can select the CT image cross-section (320) or CT image screen (330) to be observed through the first GUI (340), enlarge or reduce the size of the CT image screen (330) through the second GUI (350), and rotate and edit the virtual 3D body model (310) or display specific areas on the CT image screen (330) through the third GUI (360). The CT image cross-section (320) and the CT image screen (330) are linked to each other so that information about a specific area that is edited or displayed in the CT image screen (330) can also be displayed in the CT image cross-section (320) and the corresponding user input can be reflected in a virtual 3D body model including the CT image cross-section.
[0050]
[0051] FIG. 4 is a drawing illustrating an interface for displaying a predefined treatment effect event on a virtual three-dimensional body model according to one embodiment.
[0052] Referring to FIG. 4, a virtual three-dimensional body model (410) containing virtual body parts is configured to display only virtual bones. This can be achieved by a data conversion device (e.g., the data conversion device (120) of FIG. 1) extracting only the location coordinate data mapped to the virtual bones from a location dataset. When a user selects a CT image cross-section (420) to observe and a CT image screen (430) is provided according to the selected CT image cross-section (420), the user can perform a surgical simulation on the virtual bones appearing in at least one of the CT image cross-section (420) and the CT image screen (430) using a virtual surgical tool (not shown). When a user approaches a virtual surgical tool at a point (440) to perform a procedure, that is, for example, when the position coordinates of the virtual surgical tool and the position coordinates of the virtual bone match (200, 200, 50), the data conversion device (120) can generate an event in which the virtual surgical tool is not inserted into the interior of the virtual three-dimensional body model (410).
[0053]
[0054] FIG. 5 is a block diagram illustrating the configurations of a medical image data conversion device according to one embodiment.
[0055] The configurations of the data conversion device (500) are described with reference to FIG. 5. The data conversion device (500) is a device that performs the data conversion method described in the present disclosure.
[0056] The data conversion device (500) may include one or more processors (510), memory (520), and a display unit (530). In one embodiment, some of these components may be omitted or other components may be added to the data conversion device (500).
[0057] The processor (510) can control other components (e.g., hardware or software components) of the data conversion device (600) and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (510) can store instructions or data received from other components in memory (520), process the instructions or data stored in memory (520), and store result data in memory (520). The processor (510) may include a main processor (e.g., a central processing unit or an application processor) or an auxiliary processor (e.g., a graphics processing unit, a neural network processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with it.
[0058] The processor (510) can generate a virtual three-dimensional body model including virtual body parts corresponding to each body part based on medical images received by a communication module (not shown). To generate a virtual three-dimensional body model including virtual body parts from the received medical images, the processor (510) can classify pixels corresponding to each body part of the subject among all pixels of the medical images. Pixels corresponding to each body part of the subject among all pixels of the medical images can be classified based on Hounsfield units (HU), which indicate the degree of radiation absorption according to the type of medical images, or signal intensity, which indicates the intensity of the emitted magnetic resonance signal. The processor (510) can generate voxel data of the medical images based on the medical images and can generate a virtual three-dimensional body model based on the voxel data.
[0059] The processor (510) can generate location coordinate data corresponding to the location of each pixel included in each of the medical images. The processor (510) can generate location coordinate data by arranging the locations of each pixel sequentially. The processor (510) can generate a location dataset related to the locations of virtual body parts by mapping the location coordinate data to a virtual 3D body model. The processor (510) can label the generated 3D location coordinate data to the virtual 3D body model one-to-one.
[0060] The processor (510) can determine whether to generate a predefined surgical effect event based on a position dataset related to the position of a virtual surgical tool and the position of virtual body parts that move according to user input. The processor (510) can detect whether the virtual surgical tool is located on a virtual 3D body model, that is, whether the position coordinates of the virtual body parts included in the position dataset match the position coordinates of the virtual surgical tool, and can detect the movement of the virtual surgical tool in the surgical simulation.
[0061] The operations executed by the processor (510) may include, in addition to the operations described in FIG. 5, operations performed in the data conversion device (e.g., the data conversion device (120) of FIG. 1) described with reference to FIG. 1 to FIG. 4.
[0062] Memory (520) may store instructions executable by one or more processors (510). Data may include, for example, a program (e.g., an application), input or output data for a related command, and log data of a computing system. Memory (520) may include volatile memory such as DRAM (Dynamic RAM) and SRAM (Static RAM), or non-volatile memory such as a magnetic hard disc, an optical disc, a floppy disc, a flash memory, an EPROM (electrically programmable memories), or an EEPROM (electrically erasable and programmable). A storage (not shown) for storing various types of data (e.g., information regarding treatment effect events) used by components of the data conversion device (500) separately from the instructions may be added as another component of the data conversion device (500), and various types of data may be stored in the storage (not shown) separately from the instructions stored in memory (520).
[0063] The display unit (530) can provide the user with an interface provided by the data conversion device (500). The display unit (530) may include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a micro-electromechanical system (MEMS) display, or an electronic paper display. Although the display unit (530) is shown as being included in the data conversion device (500) in FIG. 5, the display unit (530) may be provided externally separately from the data conversion device (500).
[0064] The data conversion device can generate a virtual 3D body model based on received medical images and map position coordinate data to the generated virtual 3D model to detect the degree to which the position coordinates of a virtual surgical tool match the position coordinates of the virtual 3D model, thereby enabling an immediate surgical effect event to occur. In addition, the data conversion device can directly apply DICOM data of medical images to a virtual simulator, thereby simplifying the process of having to use different programs to apply existing DICOM data to a virtual simulator, and thus improving the accuracy and speed of surgical simulation.
[0065]
[0066] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.
[0067] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be stored on any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.
[0068] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either alone or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.
[0069] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.
[0070] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, appropriate results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.
[0071] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.
Claims
1. A method for converting medical image data for a virtual procedure simulation performed by a data conversion device, The action of receiving medical images of the body parts of the subject of the procedure; The operation of generating a virtual three-dimensional body model including virtual body parts corresponding to each of the body parts based on the medical images above; The operation of generating position coordinate data corresponding to the position of each pixel included in each of the above medical images; The operation of generating a location dataset related to the locations of the virtual body parts by mapping the location coordinate data to the virtual 3D body model; and Action of determining whether to trigger a predefined treatment effect event based on a position dataset related to the position of a virtual treatment tool moving according to user input and the positions of the virtual body parts. including, Data conversion method.
2. In Paragraph 1, The action of determining whether to trigger the above-mentioned predefined procedure effect event is, An action of determining whether to trigger the predefined procedure effect event based on the movement of the virtual procedure tool moving into the interior of the virtual 3D body model after contacting the virtual 3D body model in the above procedure simulation. including, Data conversion method.
3. In Paragraph 1, The operation of generating the above-mentioned virtual three-dimensional body model is, The operation of classifying pixels corresponding to each of the body parts of the subject of the procedure from among all the pixels of the medical images; and The operation of determining the location and shape of virtual body parts corresponding to each of the body parts of the subject of the procedure based on the above-described classified pixels. including, Data conversion method.
4. In Paragraph 3, The operation of classifying pixels corresponding to each body part of the subject of the procedure is, An operation of classifying pixels corresponding to each of the body parts of the subject among all pixels of the medical images based on Hounsfield units (HU), which indicate the degree of radiation absorption, or signal intensity, which indicates the intensity of the emitted magnetic resonance signal, according to the type of the medical images. including, Data conversion method.
5. In Paragraph 3, The operation of determining the position and shape of the above-mentioned virtual body parts is, The operation of generating the contours of the virtual body parts by isosurface extraction of pixels corresponding to each of the body parts of the subject of the procedure. including, Data conversion method.
6. In Paragraph 1, The above body parts are, including at least one of the bone, skin, spinal cord, nerve, blood vessel, and organ of the subject of the procedure, Data conversion method.
7. In Paragraph 1, The above user input is, A user input corresponding to a motion of piercing at least one of a virtual bone, virtual skin, virtual spinal cord, virtual nerve, virtual blood vessel, and virtual organ of the subject of the procedure using the virtual surgical tool. Data conversion method.
8. In Paragraph 7, The above-defined predefined treatment effect event is, An event in which the virtual surgical tool is not inserted into the interior of the virtual 3D body model when the position data of the virtual surgical tool matches part of the position dataset corresponding to the virtual bone; An event in which the virtual surgical tool is fixed to the virtual skin when the position data of the virtual surgical tool matches a portion of the position dataset corresponding to the virtual skin; An event in which virtual cerebrospinal fluid is ejected from the virtual spinal cord when the position data of the virtual surgical tool matches part of the position dataset corresponding to the virtual spinal cord; and An event in which virtual blood is ejected from the virtual blood vessel when the position data of the virtual surgical tool matches part of the position dataset corresponding to the virtual blood vessel including at least one of, Data conversion method.
9. In Paragraph 1, The above data conversion method is, The operation of extracting a location dataset corresponding to the virtual body parts where the predefined treatment effect event occurs when the predefined treatment effect event occurs. including more, Data conversion method.
10. A computer program stored on a computer-readable recording medium in combination with hardware to execute the method of claim 1.
11. A data conversion device for medical images for virtual procedure simulation, Memory containing instructions; and It includes a processor connected to the memory and for executing the instructions, When the above instructions are executed by the processor, the data conversion device, Receive medical images of the body parts of the subject of the procedure, and Based on the medical images above, a virtual 3D body model is generated that includes virtual body parts corresponding to each of the body parts, and Generate location coordinate data corresponding to the location of each pixel included in each of the above medical images, and A location dataset related to the locations of the virtual body parts is generated by mapping the location coordinate data to the virtual 3D body model, and Determining whether to trigger a predefined treatment effect event based on a position dataset related to the position of a virtual treatment tool moving according to user input and the positions of the virtual body parts. Data conversion device.
12. In Paragraph 11, When the above instructions are executed by the processor, the data conversion device, Determining whether to trigger the predefined procedure effect event based on the movement of the virtual procedure tool moving into the interior of the virtual 3D body model after contacting the virtual 3D body model in the above procedure simulation. Data conversion device.
13. In Paragraph 11, When the above instructions are executed by the processor, the data conversion device, Among all the pixels of the above medical images, pixels corresponding to each of the body parts of the subject of the procedure are classified, and Determining the location and shape of virtual body parts corresponding to each of the body parts of the subject of the procedure based on the above-described classified pixels, Data conversion device.
14. In Paragraph 13, When the above instructions are executed by the processor, the data conversion device, Classifying pixels corresponding to each of the body parts of the subject of the procedure from among all pixels of the medical images based on Hounsfield units (HU), which indicate the degree of radiation absorption according to the type of the medical images, or signal intensity, which indicates the intensity of the emitted magnetic resonance signal. Data conversion device.
15. In Paragraph 13, When the above instructions are executed by the processor, the data conversion device, Generating the contours of the virtual body parts by extracting equivalent surfaces from pixels corresponding to each of the body parts of the subject of the procedure, Data conversion device.
16. In Paragraph 11, The above body parts are, including at least one of the bone, skin, spinal cord, nerve, blood vessel, and organ of the subject of the procedure, Data conversion device.
17. In Paragraph 11, The above user input is, A user input corresponding to a motion of piercing at least one of a virtual bone, virtual skin, virtual spinal cord, virtual nerve, virtual blood vessel, and virtual organ of the subject of the procedure using the virtual surgical tool. Data conversion device.
18. In Paragraph 17, The above-defined predefined treatment effect event is, An event in which the virtual surgical tool is not inserted into the interior of the virtual 3D body model when the position data of the virtual surgical tool matches part of the position dataset corresponding to the virtual bone; An event in which the virtual surgical tool is fixed to the virtual skin when the position data of the virtual surgical tool matches a portion of the position dataset corresponding to the virtual skin; An event in which virtual cerebrospinal fluid is ejected from the virtual spinal cord when the position data of the virtual surgical tool matches part of the position dataset corresponding to the virtual spinal cord; and An event in which virtual blood is ejected from the virtual blood vessel when the position data of the virtual surgical tool matches part of the position dataset corresponding to the virtual blood vessel including at least one of, Data conversion device.
19. In Paragraph 11, When the above instructions are executed by the processor, the data conversion device, When the above-mentioned predefined procedure effect event occurs, further extracting a location dataset corresponding to the above-mentioned virtual body parts where the above-mentioned predefined procedure effect event occurs, Data conversion device.