Virtual-reality-based dissection-assisting device using lesion-customized clipping, method, and computer-readable recording medium
The virtual reality-based dissection assistance device addresses the need for precise surgical training by allowing medical professionals to interact with customized 3D models of internal body areas, enhancing training and treatment precision through lesion-specific virtual reality simulations.
Patent Information
- Application Number
- PCT/KR2024/002627
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-02
- Filing Date
- 2024-02-29
- Publication Date
- 2024-11-07
AI Technical Summary
Medical professionals face challenges in obtaining sufficient practice opportunities for surgical training due to the high precision required in surgery, while virtual reality technology offers an efficient and cost-effective means for immersive simulations, necessitating the development of tools that can create customized 3D models for lesion examination within the body based on specific lesion information.
A virtual reality-based dissection assistance device and method that includes a model output unit, a model determination unit to clip a 3D model based on lesion information, and a clipping unit to position the virtual model at specific points within the 3D model, allowing for the efficient observation of internal body areas containing lesion information during training or treatment.
Enables medical professionals to efficiently observe and interact with customized 3D models of internal body areas, enhancing training and treatment precision by allowing for real-time modification and visualization of lesion information using VR, AR, or MR technologies.
Smart Images

Figure KR2024002627_07112024_PF_FP_ABST
Abstract
Description
Virtual reality-based dissection assistance device, method, and computer-readable recording medium using lesion-specific clipping
[0001] The present invention relates to a virtual reality-based dissection assistance device, method, and computer-readable recording medium using lesion-specific clipping.
[0002]
[0003] Virtual reality technology is a field of the 4th industrial revolution that can be applied to various fields such as entertainment, education, medicine, and national defense. Recently, virtual reality technologies such as VR, AR, and MR are diversifying.
[0004] Recently, with the proliferation of VR content creation tools like MRTK2-Unity and VR headsets like Hololens across various fields, VR technology is expanding into specialized fields of education. In particular, medical fields, such as anatomy training and surgical practice, and engineering fields, including design and architecture, are leading areas utilizing VR technology.
[0005] Meanwhile, the work of medical professionals performing surgeries requires a high degree of precision, requiring extensive training. However, it's difficult to provide students with the hands-on training they desire. Conversely, medical education utilizing virtual reality technology is highly efficient in terms of time and cost, and offers highly immersive simulations. Therefore, there's a growing demand for diverse educational content utilizing virtual reality technology and content control technologies tailored to the specific needs of the medical field.
[0006]
[0007] The problem to be solved in one embodiment is to provide a technology that efficiently outputs the internal body area that needs to be observed to examine a patient according to the lesion information into a virtual space based on VR, AR, or MR by clipping a 3D model into a customized form according to the lesion information.
[0008] However, the technical problems to be solved by the embodiments of the present invention are not limited to the problems mentioned above, and various technical problems can be derived from the contents described below within a scope obvious to those skilled in the art.
[0009]
[0010] An anatomical assistance device according to one embodiment may include a model output unit that outputs a three-dimensional model modeling a body to an output device; a model determination unit that determines a virtual model to clip the three-dimensional model based on a setting of lesion information; and a clipping unit that clips an area occupied by the virtual model in the three-dimensional model by positioning the virtual model at a specific point in the three-dimensional model by a user.
[0011] Additionally, the three-dimensional model may be composed of multiple layers including an outer layer and an inner layer of the body.
[0012] Additionally, the lesion information can be set based on user input or patient diagnosis.
[0013] Additionally, the model determination unit may preset the size or shape of the virtual model to be determined based on the lesion information.
[0014] Additionally, the model determination unit may obtain information about the shape of the lesion of the target patient, and the size or shape of the virtual model may be preset based on the obtained information about the shape of the lesion.
[0015] In addition, the model determination unit may store information on the location, size, shape, and appearance of the lesion according to the type of lesion, the progression according to the patient's information, and the shape of the lesion according to the progression period, and determine the virtual model based on this.
[0016] In addition, the model determination unit may determine to additionally modify the size or shape of the virtual model according to the patient's body information in addition to the lesion information.
[0017] Additionally, the virtual model may include a sphere.
[0018] Additionally, the model output unit can output layer information of a portion corresponding to the surface of the virtual model in the three-dimensional model to the output device.
[0019] A method for controlling an anatomy assistance device according to one embodiment of the present invention may include the steps of: outputting a three-dimensional model modeling a body to an output device; determining a virtual model to clip the three-dimensional model based on a setting of lesion information; and clipping an area occupied by the virtual model in the three-dimensional model by having the user position the virtual model at a specific point in the three-dimensional model.
[0020] Additionally, the three-dimensional model may be composed of multiple layers including an outer layer and an inner layer of the body.
[0021] Additionally, the lesion information can be set based on user input or patient diagnosis.
[0022] Additionally, the size or shape of the virtual model may be preset to be determined based on the lesion information.
[0023] Additionally, the virtual model may be configured to be additionally modified in size or shape according to the patient's physical information in addition to the lesion information.
[0024] Additionally, the virtual model may include a sphere.
[0025] In addition, after the clipping step, a step of outputting layer information of a portion corresponding to the surface of the virtual model in the 3D model to the output device may be further included.
[0026] A computer-readable recording medium storing a computer program according to one embodiment of the present invention may include instructions for causing a processor to perform a control method of an anatomy assistance device, the method comprising the steps of: outputting a three-dimensional model modeling a body to an output device; determining a virtual model to clip the three-dimensional model based on a setting of lesion information; and clipping an area occupied by the virtual model in the three-dimensional model by having a user position the virtual model at a specific point in the three-dimensional model.
[0027]
[0028] In one embodiment, by determining a virtual model to clip a 3D model based on lesion information settings, a user can clip a 3D model into a customized shape based on lesion information at a specific point. This allows for efficient observation of internal body regions with specific lesion information during actual patient care or medical professional training.
[0029] In addition, various effects may be provided, either directly or indirectly, through this document.
[0030]
[0031] Figure 1 is a functional block diagram of a dissection assistance device according to one embodiment.
[0032] Figure 2 is an example diagram of a three-dimensional model according to one embodiment.
[0033] FIG. 3 is an example of an operation of setting up a virtual model having a plane model according to one embodiment and clipping a three-dimensional model by specifying an area using a linkage control device.
[0034] FIG. 4 is an example of an operation of setting up a virtual model having a concrete model according to one embodiment and clipping a three-dimensional model by specifying an area using a linkage control device.
[0035] FIG. 5 is an example of an operation in which a user clips a 3D model by specifying an area different from the above-described area using a linkage control device during the exemplary operation of FIG. 4.
[0036] FIG. 6 is a flowchart illustrating the operation of a control method of an anatomy assistance device performed by an anatomy assistance device according to one embodiment.
[0037]
[0038] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the scope of the present invention is defined solely by the claims.
[0039] In describing embodiments of the present invention, specific descriptions of known functions or configurations will be omitted unless truly necessary. Furthermore, the terms described below are defined based on their functions in the embodiments of the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of this specification.
[0040] The functional blocks depicted in the drawings and described below are merely examples of possible implementations. Other implementations may utilize other functional blocks without departing from the spirit and scope of the detailed description. Furthermore, while one or more functional blocks of the present invention are depicted as individual blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software configurations that perform the same function.
[0041] Furthermore, the expression "including certain components" is an open expression, meaning that it simply indicates the presence of those components and should not be understood as excluding additional components.
[0042] Furthermore, when it is said that a component is connected or connected to another component, it should be understood that it may be directly connected or connected to that other component, but that there may also be other components in between.
[0043] Additionally, expressions such as 'first, second' are used only to distinguish between multiple components, and do not limit the order or other characteristics between the components.
[0044] The dissection assistance device (100) of this document is a device that calculates data of a three-dimensional model, including a memory that stores data and commands and one or more processors. The dissection assistance device (100) can be implemented as various devices that output a three-dimensional model to a predetermined output device through a processor calculation. For example, the dissection assistance device (100) can be implemented as an HMD (Head Mounted Display) device that outputs data of a calculated three-dimensional model in a virtual space or a real space using virtual / augmented reality technologies such as VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality). For example, the dissection assistance device (100) can be implemented as a device that is connected to an external output device (e.g., a monitor, a smartphone, a projector, a video transmitter, etc.) by wire or wirelessly and outputs data of a calculated three-dimensional model to the external output device.
[0045] Figure 1 is a functional block diagram of a dissection assistance device (100) according to one embodiment.
[0046] Referring to FIG. 1, the processor of the dissection assistance device (100) can execute commands by implementing function blocks (modules) so that operations are performed according to the commands in the memory. The function blocks implemented by the processor of the dissection assistance device (100) can include a model output unit (110), a model determination unit (120), and a clipping unit (130).
[0047] The model output unit (110) can output a three-dimensional model modeling all or part of the body to an output device. The output device may include an internal output device included in the dissection assistance device (100) itself, or an external output device connected to the dissection assistance device (100). For example, the output device may include an output device exemplified in the introduction (e.g., a display of an HMD, a monitor, a smartphone, a projector, a video transmitter, etc.).
[0048] Figure 2 is an example diagram of a three-dimensional model according to one embodiment.
[0049] Referring to FIG. 2, a three-dimensional model of one embodiment may be composed of multiple layers, including an inner layer (FIG. 2(a)) and an outer layer (FIG. 2(b)) of the body. The three-dimensional model may be implemented in various ways, and examples of ways to implement the three-dimensional model may include the technology described in the "Patent Documents" section of the "Prior Art Documents" section of this specification.
[0050] The model determination unit (120) can determine a virtual model to be used for clipping a 3D model based on the lesion information settings. In this document, clipping refers to an operation of cutting out a specific area of a 3D model or deleting a specific area of a 3D model to observe the interior of the 3D model. The lesion information may include information about the location of the disease or lesion, the type of disease, the degree of the disease, etc. For example, the lesion information may be set based on a user's input or a patient's diagnosis.
[0051] The model determination unit (120) can determine the size or shape of a virtual model based on lesion information. The virtual model may be preset based on lesion information. For example, the virtual model may include various shapes such as planes, curved surfaces, polyhedrons, spheres (including elliptical spheres), etc., and the virtual model to be used may be preset based on lesion information.
[0052] For example, by obtaining information about the shape of a lesion, such as a tumor or trauma, of a specific patient and determining and designing a virtual model based on this to fit the area lost due to the tumor or trauma, it is possible to accurately predict the structural defect expected in an actual patient.
[0053] As another example, by pre-storing information on lesion location, size, shape, and appearance depending on tumor type, patient progression based on age, gender, etc., and lesion shape at each stage of progression, a virtual model can be created based on this information, allowing for more accurate predictions of expected structural defects in actual patients. At this time, information on underlying lesion changes and shape can be acquired through processing of previously collected data.
[0054] The model determination unit (120) can further modify the virtual model by reflecting the patient's physical information. For example, the model determination unit (120) can determine to further modify the size or shape of the virtual model based on the patient's physical information (e.g., gender, age, height, weight, special characteristics, etc.) in addition to lesion information.
[0055] The clipping unit (130) allows the user to position a virtual model at a specific point in the 3D model, thereby clipping an area occupied by the virtual model in the 3D model. For example, the user can position the virtual model by specifying an area of the 3D model using various linkage control devices (200) that are connected to the dissection assistance device (100) via wired or wireless connection and assist in operating the dissection assistance device (100). For example, if the dissection assistance device (100) is implemented as an HMD device, the user can point to a specific point with a VR controller. For example, if the dissection assistance device (100) is implemented as a computer device, the user can point to a specific point with a mouse or touch input.
[0056] When the clipping unit (130) clips the area occupied by the virtual model in the 3D model, the model output unit (110) can output layer information of the portion corresponding to the surface of the virtual model in the 3D model to the output device.
[0057] The model output unit (110) can output layer information of a clipped 3D model as 3D information. For example, the model output unit (110) can output layer information of a clipped 3D model based on the user's perspective when the user manipulates the position of the 3D model or changes the position from which the user views the 3D model.
[0058] The model output unit (110) can output layer information of a 3D model that is clipped in response to the movement of a virtual model as the user moves the position of the virtual model in the 3D model. For example, the model output unit (110) can output layer information of the 3D model by changing the position of the clipping in response to the movement of the virtual model in an area of the 3D model as the user moves the position of the virtual model in real time.
[0059] FIG. 3 is an example of an operation of setting up a virtual model having a plane model according to one embodiment and clipping a three-dimensional model by specifying an area using a linkage control device (200).
[0060] Referring to FIG. 3, the clipping unit (130) can position a virtual model (A) having a plane model at a specific point of a 3D model by a user, and can cut out an area occupied by the virtual model in the 3D model. The model output unit (110) can output 3D information of the cut layer. The model output unit (110) can output layer information of one side or the other side of the 3D model cut out by the virtual model (A) in the 3D model to an output device. For example, the layer information can output 3D information clipped based on the user's perspective when looking at the 3D model.
[0061] FIG. 4 is an example of an operation of setting up a virtual model having a concrete model according to one embodiment and clipping a three-dimensional model by specifying an area using a linkage control device (200).
[0062] Referring to FIG. 4, the clipping unit (130) can position a virtual model (B) having a spherical shape at a specific point of a 3D model by a user, and can delete an area occupied by the virtual model in the 3D model. The model output unit (110) can output layer information of a 3D model corresponding to the surface of the virtual model (B) in the 3D model to an output device. For example, the layer information can output 3D information clipped based on the user's perspective when looking at the 3D model.
[0063] FIG. 5 is an example of an operation in which a user clips a 3D model by specifying an area different from one area of FIG. 4 using a linkage control device (200) during the exemplary operation of FIG. 4.
[0064] Referring to Fig. 5, the area where the virtual model (B) is located can be moved to the right compared to Fig. 4 by the user's manipulation. As the location of the virtual model (B) moves to the right, the model output unit (110) can output the layer information to be clipped among the 3D model by moving it to the right in response to the movement of the virtual model.
[0065] Fig. 6 is a flowchart illustrating the operation of a control method of a dissection assistance device (100) performed by a dissection assistance device (100) according to one embodiment. Each step of the control method according to Fig. 6 can be performed by the dissection assistance device (100) described through Fig. 1, and each step is described as follows.
[0066] In step S1010, the model output unit (110) can output a three-dimensional model modeling the body to a predetermined output device.
[0067] In step S1020, the model determination unit (120) can determine a virtual model to clip the 3D model based on the setting of the lesion information.
[0068] In step S1030, the clipping unit (130) can clip an area occupied by the virtual model in the 3D model by positioning the virtual model at a specific point in the 3D model.
[0069] Meanwhile, in addition to the steps illustrated in FIG. 6, since the above-described information model output unit (110), model determination unit (120), and clipping unit (130) are configured in various embodiments to perform the operations described with FIG. 1, new steps performed by each functional block may be added even in the steps of FIG. 6, and the configuration of additional steps and the operations of the components that are the subjects of each step to perform the corresponding steps have been described in FIGS. 1 to 5, and therefore, duplicate descriptions are omitted.
[0070] According to the above-described embodiment, by determining a virtual model to clip the 3D model based on the lesion information settings, the user can clip the 3D model in a customized form based on the lesion information at a specific point. This allows for efficient observation of internal body regions with specific lesion information during actual patient treatment or medical professional training.
[0071] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented through hardware, firmware, software, or a combination thereof.
[0072] In the case of hardware implementation, the method according to embodiments of the present invention may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.
[0073] When implemented via firmware or software, the methods according to embodiments of the present invention may be implemented in the form of modules, procedures, or functions that perform the functions or operations described above. A computer program containing software codes, etc. may be stored on a computer-readable recording medium or a memory unit and executed by a processor. The memory unit may be located within or outside the processor and may exchange data with the processor via various known means.
[0074] Furthermore, the combinations of each block of the block diagram and each step of the flowchart attached to the present invention may be performed by computer program instructions. These computer program instructions may be installed in an encoding processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions executed by the encoding processor of the computer or other programmable data processing equipment create a means for performing the functions described in each block of the block diagram or each step of the flowchart. These computer program instructions may also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing equipment to implement a function in a specific manner, so that the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes an instruction means for performing the functions described in each block of the block diagram or each step of the flowchart. Since the computer program instructions can also be installed on a computer or other programmable data processing device, a series of operational steps are performed on the computer or other programmable data processing device to create a computer-executable process, and the instructions that cause the computer or other programmable data processing device to perform the steps for executing the functions described in each block of the block diagram and each step of the flowchart can also provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.
[0075] Additionally, each block or step may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specific logical function. It should also be noted that in some alternative embodiments, the functions described in the blocks or steps may occur out of order. For example, two blocks or steps depicted in succession may actually be performed substantially concurrently, or the blocks or steps may sometimes be performed in reverse order, depending on the corresponding functionality.
[0076] As such, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering its technical spirit or essential characteristics. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims below rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included within the scope of the present invention.
Claims
1. A model output unit that outputs a 3D model of a body to an output device; A model determination unit that determines a virtual model to clip the 3D model based on the setting of lesion information; and In the above 3D model, a clipping unit is included that allows a user to position the virtual model at a specific point and clips the area occupied by the virtual model in the 3D model. Dissecting aids.
2. In paragraph 1, The above 3D model It consists of multiple layers including the outer layer and inner layer of the body. Dissecting aids.
3. In paragraph 1, The above lesion information Set based on user input or patient diagnosis Dissecting aids.
4. In paragraph 1, The above model decision part The size or shape of the virtual model to be determined based on the above lesion information is preset. Dissecting aids.
5. In paragraph 4, The above model decision part In addition to the above lesion information, it is decided to further modify the size or shape of the virtual model according to the patient's body information. Dissecting aids.
6. In paragraph 1, The above virtual model Containing a sphere Dissecting aids.
7. In paragraph 1, The above model output section Outputting layer information of a portion corresponding to the surface of the virtual model in the above 3D model to the output device Dissecting aids.
8. In a method for controlling an anatomy assist device performed by an anatomy assist device, A step of outputting a three-dimensional model of a body to an output device; A step of determining a virtual model to clip the 3D model based on the setting of lesion information; and A step of clipping an area occupied by the virtual model in the 3D model by allowing the user to position the virtual model at a specific point in the 3D model. Method for controlling an anatomy auxiliary device.
9. In paragraph 8, The above 3D model It consists of multiple layers including the outer layer and inner layer of the body. Method for controlling an anatomy auxiliary device.
10. In paragraph 8, The above lesion information Set based on user input or patient diagnosis Method for controlling an anatomy auxiliary device.
11. In paragraph 8, The above virtual model is The size or shape of the virtual model to be determined based on the above lesion information is preset. Method for controlling an anatomy auxiliary device.
12. In paragraph 11, The above virtual model is In addition to the above lesion information, the size or shape is determined to be additionally modified based on the patient's physical information. Method for controlling an anatomy auxiliary device.
13. In paragraph 8, The above virtual model is Containing a sphere Method for controlling an anatomy auxiliary device.
14. In paragraph 8, After the above clipping step, Further comprising a step of outputting layer information of a portion corresponding to the surface of the virtual model in the three-dimensional model to the output device. Method for controlling an anatomy auxiliary device.
15. A computer-readable recording medium storing a computer program, A step of outputting a three-dimensional model of a body to an output device; A step of determining a virtual model to clip the 3D model based on the setting of lesion information; and A step of clipping an area occupied by the virtual model in the 3D model by allowing the user to position the virtual model at a specific point in the 3D model. A computer-readable recording medium comprising instructions for causing a processor to perform a method of controlling an anatomy assistance device.
Citation Information
Patent Citations
Medical image processing device, and 3D image processing method
JP2012165780A
Medical image display control device and method, and program
JP2014144156A
Augmented Reality Surgical Navigation
JP2022017422A
Automatic Cleaning System of Water Tank for Water Quality Measuring Instrument
KR102134269B1
