Semi-automatic process-based virtual pneumoperitoneum model generating apparatus and method

Abstract

The present disclosure relates to a semi-automatic process-based virtual pneumoperitoneum model generating apparatus and method. The apparatus may comprise: a memory storing a process for generating a virtual pneumoperitoneum model; and a processor for performing an operation according to the process, wherein the processor is configured to perform mask processing on a CT image of a patient when generation of a pneumoperitoneum model of the patient is requested from a user terminal, generate the pneumoperitoneum model on the basis of the result of the mask processing, perform inspection on the generated pneumoperitoneum model, display, on a screen of the user terminal, a first user interface for the mask processing, and display, on a screen of an inspector terminal, a second user interface for the inspection.

Description

Device and method for generating a virtual undulation model based on a semi-automatic process

[0001] The present disclosure relates to a device and method for generating a virtual undulation model based on a semi-automatic process.

[0002] There is a growing need for devices and software that enable medical professionals to train in realistic situations. Typically, simulation devices for medical professionals are designed to mimic patient situations and then used for training.

[0003] In particular, when performing minimally invasive surgery (e.g., robotic surgery or laparoscopic surgery), it is important to perform training under the same conditions and environment as the actual surgery. However, depending on the position and shooting direction of the camera, the intra-abdominal structures seen in the image may differ, which may lead to situations where the training effect is not sufficiently obtained.

[0004] Therefore, a process is needed to predict the patient's actual pneumoperitoneum (a state in which the patient's abdomen is inflated by injecting gas into the patient's body to facilitate surgery) so that the medical staff can receive the same images as during an actual surgery during a surgical simulation.

[0005] The purpose of the embodiment disclosed in the present disclosure is to provide a device and method for generating a virtual undulation model based on a semi-automatic process.

[0006] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0007] In order to achieve the above-described technical problem, a virtual undulation model generation device based on a semi-automatic process according to the present disclosure comprises: a memory storing a process for generating a virtual undulation model; and a processor performing an operation according to the process, wherein when a request for generation of a patient's undulation model is made from a user's terminal, the processor performs mask processing on a CT image of the patient, generates the undulation model based on a result of the mask processing, performs inspection on the generated undulation model, and causes a first user interface for the mask processing to be displayed on a screen of the user's terminal, and causes a second user interface for the inspection to be displayed on a screen of the inspector's terminal.

[0008] In addition, a method for generating a virtual undulation model based on a semi-automatic process performed by a processor of a device according to the present disclosure includes, when a request for generation of a patient's undulation model is made from a user's terminal, the steps of: performing mask processing on a CT image of the patient; generating the undulation model based on a result of the mask processing; and performing inspection on the generated undulation model, wherein the step of performing the mask processing may cause a first user interface for the mask processing to be displayed on a screen of the user's terminal, and the step of performing the inspection may cause a second user interface for the inspection to be displayed on a screen of the inspector's terminal.

[0009] In addition, a computer program stored in a computer-readable recording medium for executing a method for implementing the present disclosure may be further provided.

[0010] In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

[0011] According to the aforementioned problem solving means of the present disclosure, by performing a masking operation on the patient's anatomical structure in a CT image, the accuracy of the resulting virtual undulation model can be increased.

[0012] Additionally, the inspection process for the virtual volatility model allows inspectors to perform inspections efficiently, and accuracy can be improved by applying the inspection results to the volatility model.

[0013] This has the effect of increasing the success rate of surgery because medical staff can train using the same images as during actual surgery when performing surgical simulation.

[0014] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0015] FIG. 1 is a block diagram of a virtual undulation model generation device based on a semi-automatic process according to one embodiment of the present disclosure.

[0016] FIG. 2 is a flowchart of a method for generating a virtual undulation model based on a semi-automatic process according to one embodiment of the present disclosure.

[0017] FIG. 3 is a drawing for explaining the entire process of creating a 3D abdominal model of a patient according to one embodiment of the present disclosure.

[0018] FIG. 4 is a drawing for explaining an inspection method through comparison of a 3D relief model and a CT image according to one embodiment of the present disclosure.

[0019] FIGS. 5 to 13 are drawings for explaining a user interface for checking a undulation model according to one embodiment of the present disclosure.

[0020] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and any content that is common in the technical field to which this disclosure belongs or that overlaps between embodiments is omitted. The terms "part, module, element, block" used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple "parts, modules, elements, blocks" may be implemented as a single component, or a single "part, module, element, block" may include multiple components. Throughout the specification, when a part is said to be "connected" to another part, this includes not only cases where it is directly connected, but also cases where it is indirectly connected, and an indirect connection includes a connection via a wireless communication network.

[0021] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0022] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0023] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0024] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0025] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0026] The operating principle and embodiments of the present disclosure are described below with reference to the attached drawings.

[0027] Before proceeding, the meanings of terms used in this specification will be briefly explained. However, it should be noted that the explanation of terms is intended to aid understanding of this specification and, unless explicitly stated to limit the disclosure, is not intended to limit the technical concepts of this disclosure.

[0028] In this specification, the term "device" encompasses a variety of devices capable of performing computational processing and providing results to a user. For example, a device may include a computer, a server, or a portable terminal, or may be any one of these.

[0029] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0030] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0031] The above portable terminal may include, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as a PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, a smart phone, and a wearable device such as a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD).

[0032] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0033] FIG. 1 is a block diagram of a virtual undulation model generation device based on a semi-automatic process according to one embodiment of the present disclosure.

[0034] A virtual tummy tuck model generation device (10) (hereinafter, tummy tuck model generation device) based on a semi-automatic process according to one embodiment of the present disclosure can perform a masking operation on a patient's CT image and produce a final 3D tummy tuck model through inspection of a 3D tummy tuck model of the patient generated based on the masking result.

[0035] Here, the masking operation may include an operation performed automatically by a computer and an operation performed manually by a user.

[0036] The undulation model generation device (10) can request a masking task to a user terminal (20) for a task performed manually during a masking task, and receive the task result from the user terminal (20).

[0037] The undulation model generation device (10) can request an inspection of the 3D undulation model of the patient generated according to the masking operation to the inspector's terminal (30), and can produce a final 3D undulation model by reflecting the inspection result received from the inspector's terminal (30).

[0038] Here, the 3D undulation model (virtual undulation model) may be a model that predicts the actual undulation state of a patient scheduled for surgery (procedure).

[0039] Here, the term "surgery" (procedure) can refer to minimally invasive surgery (e.g., laparoscopic surgery or robotic surgery). Minimally invasive surgery involves viewing a portion of the patient's body via a camera inserted through a trocar, while surgical instruments are inserted through one or more trocars inserted elsewhere.

[0040] Minimally invasive surgery relies solely on internal visualization through a camera (i.e., an endoscope) inserted into the body. Therefore, if medical staff practice with images displayed in a completely different position or orientation during a virtual surgical simulation before performing the actual surgery, the medical staff will receive a different image during the actual surgery, effectively eliminating the training effect. Therefore, it is crucial to improve the accuracy of patient mood prediction data used in practice.

[0041] The undulation model generation device (10) may be a server device that provides a 3D undulation model production service. The 3D undulation model production service may be provided to users in the form of an online web or app.

[0042] The user terminal (20) and inspector terminal (30) may refer to terminal devices of users utilizing the service. Users can use the service in the form of an online web or app by installing a service application (program) provided by the service server (10) on their terminals.

[0043] At this time, a user application and an inspector application may be provided separately. Users can use the service by installing the user application on their user terminal (20), and inspectors can use the service by installing the inspector application on their inspector terminal (30).

[0044] Here, the user may be the person performing the masking task during the creation of the undulation model, and the inspector may be the person performing the inspection during the creation of the undulation model. In this case, the user may be the person who requested the creation of the undulation model using the undulation model creation device (10). However, this is not limited to this, and the request for the creation of the undulation model may be made by the inspector, or a third party other than the user or inspector.

[0045] The user terminal (20) and the inspector terminal (30) may be applied with information processing means such as a computer, and may include a processor such as a control unit, a photographing means such as a camera, an input / output means including a touch screen, and may refer to any device with a communication function. In other words, any device such as a smartphone, tablet, PDA, laptop, or desktop may be applied.

[0046] The ups and downs model generation device (10) acts as a server, and the user terminal (20) and the inspector terminal (30) act as clients. The server and the client can seamlessly link data between the two to shorten the time required for creating a ups and downs model. An automated process is performed during the ups and downs model creation process in the ups and downs model generation device (10), which is a server, and manual work can be performed during the ups and downs model creation process in the user terminal (20) and the inspector terminal (30), which are clients.

[0047] Referring to FIG. 1, the undulation model generation device (10) may include a communication unit (11), a memory (12), and a processor (13). The processor (13) may include a masking module (131), a undulation model generation module (132), and an inspection management module (133). However, in some embodiments, the undulation model generation device (10) and the processor (13) may include fewer or more components than the components illustrated in FIG. 1.

[0048] The communication unit (11) may include one or more modules that enable wireless or wired communication between the undulation model generation device (10) and the user terminal (20), between the undulation model generation device (10) and the inspector terminal (30), between the undulation model generation device (10) and an external server (not shown), and between the undulation model generation device (10) and a communication network. For example, it may include at least one of a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0049] The external server (not shown) may be a hospital server, or a server storing medical data for multiple patients. The medical data may include, but is not limited to, image data from before the patient's induction.

[0050] The communication network can use various types of communication networks, for example, wireless communication methods such as WLAN (Wireless LAN), Wi-Fi, Wibro, WiMAX, and HSDPA (High Speed ​​Downlink Packet Access), or wired communication methods such as Ethernet, xDSL (ADSL, VDSL), HFC (Hybrid Fiber Coax), FTTC (Fiber to The Curb), and FTTH (Fiber to The Home) can be used.

[0051] Meanwhile, the communication network is not limited to the communication methods presented above, and may include all other widely known or future-developed communication methods in addition to the above-described communication methods.

[0052] The wired communication module may include various wired communication modules such as a Local Area Network (LAN) module, a Wide Area Network (WAN) module, or a Value Added Network (VAN) module, as well as various cable communication modules such as a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a Digital Visual Interface (DVI), RS-232 (recommended standard 232), power line communication, or plain old telephone service (POTS).

[0053] The wireless communication module may include a wireless communication module that supports various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G, in addition to a WiFi module and a Wireless Broadband module.

[0054] The short-range communication module is for short-range communication, and can support short-range communication using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0055] The memory (12) may store at least one process for generating a virtual model of the patient's mood.

[0056] The memory (12) can store data supporting various functions of the present undulation model generation device (10), a program for the operation of the processor (13), can store input / output data (e.g., music files, still images, moving images, etc.), and can store a plurality of application programs (or applications) run by the present undulation model generation device (10), data for the operation of the present undulation model generation device (10), and commands. At least some of these application programs can be downloaded from an external server via wireless communication.

[0057] The memory (12) may include at least one type of storage medium among a flash memory type, a hard disk type, an SSD (Solid State Disk type), an SDD (Silicon Disk Drive type), a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory (12) is separate from the present undulation model generation device (10), but may be a database connected by wire or wirelessly.

[0058] The processor (13) can perform the aforementioned operations using a memory that stores data regarding an algorithm for controlling the operations of components within the present undulation model generation device (10) or a program that reproduces the algorithm, and the data stored in the memory. In this case, the memory (12) and the processor (13) may be implemented as separate chips. Alternatively, the memory (12) and the processor (13) may be implemented as a single chip.

[0059] When a masking module (131) of a processor (13) receives a request for generating a patient's undulation model from a user terminal (20), it can perform mask processing on the patient's CT image.

[0060] Here, CT images are medical imaging data that depict the patient's body in 3D before and after surgery. In addition to CT images, other imaging data, such as magnetic resonance imaging (MRI) and positron emission tomography (PET), may also be utilized.

[0061] The undulation model generation module (132) of the processor (13) can generate a 3D undulation model of the patient based on the mask processing result.

[0062] The inspection management module (133) of the processor (13) can request an inspection of the undulation model generated by the inspector terminal (30), receive the inspection result from the inspector terminal (30), store and manage the received inspection result, and reflect the received inspection result in the generated undulation model to produce the final undulation model.

[0063] In addition, the processor (13) can control one or more of the components discussed above in combination to implement various embodiments according to the present disclosure described in FIGS. 2 to 13 below on the undulation model generation device (10).

[0064] Hereinafter, with reference to FIGS. 2 to 13, a method for generating a virtual undulation model based on a semi-automatic process will be described in detail.

[0065] FIG. 2 is a flowchart of a method for generating a virtual undulation model based on a semi-automatic process according to one embodiment of the present disclosure.

[0066] FIG. 3 is a drawing for explaining the entire process of creating a 3D abdominal model of a patient according to one embodiment of the present disclosure.

[0067] FIG. 4 is a drawing for explaining an inspection method through comparison of a 3D relief model and a CT image according to one embodiment of the present disclosure.

[0068] FIGS. 5 to 13 are drawings for explaining a user interface for checking a undulation model according to one embodiment of the present disclosure.

[0069] For convenience of explanation, the following description describes each step as being performed by the processor (13), but it can be understood that each step is performed by any one of the masking module (131), the undulation model generation module (132), the inspection management module (133), and modules not shown in FIG. 1 included in the processor (13).

[0070] Referring to FIG. 2, when the processor (13) of the undulation model generation device (10) receives a request for generation of a patient's undulation model from a user terminal (20) through a communication unit (11), it can perform mask processing on the patient's CT image (S210).

[0071] The processor (13) can receive a patient's CT image (or a user terminal local CT file) from a cloud server at the hospital (31 in FIG. 3). At this time, the CT image can be pseudonymized or anonymized to protect personal information.

[0072] The processor (13) can perform mask processing on the received CT image.

[0073] Here, the mask processing may include an automatic mask segmentation step (32 in FIG. 3), a mask inspection supplementation step (33 in FIG. 3), a blood vessel mask classification step (34 in FIG. 3), a CT mask automatic alignment step (35 in FIG. 3), and an automatic recognition step (36 in FIG. 3).

[0074] The processor (13) can automatically process the first task among the mask automatic segmentation step (32 in FIG. 3), the blood vessel mask classification step (34 in FIG. 3), the CT mask automatic alignment step (35 in FIG. 3) and the automatic recognition step (36 in FIG. 3), but can request a masking task to the user terminal (20) for the second task among the mask inspection supplement step (33 in FIG. 3) and the blood vessel mask classification step (34 in FIG. 3). The user can perform the requested task through a user interface (UI) for the masking task displayed on the screen of the user terminal (20).

[0075] In the automatic mask segmentation step (32 in FIG. 3), the processor (13) can automatically segment the patient's anatomical structures, such as organs and blood vessels, in the CT image using artificial intelligence in the service server.

[0076] In the mask inspection supplementary step (33 in FIG. 3), the processor (13) transmits the automatic segmentation result from the service server to the client (user terminal (20)) through the communication unit (11), so that the user can inspect the automatic segmentation result.

[0077] When the automatic segmentation result page is displayed on the screen of the user terminal (20), the user can perform manual supplementation work on the automatic segmentation result through the user terminal (20). The mask supplemented through the manual supplementation work can be transmitted to the processor (13). Alternatively, according to an embodiment, the processor (13) can apply the supplementation request input through the user terminal (20) to generate a supplemented mask.

[0078] In the blood vessel mask classification step (34 in FIG. 3), the processor (13) automatically classifies the masks of the entire blood vessel by branch, assigns a unique ID to the classified mask by branch, and requests the user terminal (20) to match name information to the unique ID assigned to the mask by branch.

[0079] That is, the processor (13) classifies the entire blood vessel into detailed blood vessels by dividing the mask of the entire blood vessel into each branch and providing a unique ID to each divided mask to the user terminal (20), so that the user can assign name information to each branch mask. The processor (13) can match and store the unique ID and name information for each branch mask.

[0080] The processor (13) can create a remodeled branch mask using the blood vessel name as the file name using the branch-specific ID and name information. Furthermore, the entire blood vessel can be remodeled to create a mask. This blood vessel remodeling may be a process performed to analyze the structure of the blood vessel and automatically reproduce the blood vessel mask to minimize losses that occur during 3D reconstruction of the blood vessel mask.

[0081] In the CT mask automatic alignment step (35 in FIG. 3), the processor (13) can perform a mask-by-mask alignment process for each CT image when creating a mask using multiple CT images for a patient. After the vascular mask classification step (34 in FIG. 3), a matching process can be performed to create a mask in a aligned form. For example, when two CT images are used, only one CT mask among the two can be modified through alignment of the two images.

[0082] In the automatic recognition step (36 in FIG. 3), the processor (13) can perform an automatic recognition process for elements required when creating a undulation model. That is, the processor (13) can automatically recognize the locations of reference points, such as the navel and the pubic bone, and create them in the form of a mask.

[0083] The reason for producing the results of the CT mask automatic alignment step (35 in Fig. 3) and automatic recognition step (36 in Fig. 3) in the form of a mask is to process options and position transformations under the same conditions in the subsequent step, the mask 3D reconstruction step (37 in Fig. 3), so that the result is 3D reconstructed in an accurate position.

[0084] The processor (13) of the undulation model generation device (10) can generate the undulation model based on the mask processing result (S220).

[0085] The processor (13) can 3D reconstruct the mask finally produced through each step (31 to 36 of FIG. 3) included in the mask processing result (37 of FIG. 3), and generate the undulation model based on the 3D reconstruction result (38 of FIG. 3).

[0086] In the mask 3D reconstruction step (37 in FIG. 3), the processor (13) can automatically perform a 3D reconstruction task using the finally produced mask. 3D model data can be generated through the 3D reconstruction task.

[0087] In the 3D relief modeling step (38 in FIG. 3), the processor (13) can apply texture and relief algorithms to 3D polygon data, i.e., 3D model, which is 3D reconstructed from a mask, to create a final 3D model.

[0088] Here, the ups and downs algorithm can be implemented based on the patient's pre- and post-ups and downs scan model (scan data), CT model (CT image data), and the patient's characteristic values.

[0089] By performing masking processing using CT images before reconstructing into a 3D model in this way, unlike the existing method, processing for creating a 3D model can be performed in advance in CT space without spatial transformation, thereby increasing the accuracy of the resulting 3D model.

[0090] According to an embodiment, the processor (13) may further perform a step of imparting physical properties to the organ / blood vessel (39 in FIG. 3) after the 3D relief modeling step (38 in FIG. 3).

[0091] The processor (13) of the undulation model generation device (10) can request an inspection of the generated undulation model from the inspector terminal (30) (S230).

[0092] Specifically, the inspector's inspection of the generated 3D relief model can be performed through a function that links the 3D relief model (41) and the CT image (42), as illustrated in Fig. 4. That is, a mechanism is provided that allows easy inspection by matching the anatomical structure of the 3D model with the CT image.

[0093] The examiner can check the shape and location of blood vessels by comparing the CT and 3D models through the examiner terminal (30). At this time, the blood vessels to be inspected can be determined by the examiner's selection, but are not limited thereto and can also be set automatically.

[0094] When the examiner selects one of multiple blood vessels, a 3D relief model image showing only the selected blood vessel and a CT image showing only the selected blood vessel are displayed on the screen, and the examiner compares the two images showing only the selected blood vessel to determine whether the shape and location of the blood vessel are accurate.

[0095] Alternatively, when the examiner selects two or more blood vessels from among multiple blood vessels, a 3D relief model image showing only the two or more selected blood vessels and a CT image showing only the two or more selected blood vessels are displayed on the screen, and the examiner compares the two images showing only the two or more selected blood vessels to determine whether the shape and location of the blood vessels are accurate.

[0096] That is, the examiner can examine multiple blood vessels one by one or in combination of two or more.

[0097] The examiner can compare the CT and 3D models through the examiner terminal (30) to identify the key location (TP: Target Point) during surgery. At this time, the TP to be inspected can be determined by the examiner's selection, but is not limited thereto and may also be set automatically.

[0098] When the examiner selects one TP among multiple TPs, a 3D relief model image showing only the selected TP and a CT image showing only the selected TP are displayed on the screen, and the examiner compares the two images showing only the selected TP to determine whether the TP is accurately expressed.

[0099] Alternatively, if the examiner selects two or more TPs from among multiple TPs, a 3D relief model image showing only the two or more selected TPs and a CT image showing only the two or more selected TPs are displayed on the screen, and the examiner compares the two images showing only the two or more selected TPs to determine whether the TPs are accurate.

[0100] That is, the inspector can inspect multiple TPs one by one or in combination of two or more.

[0101] The inspector can confirm a POI (Point of Interest) by comparing the CT and 3D models through the inspector terminal (30). At this time, the POI to be inspected may be determined by the inspector's selection, but is not limited thereto and may also be set automatically.

[0102] When the inspector selects one POI among multiple POIs, a 3D relief model image showing only the selected POI and a CT image showing only the selected POI are displayed on the screen, and the inspector compares the two images showing only the selected POI to determine whether the POI is accurately expressed.

[0103] Alternatively, when the inspector selects two or more POIs from among multiple POIs, a 3D relief model image showing only the two or more selected POIs and a CT image showing only the two or more selected POIs are displayed on the screen, and the inspector compares the two images showing only the two or more selected POIs to determine whether the POIs are accurate.

[0104] That is, the inspector can inspect multiple POIs one by one or in combination of two or more.

[0105] Hereinafter, with reference to FIGS. 5 to 13, an inspection performed by an inspector through a user interface (UI) for inspection displayed on the screen of an inspector terminal (30) will be described.

[0106] The user interface for review may include a category-specific review page, a review list page, and a review results confirmation page. Categories may include organs, blood vessels, TPs, and POIs.

[0107] Here, TP refers to a critical location during surgery, and POI (point of interest) may include areas with a high probability of error when creating a undulation model, areas adjacent to muscles, areas with complexly intertwined blood vessels, etc.

[0108] The inspection list page may include a complete list of inspections assigned to the inspector, and the inspection result confirmation page may include an inspection status by category for inspection items selected from the inspection list.

[0109] That is, the inspector can check the inspections assigned to him / her through the inspection list within the inspection list page and perform inspections for each one.

[0110] At this time, inspection can be performed on 3D models by category through the category-specific inspection page. The inspector can check for abnormalities in each of the organs, blood vessels, TPs, and POIs through the category-specific inspection page displayed on the inspector terminal (30), and if no abnormalities are found, the 3D model can be approved. If an abnormality is found, the 3D model can be rejected for approval.

[0111] Figure 5 is a drawing showing an inspection list page, and Figures 6 to 8 are drawings showing an inspection result confirmation page.

[0112] Referring to Figure 5, it can be seen that the inspector has been assigned to inspect five 3D relief models. In the inspection list, items 1 and 2 are in a completed inspection state, 3 is in progress, and items 4 and 5 are newly assigned. Here, the "in progress" status can mean that the "save inspection" button has been pressed without pressing the "confirm" or "reject" button.

[0113] Although both No. 1 and No. 2 have completed the review, the review check marks (51, 52) are displayed differently, indicating that the review results for No. 1 and No. 2 differ. No. 1 may have completed the review because there were no abnormalities in the oscillation model, while No. 2 may have completed the review because there were abnormalities in the oscillation model.

[0114] If the review button (53) is selected while the review is in progress or has been newly assigned, you will be taken to the review page for the 3D model.

[0115] When inspection item 1 (54) is selected from the inspection list illustrated in FIG. 5, the modal window illustrated in FIG. 6 is displayed on the screen. The inspector can check the inspection results of item 1 (ID 00_12) by category through the displayed modal window. As illustrated in FIG. 6, it can be seen that the 3D model of item 1 (ID 00_12) has been checked to be free of any abnormalities in the detailed items for all categories.

[0116] When inspection item 2 (55) is selected from the inspection list illustrated in FIG. 5, the modal window illustrated in FIG. 7 is displayed on the screen. The inspector can check the inspection results of item 2 (ID 00_135) through the displayed modal window. As illustrated in FIG. 7, the 3D model of item 2 (ID 00_135) shows that there is an abnormality in one sub-item (AORTA) in the artery and in three sub-items (PV, LGV, SV) in the vein.

[0117] When inspection item 5 (57) is selected from the inspection list illustrated in FIG. 5, the modal window illustrated in FIG. 8 is displayed on the screen. The inspector can check the inspection results for item 5 (ID 00_2) through the displayed modal window. As illustrated in FIG. 8, it can be seen that the inspection for 3D model 5 (ID 00_2) has not yet been performed.

[0118] Figures 9 to 13 are drawings showing inspection pages.

[0119] Users can check for abnormalities in the organs of a 3D model through the organ inspection page shown in FIGS. 9 and 10.

[0120] The long-term inspection page illustrated in FIG. 9 may include an area (91) where a 3D relief model is displayed. Depending on the embodiment, the 3D relief model displayed in the area (91) may be repositioned and enlarged by clicking or dragging with a mouse.

[0121] The inspector can check multiple detailed items (92) for each organ.

[0122] In addition, the examiner can simultaneously check the 3D image of the relief model and the CT image as shown in Fig. 4 by selecting the 'view CT button (93)'. That is, when the examiner selects the 'view CT button (93)' through the examiner terminal (30), the CT image can be opened in a new window. Alternatively, depending on the embodiment, the 3D relief model image and the patient's CT image can be displayed together in the area (91) on the organ inspection page.

[0123] Additionally, the long-term inspection page may include an area for checking the progress of the long-term inspection (94), a button for selecting to view / turn off all organs (95), a button for checking / disabling all inspections (96), a button for saving the inspection details in the middle (97), a window for entering an inspection opinion (98), and a button for returning to the inspection list (99).

[0124] The inspection progress check area (94) can check the inspection progress for each of multiple detailed items for the organ.

[0125] If you select 'on' among the Show / Off All Organs selection buttons (95), all organs corresponding to the detailed items will be displayed in the 3D image, and if you select 'off', all organs corresponding to the detailed items will not be displayed in the 3D image.

[0126] If you select 'on' among the full inspection check / disassembly buttons (96), you can check that there are no abnormalities in all organs corresponding to the detailed items.

[0127] The inspection history intermediate save button (97) can intermediately save the inspection results that have been performed so far.

[0128] The review opinion input window (98) allows the reviewer to input his / her review opinion.

[0129] The Return to Inspection List button (99) can return to the inspection list page of the corresponding inspector.

[0130] As described above, you can check for all sub-items to be free of abnormalities by selecting 'on' among the overall inspection check / dismantle buttons (96), but you can also check for abnormalities for each sub-item as shown in 103 of Fig. 10. That is, if there is no abnormality in the skin sub-item, you can check the item, and if there is an abnormality in the liver sub-item, you can uncheck the item.

[0131] In addition, as shown in 102 of FIG. 10, it is also possible to select whether to display the undulation model within the 3D image by toggling multiple detailed items. That is, if the toggle for skin among the detailed items is turned on, only the skin can be displayed within the area (101) where the 3D model is displayed, and the remaining items can be hidden.

[0132] Additionally, the degree of undulation of the 3D undulation model can be adjusted through 104 and 105 of FIG. 10. The degree of undulation can be set to a different maximum value depending on the 3D undulation model, and can be automatically set to, for example, 100% or 150%. The user can adjust the degree of undulation within the maximum value through the slider (104), or adjust the degree of undulation within the maximum value through the percentage setting (105).

[0133] Additionally, when the inspector completes the inspection of all detailed items (if there is no problem with all detailed items), he or she can select the confirm button (106) to transmit the inspection results to the processor (13). That is, when the confirm button (106) is selected, the inspection results are saved and the inspector's screen switches to a list page.

[0134] At this time, the confirm button (106) can be activated only when all detailed items are checked.

[0135] The inspector can check for abnormalities in the artery of the 3D model through the artery inspection page shown in Fig. 11.

[0136] The arterial inspection page illustrated in FIG. 11 may include an area (111) where a 3D relief model is displayed. Depending on the embodiment, the 3D relief model displayed in the area (111) may be repositioned and enlarged by clicking or dragging with a mouse.

[0137] The examiner can check multiple details (112) for each artery.

[0138] Additionally, as with the organ inspection page, the arterial inspection page may also include anomaly check buttons for each of the multiple detailed items, and a toggle for each of the multiple detailed items may be used to select whether or not to display the undulation model within the 3D image.

[0139] The various functions included in the other arterial inspection pages are the same as those of the organ inspection pages described above, so detailed descriptions will be omitted.

[0140] The inspector can check for abnormalities in the veins of the 3D model through the vein inspection page shown in Fig. 12.

[0141] The vein inspection page illustrated in FIG. 12 may include an area (121) where a 3D undulation model is displayed. Depending on the embodiment, the 3D undulation model displayed in the area (121) may be repositioned and enlarged by clicking or dragging with a mouse.

[0142] The examiner can check multiple details (122) for each vein.

[0143] Additionally, as with the organ inspection page, the vein inspection page may also include anomaly check buttons for each of multiple detailed items, and a toggle for each of the multiple detailed items may be used to select whether or not to display the undulation model within the 3D image.

[0144] The various functions included in the vein inspection page are the same as those of the organ inspection page described above, so a detailed description will be omitted.

[0145] The inspector can check for abnormalities in points of interest (POI) of the 3D model through the point of interest (POI) inspection page shown in Fig. 13.

[0146] The point of interest inspection page illustrated in FIG. 13 may include an area (131) where a 3D relief model is displayed. Depending on the embodiment, the 3D relief model displayed in the area (131) may be repositioned and enlarged by clicking or dragging with a mouse.

[0147] Here, points of interest (POI) may include areas where errors are likely to occur when generating a undulation model, areas adjacent to muscles, areas where blood vessels are intricately intertwined, etc.

[0148] The inspector can check multiple detailed items (132) for each point of interest.

[0149] Additionally, as with the long-term inspection page, the point of interest inspection page may also include an anomaly check button (133) for each of multiple detailed items, and whether or not to display the undulation model in the 3D image may be selected through a toggle (134) for each of the multiple detailed items.

[0150] The inspector can check the detailed information (POI Description) of each detailed item by clicking the toggle button (134 in Figure 13) for each of the multiple detailed items.

[0151] The various functions included in the other points of interest inspection pages are the same as those of the long-term inspection page described above, so a detailed description will be omitted.

[0152] Although not shown in the drawing, when an inspector selects a specific detail on the point of interest inspection page, a separate page for inspecting TP for that specific detail may be displayed on the inspector terminal (30).

[0153] Each point of interest (POI) can have one or more TPs assigned to it. The inspector can select each POI, check one or more TPs assigned to that POI, and verify that each TP is properly displayed within the image.

[0154] The processor (13) of the fluctuation model generation device (10) can receive the inspection result from the inspector terminal (30).

[0155] If the inspection result is 'confirm', the processor (13) can provide the 3D undulation model to the requesting user terminal (20) without modification.

[0156] If the inspection result is 'reject', the processor (13) may perform a modification on the 3D relief model based on the inspector's inspection opinion and request inspection again on the modified 3D relief model.

[0157] Although FIG. 2 describes the steps as being executed sequentially, this is merely an example of the technical idea of ​​the present embodiment, and a person having ordinary skill in the technical field to which the present embodiment belongs can modify and apply various modifications and variations by changing the order described in FIG. 2 or executing them in parallel without departing from the essential characteristics of the present embodiment, and therefore FIG. 2 is not limited to a chronological order.

[0158] Meanwhile, in the above description, the steps described in FIG. 2 may be further divided into additional steps or combined into fewer steps, depending on the implementation example of the present disclosure. Furthermore, some steps may be omitted as needed, and the order of the steps may be changed.

[0159] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0160] Computer-readable storage media include all types of storage media that store instructions that can be deciphered by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0161] The disclosed embodiments have been described with reference to the attached drawings as described above. Those skilled in the art will understand that the present disclosure can be implemented in forms other than the disclosed embodiments without altering the technical spirit or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. Memory storing the process for creating a virtual undulation model; and A processor comprising: a processor for performing operations according to the above process; The above processor, When a request is made to create a patient's undulation model from the user's terminal, mask processing is performed on the patient's CT image, Based on the above mask processing results, the above undulation model is generated, Perform a review of the generated undulation model, The first user interface for the above mask processing is displayed on the screen of the user terminal, A virtual undulation model generation device based on a semi-automatic process, which causes a second user interface for the above inspection to be displayed on the screen of the inspector's terminal.

2. In paragraph 1, The above mask treatment is, A virtual undulation model generation device based on a semi-automatic process, comprising a mask automatic segmentation step, a mask inspection supplementation step, a vascular mask classification step, a CT mask automatic alignment step, and an automatic recognition step.

3. In paragraph 2, The above vascular mask classification step is: Automatically classifies the mask of the entire blood vessel by branch, Assign a unique ID to each mask classified by branch above, A virtual volatility model generation device based on a semi-automatic process that requests name information matching to a unique ID assigned to a branch-specific mask by the user terminal.

4. In paragraph 2, The above processor, The mask that is finally produced through each step included in the above mask processing result is reconstructed in 3D, A virtual relief model generation device based on a semi-automatic process that generates the relief model based on the above 3D reconstruction results.

5. In paragraph 1, The second user interface includes a category-specific review page, The above categories include organs, blood vessels, points of interest (POI) and target points (TP) during surgery. A virtual undulation model generation device based on a semi-automatic process, wherein the above category-specific inspection page includes an abnormality check button for each of a plurality of detailed items, and whether or not to display the undulation model in a 3D image is selected through a toggle for each of the plurality of detailed items.

6. In paragraph 5, The second user interface further includes a review result confirmation page, The above inspection result confirmation page is a virtual volatility model creation device based on a semi-automatic process, which includes the inspection status by category for the inspection selected from the inspection list.

7. In paragraph 1, The second user interface is a virtual relief model generation device based on a semi-automatic process that simultaneously displays a 3D image of the relief model and the CT image at the request of the inspector terminal.

8. A method for generating a virtual undulation model based on a semi-automatic process performed by a processor of a device, When a request is made to create a patient's abdominal model from a user's terminal, a step of performing mask processing on the patient's CT image; A step of generating the undulation model based on the mask processing result; and Including a step of performing a review on the generated undulation model, The step of performing the above mask processing is to cause the first user interface for the mask processing to be displayed on the screen of the user terminal, The step of performing the above inspection is a method in which a second user interface for the inspection is displayed on the screen of the inspector's terminal.

9. In paragraph 8, The above mask treatment is, A method comprising a mask automatic segmentation step, a mask inspection supplementation step, a vascular mask classification step, a CT mask automatic alignment step, and an automatic recognition step.

10. In paragraph 9, The above vascular mask classification step is: Automatically classifies the mask of the entire blood vessel by branch, Assign a unique ID to each mask classified by branch above, A method for requesting name information matching to a unique ID assigned to a branch-specific mask by the user terminal.

11. In paragraph 9, The mask that is finally produced through each step included in the above mask processing result is reconstructed in 3D, A method in which the undulation model is generated based on the above 3D reconstruction results.

12. In paragraph 8, The second user interface includes a category-specific review page, The above categories include organs, blood vessels, points of interest (POI) and target points (TP) during surgery. The above category-specific inspection page includes an abnormality check button for each of a plurality of detailed items, and a method in which whether or not to display the above-mentioned fluctuation model in a 3D image is selected through a toggle for each of the plurality of detailed items.

13. In paragraph 12, The second user interface further includes a review result confirmation page, The above inspection result confirmation page is a method that includes the inspection status by category for the inspection selected from the inspection list.

14. In paragraph 8, A method in which the second user interface simultaneously displays a 3D image of the relief model and the CT image at the request of the inspector terminal.

15. A computer-readable recording medium that is combined with a computer as hardware and stores a computer program that executes the method of claim 8.

Images