Apparatus and method for generating a virtual pneumoperitoneum model of a patient

Abstract

The present invention relates to an apparatus and method for generating a virtual pneumoperitoneum model of a patient based on condition data such as the patient's age, sex, height, weight, body mass index, and parity, physical data such as the patient's height-to-width ratio, skin circumference, anterior-posterior distance of the body, fat area, and muscle area, and landmark data displayed in the patient's abdominal 3D image data before pneumoperitoneum.

Description

Technical Field

[0001] The present invention relates to an apparatus and method for generating a virtual pneumoperitoneum model of a patient.

Background Art

[0002] In recent years, there has been a need for devices and software that enable medical staff to train in a situation similar to the actual one. Generally, a simulation device for medical staff is manufactured to be similar to the patient's situation and then training is conducted. Such a simulation device has a problem that it cannot provide various situations that occur to patients and the sense of reality decreases during simulation. In addition, in the case of surgical operations, there has been a problem that medical staff cannot simulate under the same conditions as the actual operation.

[0003] Furthermore, in the surgical process, the development of a technology that can provide information for assisting a doctor's surgery is required. In order to provide information for assisting surgery, surgical acts must be recognized. Conventionally, for scenario planning to optimize the surgical process, pre-shot medical videos were referred to or advice was received from highly skilled doctors. However, it was difficult to judge unnecessary processes only from medical videos, and there was a problem that it was difficult to receive advice from highly skilled doctors that was suitable for a specific patient. Therefore, medical videos and advice from highly skilled doctors were often difficult to be utilized as auxiliary means for optimizing the surgical process for the surgical target patient.

[0004] When performing surgical simulation using virtual reality, the surgical simulation cannot serve as a rehearsal unless training is conducted under the same conditions as the actual surgery.

[0005] In particular, when performing minimally invasive surgery (e.g., robotic surgery or laparoscopic surgery), if the camera shooting direction is different during actual surgery and during surgical simulation, the video seen by the medical team during the actual surgery will be different from the video seen during the simulation process, which may result in the inability to obtain the effect of training in the same way as the actual surgery. That is, since the intra-abdominal structure can vary depending on the position of the camera for viewing the intra-abdominal structure, it is necessary to achieve the entry of the camera into the body in the same way as during actual surgery during surgical simulation.

[0006] Therefore, in order for the camera to provide the same video during surgical simulation as during actual surgery, it is necessary to realize a virtual pneumoperitoneum model to which a pneumoperitoneum state (pneumoperitoneum: a state in which the patient's abdomen is inflated by injecting gas into the patient's body to facilitate surgery) is applied in the same way as during actual surgery.

Summary of the Invention

Problems to be Solved by the Invention

[0007] The present invention has been made in view of the above circumstances, and its object is to generate a virtual pneumoperitoneum model of the patient based on state data such as the patient's age, gender, height, weight, body mass index (BMI), and presence or absence of childbirth experience, the ratio of height to width of the patient's body, skin perimeter, distance with respect to the front-rear of the body, body data such as fat area and muscle area, and landmark data displayed in the abdominal 3D video data of the patient before pneumoperitoneum.

[0008] Another object of the present invention is to provide a surgical simulation environment based on the virtual pneumoperitoneum model in which the patient's actual pneumoperitoneum state is predicted, so that the surgical simulation can serve as an excellent rehearsal for the actual surgery.

[0009] The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those of ordinary skill in the art from the following description.

Means for Solving the Problem

[0010] A method for generating a virtual pneumoperitoneum model of a patient according to the present invention for achieving the above-described technical problem includes a step of obtaining the state data of the patient - the state data includes data for at least one of the patient's age, gender, height, weight, Body Mass Index, and presence or absence of childbirth experience - and a step of obtaining a plurality of landmark data of the patient, wherein the plurality of landmark data is displayed in the abdominal 3D video data of the patient, and a step of obtaining body data extracted from a plurality of cross-sectional video data of the patient, wherein the plurality of cross-sectional video data is a cross-section at a position where the plurality of landmark data is displayed, and the body data includes at least one of a ratio of height to width with respect to the body of the patient in the plurality of cross-sectional video data, skin circumference, a distance with respect to the front-back of the body, a fat region, and a muscle region, and a step of generating a virtual pneumoperitoneum model for predicting an actual pneumoperitoneum state of the patient based on the state data, the plurality of landmark data, and the body data.

[0011] At this time, the step of obtaining the landmark data can generate one reference landmark data at a position already set based on the umbilicus of the patient based on the abdominal 3D video data of the patient, and further generate the plurality of landmark data based on the reference landmark data.

[0012] In addition, in the step of generating the virtual pneumoperitoneum model, among the plurality of pneumoperitoneum morphology data already stored based on the first algorithm, the specific pneumoperitoneum morphology data with the highest matching rate among the state data, the landmark data, and the body data can be selected. Specifically, in the step of generating the virtual pneumoperitoneum model, among the plurality of stored pneumoperitoneum morphology classes, the specific pneumoperitoneum morphology class having the highest similarity with the state data, the landmark data, and the body data is selected, and the specific pneumoperitoneum morphology data is selected from within the selected specific pneumoperitoneum morphology class based on the first algorithm. The plurality of stored pneumoperitoneum morphology classes can be generated by clustering the state data, the body data, and the pneumoperitoneum morphology data for a plurality of existing patients.

[0013] Furthermore, in the step of generating the virtual pneumoperitoneum model, the virtual pneumoperitoneum model can be generated based on the state data and the body data through a machine learning model. Specifically, in the step of generating the virtual pneumoperitoneum model, the position information of the landmark data for the patient after pneumoperitoneum is calculated by the machine learning model, the body surface of the patient is generated based on the position information after pneumoperitoneum, and the virtual pneumoperitoneum model is output. The machine learning model constructs a training dataset based on the state data, the body data, the landmark data, and the landmark data after pneumoperitoneum for a plurality of existing patients, is machine-learned based on the constructed training dataset, and the landmark data after pneumoperitoneum can be obtained based on the actual pneumoperitoneum result during the surgery of existing patients.

[0014] In addition, in the step of obtaining the body data, the fat region is obtained based on the plurality of cross-sectional video data through a fat extraction model, and the muscle region is obtained based on the plurality of cross-sectional video data through a muscle extraction model. The fat extraction model is a machine learning model trained by designating only the fat region in abdominal medical video data as the region of interest, and the muscle extraction model can be a machine learning model trained by designating only the muscle region in abdominal medical video data as the region of interest.

[0015] Furthermore, an apparatus for generating a virtual pneumoperitoneum model of a patient according to the present invention for achieving the above-described technical problem includes a memory storing a plurality of processes for generating the virtual pneumoperitoneum model of the patient, and a processor for generating the virtual pneumoperitoneum model of the patient based on the plurality of processes. The processor acquires state data of the patient, and the state data includes data for at least one of the patient's age, gender, height, weight, Body Mass Index, and presence or absence of childbirth experience. The processor acquires a plurality of landmark data of the patient, and the plurality of landmark data is displayed in the abdominal 3D video data of the patient. The processor acquires body data extracted from a plurality of cross-sectional video data of the patient, and the plurality of cross-sectional video data is a cross-section at a position where the plurality of landmark data is displayed. The body data includes at least one of a ratio of height to width of the patient's body in the plurality of cross-sectional video data, a skin circumference, a distance from the front to the back of the body, a fat region, and a muscle region. The processor is characterized by generating a virtual pneumoperitoneum model for predicting an actual pneumoperitoneum state of the patient based on the state data, the plurality of landmark data, and the body data.

[0016] In addition, another method, another apparatus, another system for realizing the present invention, and a computer-readable recording medium for recording a computer program for executing the method can be further provided.

Advantages of the Invention

[0017] According to the present invention as described above, by generating a virtual pneumoperitoneum model of the patient based on state data such as the patient's age, gender, height, weight, Body Mass Index, and presence or absence of childbirth experience, body data such as a ratio of height to width of the patient's body, a skin circumference, a distance from the front to the back of the body, a fat region, and a muscle region, and landmark data displayed in the abdominal 3D video data of the patient before pneumoperitoneum, there is an effect that the actual pneumoperitoneum state of the patient can be predicted with high accuracy.

[0018] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

Brief Description of the Drawings

[0019]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0020] The advantages, features, and the methods for achieving them of the present invention will become clear by referring to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and can be realized in various different forms. However, this embodiment is provided to make the disclosure of the present invention complete and to enable those of ordinary skill in the technical field to which the present invention pertains to fully understand the scope of the present invention. The present invention is only defined by the scope of the claims.

[0021] The terms used in this specification are for the purpose of explaining the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless otherwise specifically mentioned. The "comprises" and / or "comprising" used in the specification do not exclude the existence or addition of one or more other components in addition to the recited components. Throughout the specification, the same reference numerals indicate the same components, and "and / or" includes each of the recited components and all combinations of one or more of them. Even if terms such as "first", "second", etc. are used to describe various components, these components are of course not limited by these terms. These terms are only used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below can also be the second component within the technical idea of the present invention.

[0022] Unless otherwise defined, all terms (including technical and scientific terms) used in this specification are used as meanings commonly understood by those of ordinary skill in the technical field to which the present invention pertains. Also, terms defined in commonly used dictionaries are not ideally or excessively interpreted unless specifically defined otherwise.

[0023] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0024] FIG. 1 is a diagram for explaining an apparatus 10 for generating a virtual pneumoperitoneum model of a patient according to the present invention.

[0025] FIG. 2 is an exemplary diagram for explaining a plurality of landmark data according to the present invention.

[0026] FIG. 3 is an exemplary diagram showing the ratio of height to width with respect to the body of a patient in cross-sectional video data according to the present invention.

[0027] FIG. 4 is an exemplary diagram showing the skin perimeter with respect to the body of a patient in cross-sectional video data according to the present invention.

[0028] FIG. 5 is an exemplary diagram showing the distance in the front-back direction of the body of a patient in cross-sectional video data according to the present invention.

[0029] FIG. 6 is an exemplary diagram showing a fat region in cross-sectional video data according to the present invention.

[0030] FIG. 7 is an exemplary diagram showing a muscle region in cross-sectional video data according to the present invention.

[0031] FIG. 8 is a diagram for explaining generating a previously stored pneumoperitoneum form class according to the present invention.

[0032] FIG. 9 is an exemplary diagram showing pneumoperitoneum form data included in the first class among the pneumoperitoneum form classes according to the present invention.

[0033] FIG. 10 is an exemplary diagram showing pneumoperitoneum form data included in the second class among the pneumoperitoneum form classes according to the present invention.

[0034] FIG. 11 is an exemplary diagram showing pneumoperitoneum form data included in the third class among the pneumoperitoneum form classes according to the present invention.

[0035] FIG. 12 is a diagram for explaining generation of a virtual pneumoperitoneum model based on a pneumoperitoneum form class already stored according to the present invention.

[0036] FIG. 13 is an exemplary diagram for explaining generation of a virtual pneumoperitoneum model according to the present invention.

[0037] Hereinafter, with reference to FIGS. 1 to 13, an apparatus 10 for generating a virtual pneumoperitoneum model of a patient according to the present invention will be described.

[0038] The apparatus 10 according to the present invention can acquire the state data of the patient, acquire a plurality of landmark data of the patient, acquire body data extracted from a plurality of cross-sectional video data of the patient, and generate a virtual pneumoperitoneum model for predicting an actual pneumoperitoneum state of the patient based on the state data, the plurality of landmark data, and the body data.

[0039] Specifically, when performing minimally invasive surgery (for example, laparoscopic surgery or robotic surgery), surgery can be performed with a surgical instrument that has entered through one or more trocars inserted at other positions while confirming a partial range inside the body with a camera that has entered the inside of the body through a trocar that has penetrated the body.

[0040] At this time, in order to secure a space for the surgical instrument to move inside the body during minimally invasive surgery, a gas (for example, carbon dioxide) already set inside the body (for example, the space between the abdominal walls when performing abdominal surgery) can be injected.

[0041] Medical staff desire to simulate the surgery in advance before the actual surgery to ensure countermeasures against various variables that may occur during the actual surgery, and as a countermeasure for this, a virtual surgery simulation can be provided in the same virtual surgery environment as the actual surgery.

[0042] Minimally invasive surgery involves performing surgery while only observing through a camera (i.e., an endoscope) inserted into the body. Therefore, when medical staff practice using images displayed in completely different positions or directions during virtual simulation and then perform the actual surgery, the images provided during the actual surgery are completely different from those during practice, and thus no practice effect can be obtained. In particular, even if a virtual body model is modeled to be the same as the patient's physical state during body surgery, when the camera enters at different positions or directions, practice is carried out while observing completely different images, and thus no practice effect can be obtained.

[0043] When pneumoperitoneum is applied to the patient's body, the shape of the body surface (e.g., the abdomen) deforms. As a result, even if the same position as the body surface with pneumoperitoneum is specified in a 3D body model without pneumoperitoneum, the angle at which the camera is inserted changes.

[0044] Therefore, when generating a virtual body model for surgical simulation, it is necessary to realize a virtual pneumoperitoneum model to which pneumoperitoneum is applied in the same manner as during actual surgery.

[0045] Hereinafter, a method for generating a virtual pneumoperitoneum model that can be performed by an apparatus 10 for generating a virtual pneumoperitoneum model of a patient will be described.

[0046] The apparatus 10 can obtain the effect of accurately predicting the actual pneumoperitoneum state of the patient by generating the virtual pneumoperitoneum model of the patient based on the patient's state data, landmark data, and body data.

[0047] Specifically, the apparatus 10 can accurately predict the actual pneumoperitoneum state of the patient by generating the virtual pneumoperitoneum model of the patient based on state data such as the patient's age, gender, height, weight, body mass index, and presence or absence of childbirth experience, body data such as the ratio of height to width of the patient's body, skin perimeter, distance with respect to the front - back of the body, fat area, and muscle area, and landmark data displayed in the abdominal 3D video data of the patient before pneumoperitoneum.

[0048] Such a device 10 can include any of a variety of devices capable of performing arithmetic processing and providing results to a user.

[0049] Here, the device 10 can be in the form of a computer. More specifically, the computer can include any of a variety of devices capable of performing arithmetic processing and providing results to a user.

[0050] For example, the computer can be not only a desktop PC or a notebook computer, but also a smart phone, a tablet PC, a cellular phone, a PCS phone (Personal Communication Service phone), a synchronous / asynchronous IMT-2000 (International Mobile Telecommunication-2000) mobile terminal, a palm PC (Palm Personal Computer), a personal digital assistant (PDA), etc. Also, when a head-mounted display (HMD) device includes a computing function, the HMD device can be a computer.

[0051] Also, the computer can be a server that receives requests from clients and performs information processing.

[0052] And the device 10 can include a communication unit 110, a memory 120, and a processor 130. Here, the device 10 can include fewer components or more components than the components shown in FIG. 1.

[0053] The communication unit 110 can include one or more modules that enable wireless communication between the device 10 and an external device (not shown), between the device 10 and an external server (not shown), or between the device 10 and a communication network (not shown).

[0054] Here, the external device (not shown) can be a medical imaging device that captures medical imaging data (hereinafter, abdominal 3D imaging data). Here, the medical imaging data can include all medical images that can realize the patient's body as a three-dimensional model.

[0055] In addition, the medical imaging data can include at least one of Computed Tomography (CT) images, Magnetic Resonance Imaging (MRI), and Positron Emission Tomography (PET) images.

[0056] Furthermore, the external server (not shown) can be a server that stores patient-specific medical data for a plurality of patients.

[0057] Also, the communication network (not shown) can transmit and receive various information among the device 10, the external device (not shown), and the external server (not shown). The communication network can use various forms of communication networks, for example, wireless communication methods such as WLAN (Wireless LAN), Wi-Fi (registered trademark), Wibro, WiMAX (registered trademark), HSDPA (High Speed Downlink Packet Access), or wired communication methods such as Ethernet (registered trademark), xDSL (ADSL, VDSL), HFC (Hybrid Fiber Coax), FTTC (Fiber to The Curb), FTTH (Fiber To The Home).

[0058] On the other hand, the communication network is not limited to the communication methods presented above, and can include all forms of communication methods that are widely known or will be developed in the future in addition to the communication methods described above.

[0059] The communication unit 110 can include one or more modules that connect the device 10 to one or more networks.

[0060] The memory 120 can store data that supports various functions of the device 10. The memory 120 can store a number of application programs (application program or application) driven by the device 10, data for the operation of the device 10, and instruction words. At least a part of such application programs may exist for the basic functions of the device 10. On the other hand, the application program can be stored in the memory 120, installed on the device 10, and driven by the processor 130 to perform the operation (or function) of the device 10.

[0061] Also, the memory 120 can include a plurality of processes for generating the virtual pneumoperitoneum model according to the present invention. Here, the plurality of processes will be described later when explaining the operation of the processor 130.

[0062] In addition to the operations related to the application program, the processor 130 can generally control the overall operation of the device 10. The processor 130 processes signals, data, information, etc. input or output through the above-described components, or drives the application program stored in the memory 120 to provide or process appropriate information or functions to the user.

[0063] Also, the processor 130 can control at least a part of the components shown in FIG. 1 to drive the application program stored in the memory 120. Further, the processor 130 can operate at least two or more of the components included in the device 10 in combination with each other for driving the application program.

[0064] The processor 130 can acquire the state data of the patient. Here, the processor 130 can acquire the state data of the patient from an external server (not shown) or the memory 120 based on the first process among the plurality of processes.

[0065] Here, the state data can include data for at least one of the patient's age, gender, height, weight, Body Mass Index, and presence or absence of childbirth experience.

[0066] Such state data can be clinical information that can affect the formation of pneumoperitoneum when pneumoperitoneum is created in the patient's abdomen.

[0067] The processor 130 can acquire a plurality of landmark data of the patient. Here, the processor 130 can acquire a plurality of landmark data of the patient based on a second process among a plurality of processes. Here, the plurality of landmark data can be those displayed in the 3D abdominal image data of the patient. The plurality of landmark data can be data in which landmarks are mapped to the 3D abdominal image data of the patient or coordinate data of each landmark.

[0068] Such a plurality of landmark data can be used as a reference point for inflating the abdomen around the corresponding landmark to form pneumoperitoneum or for measuring how much deformation has occurred after the formation of pneumoperitoneum.

[0069] More specifically, the processor 130 can generate one reference landmark data at a position already set based on the umbilicus of the patient based on the 3D abdominal image data of the patient, and can further generate the plurality of landmark data based on the reference landmark data.

[0070] Here, the plurality of landmark data can be directly generated by the user through the device 10 or automatically generated by the device 10 at intervals already set around the umbilicus (for example, 5 cm). Here, the user can be a doctor, nurse, clinical pathologist, medical imaging expert, etc. as a medical professional, and can be a technician who repairs medical devices, but is not limited thereto.

[0071] Referring to FIG. 2, the processor 130 can generate one reference landmark data 201 at a position already set with reference to the navel of the first patient based on the 3D video data 20 of the abdomen of the first patient.

[0072] Thereafter, the processor 130 can further generate the plurality of landmark data with reference to the reference landmark data 201.

[0073] Here, the processor 130 can further generate the plurality of landmark data at equal intervals already set with reference to the reference landmark data 201, but is not necessarily limited thereto.

[0074] The processor 130 can acquire body data extracted from the plurality of cross-sectional video data of the patient. Here, the processor 130 can acquire body data extracted from the plurality of cross-sectional video data of the patient based on the third process among the plurality of processes.

[0075] Here, the plurality of cross-sectional video data can be video data for a cross-section at any one of the positions where the plurality of landmark data are displayed, or can also be video data for a cross-section of a portion connecting two or more positions together.

[0076] Here, the body data can include at least one of the ratio of the height to the width of the patient's body in the plurality of cross-sectional video data, the skin perimeter, the distance with respect to the front-back of the body, the fat region, and the muscle region.

[0077] Here, the distance with respect to the front-back of the body can be the distance from the front of the navel part to the back of the back part.

[0078] Referring to FIG. 3, the processor 130 can obtain the ratio of the height 301 to the width 302 with respect to the patient's body based on the plurality of cross-sectional video data generated in the above process. Here, the ratio of the height 301 to the width 302 may be different for each position where the cross-sectional video data is obtained. Alternatively, the ratio of the height 301 to the width 302 may be different for each patient.

[0079] Referring to FIG. 4, the processor 130 can obtain the skin enclosure 401 with respect to the patient's body based on the plurality of cross-sectional video data. Here, the skin enclosure 401 with respect to the patient's body may be different for each position where the cross-sectional video data is obtained. Alternatively, the skin enclosure 401 with respect to the patient's body may be different for each patient.

[0080] Referring to FIG. 5, the processor 130 can obtain the distance with respect to the front-back of the body with respect to the patient's body based on the plurality of cross-sectional video data. Here, the distance with respect to the front-back of the body with respect to the patient's body may be different for each patient.

[0081] Referring to FIG. 6, the processor 130 can obtain the fat region 601 with respect to the body of the first patient based on the plurality of cross-sectional video data through the fat extraction model provided in the memory 120. Here, the fat region 601 with respect to the body of the first patient may be different for each position where the cross-sectional video data is obtained.

[0082] Also, the processor 130 can obtain the fat region 602 with respect to the body of the second patient based on the plurality of cross-sectional video data through the fat extraction model. Here, the fat region 602 with respect to the body of the second patient may be different for each position where the cross-sectional video data is obtained.

[0083] Such a fat region with respect to the patient's body may be different for each patient.

[0084] Here, the fat region can include at least one of a visceral fat region and a subcutaneous fat region.

[0085] Here, the fat extraction model may be a machine learning model trained by designating only the fat region as the region of interest from the abdominal 3D video data.

[0086] Such a machine learning model can include a convolutional neural network (hereinafter referred to as CNN), but is not necessarily limited thereto, and can be formed by neural networks with various structures.

[0087] Referring to FIG. 7, the processor 130 can obtain the muscle region 701 for the body of the first patient based on the plurality of cross-sectional video data through the muscle extraction model provided in the memory 120. Here, the muscle region 701 for the body of the first patient may vary for each position where the cross-sectional video data is obtained.

[0088] Further, the processor 130 can obtain the muscle region 702 for the body of the second patient based on the plurality of cross-sectional video data through the muscle extraction model. Here, the muscle region 702 for the body of the second patient may vary for each position where the cross-sectional video data is obtained.

[0089] Such a muscle region for the body of the patient may vary for each patient.

[0090] Here, the muscle extraction model may be a machine learning model trained by designating only the muscle region as the region of interest from the abdominal 3D video data.

[0091] Such a machine learning model can include a convolutional neural network (hereinafter referred to as CNN), but is not necessarily limited thereto, and can be formed by neural networks with various structures.

[0092] The processor 130 can generate a virtual pneumoperitoneum model for predicting the actual pneumoperitoneum state of the patient based on the state data, the plurality of landmark data, and the body data.

[0093] Here, the processor 130 can generate a virtual pneumoperitoneum model for predicting the actual pneumoperitoneum state of the patient based on the fourth process among the plurality of processes.

[0094] More specifically, the processor 130 can generate the virtual pneumoperitoneum model through a plurality of pre-stored pneumoperitoneum morphology classes or machine learning models based on the state data, the plurality of landmark data, and the body data.

[0095] First, the processor 130 can select specific pneumoperitoneum morphology data with the highest coincidence rate with the state data, the landmark data, and the body data among the plurality of pre-stored pneumoperitoneum morphology data based on the first algorithm to generate the virtual pneumoperitoneum model.

[0096] Here, when the state data, the body data, and the pneumoperitoneum morphology data of a plurality of existing patients increase, the processor 130 can cluster such data to generate a plurality of pneumoperitoneum morphology classes.

[0097] Here, the pneumoperitoneum morphology data can be the actual pneumoperitoneum morphology data of the patient generated by 3D scanning the body of the patient after pneumoperitoneum for each actual existing patient.

[0098] That is, referring to FIG. 8, the processor 130 can cluster the state data, the body data, and the pneumoperitoneum morphology data of a plurality of existing patients through the k-mean clustering algorithm to generate k (for example, k = 3) pneumoperitoneum morphology classes.

[0099] Here, the algorithm used for clustering can include at least one of the k-mean clustering algorithm, the adaptive breaking detection clustering algorithm, and the Hough Transform algorithm.

[0100] That is, when the state data, the body data, and the pneumoperitoneum form data for existing patients increase, the processor 130 can generate the plurality of pneumoperitoneum form classes through a specific algorithm (for example, the k-mean clustering algorithm).

[0101] Here, the plurality of pneumoperitoneum form data shown in FIG. 9 can be the pneumoperitoneum form data for existing patients included in the first class (class 1) when there are three pneumoperitoneum form classes. That is, in the case of a plurality of existing patients included in the first class, they can be patients classified into one class with similar state data, landmark data, and body data.

[0102] Also, the plurality of pneumoperitoneum form data shown in FIG. 10 can be the pneumoperitoneum form data for existing patients included in the second class (class 2) when there are three pneumoperitoneum form classes. That is, in the case of a plurality of existing patients included in the second class, they can be patients classified into one class with similar state data, landmark data, and body data.

[0103] Furthermore, the plurality of pneumoperitoneum form data shown in FIG. 11 can be the pneumoperitoneum form data for existing patients included in the third class (class 3) when there are three pneumoperitoneum form classes. That is, in the case of a plurality of existing patients included in the third class, they can be patients classified into one class with similar state data, landmark data, and body data.

[0104] As a result, when the data for a plurality of existing patients are clustered by class according to the similarity and the amount of data increases, the processor 130 can quickly select the pneumoperitoneum form data by first finding the most similar class and then selecting the pneumoperitoneum form data that is most similar to the new patient from the most similar class.

[0105] Referring to FIG. 12, the processor 130 can select a specific pneumoperitoneum form class that has the highest similarity to the medical video data in which the state data, the body data, and the landmark are mapped among the plurality of already stored pneumoperitoneum form classes.

[0106] Specifically, the processor 130 can select the specific pneumoperitoneum form class that has the highest similarity to the state data, the landmark data, and the body data for the patient based on an SVM (Support Vector Machine) classification model among the plurality of already stored pneumoperitoneum form classes.

[0107] Then, the processor 130 can select the specific pneumoperitoneum form data from within the selected specific pneumoperitoneum form class based on the first algorithm.

[0108] Specifically, the processor 130 can select the specific pneumoperitoneum form data in which the state data, the body data, and the specific pneumoperitoneum form data of the patient are most similar from within the selected specific pneumoperitoneum form class based on the cosine similarity algorithm.

[0109] That is, the processor 130 can select the specific pneumoperitoneum form data that has the highest cosine similarity to the state data, the body data, and the specific pneumoperitoneum form data of the patient from within the selected specific pneumoperitoneum form class based on the cosine similarity algorithm.

[0110] Here, the formula for the cosine similarity algorithm can be explained by the following formula 1.

[0111]

Number

[0112] Here, A is a vector for the state data with respect to the patient data and the body data, and B can be a vector for the state data and the body data with respect to the data within a selected specific class.

[0113] Therefore, the denominator part indicates the magnitude of each vector, and the numerator part can be calculated as the product between the two vectors. When the two vectors match, the cosine similarity has a value of 1, and the more different they are from each other, the closer the cosine similarity can have a value close to 0.

[0114] Next, the processor 130 can generate the virtual pneumoperitoneum model based on the state data, the landmark data, and the body data through a machine learning model.

[0115] Here, the machine learning model can include a convolutional neural network (CNN, Convolutional Neural Network, hereinafter referred to as CNN), but is not necessarily limited thereto, and can be formed by neural networks with various structures.

[0116] More specifically, the processor 130 can calculate the post-pneumoperitoneum position information of the landmark data for the patient by the machine learning model.

[0117] Specifically, the processor 130 can calculate the post-pneumoperitoneum position information of the landmark data for the patient based on a regression model among the machine learning models.

[0118] Here, the equation for the regression model can be explained by Equation 2 below.

[0119]

Number

[0120] Here, y is the post - pneumoperitoneum position information of the landmark data, x (independent variable) is the state data and body data, p is the total number of the state data and the body data, and β (beta) can be information indicating the influence on the x (independent variable).

[0121] More specifically, the processor 130 can calculate β (beta) based on the training data of the regression model.

[0122] That is, since the processor 130 knows both x and y values through the training data, it can calculate the β (beta). Here, y (dependent variable) can be the movement vector before and after pneumoperitoneum of the training data.

[0123] After that, the y (position information) can be predicted by calculating x for the new data and then calculating it together with the β (beta).

[0124] And the processor 130 can generate the body surface of the patient through TPS warping (Thin Plate Spline Warping) based on the post - pneumoperitoneum position information and output the virtual pneumoperitoneum model.

[0125] Here, referring to FIG. 13, the upper left shows the side view of the virtual model generated based on the abdominal 3D video data of the patient, and the plurality of landmark data can be displayed on the virtual model.

[0126] Here, the lower left shows the front view of the virtual model for the patient, and the plurality of landmark data can be displayed on the virtual model.

[0127] On the upper right, it shows the virtual pneumoperitoneum model after pneumoperitoneum for the patient by the machine learning model, and the virtual pneumoperitoneum model can display the positions of the plurality of landmark data after pneumoperitoneum.

[0128] Here, on the lower right, it shows the front view of the virtual pneumoperitoneum model after pneumoperitoneum for the patient, and the virtual pneumoperitoneum model after pneumoperitoneum can display the positions of the plurality of landmark data after pneumoperitoneum.

[0129] Comparing the left and right of FIG. 13, the virtual model showing the state of the patient before pneumoperitoneum and the virtual pneumoperitoneum model showing the state of the patient after pneumoperitoneum have changed in the positions of the plurality of landmark data, which is a positional change due to pneumoperitoneum. Since the patient is pneumoperitoneum in various forms for each patient, it can change to various positions.

[0130] FIG. 14 is a flowchart showing the process of generating a virtual pneumoperitoneum model of a patient according to the present invention. Here, the operation of the processor 130 can be executed by the server 10.

[0131] The processor 130 can acquire the state data of the patient (S1401).

[0132] Specifically, the processor 130 can acquire the state data of the patient from an external server (not shown) or the memory 120.

[0133] Here, the state data can include data on at least one of the patient's age, gender, height, weight, body mass index, and presence or absence of childbirth experience.

[0134] The processor 130 can acquire a plurality of landmark data of the patient (S1402).

[0135] Here, the plurality of landmark data may be those displayed in the abdominal 3D video data of the patient. The plurality of landmark data may be data in which landmarks are mapped to the abdominal 3D video data of the patient, or coordinate data of each landmark.

[0136] The processor 130 can acquire body data extracted from the plurality of cross-sectional video data of the patient (S1403).

[0137] Here, the plurality of cross-sectional video data may be cross-sections at positions where the plurality of landmark data are displayed.

[0138] Also, the body data can include at least one of the ratio of the height to the width of the patient's body in the plurality of cross-sectional video data, the skin perimeter, the distance in the front-back direction of the body (A-P direction distance), the fat area, and the muscle area.

[0139] Then, the processor 130 can acquire the fat area and the muscle area respectively based on the plurality of cross-sectional video data through a fat extraction model and a muscle extraction model.

[0140] The processor 130 can generate a virtual pneumoperitoneum model for predicting the actual pneumoperitoneum state of the patient based on the state data, the plurality of landmark data, and the body data (S1404).

[0141] First, the processor 130 can select specific pneumoperitoneum form data with the highest coincidence rate with the state data, the landmark data, and the body data among the plurality of stored pneumoperitoneum form data based on the first algorithm and generate the virtual pneumoperitoneum model.

[0142] Here, the processor 130 can select the specific pneumoperitoneum morphology class having the highest similarity to the state data, the landmark data, and the body data for the patient from among the plurality of stored pneumoperitoneum morphology classes based on an SVM (Support Vector Machine) classification model.

[0143] Then, the processor 130 can select the specific pneumoperitoneum morphology data from within the selected specific pneumoperitoneum morphology class based on the first algorithm.

[0144] The processor 130 can generate the virtual pneumoperitoneum model based on the state data, the landmark data, and the body data through a machine learning model (S1404).

[0145] More specifically, the processor 130 can calculate the post-pneumoperitoneum position information of the landmark data for the patient by the machine learning model.

[0146] Then, the processor 130 can generate the body surface of the patient through TPS warping (Thin Plate Spline Warping) based on the post-pneumoperitoneum position information and output the virtual pneumoperitoneum model.

[0147] FIG. 14 describes that steps S1401 to S1404 are sequentially executed, but this is only an exemplary explanation of the technical idea of this embodiment. For those with ordinary knowledge in the technical field to which this embodiment belongs, within the scope not departing from the essential characteristics of this embodiment, it is possible to modify and apply it in various ways by changing the order described in FIG. 14 or executing one or more of steps S1401 to S1404 in parallel. Therefore, it is not limited to the chronological order shown in FIG. 14.

[0148] The method according to an embodiment of the present invention described above can be embodied in a program (or application) and stored in a medium for being executed in combination with a computer which is hardware. Here, the computer can be the device 10 described above.

[0149] The program described above can include code (Code) encoded in a computer language such as C, C++, JAVA (registered trademark), machine language, etc., which is read by a processor (CPU) of the computer via a device interface of the computer so that the computer reads the program and executes the method realized by the program. Such code can include functional code (Functional Code) related to functions such as functions that define the functions necessary to execute the method, and can include control code related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. Also, such code can further include code related to memory references regarding at which position (address) in the internal or external memory of the computer additional information and media necessary for the processor of the computer to execute the functions should be referenced. Furthermore, when communication with any other remote computer, server, etc. is required for the processor of the computer to execute the functions, the code can further include code related to communication regarding how to communicate with any other remote computer, server, etc. using a communication module of the computer, and what information and media should be transmitted and received during communication.

[0150] The steps of the methods or algorithms described in connection with the embodiments of the present invention can be implemented directly in hardware, implemented by software modules executed by the hardware, or implemented by a combination thereof. The software modules can always be present in a RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, hard disk, removable disk, CD-ROM, or any form of computer-readable recording medium well-known in the technical field to which the present invention pertains.

[0151] As described above, the embodiments of the present invention have been described with reference to the accompanying drawings. However, those of ordinary skill in the technical field to which the present invention pertains can understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. Therefore, the embodiments described above should be understood as exemplary in all aspects and not restrictive.

Claims

1. A method for generating a virtual pneumoperitoneum model of a patient, performed by a device, comprising: obtaining the patient's state data, wherein the state data includes data for at least one of the patient's age, gender, height, weight, Body Mass Index (BMI), and presence or absence of childbirth experience; obtaining a plurality of landmark data of the patient displayed in the patient's abdominal 3D video data; obtaining body data extracted from a plurality of cross-sectional video data of the patient, wherein the plurality of cross-sectional video data are cross-sections at positions where the plurality of landmark data are displayed, and the body data includes at least one of a ratio of the height to the width of the patient's body in the plurality of cross-sectional video data, skin circumference, a distance with respect to the front-rear of the body, a fat region, and a muscle region; classifying and storing a plurality of pneumoperitoneum morphology data representing the bodies of a plurality of existing patients after pneumoperitoneum for each of the state data, landmark data, and body data of the existing patients; selecting, from the plurality of pneumoperitoneum morphology data, the pneumoperitoneum morphology data classified according to the state data, landmark data, and body data of the existing patient having the highest coincidence rate with the state data, landmark data, and body data of the patient, and generating the selected pneumoperitoneum morphology data as a virtual pneumoperitoneum model for predicting the actual pneumoperitoneum state of the patient; A method comprising the above steps.

2. A method for generating a virtual pneumoperitoneum model of a patient, performed by a device, comprising: obtaining the patient's state data, wherein the state data includes data for at least one of the patient's age, gender, height, weight, Body Mass Index (BMI), and presence or absence of childbirth experience; obtaining a plurality of landmark data of the patient displayed in the patient's abdominal 3D video data; obtaining body data extracted from a plurality of cross-sectional video data of the patient, wherein the plurality of cross-sectional video data are cross-sections at positions where the plurality of landmark data are displayed, and the body data includes at least one of a ratio of the height to the width of the patient's body in the plurality of cross-sectional video data, skin circumference, a distance with respect to the front-rear of the body, a fat region, and a muscle region; Calculating, by a machine learning model, position information of the landmark data for the patient after pneumoperitoneum; Generating a virtual pneumoperitoneum model for predicting an actual pneumoperitoneum state of the patient by generating a body surface of the patient based on the position information after pneumoperitoneum; comprising; The machine learning model constructs a learning dataset based on state data, body data, the landmark data, and landmark data after pneumoperitoneum for a plurality of existing patients, is machine-learned based on the constructed learning dataset, and the landmark data after pneumoperitoneum is obtained from existing patients who were actually pneumoperitoneum during surgery. A method characterized by that.

3. The step of obtaining the landmark data is generating one reference landmark data at a position already set with reference to the navel of the patient based on the abdominal 3D video data of the patient, The method according to claim 1 or 2, further comprising generating the plurality of landmark data with reference to the reference landmark data.

4. The step of generating the virtual pneumoperitoneum model is classifying the plurality of pneumoperitoneum form data into a plurality of pneumoperitoneum form classes for each of the similar state data, landmark data, and body data of the existing patients, selecting a specific pneumoperitoneum form class classified into the state data, landmark data, and body data of the existing patient having the highest matching rate with the state data, landmark data, and body data of the patient among the plurality of pneumoperitoneum form classes, selecting specific pneumoperitoneum form data classified into the state data, landmark data, and body data of the existing patient having the highest matching rate with the state data, landmark data, and body data of the patient among the plurality of pneumoperitoneum form data classified into the specific pneumoperitoneum form class, The method according to claim 1, comprising generating the specific pneumoperitoneum form data as the virtual pneumoperitoneum model.

5. The step of obtaining the body data is obtaining the fat region based on the plurality of cross-sectional video data through a fat extraction model, obtaining the muscle region based on the plurality of cross-sectional video data through a muscle extraction model, The fat extraction model is a machine learning model trained by designating only the fat region in abdominal medical video data as the region of interest. The muscle extraction model is a machine learning model trained by designating only the muscle region in abdominal medical video data as the region of interest, according to the method of claim 1 or 2. **Claim 6** A computer program stored in a computer-readable recording medium which is hardware, for executing the method of generating a virtual pneumoperitoneum model of a patient according to claim 1 or 2. **Claim 7** In an apparatus for generating a virtual pneumoperitoneum model of a patient, a memory storing a plurality of processes for generating the virtual pneumoperitoneum model of the patient; a processor for generating the virtual pneumoperitoneum model of the patient based on the plurality of processes, wherein the processor acquires the state data of the patient, and the state data includes data for at least one of the age, gender, height, weight, body mass index (BMI), and presence or absence of childbirth experience of the patient; acquires a plurality of landmark data of the patient, and the plurality of landmark data are displayed in the abdominal 3D video data of the patient; acquires body data extracted from a plurality of cross-sectional video data of the patient, the plurality of cross-sectional video data being cross-sections at positions where the plurality of landmark data are displayed, and the body data includes at least one of the ratio of the height to the width of the patient's body in the plurality of cross-sectional video data, the skin circumference, the distance from the front to the back of the body, the fat region, and the muscle region; wherein the processor classifies and stores a plurality of pneumoperitoneum morphology data representing the bodies of a plurality of existing patients after pneumoperitoneum for each of the state data, landmark data, and body data of the existing patients; selects the pneumoperitoneum morphology data classified into the state data, landmark data, and body data of the existing patient with the highest matching rate among the plurality of pneumoperitoneum morphology data with the state data, landmark data, and body data of the patient, and generates the selected pneumoperitoneum morphology data as a virtual pneumoperitoneum model for predicting the actual pneumoperitoneum state of the patient. **Claim 8** In an apparatus for generating a virtual pneumoperitoneum model of a patient, a memory storing a plurality of processes for generating the virtual pneumoperitoneum model of the patient; A processor that generates a virtual pneumoperitoneum model of the patient based on the plurality of processes, and the processor acquires the state data of the patient, and the state data includes data for at least one of the age, gender, height, weight, body mass index (BMI), and presence or absence of childbirth experience of the patient, acquires a plurality of landmark data of the patient, and the plurality of landmark data is displayed in the abdominal 3D video data of the patient, acquires body data extracted from a plurality of cross-sectional video data of the patient, the plurality of cross-sectional video data is a cross-section at a position where the plurality of landmark data is displayed, and the body data includes at least one of a ratio of the height and width of the patient's body in the plurality of cross-sectional video data, a skin circumference, a distance with respect to the front-back of the body, a fat region, and a muscle region, the processor calculates the post-pneumoperitoneum position information of the landmark data for the patient by a machine learning model, generates the body surface of the patient based on the post-pneumoperitoneum position information, and generates a virtual pneumoperitoneum model for predicting the actual pneumoperitoneum state of the patient, The machine learning model constructs a learning data set based on the state data, the body data, the landmark data, and the post-pneumoperitoneum landmark data for a plurality of existing patients, is machine-learned based on the constructed learning data set, and the post-pneumoperitoneum landmark data is obtained from existing patients who were actually pneumoperitoneum during surgery. The apparatus is characterized by this.

9. When acquiring the landmark data, the processor generates one reference landmark data at a position already set with reference to the umbilicus of the patient based on the abdominal 3D video data of the patient, The apparatus according to claim 7 or 8, further comprising generating the plurality of landmark data with reference to the reference landmark data.

10. When generating the virtual pneumoperitoneum model, the processor classifies the plurality of pneumoperitoneum form data into a plurality of pneumoperitoneum form classes for each of the similar state data, landmark data, and body data of the existing patients, Among the plurality of pneumoperitoneum form classes, select the specific pneumoperitoneum form class classified into the state data, the landmark data, and the body data of the existing patient with the highest coincidence rate with the state data, the landmark data, and the body data of the patient, Among the plurality of pneumoperitoneum form data classified into the specific pneumoperitoneum form class, select the specific pneumoperitoneum form data classified into the state data, the landmark data, and the body data of the existing patient with the highest coincidence rate with the state data, the landmark data, and the body data of the patient, The apparatus according to claim 7, wherein the specific pneumoperitoneum form data is generated as the virtual pneumoperitoneum model.

11. When acquiring the body data, the processor acquires the fat region based on the plurality of cross-sectional video data through a fat extraction model, acquires the muscle region based on the plurality of cross-sectional video data through a muscle extraction model, The fat extraction model is a machine learning model trained by designating only the fat region in abdominal medical video data as the region of interest, The apparatus according to claim 7 or 8, wherein the muscle extraction model is a machine learning model trained by designating only the muscle region in abdominal medical video data as the region of interest.

Images