Medical information management server, surgical training device, image transmission method and program

JP7579698B2Active Publication Date: 2024-11-08SONY OLYMPUS MEDICAL SOLUTIONS
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2020215826
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-03-19
Filing Date
2020-12-24
Publication Date
2024-11-08
Estimated Expiration
2040-12-24

AI Technical Summary

Benefits of technology

【0013】 本開示によれば、実手術に相当する経験を積むことができるという効果を奏する。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007579698000001
    Figure 0007579698000001
  • Figure 0007579698000002
    Figure 0007579698000002
  • Figure 0007579698000003
    Figure 0007579698000003
Patent Text Reader

Abstract

To provide a medical information management server, surgery training apparatus, surgery training system, image transmission method, and display method making it possible to gain experience equivalent to an actual surgery.SOLUTION: A medical information management server 70 includes a simulation data recording unit 75 in which at least plural surgical techniques are recorded in association with plural simulation data items including at least data items concerning living tissues, and a server control unit 79 that, when a client side selects one of the plural surgical techniques, transmits simulation data associated with the selected surgical technique to the client side.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a medical information management server that manages and distributes simulation data, a surgical training device, a surgical training system, an image transmission method, a display method, and a program. [Background technology]

[0002] Conventionally, a technology has been known for a surgical training device that reproduces the appearance of a teaching material video accompanying the movement of an endoscope in a display during practice (see, for example, Patent Document 1). With this technology, the movement history of the teaching material endoscope during the process of capturing teaching material videos by the teaching material endoscope is reproduced in a training endoscope inserted in a training device that provides an internal surgical practice space, and the teaching material video and the video captured by the training endoscope are synthesized and displayed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-148765 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in order to become a skilled surgeon, medical students and interns must attend many actual surgeries and cases and gain a variety of surgeries and experiences. However, in the above-mentioned Patent Document 1, the movement history of one educational endoscope is merely reproduced by a training endoscope, so it is not possible to handle various surgeries and cases, and even for the same case, medical students and interns cannot trace the procedure performed by the surgeon or professor, or try a procedure using a different approach.

[0005] The present disclosure has been made in consideration of the above, and aims to provide a medical information management server, a surgical training device, a surgical training system, an image transmission method, and a display method, which allow users to gain experience equivalent to that of actual surgery. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the objectives, the medical information management server of the present disclosure includes a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data in association with each other, and a control unit that, when any one of the plurality of surgical procedures is selected on the client side, transmits the simulation data corresponding to the selected surgical procedure to the client side.

[0007] In addition, the surgical training device of the present disclosure includes a display unit capable of displaying a simulation video based on simulation data transmitted from a medical information management server that records a plurality of simulation data corresponding to a plurality of surgical procedures, one or more medical instruments, a detection unit that detects operations performed by an operator on the medical instruments, and a control unit that causes a medical instrument image related to the medical instruments to be displayed on the display unit on the simulation video, and the control unit controls the display mode of the medical instrument image and the simulation video based on the operation content detected by the detection unit.

[0008] Moreover, a surgical training system according to the present disclosure comprises a medical information management server and a surgical training apparatus capable of communicating with the medical information management server via a network, the medical information management server comprising a recording unit which records at least a plurality of surgical procedures and a plurality of simulation data in association with each other, and a first control unit which, when any one of the plurality of surgical procedures is selected on a client side, transmits simulation data corresponding to the selected surgical procedure to the client side, the surgical training apparatus comprising: a display unit which can display a simulation video based on the simulation data transmitted from the medical information management server, one or more medical instruments which can be used to treat a model imitating a subject, a detection unit which detects operation contents by an operator on the medical instruments, and a second control unit which causes a medical instrument image related to the medical instrument to be displayed on the display unit on the simulation video, and the second control unit controls the display manner of the medical instrument image and the simulation video based on the operation contents detected by the detection unit.

[0009] In addition, the image transmission method of the present disclosure is an image transmission method executed by a medical information management server having a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data in association with each other, and when any one of the plurality of surgical procedures is selected on the client side, the simulation data corresponding to the selected surgical procedure is transmitted to the client side.

[0010] Furthermore, the display method according to the present disclosure is a display method executed by a surgical training device including a display unit capable of displaying a simulation video based on simulation data transmitted from a medical information management server that records a plurality of simulation data corresponding to a plurality of surgical procedures, and one or more medical instruments capable of performing treatment on a model imitating a subject, the display method detecting an operation performed by an operator on the medical instruments, causing the display unit to display a medical instrument image related to the medical instruments on the simulation video, and controlling the display mode of the medical instrument image and the simulation video based on the operation content.

[0011] In addition, the program disclosed herein is a program executed by a medical information management server having a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data in association with each other, and when any one of the plurality of surgical procedures is selected on the client side, the program causes the simulation data corresponding to the selected surgical procedure to be transmitted to the client side.

[0012] In addition, the program disclosed herein is a program to be executed on a surgical training device that includes a display unit capable of displaying a simulation video based on simulation data transmitted from a medical information management server that records a plurality of simulation data corresponding to a plurality of surgical procedures, and one or more medical instruments capable of treating a model imitating a subject, the program detecting operations performed by an operator on the medical instruments, displaying a medical instrument image related to the medical instruments on the simulation video on the display unit, and controlling the display mode of the medical instrument image and the simulation video based on the operations. Effect of the Invention

[0013] According to the present disclosure, it is possible to gain experience equivalent to that of an actual surgery. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a surgery training system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing the functional configuration of the main parts of a surgery training device according to one embodiment. [Diagram 3] FIG. 3 is a block diagram showing a functional configuration of a medical information management server according to an embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a setting screen displayed when performing surgery training on the display device according to one embodiment. [Diagram 5]FIG. 5 is a diagram showing an example of a setting screen displayed when performing surgery training on the display device according to one embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a setting screen displayed when performing surgery training on the display device according to one embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a setting screen displayed when performing surgery training on the display device according to one embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a setting screen displayed when performing surgery training on the display device according to one embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a setting screen displayed when performing surgery training on a display device according to an embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a setting screen displayed when performing surgery training on the display device according to one embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a setting screen displayed when performing surgery training on a display device according to an embodiment. [Figure 12] FIG. 12 is a flowchart showing an outline of the process executed by the medical information management server according to one embodiment. [Figure 13] FIG. 13 is a flowchart showing an outline of the simulation data transmission process of FIG. [Figure 14] FIG. 14 is a flowchart showing an outline of the simulation data generation process of FIG. [Figure 15] FIG. 15 is a flowchart showing an overview of the processing executed by the surgery training device according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] The embodiments for carrying out the present disclosure will be described in detail with the drawings. The present disclosure is not limited to the following embodiments. In the following description, each of the drawings is merely a schematic representation of the shape, size, and positional relationship to the extent that the contents of the present disclosure can be understood. That is, the present disclosure is not limited to only the shape, size, and positional relationship exemplified in each of the drawings. Furthermore, in the description of the drawings, the same parts are denoted by the same reference numerals. Furthermore, as an example of a surgical training system according to the present disclosure, a surgical training system using an endoscopic system using a laparoscope will be described, but it can also be applied to a surgical training system using a surgical microscope system.

[0016] [Overview of the surgical training system] FIG. 1 is a diagram showing a schematic configuration of a surgery training system according to an embodiment. The surgery training system 1 shown in FIG. 1 is a system in which a plurality of holes are formed in the abdomen of a model O1 imitating a subject such as a human or animal using a trocar, and the abdomen of the model O1 is inflated with carbon dioxide or the like, and an operator U1 (trainee) performs training in laparoscopic surgery while observing and treating the model O1 by inserting one or more medical instruments, such as a medical imaging device (a videoscope such as a rigid mirror), forceps, and an energy device such as an electric scalpel, into the model O1. Note that, although two holes are formed in the model O1 in FIG. 1, they can be appropriately changed according to the training contents, for example, the case to be trained, the surgical procedure, and the surgical technique.

[0017] 1 includes a trocar 10, a trocar 20, a medical instrument 30, a medical instrument 40, a display device 50, a control device 60, a medical information management server 70, and a plurality of hospital management servers 80. In one embodiment, the trocar 10, the trocar 20, the medical instrument 30, the medical instrument 40, the display device 50, and the control device 60 function as a surgical training device 2.

[0018] The trocars 10 and 20 are cylindrical and are inserted into the model O1. The trocars 10 and 20 have a medical instrument 30 or a medical instrument 40, which will be described later, inserted therein.

[0019] The medical instrument 30 is inserted into the model O1 via the trocar 10. The medical instrument 30 performs a simulated treatment on the subject of observation in response to the operation of an operator such as a doctor or a medical intern. The medical instrument 30 also detects the operation of the operator and outputs the detection result to the control device 60 via a transmission cable. The medical instrument 30 is, for example, an injection, forceps, a knife, a high-frequency snare, a treatment tool, or an energy device such as a high-frequency scalpel or an electric scalpel.

[0020] The medical instrument 40 is inserted into the model O1 via the trocar 20. The medical instrument 40 performs a simulated treatment on the subject of observation in response to the operation of an operator such as a doctor or a medical intern. The medical instrument 40 also detects the operation of the operator and outputs the detection result to the control device 60 via a transmission cable. The medical instrument 40 is, for example, an injection, forceps, a knife, a high-frequency snare, a treatment tool, or an energy device such as a high-frequency scalpel or an electric scalpel.

[0021] Under the control of the control device 60, the display device 50 displays a two-dimensional image or a three-dimensional image based on image data input from the control device 60. The display device 50 is realized using, for example, a liquid crystal display, an organic electroluminescent display (EL display), an audio speaker, etc.

[0022] The control device 60 controls each part of the surgery training device 2. The control device 60 performs various image processing on the simulation data input from the medical information management server 70 via the network 90, and outputs the processed data, thereby displaying a simulation video based on the simulation data on the display device 50. The control device 60 is realized using a processor including a memory and hardware such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field-Programmable Gate Array). Here, the simulation data is data for performing a surgery simulation in a three-dimensional space using an organ, and includes data related to the organ or medical instrument. The simulation data for each organ and medical instrument includes, for example, the following parameters, and is defined by polygon data in the three-dimensional space. (1) Location (organ location (coordinate information)) (2) Shape (shape of organs, shape of medical devices) Furthermore, the simulation data for each organ and medical instrument may include the following parameters (3) to (7). (3) Mass density (4) Rigidity (5) Friction (6) Viscosity (7) Touch The simulation data may further include information on time series changes regarding the motion and deformation of the medical instrument or organ over time.

[0023] The medical information management server 70 transmits the simulation data requested by the control device 60 via the network 90. ​​The medical information management server 70 also records two-dimensional image data or three-dimensional image data including at least data related to organs acquired for each of a plurality of patients by a CT (Computed Tomography) device or an MRI (Magnetic Resonance Imaging) device, etc. The medical information management server 70 also records simulation image data based on intraoperative video data captured by a medical imaging device during surgery of the subject, and at least one of two-dimensional image data and three-dimensional image data including at least data related to organs captured by a CT device or an MRI device. The medical information management server 70 is realized using a processor including hardware such as a CPU, a GPU, and an FPGA, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc.

[0024] The hospital management servers 80 transmit educational material data for simulation data generated in the medical information management server 70 to the medical information management server 70 via the network 90. ​​Here, the educational material data is a video file in which two-dimensional image data or three-dimensional image data of each of the subjects (patients) acquired by a CT device or an MRI device, intraoperative video data including at least organs (e.g., stomach, intestines, etc.) imaged during surgery of each of the subjects (patients) by a medical imaging device (e.g., a video microscope such as an endoscope or a rigid endoscope), subject information (patient information) regarding each of the subjects, multiple surgeons (surgeons and hospital names) who operated on each of the subjects, procedure information regarding the surgical procedure of each of the subjects, and medical instrument information regarding each of the medical instruments used in the surgical procedure are associated with each other. The hospital management servers 80 are realized using a processor having hardware such as a CPU, a GPU, and an FPGA, and a HDD, an SSD, etc.

[0025] [Functional configuration of the main parts of the surgical training device] Next, a description will be given of the functional configuration of the main parts of the surgery training device 2. Fig. 2 is a block diagram showing the functional configuration of the main parts of the surgery training device 2.

[0026] [Configuration of medical device] First, a description will be given of the configuration of the medical instrument 30. The medical instrument 30 includes a first detection unit 31, a second detection unit 32, a recording unit 33, and a communication unit 34. Note that the medical instrument 40 has the same configuration as the medical instrument 30, and therefore detailed description thereof will be omitted.

[0027] The first detection unit 31 detects the insertion and removal of the medical instrument 30 held by the surgeon into and from the trocar 10, for example, the amount of pushing or pulling, and outputs the detection result to the communication unit 34. The first detection unit 31 is realized using, for example, a gyro sensor and an acceleration sensor.

[0028] The second detection unit 32 detects a pressure value (pressure value) or a speed associated with the opening and closing operation of the medical instrument 30 operated by the surgeon, and outputs the detection result to the communication unit 34. The second detection unit 32 is realized using a pressure sensor, a gyro sensor, an acceleration sensor, and the like.

[0029] The recording unit 33 is realized by using a volatile memory or a non-volatile memory, and records medical instrument information related to the medical instrument 30. Here, the medical instrument information includes the model name, shape, product number, type information of the medical instrument 30 (for example, type of forceps, electric scalpel, etc.), and manufacturing date, etc. The recording unit 33 outputs the medical instrument information to the communication unit 34.

[0030] The communication unit 34 is capable of communicating with the first communication unit 61, and transmits the detection result of the first detection unit 31, the detection result of the second detection unit 32, and the medical instrument information recorded by the recording unit 33 to the control device 60. The communication unit 34 is realized using a communication module or the like.

[0031] [Configuration of the control device] Next, a description will be given of the configuration of the control device 60. The control device 60 includes a first communication unit 61, an image processing unit 62, an input unit 63, a recording unit 64, a second communication unit 65, and a control unit 66.

[0032] The first communication unit 61 transmits various control data to the medical instruments 30, 40 under the control of the control unit 66, and outputs various data transmitted from the medical instruments 30, 40 and the medical information management server 70 to the image processing unit 62 or the control unit 66. The first communication unit 61 is realized by using, for example, an S / P conversion circuit and a communication module. The first communication unit 61 may transmit various data to the medical instruments 30, 40 by wireless communication or optical communication.

[0033] The image processing unit 62, under the control of the control unit 66, displays a simulation moving image based on the simulation data (polygon moving image data) input from the medical information management server 70 via the first communication unit 61. Specifically, the image processing unit 62 generates a simulation image of an observation target inside a subject (a simulation moving image obtained by applying graphics to the polygon data moving image data) and displays the simulation moving image by outputting the simulation moving image to the display device 50. The image processing unit 62 is realized by using a processor having a memory and hardware such as a GPU (Graphics Processing Unit) and an FPGA.

[0034] The input unit 63 receives operation input from the user, and outputs a signal according to the received operation to the control unit 66. The input unit 63 is realized using a mouse, a keyboard, a touch panel, a switch, a button, and the like.

[0035] The recording unit 64 records various data related to the surgery training device 2. The recording unit 64 has a program recording unit 641 that records various programs executed by the surgery training device 2. The recording unit 64 is realized using a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.

[0036] The second communication unit 65, under the control of the control unit 66, acquires simulation data from the medical information management server 70 via the network 90, and outputs the acquired simulation data to the control unit 66 and the image processing unit 62. The second communication unit 65 is realized using a communication module or the like.

[0037] The control unit 66 controls each unit constituting the control device 60. The control unit 66 causes the display device 50 to display a medical instrument image related to the medical instrument on a simulation video based on simulation data acquired from the medical information management server 70 via the second communication unit 65. Specifically, the control unit 66 controls the display mode of the medical instrument image and the simulation video displayed by the display device 50 based on the operation content of the medical instrument 30 detected by each of the first detection unit 31 and the second detection unit 32. In addition, when a stop signal for stopping the simulation video displayed by the display device 50 is input from the input unit 63, the control unit 66 changes the viewpoint of the simulation video based on the operation content detected by each of the first detection unit 31 and the second detection unit 32. Specifically, the control unit 66 causes the display device 50 to display a simulation video of a viewpoint rotated 90 degrees clockwise from a lower layer of the simulation video or a visual field area of ​​the current viewpoint. The control unit 66 is realized by using a processor having a memory and hardware such as a CPU. In one embodiment, the control unit 66 functions as a second control unit.

[0038] [Configuration of medical information management server] Next, a description will be given of the configuration of the medical information management server 70. Fig. 3 is a block diagram showing the functional configuration of the medical information management server 70. The medical information management server 70 shown in Fig. 3 includes a communication unit 71, a recording unit 72, a generation unit 73, an inference unit 74, a simulation data recording unit 75, an image data recording unit 76, a user data recording unit 77, a medical instrument data recording unit 78, and a server control unit 79.

[0039] The communication unit 71, under the control of the server control unit 79, transmits simulation data requested by the control device 60 (client side) to the control device 60 via the network 90. ​​The communication unit 71 is realized using a communication module or the like.

[0040] The recording unit 72 records various data related to the medical information management server 70. The recording unit 72 has a program recording unit 721 that records various programs executed by the medical information management server 70. The recording unit 72 is realized using a RAM, a ROM, etc.

[0041] The generating unit 73 generates simulation data based on intraoperative video data captured during surgery of the subject transmitted from each of the multiple hospital management servers 80 via the network 90 and at least one of two-dimensional image data and three-dimensional data captured in advance by a CT device or an MRI device, under the control of the server control unit 79, and records the simulation data in the simulation data recording unit 75. The generating unit 73 also generates simulation data by aligning the positions of organs included in the intraoperative video data and at least one of the two-dimensional image data and three-dimensional data, based on the inference result of the inference unit 74 described later. Furthermore, the generating unit 73 generates simulation data reflecting the orientation and crushing state of organs included in the intraoperative video data and at least one of the two-dimensional image data and three-dimensional data, based on the inference result of the inference unit 74 described later. The generating unit 73 is realized by using a processor having a memory and hardware such as a GPU.

[0042] The inference unit 74, under the control of the server control unit 79, estimates the organs and positions of the subject appearing in the intraoperative video data based on the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90 and at least one of the two-dimensional image data and the three-dimensional data, and outputs the estimation result to the generation unit 73. The inference unit 74 is realized using a trained model that uses the intraoperative video data, the organs and positions appearing in the intraoperative video data, distance information indicating the distance from the tip of the medical imaging device (videoscope) to the organs, the feature amount of the intraoperative video data, and multiple simulation data (e.g., polygon data based on at least one of CT image data and MRI image data) as learning parameters (teaching data), machine-learns the learning parameters by deep learning or the like, and outputs the estimation result of the organs and positions of the subject appearing in the intraoperative video data as output parameters. Here, DNN (Deep Neural Network), CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), etc. are used as the machine learning. Furthermore, when the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90 is multispectral image data irradiated with multispectral light by a light source device, the inference unit 74 estimates the biological tissue types (e.g., organs such as the gallbladder and pancreas, blood vessels, nerves, lymph, ureters, and fat, etc.) and their positions (areas) of the subject's tissues in the multispectral image data by using the difference in reflected light frequency (for each spectrum) contained in the multispectral image data, and outputs the estimation result to the generation unit 73. Furthermore, the inference unit 74 estimates the orientation (direction) and crushing degree of the organs, blood vessels, nerves, etc., in the intraoperative video data when the optical axis of the optical system of the medical imaging device (the insertion direction of the trocar) is used as a reference, based on the position and orientation of the medical imaging device, the insertion position of the subject, and the operation amount (distance) inserted into the subject, which are associated with the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90, and outputs the estimation result to the generation unit 73.The inference unit 74 is realized using a processor having a memory and hardware such as a GPU.

[0043] The simulation data recording unit 75 records a plurality of surgical procedures and a plurality of simulation data in association with each other. Specifically, as shown in Table T1 of Fig. 4, the simulation data recording unit 75 records a surgical procedure, simulation data, surgeon information for identifying the surgeon who performed the simulation data, medical instrument information regarding the medical instrument used in the simulation data, and level information indicating the level of the simulation data in association with each other. The simulation data recording unit 75 is realized by using a HDD, an SSD, or the like.

[0044] The image data recording unit 76 records two-dimensional image data captured by a CT device or three-dimensional image data captured by an MRI examination, which are transmitted from each of the multiple hospital management servers 80 via the network 90, in association with hospital identification information indicating the name of the hospital and subject information (patient information) regarding the subject. Here, the subject information refers to the subject's name, disease name, medical history, biological information, doctor in charge, surgeon, etc. The image data recording unit 76 is realized using an HDD, SSD, etc.

[0045] The user data recording unit 77 records identification information for each of a plurality of operators U1 who have trained using the surgery training device 2 in the medical information management server 70 via the network 90. ​​Specifically, as shown in Table T2 in FIG. 5, the user data recording unit 77 records the user ID, the level (difficulty) of the surgery training of the operator U1, and the training content as identification information for identifying the operator U1, in association with each other. Furthermore, the user data recording unit 77 may record the number of training sessions, the hospital or academic society to which the user ID belongs, the address, and the bank account in association with the user ID. The user data recording unit 77 is realized using an HDD, an SSD, or the like.

[0046] The medical instrument data recording unit 78 records medical instrument information related to the medical instrument. Here, the medical instrument information includes the model name, model number, animation data, CAD data (polygon data), etc. of the medical instrument.

[0047] The server control unit 79 controls each unit constituting the medical information management server 70. The server control unit 79 is realized by using a processor having a memory and hardware such as a CPU. When any one of a plurality of surgical procedures is selected by the control device 60 (client side), the server control unit 79 transmits simulation data corresponding to the selected surgical procedure to the control device 60 (client) side. The server control unit 79 has a determination unit 791, a billing unit 792, and a point assignment unit 793. In one embodiment, the server control unit 79 functions as a first control unit.

[0048] The judgment unit 791 judges the identification information of the operator U1 who accessed the medical information management server 70 via the control device 60 and the network 90 from the identification information recorded by the user data recording unit 77, and outputs this judgment result to the billing unit 792 and the point assignment unit 793.

[0049] Based on the judgment result of the judgment unit 791, the billing unit 792 performs billing by charging a fee based on the level or number of times the simulation data was downloaded to the control device 60 to the registered address or affiliated hospital registered by the user corresponding to the user ID recorded in the user data recording unit 77.

[0050] Based on the judgment result of the judgment unit 791, the score assignment unit 793 assigns points to the user ID recorded by the user data recording unit 77 according to the level associated with the simulation video or the number of times training was performed using the simulation data.

[0051] [Training method overview] Next, there will be described an outline of a surgical training method using the surgical training system 1. Figures 6 to 11 are diagrams showing examples of setting screens displayed on the display device 50 when performing surgical training.

[0052] First, the operator U1 uses the input unit 63 of the control device 60 to access the browser of the medical information management server 70 and inputs identification information (such as a user ID and a password) for identifying the operator U1. In this case, the medical information management server 70 transmits training screen data to the control device 60 (client side) that has accessed the browser. As a result, as shown in FIG. 6, the display device 50 displays a training screen W1 corresponding to the training screen data under the control of the control device 60. The training screen W1 includes at least the level of the surgical technique, the surgical target of the surgical technique (e.g., the rectum), the mission (e.g., tumor removal and suturing), and a thumbnail image SM1 corresponding to the simulation data. As shown in FIG. 6, the operator U1 operates a pointer or the like using the input unit 63 to set a desired level and a desired surgical target in each predetermined cell. In this case, the control device 60 transmits the setting contents set by the operator U1 to the medical information management server 70.

[0053] 7, when a request signal is input from the control device 60 to request the setting of realism setting of surgical procedures and various modes during training, the medical information management server 70 transmits realism setting and panic mode setting screen data to the control device 60. As a result, as shown in Fig. 7, the display device 50 displays the realism setting and panic mode setting screen W2 under the control of the control device 60. The realism setting and panic mode setting screen W2 includes at least bleeding speed, blood pressure drop speed, O2 drop, malformed organs and blood vessels, hardness and fragility, vital signs, and the presence or absence of panic during surgery.

[0054] Furthermore, as shown in FIG. 8, when a request signal for requesting tool selection and setting for selecting a medical tool is input from the control device 60, the medical information management server 70 transmits tool selection and setting setting screen data to the control device 60. As a result, as shown in FIG. 8, the display device 50 displays a tool selection and setting setting screen W3 under the control of the control device 60. The tool selection and setting setting screen W3 includes a plurality of medical tools M1 to M5 (a plurality of treatment tools) used in the surgical procedure selected by the operator U1. For example, the tool selection and setting setting screen W3 includes forceps, an electric scalpel, a medical imaging device (rigid scope), and the like. The operator U1 selects a medical tool to be used in the surgical procedure to be performed in training. Furthermore, the operator U1 can increase the number of medical tools (selected items) that can be selected by charging the medical information management server 70. In this case, the medical information management server 70 charges a fee according to the number of selected medical tools based on the identification information of the operator U1.

[0055] Also, as shown in FIG. 9, when a request signal for requesting a surgical procedure or case base is input from the control device 60, the medical information management server 70 transmits case selection screen data displaying a list of thumbnail images of multiple simulation data corresponding to each of multiple surgical procedures or multiple cases to the control device 60. As a result, as shown in FIG. 9, the display device 50 displays a case selection screen W4 under the control of the control device 60. The case selection screen W4 includes thumbnail images P1 to P5 corresponding to each of the multiple simulation data. The operator U1 selects a desired thumbnail image. In this case, the medical information management server 70 transmits to the control device 60 surgeon data related to the surgeon (surgeon) who performed the operation of the simulation data of the thumbnail image according to the selection signal input from the control device 60. As a result, as shown in FIG. 10, the display device 50 displays a surgeon screen W5 related to the surgeon who is a supervising professor or the like under the control of the control device 60. The surgeon screen W5 includes a portrait of the surgeon (surgeon), the case operated by the surgeon, the surgical procedure, a level and a message by the surgery training system 1, and the like.

[0056] 11, when a request signal for requesting simulation data of a new approach is input from the control device 60, the medical information management server 70 transmits simulation data screen data of the new approach to the control device 60. As a result, as shown in Fig. 10, the display device 50 displays a simulation data screen W6 of the new approach under the control of the control device 60. The simulation data screen W6 includes a thumbnail image SM2 corresponding to the simulation moving image, etc.

[0057] In this way, the operator U1 can perform training by using the input unit 63 of the control device 60 to download simulation data of a desired surgical case according to the level from the medical information management server 70 to the control device 60.

[0058] [Medical information management server processing] Next, a description will be given of the processing executed by the medical information management server 70. Fig. 12 is a flowchart showing an outline of the processing executed by the medical information management server 70.

[0059] First, the server control unit 79 judges whether or not a request has been made from the client side (step S1). Specifically, the server control unit 79 judges whether or not a request signal requesting various types of simulation data has been input from the control device 60 (client side) via the network 90. ​​If the server control unit 79 judges that a request has been made from the client side (step S1: Yes), the medical information management server 70 proceeds to step S2, which will be described later. On the other hand, if the server control unit 79 judges that no request has been made from the client side (step S1: No), the medical information management server 70 proceeds to step S4, which will be described later.

[0060] In step S2, the medical information management server 70 executes a simulation data transmission process for transmitting simulation data to the client side. The details of the simulation moving image transmission process will be described later.

[0061] Next, the server control unit 79 judges whether the access from the client side has ended (step S3). If the server control unit 79 judges that the access from the client side has ended (step S3: Yes), the medical information management server 70 ends this process. On the other hand, if the server control unit 79 judges that the access from the client side has not ended (step S3: No), the medical information management server 70 returns to the above-mentioned step S1.

[0062] In step S4, the server control unit 79 judges whether or not intraoperative video data, two-dimensional image data, and three-dimensional image data have been received from any one of the multiple hospital management servers 80 via the network 90. ​​If the server control unit 79 judges that intraoperative video data, two-dimensional image data, and three-dimensional image data have been received from any one of the multiple hospital management servers 80 (step S4: Yes), the medical information management server 70 proceeds to step S5, which will be described later. On the other hand, if the server control unit 79 judges that intraoperative video data, two-dimensional image data, and three-dimensional image data have not been received from any one of the multiple hospital management servers 80 (step S4: No), the medical information management server 70 proceeds to step S3.

[0063] In step S5, the medical information management server executes a simulation data generation process to generate simulation data based on the intraoperative video data and either one of the two-dimensional image data and the three-dimensional image data. The details of the simulation data generation process will be described later. After step S5, the medical information management server 70 proceeds to step S3.

[0064] [Outline of simulation data transmission process] Next, an outline of the simulation data transmission process explained in step S2 of Fig. 12 will be described below. Fig. 13 is a flowchart showing an outline of the simulation data transmission process.

[0065] 13, first, the server control unit 79 acquires identification information for identifying the operator U1 transmitted from the control device 60 (client side) accessed via the network 90 and the communication unit 71 (step S101). In this case, the determination unit 791 determines whether the identification information recorded by the user data recording unit 77 is the identification information of the operator U1 transmitted from the control device 60.

[0066] Next, the determination unit 791 determines whether or not a setting signal for setting a mission in the surgical training has been input from the control device 60 (step S102). If the determination unit 791 determines that a setting signal for setting a mission in the surgical training has been input from the control device 60 (step S102: Yes), the medical information management server 70 proceeds to step S103, which will be described later. On the other hand, if the determination unit 791 determines that a setting signal for setting a mission in the surgical training has not been input from the control device 60 (step S102: No), the medical information management server 70 proceeds to step S105, which will be described later.

[0067] In step S103, the server control unit 79 causes the control device 60 to transmit simulation data corresponding to the mission set in the surgical training to the communication unit 71 (step S103). Specifically, the server control unit 79 acquires simulation data corresponding to the mission set in the surgical training from the control device 60 from among the multiple simulation data recorded by the simulation data recording unit 75, and causes the acquired simulation data to be transmitted to the communication unit 71. This allows the operator U1 to perform training according to the set mission while watching the simulation video displayed by the display device 50.

[0068] Next, the server control unit 79 charges the operator U1 and assigns points to the identification information based on the identification information and the level of the simulation data downloaded to the control device 60 (step S104). Specifically, the charging unit 792 charges the operator U1 by charging a fee based on the level or number of times of the simulation data downloaded to the control device 60 to the registered address or affiliated hospital registered by the user who matches the user ID recorded by the user data recording unit 77 determined by the determination unit 791. Furthermore, the score assigning unit 793 assigns points according to the level associated with the simulation video or the number of times training was performed with the simulation data to the user ID recorded by the user data recording unit 77 determined by the determination unit 791. After step S104, the medical information management server 70 returns to the main routine of FIG. 12. Note that the charging method by the charging unit 792 is an example, and in addition to the level of the simulation data, the charging unit 792 may also charge for downloading medical instrument information of a new medical instrument, downloading simulation data of a new surgical procedure, and downloading simulation data by a specified surgeon.

[0069] In step S105, the determination unit 791 determines whether or not a request signal for selecting a desired surgical procedure case from the multiple surgical procedure cases has been input from the control device 60. If the determination unit 791 determines that a request signal for selecting a desired surgical procedure case from the multiple surgical procedure cases has been input from the control device 60 (step S105: Yes), the medical information management server 70 proceeds to step S106, which will be described later. On the other hand, if the determination unit 791 determines that a request signal for selecting a desired surgical procedure case from the multiple surgical procedure cases has not been input from the control device 60 (step S105: No), the medical information management server 70 proceeds to step S109, which will be described later.

[0070] In step S106, the server control unit 79 causes the communication unit 71 to transmit surgical procedure case information that summarizes a plurality of surgical procedure cases corresponding to each of the plurality of simulation data recorded by the simulation data recording unit 75.

[0071] Next, the server control unit 79 acquires, from the control device 60 via the network 90 and the communication unit 71, information on the medical instruments used in the surgical training by the operator U1 (step S107).

[0072] Thereafter, the server control unit 79 causes the communication unit 71 to transmit simulation data based on the surgical procedure case and medical instrument information selected by the operator U1 from among the multiple simulation data recorded by the simulation data recording unit 75 (step S108). This allows the operator U1 to perform surgical training for a desired surgical procedure case using the medical instruments 30, 40 while watching the simulation video displayed on the display device 50. After step S108, the medical information management server 70 proceeds to step S104.

[0073] In step S109, the determination unit 791 determines whether or not a request signal for selecting a desired surgeon from a plurality of surgeons has been input from the control device 60. If the determination unit 791 determines that a request signal for selecting a desired surgeon from a plurality of surgeons has been input from the control device 60 (step S109: Yes), the medical information management server 70 proceeds to step S110, which will be described later. On the other hand, if the determination unit 791 determines that a request signal for selecting a desired surgeon from a plurality of surgeons has not been input from the control device 60 (step S109: No), the medical information management server 70 proceeds to step S104.

[0074] In step S110, the server control unit 79 causes the communication unit 71 to transmit surgeon information that compiles a plurality of surgeons corresponding to each of the plurality of simulation data recorded by the simulation data recording unit 75.

[0075] Next, the server control unit 79 acquires the surgeon information selected by the operator U1 from the control device 60 via the network 90 and the communication unit 71 (step S111), and causes the communication unit 71 to transmit simulation data based on the surgeon information selected by the operator U1 from among the multiple simulation data recorded by the simulation data recording unit 75 (step S112). This allows the operator U1 to trace and learn the surgical technique performed by the surgeon while watching the simulation video displayed on the display device 50. After step S112, the medical information management server 70 proceeds to step S104.

[0076] [Outline of simulation data generation process] Next, an overview of the simulation data generation process explained in step S5 of Fig. 12 will be described. Fig. 14 is a flowchart showing the overview of the simulation data generation process.

[0077] 14, first, the inference unit 74 estimates the organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90 and at least one of the two-dimensional image data and the three-dimensional data including at least data related to the organs (step S201). Specifically, when the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90 is multispectral image data irradiated with multispectral light by a light source device, the inference unit 74 estimates the types of biological tissues (e.g., organs such as the gallbladder and pancreas, blood vessels, nerves, lymph, ureters, and fat) and their positions (areas) of each part of the subject to be observed shown in the multispectral image data using the difference in the reflected light frequency (for each spectrum) included in the multispectral image data, and outputs the estimation result to the generation unit 73. In addition, the inference unit 74 may estimate the orientation (direction) and degree of crushing of organs, blood vessels, nerves, etc. depicted in the intraoperative video data when referenced to the optical axis of the optical system of the medical imaging device (insertion direction of the trocar) based on the position, orientation, insertion position of the subject, and the amount of operation (distance) inserted into the subject of the medical imaging device associated with the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90, and output this estimation result to the generation unit 73.

[0078] Next, the generating unit 73 aligns the positions of the organs included in the intraoperative video data and at least one of the two-dimensional image data and three-dimensional data based on the inference result of the inference unit 74 to generate simulation data (step S202). Specifically, the generating unit 73 performs orientation, position, and identification of the organs included in the intraoperative video data and at least one of the two-dimensional image data and three-dimensional data based on the inference result of the inference unit 74, and then aligns the positions of the organs to generate simulation data that reflects the degree of crushing of the organs expected from the current viewpoint, etc.

[0079] Thereafter, the server control unit 79 acquires surgeon information from metadata (header information) included in the intraoperative video file that stores the intraoperative video data (step S203), and acquires medical instrument information from metadata (header information) included in the intraoperative video file that stores the intraoperative video data (step S204).

[0080] Next, the server control unit 79 generates a simulation file in which the simulation data generated by the generation unit 73 is associated with the surgeon information and the medical instrument information, and records the simulation file in the simulation data recording unit 75 (step S205). After step S205, the medical information management server 70 returns to the main routine of FIG.

[0081] Processing of surgical training equipment Next, a description will be given of the processing executed by the surgery training apparatus 2. Fig. 15 is a flow chart showing an outline of the processing executed by the surgery training apparatus 2.

[0082] First, the control unit 66 causes the second communication unit 65 to transmit the identification information of the operator U1 input via the input unit 63 (step S301).

[0083] Next, when the operator U1 selects the mission mode via the input unit 63 (step S302: Yes), the control unit 66 causes the display device 50 to display mission information that can be set in the surgical training (step S303).

[0084] After that, the control unit 66 causes the second communication unit 65 to transmit the mission information set by the operator U1 using the input unit 63 (step S304).

[0085] Next, the control unit 66 causes the display device 50 to display a simulation video based on the simulation data corresponding to the mission set for the operator U1 and transmitted from the medical information management server 70 (step S305).

[0086] Thereafter, the control unit 66 determines whether or not a stop signal to stop the playback of the simulation moving image has been input from the input unit 63 (step S306). If the control unit 66 determines that a stop signal to stop the playback of the simulation moving image has been input from the input unit 63 (step S306: Yes), the surgery training apparatus 2 proceeds to step S307, which will be described later. On the other hand, if the control unit 66 determines that a stop signal to stop the playback of the simulation moving image has not been input from the input unit 63 (step S306: No), the surgery training apparatus 2 proceeds to step S309, which will be described later.

[0087] In step S307, the control unit 66 controls the viewpoint of the simulation video based on the operation details of the medical instruments 30, 40. Specifically, the control unit 66 controls the image processing unit 62 based on the operation details of the medical instruments 30, 40 to generate an image of a display layer lower than the image of the current display layer as the viewpoint of the simulation video, or to generate an image of a display layer that can be observed in a state where the viewpoint is changed 90 degrees clockwise from the current viewpoint, thereby controlling the viewpoint of the simulation video displayed by the display device 50. Of course, the control unit 66 may also perform, for example, enlargement or reduction in addition to changing the viewpoint of the simulation video by controlling the image processing unit 62 based on the operation details of the medical instruments 30, 40.

[0088] Next, if the playback of the simulation video has ended (step S308: Yes), the surgery training apparatus 2 ends this process. On the other hand, if the playback of the simulation video has not ended (step S308: No), the surgery training apparatus 2 proceeds to step S306 described above.

[0089] In step S309, the control unit 66 controls the display mode of the simulation video based on the simulation data, based on the operation content of the medical instruments 30, 40. Specifically, the control unit 66 moves the medical instruments 30, 40 shown in the simulation video in accordance with the operation content of the operator U1, based on the operation content of the medical instruments 30, 40, and reflects the treatment by the medical instruments 30, 40 in the simulation video. After step S309, the surgery training apparatus 2 proceeds to step S308.

[0090] Next, a description will be given of a case where the operator U1 has not selected the mission mode via the input unit 63 in step S302 (step S302: No). In this case, the surgery training apparatus 2 proceeds to step S310.

[0091] Next, if the operator U1 selects the surgical procedure case mode via the input unit 63 (step S310: Yes), the surgical training apparatus 2 proceeds to step S311, which will be described later. On the other hand, if the operator U1 does not select the surgical procedure case mode via the input unit 63 (step S310: No), the surgical training apparatus 2 proceeds to step S315, which will be described later.

[0092] In step S311, the control unit 66 causes the display device 50 to display surgical procedure case information that can be selected in the surgical training.

[0093] Next, the control unit 66 causes the second communication unit 65 to transmit the surgical procedure case information selected by the operator U1 using the input unit 63 (step S312).

[0094] Thereafter, the control unit 66 causes the second communication unit 65 to transmit the medical instrument information relating to the medical instruments 30, 40 (step S313).

[0095] Next, the control unit 66 causes the display device 50 to display a simulation video based on the simulation data corresponding to the surgical procedure case selected by the operator U1 and transmitted from the medical information management server 70 (step S314). After step S314, the surgical training device 2 proceeds to step S306.

[0096] In step S315, if the operator U1 selects the surgeon mode via the input unit 63 (step S315: Yes), the surgery training apparatus 2 proceeds to step S316, which will be described later.

[0097] In step S316, the control unit 66 causes the display device 50 to display surgeon information from which a plurality of surgeons can be selected in the surgery mode.

[0098] Next, the operator information selected by the operator U1 is transmitted to the second communication unit 65 via the input unit 63 (step S317).

[0099] Thereafter, the control unit 66 causes the display device 50 to display a simulation video based on the simulation data for the surgeon information selected by the operator U1 and transmitted from the medical information management server 70 (step S318). After step S318, the surgery training apparatus 2 proceeds to step S306.

[0100] According to the embodiment described above, when one of a plurality of surgical procedures is selected by the control device 60 (client side), the server control unit 79 transmits simulation data corresponding to the selected surgical procedure to the control device 60, so that the operator U1 can gain experience equivalent to an actual surgery, thereby improving the learning curve of the operator U1.

[0101] Furthermore, according to one embodiment, training in surgical techniques can be carried out in one's spare time, regardless of location.

[0102] Furthermore, according to one embodiment, the generation unit 73 generates simulation data based on intraoperative video data captured by a medical imaging device during surgery on the subject and transmitted from each of the hospital management servers 80, and at least one of two-dimensional image data and three-dimensional data including data on at least the organs of the subject captured in advance by a CT device or an MRI device, so that simulation data for multiple surgical procedures and multiple cases can be easily collected.

[0103] Furthermore, according to one embodiment, the generation unit 73 generates simulation data by aligning the organs contained in the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data based on the estimation result of the inference unit 74, so that realistic simulation data can be generated.

[0104] Furthermore, according to one embodiment, the simulation data recording unit 75 records multiple pieces of simulation data in association with multiple pieces of surgeon information regarding the multiple surgeons who performed each of the multiple simulation data, so that the operator U1 can select the simulation data performed by the desired surgeon and perform training.

[0105] Furthermore, according to one embodiment, the simulation data recording unit 75 records a plurality of pieces of simulation data in association with a plurality of pieces of medical instrument information relating to a plurality of medical instruments used in each of the plurality of simulation data, so that the operator U1 can select simulation data using a desired medical instrument to perform training.

[0106] Furthermore, according to one embodiment, when one or more of the multiple pieces of medical instrument information stored in the control device 60 (client side) are selected, the server control unit 79 transmits the selected medical instrument information to the control device 60 side, so that the operator U1 can train with simulation data using the desired medical instrument.

[0107] Furthermore, according to one embodiment, when the billing unit 792 of the server control unit 79 transmits simulation data to the control device 60 (client side), billing is performed on the operator U1 operating the client side based on the identification information of the operator U1, thereby making it possible to obtain profits.

[0108] Furthermore, according to one embodiment, the server control unit 79 transmits surgical procedure information regarding surgical procedures selectable by the control device 60 to the control device 60 based on the level associated with the identification information of the operator U1 transmitted from the control device 60, thereby enabling the selection of simulation data appropriate to the level of the operator U1.

[0109] Furthermore, according to one embodiment, the server control unit 79 assigns points to the number of times the operator U1 virtually trains using simulation data based on the identification information of the operator U1, so that the level of the technique performed by the operator U1 can be easily grasped.

[0110] Furthermore, according to one embodiment, the simulation data recording unit 75 records the plurality of simulation data in association with each level, so that the operator U1 can select simulation data according to his / her own level and perform training.

[0111] Furthermore, according to one embodiment, the control unit 66 controls the display mode of the medical instrument image and the simulation video displayed by the display device 50 based on the operation content of the operator U1 detected by each of the first detection unit 31 and the second detection unit 32, thereby enabling intuitive operation.

[0112] Furthermore, according to one embodiment, when the control unit 66 receives a stop signal from the input unit 63 to stop the simulation video displayed by the display device 50, and detects the operation content detected by each of the first detection unit 31 and the second detection unit 32, the control unit 66 changes the viewpoint of the simulation video based on the operation content, so that the operator U1 can observe the simulation video from the viewpoint desired.

[0113] Furthermore, according to one embodiment, the control unit 66 causes the display device 50 to display multiple pieces of surgical procedure information relating to each of the multiple surgical procedures transmitted from the medical information management server 70, and when any one or more of the multiple pieces of surgical procedure information is selected via the input unit 63, the control unit 66 requests the medical information management server 70 to transmit simulation data corresponding to the selected surgical procedure information, so that simulation data corresponding to the case desired by the operator U1 can be obtained from the medical information management server 70.

[0114] Furthermore, according to one embodiment, the control unit 66 transmits the identification information of the operator U1 to the medical information management server 70, so that it is possible to acquire simulation data corresponding to the operator U1.

[0115] Furthermore, according to one embodiment, the control unit 66 transmits medical instrument information regarding each of the medical instruments 30, 40 used by the operator U1 to the medical information management server 70, so that simulation data suitable for the medical instruments 30, 40 used by the operator U1 for training can be obtained.

[0116] In one embodiment, one operator U1 performs the training, but this is not limited to the above, and multiple operators U1 may perform training while watching the same simulation video displayed by the display device 50, or multiple operators U1 may perform training using surgical training devices installed in different locations.

[0117] (Other embodiments) Various inventions can be formed by appropriately combining the multiple components disclosed in the above-mentioned surgical training system according to one embodiment of the present disclosure. For example, some components may be deleted from all the components described in the above-mentioned surgical training system according to one embodiment of the present disclosure. Furthermore, the components described in the above-mentioned medical observation system according to one embodiment of the present disclosure may be appropriately combined.

[0118] Furthermore, in the surgical training system according to an embodiment of the present disclosure, the above-mentioned "unit" can be read as "means" or "circuit," etc. For example, the control unit can be read as control means or control circuit.

[0119] In addition, the program to be executed by the surgical training system according to one embodiment of the present disclosure is provided as file data in an installable or executable format recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, a digital versatile disk (DVD), a USB medium, or a flash memory.

[0120] Furthermore, the program to be executed by the surgical training system according to one embodiment of the present disclosure may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0121] In the description of the flowcharts in this specification, the order of processing between steps is clearly indicated using expressions such as "first," "then," and "continue," but the order of processing required to implement the present invention is not uniquely determined by these expressions. In other words, the order of processing in the flowcharts described in this specification can be changed as long as there is no contradiction.

[0122] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present disclosure can be implemented in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the present disclosure section.

[0123] The present technology can also be configured as follows. (Appendix 1) a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other; a control unit that, when any one of the plurality of surgical procedures is selected on a client side, transmits simulation data corresponding to the selected surgical procedure to the client side; Equipped with Medical information management server. (Appendix 2) A medical information management server according to (Supplementary Note 1), A generating unit is further provided for generating the simulation data based on intraoperative video data captured during an operation on a subject and at least one of two-dimensional image data and three-dimensional data captured in advance by a CT device or an MRI device on the subject. Medical information management server. (Appendix 3) A medical information management server according to (Appendix 2), An inference unit that estimates organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data, The generation unit is and generating the simulation data by performing registration of organs included in the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data based on the estimation result of the inference unit. Medical information management server. (Appendix 4) A medical information management server according to (Appendix 3), The intraoperative video data is a multispectral light source is used to irradiate the subject with multispectral light, and a return light is received from the subject, thereby obtaining spectral image data including at least a reflection frequency for each biological tissue; The inference unit is estimating at least the type and location of the biological tissue using a reflection frequency for each of the biological tissues included in the spectral image data; Medical information management server. (Appendix 5) A medical information management server according to any one of (Supplementary Note 1) to (Supplementary Note 4), The recording unit is a plurality of pieces of simulation data are recorded in association with a plurality of pieces of surgeon information relating to a plurality of surgeons who performed the plurality of pieces of simulation data; The control unit is When any one of the plurality of pieces of surgeon information is selected on the client side, the plurality of pieces of simulation data corresponding to the selected surgeon information are transmitted to the client side. Medical information management server. (Appendix 6) A medical information management server according to any one of (Supplementary Note 1) to (Supplementary Note 5), The recording unit is a plurality of pieces of medical instrument information related to a plurality of medical instruments used in each of the plurality of pieces of simulation data is further associated with the plurality of pieces of simulation data and recorded; The control unit is When transmitting the simulation data to the client side, the medical device information associated with the simulation data is also transmitted to the client side. Medical information management server. (Appendix 7) A medical information management server according to (Supplementary Note 6), The control unit is When one or more of the plurality of pieces of medical instrument information are selected on the client side, the selected pieces of medical instrument information are transmitted to the client side. Medical information management server. (Appendix 8) A medical information management server according to any one of (Supplementary Note 1) to (Supplementary Note 7), The control unit is when the simulation data is transmitted to the client side, a fee is charged to the operator operating the client side based on identification information of the operator. Medical information management server. (Appendix 9) A medical information management server according to (Supplementary Note 8), The control unit is transmitting surgical procedure information regarding the surgical procedure selectable on the client side to the client side based on the level associated with the identification information; Medical information management server. (Appendix 10) A medical information management server according to (Supplementary Note 9), The control unit is giving a score to the number of times the operator has virtually performed training using the simulation data based on the identification information; Medical information management server. (Appendix 11) A medical information management server according to any one of (Supplementary Note 8) to (Supplementary Note 10), The recording unit is further correlating and recording each level of the plurality of simulation data; Medical information management server. (Appendix 12) a display unit capable of displaying a simulation video based on a plurality of simulation data including at least data related to a biological tissue and a plurality of surgical procedures transmitted from a medical information management server that records the plurality of simulation data in association with each other; One or more medical devices; A detection unit that detects an operation performed by an operator on the medical instrument; a control unit that causes a medical instrument image related to the medical instrument to be displayed on the display unit on the simulation video; Equipped with The control unit controls a display mode of the medical instrument image and the simulation video based on the operation content detected by the detection unit. Surgical training equipment. (Appendix 13) A surgical training device according to claim 12, The control unit is when a stop signal for stopping the simulation moving image displayed by the display unit is input, and when the detection unit detects the operation content, the viewpoint of the simulation moving image is changed based on the operation content. Surgical training equipment. (Appendix 14) A surgical training device according to claim 13, The control unit is displaying a plurality of pieces of surgical procedure information related to each of the plurality of surgical procedures on the display unit, and when any one or more of the plurality of pieces of surgical procedure information is selected, requesting the medical information management server to transmit the simulation data corresponding to the selected surgical procedure information; Surgical training equipment. (Appendix 15) A surgical training apparatus as described in (Appendix 14), an input unit for receiving input of identification information of the operator, The control unit is transmitting the identification information to the medical information management server; Surgical training equipment. (Appendix 16) A surgical training apparatus according to any one of (Appendix 12) to (Appendix 15), The control unit is transmitting medical instrument information relating to the medical instrument to the medical information management server; Surgical training equipment. (Appendix 17) A medical information management server; a surgery training device capable of communicating with the medical information management server via a network; Equipped with The medical information management server includes: a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other; a first control unit that, when any one of the plurality of surgical procedures is selected on a client side, transmits simulation data corresponding to the selected surgical procedure to the client side; Equipped with The surgical training device comprises: a display unit capable of displaying a simulation video based on the simulation data transmitted from the medical information management server; one or more medical instruments capable of performing treatment on a model that mimics a subject; A detection unit that detects an operation performed by an operator on the medical instrument; a second control unit that causes the display unit to display a medical instrument image related to the medical instrument on the simulation moving image; Equipped with The second control unit controls a display mode of the medical instrument image and the simulation video based on the operation content detected by the detection unit. Surgical training system. (Appendix 18) An image transmission method executed by a medical information management server including a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other, the method comprising: When any one of the plurality of surgical procedures is selected on the client side, simulation data corresponding to the selected surgical procedure is transmitted to the client side. How to send images. (Appendix 19) A display method is executed by a surgical training device, the display device being equipped with a display unit capable of displaying a simulation video based on simulation data transmitted from a medical information management server that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissues in association with each other, and one or more medical instruments capable of treating a model simulating a subject, the display method comprising: Detecting an operation performed by an operator on the medical instrument; causing the display unit to display a medical instrument image relating to the medical instrument on the simulation video, and controlling a display mode of the medical instrument image and the simulation video based on the operation content. Display method. (Appendix 20) A program executed by a medical information management server having a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other, When any one of the plurality of surgical procedures is selected on the client side, simulation data corresponding to the selected surgical procedure is transmitted to the client side. program. (Appendix 21) A program to be executed by a surgical training device, the program comprising: a display unit capable of displaying a simulation video based on simulation data transmitted from a medical information management server that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissues in association with each other; and one or more medical instruments capable of performing treatment on a model simulating a subject, detecting an operation performed by an operator on the medical instrument; causing the display unit to display a medical instrument image relating to the medical instrument on the simulation video, and causing the display unit to control a display mode of the medical instrument image and the simulation video based on the operation content. program. [Explanation of symbols]

[0124] 1. Surgical training system 2. Surgical training equipment 10,20 Trocar 30,40 Medical equipment 31 First detection unit 32 Second detector 33,64,72 Recording section 34,71 Communications Department 50 Display device 60 Control device 61 1 Communications Department 62 Image Processing Unit 63 Input section 65 Second Communication Department 66 Control Unit 70 Medical information management server 71 Communications Department 73 Generation part 74 Reasoning part 75 Simulation Data Recording Unit 76 Image data recording section 77 User data recording section 78 Medical Instrument Data Recording Unit 79 Server control section 80 Hospital Management Server 83 Input section 90 Network 641 Program Recording Section 721 Program Recording Section 791 Judgment section 792 Billing Department 793 Scoring Department O1 model U1 Operator

Claims

1. a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other; a control unit that, when any one of the plurality of surgical procedures is selected on a client side, transmits simulation data corresponding to the selected surgical procedure to the client side; a generating unit that generates the simulation data based on intraoperative video data captured during an operation on a subject and at least one of two-dimensional image data and three-dimensional data captured in advance by a CT device or an MRI device on the subject; an inference unit that estimates organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data; Equipped with The generation unit is and generating the simulation data by performing registration of organs included in the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data based on the estimation result of the inference unit. Medical information management server.

2. 2. The medical information management server according to claim 1, The intraoperative video data is a multispectral light source is used to irradiate the subject with multispectral light, and a return light is received from the subject, thereby obtaining spectral image data including at least a reflection frequency for each biological tissue; The inference unit is estimating at least the type and location of the biological tissue using a reflection frequency for each of the biological tissues included in the spectral image data; Medical information management server.

3. 2. The medical information management server according to claim 1, The recording unit is a plurality of pieces of simulation data are recorded in association with a plurality of pieces of surgeon information relating to a plurality of surgeons who performed the plurality of pieces of simulation data; The control unit is When any one of the plurality of pieces of surgeon information is selected on the client side, the plurality of pieces of simulation data corresponding to the selected surgeon information are transmitted to the client side. Medical information management server.

4. 2. The medical information management server according to claim 1, The recording unit is a plurality of pieces of medical instrument information related to a plurality of medical instruments used in each of the plurality of pieces of simulation data is further associated with the plurality of pieces of simulation data and recorded; The control unit is When transmitting the simulation data to the client side, the medical device information associated with the simulation data is also transmitted to the client side. Medical information management server.

5. 5. The medical information management server according to claim 4, The control unit is When one or more of the plurality of pieces of medical instrument information are selected on the client side, the selected pieces of medical instrument information are transmitted to the client side. Medical information management server.

6. 2. The medical information management server according to claim 1, The control unit is when the simulation data is transmitted to the client side, a fee is charged to the operator operating the client side based on identification information of the operator. Medical information management server.

7. 7. The medical information management server according to claim 6, The control unit is transmitting, to the client side, surgical procedure information regarding the surgical procedure selectable on the client side based on the level associated with the identification information; Medical information management server.

8. The medical information management server according to claim 7, The control unit is giving a score to the number of times the operator has virtually performed training using the simulation data based on the identification information; Medical information management server.

9. 7. The medical information management server according to claim 6, The recording unit is further correlating and recording each level of the plurality of simulation data; Medical information management server.

10. An image transmission method executed by a medical information management server including a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other, the method comprising: a control step of transmitting, when any one of the plurality of surgical procedures is selected on a client side, simulation data corresponding to the selected surgical procedure to the client side; a generating step of generating the simulation data based on intraoperative video data captured during an operation on a subject and at least one of two-dimensional image data and three-dimensional data captured in advance by a CT device or an MRI device on the subject; an estimation step of estimating organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data; Including, The generating step includes: generating the simulation data by aligning organs included in the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data based on the estimation result of the estimation step; How to send images.

11. A program executed by a medical information management server having a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other, a control step of transmitting, when any one of the plurality of surgical procedures is selected on a client side, simulation data corresponding to the selected surgical procedure to the client side; a generating step of generating the simulation data based on intraoperative video data captured during an operation on a subject and at least one of two-dimensional image data and three-dimensional data captured in advance by a CT device or an MRI device on the subject; an estimation step of estimating organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data; Run the command, The generating step includes: Based on the estimation result of the estimation step, the positioning of organs included in the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data is performed to generate the simulation data. program.

Citation Information

Patent Citations

  • Operation supporting system

    JP1999197159A

  • Pseudo-experience presentation system and method for medical equipment

    JP2002006735A

  • Surgery supporting system

    JP2006006604A

  • Surgery training device

    JP2016148765A

  • Surgical video retrieval based on preoperative images

    US20190286652A1