Medical information processing apparatus, medical information processing method, and medical information processing program

Abstract

To improve the quality of a learning model for generating an operation plan used for robotic surgery.SOLUTION: The medical information processing apparatus according to the embodiment includes an operation information acquisition circuit configured to acquire a plurality of pieces of operation information including an operation record of a surgical robot, a surgery evaluation information acquisition circuit configured to acquire a plurality of pieces of surgery evaluation information corresponding to the plurality of pieces of operation information and including an evaluation of a surgery performed by the surgical robot, an operation information classification circuit configured to classify each of the plurality of pieces of operation information based on the plurality of pieces of surgery evaluation information, and a learning model construction circuit configured to construct a learning model based on the classified plurality of pieces of operation information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The embodiments disclosed in the present specification and drawings relate to a medical information processing device, a medical information processing method, and a medical information processing program. [Background technology]

[0002] In recent years, the use of robots (surgical robots) to assist in surgery has rapidly spread. The benefits of surgery using surgical robots (robotic surgery) include the ability to minimize invasiveness to the patient and the ability to perform procedures that are difficult for humans to perform using multiple rotation axes and small devices. From the perspective of insurance coverage, the scope of robotic surgery is rapidly expanding, making it easier to include robotic surgery as an option.

[0003] Research and development is underway on the remote control, automation, and semi-automation of surgical robots, and it is expected that robotic surgery will be realized in the near future through remote control and automation of surgical robots. When a surgical robot is automated or semi-automated, it is conceivable that the surgical robot will be controlled based on a motion plan created in advance. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-29274 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-104456 [Patent Document 3] Japanese Patent Publication No. 2020-168359 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the quality of a learning model for generating motion plans used in robotic surgery. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0006] The medical information processing device according to the embodiment includes a motion information acquisition unit that acquires multiple pieces of motion information including motion records of a surgical robot, a surgical evaluation information acquisition unit that acquires multiple pieces of surgical evaluation information corresponding to the multiple pieces of motion information and including evaluations of surgery performed by the surgical robot, a motion information classification unit that classifies each of the multiple pieces of motion information based on the multiple pieces of surgical evaluation information, and a learning model construction unit that constructs a learning model based on the classified multiple pieces of motion information. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a medical system according to an embodiment. [Figure 2] FIG. 1 is a block diagram showing an example of the configuration of a medical image processing apparatus according to an embodiment. [Figure 3] FIG. 1 is a block diagram showing an example of the configuration of a medical information storage device according to an embodiment. [Figure 4] FIG. 2 is a block diagram showing an example of the configuration of a terminal device according to the embodiment. [Figure 5A] 5A and 5B are schematic diagrams illustrating operation information of a surgical robot according to an embodiment. [Figure 5B] 5A and 5B are schematic diagrams illustrating operation information of a surgical robot according to an embodiment. [Figure 5C] 5A and 5B are schematic diagrams illustrating operation information of a surgical robot according to an embodiment. [Figure 6] FIG. 4 is a diagram showing an example of text data of operation information according to the embodiment. [Figure 7] FIG. 4 is a diagram showing an example of text data of peripheral information according to the embodiment. [Figure 8]FIG. 2 is a diagram showing an example of a surgery record according to the embodiment. [Figure 9] 10A and 10B are diagrams for explaining a correction process of an operation plan according to the embodiment. [Figure 10] FIG. 4 is a diagram showing an example of a screen displayed on the terminal device according to the embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a screen (support information) displayed on the terminal device according to the embodiment. [Figure 12] 1 is a flowchart showing an example of a medical information processing method according to an embodiment. [Figure 13] 1 is a flowchart showing an example of a medical information processing method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of a medical system and a medical image processing device will be described with reference to the drawings. In the following description, components having substantially the same functions and configurations are designated by the same reference numerals, and redundant explanations will be given only when necessary.

[0009] <Medical System 1> A medical system 1 according to an embodiment will be described with reference to FIG.

[0010] The medical system 1 includes a medical information processing device 10, a medical information storage device 20, a terminal device 30, and a surgical robot 40. The components of the medical system 1 are communicatively connected via a communication network. The configuration and protocol of the communication network are not particularly limited. The communication network may be an in-hospital network or may include a public network such as the Internet.

[0011] The medical information processing device 10 is configured to classify motion information based on surgical evaluation information including an evaluation of the surgery performed by the surgical robot 40, construct a learning model based on the classified motion information, generate a motion plan to be used in surgery on a target patient (an individual patient who is the target of surgery) based on the constructed learning model, and output information for controlling the surgical robot 40 based on the generated motion plan. Here, the motion information is information including a motion record or an operation record of the surgical robot 40. Details of the medical information processing device 10 will be described later using FIG. 2.

[0012] The medical information storage device 20 is a device that stores various databases. Specifically, databases such as a motion information database 21a that stores motion information of the surgical robot 40 are stored in the medical information storage device 20. Details of the medical information storage device 20 will be described later with reference to FIG. 3.

[0013] The terminal device 30 is a terminal used by a user such as a doctor, and is used, for example, to monitor surgery performed by the surgical robot 40 or to operate the surgical robot 40. The number of terminal devices 30 is not limited to one, and multiple terminal devices 30 may be provided in the medical system 1. In this case, for example, one terminal device 30 is a terminal used by a doctor who operates the surgical robot 40, and another terminal device 30 is a terminal used by another doctor who supports or guides the doctor. Details of the terminal devices 30 will be described later using Figure 4.

[0014] The surgical robot 40 is a surgical robot equipped with one or more surgical devices. Examples of surgical devices include an automatic suturer and a laparoscope. The surgical robot 40 may be manually operated by a doctor, or may perform surgery fully or semi-automatically. The surgical robot 40 is equipped with a camera (such as an endoscopic camera) and various sensors (such as a force sensor), and outputs images and sensor information during surgery to the medical information processing device 10, etc.

[0015] <Medical information processing device 10> The medical information processing apparatus 10 according to the embodiment will be described in detail with reference to FIG.

[0016] The medical information processing device 10 includes a memory circuitry 11, a communication interface 12, and a processing circuitry 13. Each component will be described in detail below.

[0017] The memory circuitry 11 is connected to the processing circuitry 13 and stores various types of information used by the processing circuitry 13. The memory circuitry 11 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, a hard disk, an optical disk, etc. The memory circuitry 11 stores various programs required for the processing circuitry 13 to execute each function, various types of data to be processed by the programs, etc. Note that the various types of data handled in this specification are typically digital data.

[0018] The communication interface 12 communicates with other components of the medical system 1 (such as the medical information storage device 20, the terminal device 30, and the surgical robot 40) via a communication network in accordance with various communication protocols.

[0019] The processing circuit 13 is an arithmetic circuit that performs various calculations and controls the operation of the medical information processing device 10. The processing circuit 13 has an information acquisition function 13a, a motion information classification function 13b, a learning model construction function 13c, a motion plan generation function 13d, and an information output function 13e. Details of each processing function of the processing circuit 13 will be explained after the medical information storage device 20 and the terminal device 30 are explained. The information acquisition function 13a is an example of a motion information acquisition unit and a surgical evaluation information acquisition unit in the claims. The motion information classification function 13b is an example of a motion information classification unit. The learning model construction function 13c is an example of a learning model construction unit. The motion plan generation function 13d is an example of a motion plan generation unit. The information output function 13e is an example of an information output unit.

[0020] In this embodiment, each processing function executed by the information acquisition function 13a, the motion information classification function 13b, the learning model construction function 13c, the motion plan generation function 13d, and the information output function 13e is stored in the form of a computer-executable program in the storage circuitry 11. The processing circuitry 13 is configured with a processor, and realizes the function corresponding to each program by reading and executing the program from the storage circuitry 11. In other words, the processing circuitry 13 in a state in which each program has been read has each function shown in the processing circuitry 13 of FIG. 2.

[0021] 2 illustrates a case where the information acquisition function 13a, the motion information classification function 13b, the learning model construction function 13c, the motion plan generation function 13d, and the information output function 13e are each realized by a single processing circuit 13, but the embodiment is not limited to this. For example, the processing circuit 13 may be configured as a combination of multiple independent processors, and each processor may execute a program to realize each processing function. Each processing function of the processing circuit 13 may be realized by being appropriately distributed or integrated into a single or multiple processing circuits.

[0022] 2 is merely an example, and the medical information processing device 10 may be configured with multiple information processing devices. For example, the medical information processing device 10 may be configured to include a first information processing device having a motion information classification function 13b and a learning model construction function 13c, and a second information processing device having a motion plan generation function 13d.

[0023] <Medical information storage device 20> The medical information storage device 20 according to the embodiment will be described in detail with reference to FIG.

[0024] The medical information storage device 20 includes a storage circuit 21, a communication interface 22, and a processing circuit 23. Each component will be described in detail below.

[0025] The memory circuitry 21 is connected to the processing circuitry 23 and stores various types of information used by the processing circuitry 23. The memory circuitry 21 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like.

[0026] The memory circuitry 21 stores a motion information database 21a, a surgery evaluation information database 21b, and a patient information database 21c.

[0027] The motion information database 21a stores motion information that records the motion of the surgical robot 40. This motion information database 21a stores a plurality of pieces of motion information for each patient. The motion information is information that indicates the motion record of the surgical robot 40. Specific examples of the motion information will be described later with reference to FIG. 5A etc.

[0028] The surgery evaluation information database 21b stores surgery evaluation information (surgery records, prognosis records, etc.) including evaluations of the surgery. Here, the surgery records are records of the surgery performed by the surgical robot 40, such as records created by a doctor after the surgery is completed. The prognosis records are records of follow-up observations after surgery, such as the results of tests (CT scans, blood tests, etc.) performed some time after the surgery, diagnostic reports, etc.

[0029] The surgical record may be automatically created by the surgical robot 40 or the like. Furthermore, the surgical record is not limited to text data, and may be video data captured during surgery or sensor information. Furthermore, the prognosis record may be data collected from the patient using a wearable device or the like.

[0030] The patient information database 21c is a database that stores patient information. The patient information includes, for example, information about the patient's name, age, sex, disease, and case. The patient information may also include information about the patient's modality image and the patient's body type and tissue shape obtained from the modality image. Here, the modality image refers to a medical image obtained by medical equipment such as a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, or an ultrasound diagnostic device.

[0031] The communication interface 22 communicates with other components of the medical system 1 (such as the medical information processing device 10, the terminal device 30, and the surgical robot 40) via a communication network in accordance with various communication protocols.

[0032] The processing circuitry 23 is an arithmetic circuit that performs various calculations and controls the operation of the medical information storage device 20. The processing circuitry 23 has an information reading function 23a. The information reading function 23a reads information from the database stored in the storage circuitry 21 in response to a request from another device such as the medical information processing device 10, and transmits the information to the requestor via the communication interface 22.

[0033] The medical information storage device 20 may be configured with multiple information storage devices. For example, the medical information storage device 20 may be configured with a first information storage device having a motion information database 21a, a second information storage device having a surgical evaluation information database 21b, and a third information storage device having a patient information database 21c. In this case, the first information processing device may constitute an information sharing service or a data service, the second information processing device may constitute a pathway service for checking a patient's medical condition and history, and the third information processing device may constitute an electronic medical record system or a PACS (Picture Archiving and Communication System).

[0034] <Terminal device 30> Next, the terminal device 30 according to the embodiment will be described in detail with reference to FIG.

[0035] The terminal device 30 includes a memory circuit 31, a communication interface 32, an output interface 33, an input interface 34, and a processing circuit 35.

[0036] The memory circuit 31 is connected to the processing circuit 35 and stores various information used by the processing circuit 35. The memory circuit 31 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like.

[0037] The communication interface 32 communicates with other components of the medical system 1 (such as the medical information processing device 10, the medical information storage device 20, and the surgical robot 40) via a communication network in accordance with various communication protocols.

[0038] The output interface 33 presents information to a user (doctor, etc.) of the terminal device 30. The output interface 33 has a display unit such as a liquid crystal display, and displays information such as images. For example, the output interface 33 displays information about the currently executing surgical process, surgical images, and a GUI (Graphical User Interface) that accepts user operations. The output interface 33 may also have an audio output means (speaker) and output alerts, operation guide information, etc. by audio. The output interface 33 may also output information, etc. to another information processing device connected to the terminal device 30 via a communication network (for example, a terminal device used by a support / guide doctor in another hospital).

[0039] The input interface 34 receives various input operations from the user, converts the received input operations into electrical signals, and outputs the signals to the processing circuit 35. The input interface 34 is realized by, for example, a mouse, a keyboard, a touch panel, a trackball, a manual switch, a foot switch, a button, a joystick, or the like.

[0040] The processing circuit 35 is an arithmetic circuit that performs various calculations and controls the operation of the terminal device 30. The processing circuit 35 has an information reception function 35a and a display control function 35b. The information reception function 35a receives information input from a user via the input interface 34. For example, the information reception function 35a receives operations on a GUI displayed on the screen of the terminal device 30 and information input via the GUI. The display control function 35b controls the output interface 33 to display desired information on the display unit. For example, the display control function 35b displays surgical images, operation information of the surgical robot 40, etc. on the display unit.

[0041] <Details of the medical information processing device 10> Next, each processing function of the processing circuitry 13 of the medical information processing apparatus 10 will be described in detail.

[0042] The information acquisition function 13a acquires various types of information from the components of the medical system 1. For example, the information acquisition function 13a acquires motion information of the surgical robot 40 from the medical information storage device 20 (motion information database 21a). In this case, the information acquisition function 13a extracts information on multiple patients (similar patients) whose cases, surgical contents, etc. are similar to each other from the patient information database 21c of the medical information storage device 20, and acquires motion information of the surgical robot 40 in the surgery of each similar patient from the motion information database 21a of the medical information storage device 20. In this way, the information acquisition function 13a acquires multiple pieces of motion information (motion information group). Note that similar patients may be extracted by searching the patient information database 21c using the patient information of the target patient.

[0043] Here, specific examples of operation information will be described. Figures 5A, 5B, and 5C are schematic diagrams showing examples of operation information related to a series of operations during surgery. In each figure, TR indicates the surgical route, and BI1, BI2, and BI3 are internal body images of the patient (surgical video, tomographic images, etc.). The operation information shown in Figure 5A includes point information Pt1, the operation information shown in Figure 5B includes point information Pt2, and the operation information shown in Figure 5C includes point information Pt3. Figure 6 shows an example of the data structure of point information Pt1, Pt2, and Pt3. In this example, point information Pt1, Pt2, and Pt3 include procedure information (e.g., clipping, resection) related to the procedure performed at that point, the type of surgical device used in the procedure (e.g., automatic stapler, laparoscope), the position of the surgical device, and the route. Here, the route is information related to the surgical route of the surgical robot 40 (surgical device), such as "bypassing the inferior vena cava from the left" or "bypassing the inferior vena cava from the right." An in-vivo image BI1 in Fig. 5A is an in-vivo image at time (t1) of point information Pt1. Similarly, an in-vivo image BI2 in Fig. 5B is an in-vivo image at time (t2) of point information Pt2, and an in-vivo image BI3 in Fig. 5C is an in-vivo image at time (t3) of point information Pt3.

[0044] In Figure 6, for point information Pt1, Operation1 indicates procedure information, Device1 indicates the type of surgical device, x1, y1, z1 indicate the position of the surgical device (x coordinate, y coordinate, z coordinate), t1 indicates time, and Route indicates information about the route. Note that the point information may also include information about the angle and strength (force applied to the treatment area) of the surgical device. Furthermore, the point information may include at least one of the above-mentioned procedure information, type of surgical device, position, angle, strength, and route of the surgical device.

[0045] The motion information acquired by the information acquisition function 13a may be information corresponding to an entire surgery (information on all points constituting the trajectory TR), or information corresponding to a part of the surgery (information on only points constituting a part of the trajectory TR). In the latter case, only motion information on a specific scene during the surgery is acquired. For example, only motion information on a specific procedure or surgical device is acquired. In this case, the information acquisition function 13a acquires multiple pieces of motion information on the specific scene for multiple patients (similar patients).

[0046] The motion information may also include peripheral information about the area around the treatment site. The peripheral information is obtained based on information acquired by the surgical robot 40's camera, sensors, etc. Figure 7 shows an example of the data structure of peripheral information Pt1α, Pt2α, and Pt3α corresponding to point information Pt1, Pt2, and Pt3. In this example, the peripheral information includes information about the area (e.g., upper renal vein, lower renal artery), condition (e.g., significant damage due to invasive surgery), and weight. The "weight" indicates the degree of contribution or influence to the prognosis. In Figure 7, for the peripheral information Pt1α, Area1 indicates the area, Remarks1 indicates the condition, and Weight1 indicates the weight. Using motion information including peripheral information for learning enables the construction of a more accurate learning model. For example, a more reliable learning model can be constructed by extracting and integrating motion information with a good "condition" or a relatively large "weight."

[0047] Furthermore, the information acquiring function 13a acquires multiple pieces of surgery evaluation information corresponding to the acquired multiple pieces of operation information from the surgery evaluation information database 21b of the medical information storage device 20. The surgery evaluation information is a record of the surgery performed by the surgical robot 40 (surgery record) and / or a record of follow-up observation after the surgery (prognosis record). Note that the surgery evaluation information may be acquired by an information acquiring function different from the information acquiring function 13a (for example, an information acquiring function that acquires only the surgery evaluation information).

[0048] Figure 8 shows an example of a surgical record. This surgical record includes "patient information," "intraoperative findings," and "surgical findings." In this example, the amount of blood loss (359 ml) in "patient information" is a relatively small value, and therefore indicates a favorable evaluation of the surgery. The "intraoperative findings" include "blood in the left ovarian artery and vein ligation and fallopian tube branch" and the "surgical findings" include "anal margin distance of approximately 6 cm was maintained and the mesorectum was dissected circumferentially," which are procedures that went as expected, and therefore indicate a favorable evaluation of the surgery. On the other hand, the "bleeding from the left adnexa during intraoperative manipulation" in "intraoperative findings" is an unexpected event, and therefore indicates a negative evaluation of the surgery.

[0049] The evaluation of a surgery or procedure is not limited to two levels, "good" and "bad," but may be three or more levels, such as "very good," "good," "bad," and "very bad." In this case, the weighting coefficient for "very good" may be set higher than the weighting coefficient for "good," and this may be reflected in the evaluation of the surgery.

[0050] Furthermore, the evaluation target for the surgery may be the entire surgical record as described above, or it may be the action information corresponding to a part of the surgical record. In the latter case, for example, only the description of the surgical findings in the surgical record may be the evaluation target. Furthermore, only the description in the "2. Anal intestinal resection and anastomosis" column of the surgical findings may be the evaluation target.

[0051] The motion information classification function 13b classifies each of the multiple pieces of motion information based on the multiple pieces of surgical evaluation information. In other words, the motion information classification function 13b classifies each of the multiple pieces of motion information based on the corresponding surgical evaluation information. In this embodiment, the motion information classification function 13b classifies each piece of motion information as "good" or "bad." For example, in the case of the surgical record of FIG. 8, since there are more pieces of information indicating good evaluations, the motion information classification function 13b classifies the surgical motion information corresponding to the surgical record as "good." Note that the classification of motion information is not limited to a method of simply comparing the number of pieces of information indicating good evaluations with the number of pieces of information indicating bad evaluations. For example, the weights of each piece of information included in the surgical evaluation information may be different, for example, by assigning a large weighting coefficient to information indicating heavy bleeding.

[0052] The motion information is not limited to being classified into two categories, "good" and "bad," but may be classified into three or more categories. For example, the motion information classification function 13b may classify the motion information into "good," "neither good nor bad," or "bad." Alternatively, the motion information classification function 13b may classify the motion information into "very good," "good," "bad," or "very bad."

[0053] Furthermore, when a prognosis record is used as the surgical evaluation information, the motion information classification function 13b may classify the motion information using information on the recovery or worsening of the affected area described in the prognosis record. For example, when the diagnostic report includes a description indicating a good evaluation such as "partial response," the motion information classification function 13b may classify (update) the motion information of the surgery corresponding to the diagnostic report as "good."

[0054] The learning model construction function 13c constructs a learning model based on multiple pieces of classified motion information. For example, the learning model construction function 13c constructs a learning model (a standard model for motion planning) using motion information classified as "good." As an example, the learning model construction function 13c constructs a learning model by integrating multiple pieces of motion information classified as "good" through statistical processing (such as averaging). At this time, optimization processing may be performed for each piece of point information included in the motion information. In this case, averaging processing may be performed for each of the position, angle, and intensity of the point information. Note that the learning model is not limited to being constructed using all of the point information included in the multiple pieces of motion information, and may be constructed using only point information of pre-extracted feature points. Furthermore, the method for constructing the learning model is not particularly limited, and the learning model may be constructed by machine learning such as deep learning. For example, the learning model construction function 13c may construct a highly rated learning model (a standard model for motion planning) by performing machine learning using motion information and its classification results as training data.

[0055] The learning model construction function 13c may construct a learning model using both the action information classified as "good" and the action information classified as "bad." For example, a learning model may be constructed that passes through the route of the action information classified as "good" as much as possible and passes through a position as far as possible from the route of the action information classified as "bad."

[0056] Furthermore, the learning model construction function 13c may construct a learning model based on the motion information classified using the prognosis record. Specifically, the learning model construction function 13c may construct a learning model based on the motion information classified using the surgery record, and then construct (reconstruct) the learning model based on the motion information classified using the prognosis record.

[0057] The motion plan generation function 13d generates a motion plan for the surgical robot 40 to perform surgery on the target patient based on the constructed learning model. For example, the motion plan includes multiple point motion information from the start to the end of the surgery. Each point motion information includes information on the procedure (e.g., clipping, resection), the surgical device (e.g., an automatic suturer, a laparoscope), and the position, angle, and strength of the surgical device. The motion plan generation function 13d may use the learning model as the motion plan. Alternatively, the motion plan generation function 13d may use a learning model reconstructed based on motion information of surgeries performed on patients whose body shapes are similar to the target patient as the motion plan. A patient similar to the target patient is a patient whose height, weight, body shape, or the shape or position of tissues and organs is similar to the target patient. For example, the motion plan generation function 13d may assign a large weighting coefficient to the motion information of patients similar to the target patient and reconstruct the learning model to create a motion plan.

[0058] The motion plan generation function 13d may generate a motion plan by correcting the learning model based on correction information for the target patient. Here, the correction information is information for correcting the learning model to suit an individual patient, such as the height, weight, body type, or shape or position of tissues and organs of the target patient. The correction information is obtained from patient information, modality images, sensor information, etc. For example, if the correction information reveals that the learning model is not suitable for the target patient, the motion plan generation function 13d corrects the learning model based on the correction information to generate a motion plan for the target patient.

[0059] 9 is a diagram for explaining the correction process of the motion plan. In this example, the position, angle, and strength of the surgical device are corrected to prevent the surgical device from hitting the internal organs of the target patient and reduce the impact on the internal organs. In this way, the motion plan generation function 13d corrects the learning model based on the correction information to generate a motion plan that can actually be applied to an individual patient.

[0060] It is expected that the motion plan may need to be revised to deal with unplanned problems detected by cameras, sensors, etc. during surgery. For this reason, the motion plan generation function 13d may correct the motion plan for the target patient in real time based on information obtained during surgery (real-time information). The real-time information may include surgical images captured by the surgical robot 40 or cameras installed in the operating room, information obtained from sensors on the surgical robot 40, etc. For example, the motion plan generation function 13d identifies similar cases of the problem that has occurred, collects motion information for the identified similar cases, and corrects the motion plan based on the collected motion information. By reviewing the motion plan using real-time information in this way, the surgery can be continued appropriately even if an unexpected problem or event occurs during surgery.

[0061] Furthermore, the motion plan generating function 13d may correct the motion plan based on the intervention of a user such as a doctor (user intervention information). User intervention is performed, for example, by a user monitoring the surgery on the terminal device 30, changing the motion plan via the terminal device 30. By correcting the motion plan based on the user intervention information in this way, it is possible to perform an appropriate surgery even when a problem occurs that cannot be handled by the surgical robot 40.

[0062] The information output function 13e outputs various information to other components (such as the terminal device 30 and the surgical robot 40) of the medical system 1. For example, the information output function 13e outputs surgical images, sensor information, or operation information obtained from the surgical robot 40 to the terminal device 30 via the communication interface 12.

[0063] FIG. 10 shows an example of a screen displayed on the terminal device 30. In this example, surgical video SV and information Inf1 and Inf2 are displayed on the screen. The surgical video SV is a real-time video showing the surgical situation, and includes surgical devices D1 and D2 and the area R currently being treated. Information Inf1 displays the current operation information (operation details) of the surgical robot 40. Information Inf2 displays the operation procedure (surgical procedure) of the surgical robot 40. In this example, Pt3 (clipping) is currently being performed.

[0064] The user of the terminal device 30 may change the content displayed on the screen. That is, the user may change the content of information Inf1 and Inf2 via the input interface 34 of the terminal device 30. For example, the user may change the value of the procedure or strength of the surgical device in information Inf1, or change the operation procedure in information Inf2. The changed information (user intervention information) is transmitted from the terminal device 30 to the medical information processing device 10. The operation plan generation function 13d of the medical information processing device 10 modifies the operation plan being executed based on the user intervention information received from the terminal device 30.

[0065] The information output function 13e outputs information (control information) for controlling the surgical robot 40 based on the generated operation plan to the surgical robot 40. Upon receiving the control information, the surgical robot 40 automatically operates in accordance with the operation plan.

[0066] The information output function 13e may also output information for displaying support information on the terminal device 30. The support information is information for supporting a user, such as a doctor, to intervene in surgery. The terminal device 30 displays the support information received from the medical information processing device 10. FIG. 11 is an example of a screen including support information displayed on the terminal device 30. In this example, support information M1 and M2 are displayed together with the surgical video SV. The support information M1 includes an alert such as a dangerous area in the surgery (in this example, "upper renal vein"). The alert may also include information such as a dangerous act. The support information M2 includes a recommended route or approach (in this example, "bypass the inferior vena cava from the right"). Such support information is generated based on real-time information obtained during surgery. Presenting the support information makes it easier for a user, such as a doctor, to intervene in the surgery appropriately.

[0067] It is also possible to generate recommended values ​​for the position, angle, strength, etc. of the surgical device based on real-time information and display them as support information. In this case, when the support information such as the recommended values ​​is approved by the user, the operation plan generation function 13d corrects the operation plan based on the approved information.

[0068] Furthermore, a reference video IM may be displayed as shown in Fig. 11. The reference video IM presents, for example, a video of a recommended surgery to the user.

[0069] <Medical information processing method> An example of the processing of the medical information processing device 10 will be described with reference to the flowcharts of Figures 12 and 13. Figure 12 shows an example of the processing up to constructing a learning model, and Figure 13 shows an example of the processing when performing surgery using the constructed learning model.

[0070] Step S11: The information acquisition function 13a acquires multiple pieces of operation information including operation records of the surgical robot 40. Specifically, the information acquisition function 13a extracts multiple similar patients by searching the patient information database 21c using the patient information of the target patient, and acquires multiple pieces of operation information of the surgical robot 40 in surgeries on the extracted multiple similar patients from the operation information database 21a of the medical information storage device 20.

[0071] Step S12: The information acquiring function 13a acquires multiple pieces of surgery evaluation information corresponding to the multiple pieces of motion information acquired in step S11. Specifically, the information acquiring function 13a acquires the surgery record and / or prognosis record of the surgery related to the motion information from the surgery evaluation information database 21b.

[0072] Step S13: The motion information classification function 13b classifies each of the multiple pieces of motion information acquired in step S11 based on the multiple pieces of surgery evaluation information acquired in step S12. For example, the motion information classification function 13b determines whether the surgery evaluation information indicates a good evaluation of the surgery, and classifies the motion information corresponding to the surgery evaluation information based on that determination.

[0073] Step S14: The learning model construction function 13c constructs a learning model based on the plurality of pieces of motion information classified in step S13.

[0074] By performing the processes of steps S11 to S14, a learning model is generated as a standard model for a motion plan. After that, the motion plan generation function 13d corrects the learning model based on the correction information of an individual patient (target patient), thereby generating a motion plan.

[0075] Next, an example of a processing flow during surgery will be described with reference to FIG.

[0076] Step S21: The information acquisition function 13a determines whether real-time information or user intervention information has been acquired. It is determined whether real-time information has been received from the surgical robot 40 and whether user intervention information has been received from the terminal device 30. If real-time information or user intervention information has been acquired (S21: Yes), proceed to step S22; if not (S21: No), proceed to step S23. Note that the determination in this step may be made by a function of the processing circuit 13 other than the information acquisition function 13a.

[0077] Step S22: The motion plan generating function 13d corrects the motion plan based on the real-time information or user intervention information acquired in step S21.

[0078] Step S23: The information output function 13e outputs information (control information) for controlling the surgical robot 40 based on the operation plan to the surgical robot 40. The surgical robot 40 performs surgical operations based on the control information received from the medical information processing device 10.

[0079] Step S24: The information output function 13e determines whether or not there are any remaining operation plans (operation procedures) to be executed. For example, if there are any unexecuted operations in the operation plan, the process returns to step S21; otherwise, the process ends. Note that the determination in this step may be performed by a function of the processing circuitry 13 other than the information output function 13e.

[0080] As described above, in this embodiment, multiple pieces of motion information are classified based on the corresponding surgical evaluation information, and a learning model is constructed based on the classified multiple pieces of motion information. This allows for the construction of a learning model (a standard model for motion planning) that takes into account the results of surgery performed by a surgical robot, thereby improving the quality of the learning model.

[0081] In this embodiment, a learning model is constructed based on the motion information classified using the prognosis record, which allows the construction of a learning model that reflects the prognosis result indicating whether the surgery contributed to the recovery of the patient's symptoms.

[0082] In this embodiment, a motion plan is generated by correcting the learning model based on correction information such as the patient's body type, tissue and organ shapes, and positions, etc. This makes it possible to generate a motion plan optimized for each individual patient.

[0083] Furthermore, in this embodiment, a motion plan to be used in surgery on a target patient is generated based on the learning model, and information for controlling the surgical robot 40 is output based on the generated motion plan. This generates a motion plan individual to the target patient, allowing surgery on the target patient to be performed by the surgical robot 40.

[0084] Furthermore, in this embodiment, the learning model is corrected during surgery based on real-time information or user intervention information, which allows for appropriate responses even when unexpected problems occur during surgery.

[0085] The term "processor" used in the above description refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). The processor realizes its function by reading and executing a program stored in the memory circuit 11. Instead of storing the program in the memory circuit 11, the processor may be configured to directly incorporate the program into its circuit. In this case, the processor realizes its function by reading and executing the program embedded in the circuit. The processor is not limited to being configured as a single circuit, but may also be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, the multiple components in FIG. 2 may be integrated into a single processor to realize its function.

[0086] 12 and 13 can be realized by executing a prepared medical information processing program on a computer such as a personal computer or a workstation. This medical information processing program can be distributed via a network such as the Internet. This medical information processing program can also be recorded on a non-transitory computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, or DVD, and executed by being read from the recording medium by a computer.

[0087] Although several embodiments have been described above, these embodiments are presented only as examples and are not intended to limit the scope of the invention. The novel apparatus and method described herein may be embodied in various other forms. Furthermore, various omissions, substitutions, and modifications may be made to the forms of the apparatus and method described herein without departing from the spirit of the invention. The appended claims and their equivalents are intended to cover such forms and modifications that fall within the scope and spirit of the invention. [Explanation of symbols]

[0088] 1 Medical Systems 10 Medical information processing device 11 Memory circuit 12 Communication Interface 13 Processing circuit 13a Information Acquisition Function 13b Operation information classification function 13c Learning model building function 13d Motion plan generation function 13e Information output function 20 Medical information storage device 21 Memory circuit 21a Operation Information Database 21b Surgery Evaluation Information Database 21c patient information database 22 Communication Interface 23 Processing circuit 23a Information reading function 30 Terminal Equipment 31 Memory circuit 32 Communication Interface 33 Output Interface 34 Input Interface 35 Processing circuit 35a Information reception function 35b Display control function 40 Surgical Robot A (Current behavior) BI1, BI2, BI3 Intra-body images D1, D2 surgical devices IM reference video Inf1, Inf2 information M1,M2 support information Pt1, Pt2, Pt3 point information R (treatment area) SV surgery video TR trajectory

Claims

1. a motion information acquisition unit that acquires a plurality of motion information including motion records of the surgical robot; a surgical evaluation information acquisition unit that acquires a plurality of pieces of surgical evaluation information corresponding to the plurality of pieces of motion information, the evaluation information including an evaluation of the surgery performed by the surgical robot; a motion information classifying unit that classifies each of the plurality of pieces of motion information based on the plurality of pieces of surgical evaluation information; a learning model construction unit that constructs a learning model based on the classified plurality of pieces of motion information; A medical information processing device comprising:

2. The medical information processing device according to claim 1 , wherein the surgical evaluation information is a record of the surgery performed by the surgical robot and / or a record of follow-up observation after the surgery.

3. The medical information processing device of claim 1, wherein the learning model construction unit constructs the learning model using a record of surgery performed by the surgical robot as the surgical evaluation information, and then reconstructs the learning model using a record of follow-up observation after surgery performed by the surgical robot as the surgical evaluation information.

4. The medical information processing device according to claim 1 , wherein the operation information has one or more pieces of point information including at least one of procedure information, a type of surgical device, and a position, angle, strength, and route of the surgical device.

5. The medical information processing device according to claim 4 , wherein the operation information further includes peripheral information relating to the periphery of the treatment area.

6. The medical information processing apparatus according to claim 1 , wherein the motion information classifying unit classifies each of the plurality of pieces of motion information into good or bad.

7. 7. The medical information processing device according to claim 1, further comprising a motion plan generation unit that generates a motion plan for the surgical robot to perform surgery on a target patient based on the learning model.

8. The medical information processing apparatus according to claim 7 , wherein the motion plan generating unit generates a motion plan by correcting the learning model based on correction information of the target patient.

9. The medical information processing apparatus according to claim 7 , wherein the operation plan generating unit corrects the generated operation plan based on real-time information acquired during surgery on the target patient or user intervention information.

10. The medical information processing device according to claim 7 , further comprising an information output unit that outputs information for controlling the surgical robot to the surgical robot based on the generated operation plan.

11. The medical information processing apparatus according to claim 10 , wherein the information output unit outputs information for displaying support information for supporting a user's intervention in the surgery on a terminal device of the user.

12. Acquire multiple pieces of motion information including motion records of the surgical robot, acquiring a plurality of pieces of surgical evaluation information corresponding to the plurality of pieces of motion information, the evaluation information including an evaluation of the surgery performed by the surgical robot; classifying the plurality of pieces of motion information based on the plurality of pieces of surgical evaluation information; A medical information processing method that constructs a learning model based on the classified plurality of pieces of motion information.

13. On the computer, Acquire multiple pieces of motion information including motion records of the surgical robot, acquiring a plurality of pieces of surgical evaluation information corresponding to the plurality of pieces of motion information, the evaluation information including an evaluation of the surgery performed by the surgical robot; classifying the plurality of pieces of motion information based on the plurality of pieces of surgical evaluation information; A medical information processing program that causes the computer to construct a learning model based on the classified plurality of pieces of motion information.

14. A medical information processing device comprising an operation plan generation unit that generates an operation plan for a surgical robot based on a learning model constructed using multiple operation information of the surgical robot classified by corresponding surgical evaluation information.

15. A medical information processing device comprising: a motion plan generation unit that generates a motion plan for a surgical robot, the motion plan generation unit correcting a learning model based on correction information for the target patient to generate a motion plan for the target patient.

Images