Medical information processing equipment

Abstract

To support a medical worker to make preparations for a procedure for the next patient as planned.SOLUTION: A medical information processing apparatus according to an embodiment comprises a data acquisition unit and a time calculation unit. The data acquisition unit sequentially acquires pieces of data related to the progress of a procedure during the execution of the procedure. The time calculation unit calculates the time required for the procedure based on the plurality of pieces of data related to the progress of the procedure acquired by the data acquisition unit at different timings during the execution of the procedure.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention is a medical information processing device. Place Regarding. [Background technology]

[0002] Conventionally, techniques such as interventional therapy are known in which procedures are performed while imaging the subject with an X-ray diagnostic device. The degree of difficulty of such procedures varies greatly depending on the location and shape of the lesion, so it can be difficult for doctors to predict the time required for the procedure. This makes it difficult to schedule procedures, and medical professionals have had difficulty in planning and preparing procedures for the next patient. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2019 / 0117087 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem that the present invention aims to solve is to assist medical personnel in planning preparations for a procedure on an upcoming patient. [Means for solving the problem]

[0005] The medical image processing apparatus according to the embodiment includes a data acquisition unit, a time calculation unit, Judgment section and The data acquisition unit sequentially acquires data related to the progress of the procedure, including medical images, in chronological order as the procedure progresses during the procedure. The time calculation unit calculates the time required for the procedure each time a medical image is acquired, based on a plurality of pieces of data related to the progress of the procedure acquired by the data acquisition unit at different times during the procedure. The determination unit determines the likelihood of success or failure of the procedure being performed based on a plurality of data related to the progress of the procedure acquired by the data acquisition unit at different times during the procedure. The data related to the progress of the procedure includes at least one of information representing the proficiency of the surgeon performing the procedure and information representing the severity of the subject of the procedure. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a medical information processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of input data and output data of a trained model according to an embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of image data and non-image data according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of input data and output data that change with the progress of a procedure according to the embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of a display of the time required for a procedure and the result of determination of the success or failure of the procedure according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of learning image data according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of non-image data for learning according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a method for generating a trained model according to an embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of a process flow for calculating the time required for a procedure and determining whether the procedure has been successful, according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, the medical information processing device will be described with reference to the drawings. Place The embodiment will now be described in detail.

[0008] (Embodiment) 1 is a block diagram showing an example of the configuration of a medical information processing system S according to this embodiment. As shown in FIG.

[0009] The X-ray diagnostic apparatus 10 includes an X-ray high voltage device 11, an X-ray tube 12, an X-ray aperture 13, a tabletop 14, a C-arm 15, an X-ray detector 16, a C-arm rotation / movement mechanism 17, a tabletop movement mechanism 18, a C-arm / tabletop mechanism control circuit 19, an aperture control circuit 20, a processing circuit 21, an input interface 22, displays 23a and 23b, and a memory circuit 24. The X-ray diagnostic apparatus 10 is, for example, an X-ray angiography apparatus. In this embodiment, the X-ray diagnostic apparatus 10 is an example of a medical information processing apparatus.

[0010] Of the components of the X-ray diagnostic apparatus 10, the X-ray high voltage device 11, the X-ray tube 12, the X-ray aperture 13, the tabletop 14, the C-arm 15, the X-ray detector 16, the C-arm rotation / movement mechanism 17, the tabletop movement mechanism 18, the C-arm / tabletop mechanism control circuit 19, the aperture control circuit 20, and the display 23b are installed in the examination room R1.

[0011] Moreover, the processing circuit 21, the input interface 22, the display 23a, and the memory circuit 24 are installed, for example, in the operation room R2.

[0012] The examination room R1 is a room where a procedure is performed on a subject (patient) P. For example, in this embodiment, as an example of a procedure, an operator such as a doctor performs catheter treatment on the subject P1, so the examination room R1 is also called a catheter room. In the examination room R1, an operator such as a doctor and multiple other medical staff members work. The examination room R1 is an example of a space where a procedure is performed on the subject P1 in this embodiment. Medical staff members include not only doctors but also nurses, engineers, etc. In this embodiment, the operator and other medical staff members are collectively referred to as medical professionals.

[0013] The operation room R2 is, for example, a separate room adjacent to the examination room R1. In the operation room R2, an operator such as a technician or a supervisor operates the input interface 22 or refers to the display 23a. The supervisor is, for example, a senior doctor. Note that a configuration may be adopted in which all components of the X-ray diagnostic apparatus 10 are installed in the examination room R1.

[0014] As shown in FIG. 1, the terminal device 30 is provided outside the examination room R1 and the operation room R2. For example, the terminal device 30 may be provided in a waiting room for the surgeon or medical staff, or may be carried by each medical worker. The terminal device 30 is, for example, a personal computer (PC) or a tablet terminal, but is not limited thereto. The number of terminal devices 30 included in the medical information processing system S is not particularly limited, and may be one or more.

[0015] Further, in FIG. 1, the terminal device 30 is illustrated as being connected to the X-ray diagnostic apparatus 10 via a wired network, but the terminal device 30 may be connected to the X-ray diagnostic apparatus 10 via a wireless network.

[0016] Further, the X-ray high voltage device 11 is a high voltage power supply that generates a high voltage under the control of the processing circuit 21 and supplies the generated high voltage to the X-ray tube 12.

[0017] The X-ray tube 12 generates X-rays using a high voltage supplied from the X-ray high voltage device 11. Under the control of the aperture control circuit 20, the X-ray aperture 13 narrows down the X-rays generated by the X-ray tube 12 so that they are selectively irradiated onto a region of interest (ROI) of the subject P1.

[0018] The top board 14 is a bed on which the subject P1 rests, and is placed on a bed device (not shown). The bed device (not shown) may be included in the X-ray diagnostic apparatus 10, or may be external to the X-ray diagnostic apparatus 10. Even if the bed device is not included in the X-ray diagnostic apparatus 10, it may be included in the medical information processing system S. The subject P1 is not included in the X-ray diagnostic apparatus 10.

[0019] The X-ray detector 16 detects the X-rays that have passed through the subject P1, and transmits the detection result to the processing circuitry 21.

[0020] The C-arm 15 holds the X-ray tube 12, the X-ray aperture 13, and the X-ray detector 16. The C-arm rotation / movement mechanism 17 is a mechanism for rotating and moving the C-arm 15 by driving a motor or the like provided on the support. The top board movement mechanism 18 is a mechanism for moving the top board 14. For example, the top board movement mechanism 18 moves the top board 14 using power generated by an actuator.

[0021] Under the control of the processing circuitry 21, the C-arm / top mechanism control circuit 19 controls the C-arm rotation / movement mechanism 17 and the top movement mechanism 18 to adjust the rotation and movement of the C-arm 15 and the movement of the top 14. Under the control of the processing circuitry 21, the aperture control circuit 20 adjusts the opening of the aperture blades of the X-ray aperture 13 to control the irradiation range of the X-rays irradiated to the subject P1.

[0022] The input interface 22 is realized by a trackball, a switch button, a mouse, a keyboard, a touch pad, a foot switch for irradiating X-rays, etc. The input interface 22 is connected to the processing circuitry 21, and converts an input operation received from an operator into an electrical signal and outputs the electrical signal to the processing circuitry 21.

[0023] The displays 23a and 23b display a GUI (Graphical User Interface) for receiving instructions from an operator and various images generated by the processing circuitry 21. In this embodiment, the displays 23a and 23b display the time required for a procedure output from a trained model described later.

[0024] More specifically, the display 23a is hung from the ceiling of the examination room R1, for example. The display 23a is not limited to having one screen, and may have multiple screens. The display 23a is also called an in-examination room display.

[0025] Moreover, the display 23b is installed in the operation room R2. The display 23b is also referred to as an operation room display. The number of displays 23b is not particularly limited, and may be one or more. Hereinafter, when there is no particular distinction between the displays 23a and 23b, they will simply be referred to as the display 23.

[0026] In addition, in the X-ray diagnostic apparatus 10, each processing function is stored in the memory circuitry 24 in the form of a program executable by a computer. The C-arm / top mechanism control circuitry 19, the aperture control circuitry 20, and the processing circuitry 21 are processors that realize the function corresponding to each program by reading and executing the program from the memory circuitry 24. In other words, each circuit in the state where each program has been read has the function corresponding to the read program.

[0027] The memory circuitry 24 stores programs corresponding to various functions read and executed by each circuit shown in Fig. 1. The memory circuitry 24 also stores logs of various processes executed by the processing circuitry 21. The memory circuitry 24 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like. The memory circuitry 24 is an example of a memory unit.

[0028] The processing circuit 21 has a data acquisition function 211, a trained model operation function 212, an output function 213, a reception function 214, an imaging function 215, a training data generation function 216, and a training function 217. The data acquisition function 211 is an example of a data acquisition unit. The trained model operation function 212 is an example of a trained model operation unit. The trained model operation function 212 also has a time calculation function and a judgment function. The time calculation function is an example of a time calculation unit. The judgment function is an example of a judgment unit. The output function 213 is an example of an output unit. The reception function 214 is an example of a reception unit. The imaging function 215 is an example of an imaging unit. The training data generation function 216 is an example of a training data generation unit. The learning function 217 is an example of a learning unit.

[0029] Moreover, the processing executed by the data acquisition function 211 is referred to as a data acquisition step. The processing executed by the learned model operation function 212 is referred to as a learned model operation step, a time calculation step, or a judgment step. The processing executed by the output function 213 is referred to as an output step. The processing executed by the reception function 214 is referred to as a reception step. The processing executed by the imaging function 215 is referred to as an imaging step. The processing executed by the learning data generation function 216 is referred to as a learning data generation step. The processing executed by the learning function 217 is referred to as a learning step.

[0030] In Figure 1, it has been described that the processing functions of the data acquisition function 211, the learned model operation function 212, the output function 213, the reception function 214, the imaging function 215, the learned data generation function 216, and the learning function 217 are realized by a single processing circuit 21. However, the processing circuit 21 may be configured by combining multiple independent processors, and each processor may realize each processing function by executing each program.

[0031] The term "processor" used in the above description refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). Instead of storing a program in the memory circuit 24, the processor may be configured to directly incorporate the program into its circuit. In this case, the processor realizes its function by reading and executing the program incorporated in the circuit.

[0032] The data acquisition function 211 sequentially acquires data related to the progress of a procedure while the procedure is being performed.

[0033] Data related to the progress of a procedure includes image data and non-image data.

[0034] The image data in this embodiment is a medical image of a subject P1 to be subjected to a procedure. More specifically, the image data in this embodiment is an X-ray image captured by the X-ray diagnostic apparatus 10.

[0035] In addition, the non-image data in this embodiment includes at least one of procedure information related to the procedure, subject information related to the subject P1, operator information related to the operator performing the procedure, measurement data measured during the procedure, or equipment information related to the equipment used in the procedure. The non-image data in this embodiment is an example of non-image information.

[0036] The procedure information is, for example, the type of procedure. The type of procedure is, specifically, the type of protocol or program executed by the X-ray diagnostic apparatus 10 when performing the procedure. For example, a protocol is a processing procedure determined for each treatment target part. Also, a program is a process executed by each protocol, and an example of the program is a program that executes a scan step such as DSA (Digital Subtraction Angiography).

[0037] The procedure information may also be information indicating the difficulty level of the procedure. The procedure information may also be information indicating the type of approach. The approach of the procedure may be, for example, an antegrade approach or a retrograde approach.

[0038] The subject information on the subject P1 is, for example, information on the severity of the condition of the subject P1, the age of the subject P1, the sex of the subject P1, etc. The subject information may also be information representing the condition of the affected area of ​​the subject P1. The information representing the condition of the affected area of ​​the subject P1 is, for example, the formation status of collateral flow, etc.

[0039] The surgeon information on the surgeon performing the procedure is, for example, information that indicates the proficiency of the surgeon. For example, the surgeon information may be a category according to the proficiency, such as "young surgeon," "veteran," or "supervising surgeon." The surgeon information may also be a numerical representation of the surgeon's proficiency. The surgeon information is not limited to this, and may include various types of information other than proficiency.

[0040] Moreover, the measurement data measured during the procedure is a measurement result related to the state of the treatment target part of the subject P1 during the procedure. For example, when the procedure is Percutaneous Coronary Intervention (PCI) for CTO (Chronic Total Occlusion), the measurement data is a measurement result at the occluded part of the coronary artery. One example of the measurement data is a measurement result of fractional flow reserve (FFR), etc.

[0041] The equipment information on the equipment used in the procedure is, for example, information on the X-ray diagnostic apparatus 10 that images the subject P1 during the procedure, or information on a device used in the procedure.

[0042] For example, the device information includes position information of a movable mechanism included in the X-ray diagnostic apparatus 10. The movable mechanism is, for example, the C-arm 15 or the tabletop 14. For example, the data acquisition function 211 acquires the position information of the C-arm 15 or the tabletop 14 from the C-arm / tabletop mechanism control circuit 19. For example, the position of the C-arm 15 during a procedure is called a working angle.

[0043] The device information may also be the operating state of an application used in a procedure. For example, when an operator places a stent in a blood vessel during a procedure, an application for generating a stent-enhanced image of the X-ray diagnostic apparatus 10 may be executed. The device information may also be information indicating whether or not an application executed by the X-ray diagnostic apparatus 10 during such a procedure is being executed.

[0044] Furthermore, for example, the equipment information may be information about a device used in a procedure. The information about a device used in a procedure may be, for example, the number of catheters used in the procedure, or the type of catheter. Note that the device is not limited to a catheter, and may be a balloon, a stent, or the like. For example, the equipment information may be information about the type of stent.

[0045] The device information may also be information that indicates the operating state of a device during a procedure, for example, the inflation time of a balloon during a procedure.

[0046] The non-image data is not limited to the above example, and may further include other information. The data acquisition function 211 sends the acquired image data and non-image data to the trained model operation function 212.

[0047] The learned model operation function 212 has a time calculation function that calculates the time required for a procedure and a judgment function that judges whether the procedure is successful or not.

[0048] For example, the learned model operation function 212 calculates the time required for a procedure based on multiple image data and multiple non-image data acquired by the data acquisition function 211 at different times during the procedure.

[0049] In this embodiment, the time required for a procedure is the remaining time until the procedure is completed. The procedure may end when the procedure is successful or when the procedure is stopped due to failure. The trained model operation function 212 may calculate the time required for a procedure from the start to the end.

[0050] In addition, the learned model operation function 212 determines the success or failure of the procedure based on multiple image data and multiple non-image data acquired by the data acquisition function 211 at different times during the procedure.

[0051] The success or failure of a procedure is determined to be either "success" or "failure." In this embodiment, a procedure is successful when the purpose of the procedure is achieved. The purpose of the procedure is, for example, recovery of blood flow in the coronary artery, but varies depending on the procedure. A procedure is unsuccessful when, for example, the operator gives up on continuing the procedure and the procedure is discontinued.

[0052] The success or failure of a procedure determined by the learned model operation function 212 is an estimated result of the outcome of the procedure currently being performed. In other words, when the learned model operation function 212 determines "success", it means that "the currently being performed procedure is highly likely to be successful". On the other hand, when the learned model operation function 212 determines "unsuccess", it means that "the currently being performed procedure is highly likely to end in failure".

[0053] More specifically, the learned model operation function 212 reads out the learned model from the memory circuitry 24 and inputs the image data and non-image data acquired by the data acquisition function 211 into the learned model, thereby causing the learned model to output the time required for the procedure and the judgment result of the success or failure of the procedure.

[0054] The time required for the procedure and the judgment result of whether the procedure was successful or not are output data of the trained model of this embodiment.

[0055] 2 is a diagram showing an example of input data and output data 80 of the trained model 90 according to this embodiment. As shown in Fig. 2, when image data 71 and non-image data 72 are input, the trained model 90 outputs, as output data 80, the time required for the procedure corresponding to the input data and the result of the success or failure of the procedure.

[0056] The trained model 90 is a trained model generated by deep learning such as a neural network. The trained model 90 is configured by a neural network and trained parameter data. As a deep learning method, for example, a recurrent neural network (RNN), a long short-term memory (LSTM), etc. can be applied. The trained model 90 may also be generated by other deep learning or machine learning. In this embodiment, the trained model 90 is generated by a learning function 217 described later and stored in the memory circuitry 24.

[0057] Next, details of the image data 71 and non-image data 72 input to the trained model 90 in this embodiment will be described.

[0058] Fig. 3 is a diagram showing an example of image data 71 and non-image data 72 according to this embodiment. The multiple image data 71a-71g and multiple non-image data 72a-72l shown in Fig. 3 are image data 71 and non-image data 72 acquired by the data acquisition function 211 at different times during the procedure.

[0059] Specifically, the multiple image data 71a to 71g shown in Fig. 3 are multiple X-ray images captured from the start to the end of a CTO procedure. In Fig. 3, the timing at which each image data 71 was acquired is displayed in association with the elapsed time from the start of the procedure.

[0060] From the captured X-ray images, information such as the progress of the procedure, the speed of the procedure, and the possibility of success can be obtained. For example, image data 71a is an X-ray image of the blood vessel at the treatment target site captured by the X-ray diagnostic device 10 with contrast before the catheter is introduced into the blood vessel of the subject P1. Also, image data 71b to 71d are X-ray images sequentially captured showing the progress of the catheter introduced into the blood vessel of the subject P1 by the operator through the blood vessel. In the example shown in FIG. 3, it takes 60 minutes to reach the state of image data 71d, but the speed of the procedure varies depending on the operator's skill, the severity of the subject P1, and the like.

[0061] Moreover, image data 71e is an X-ray image capturing an image of a balloon expanding in the blood vessel of subject P1. Moreover, image data 71f is an X-ray image capturing an image of a stent placed in the blood vessel of subject P1. If the stent is depicted on the X-ray image as in image data 71f, it becomes clear that the procedure has progressed to the placement of the stent.

[0062] Moreover, image data 71g is an X-ray image obtained by again capturing contrast images of the blood vessels at the treatment target site of subject P1 at the end of the catheter treatment. The blood vessels that were blocked midway in the image data 71a are open in the image data 71g, indicating that the procedure was successful.

[0063] Hereinafter, when the image data 71a to 71g are not differentiated from one another, they will simply be referred to as image data 71. When the non-image data 72a to 72l are not differentiated from one another, they will simply be referred to as non-image data 72.

[0064] All X-ray images captured during a procedure may be input to the trained model 90 as image data 71, or the latest X-ray image may be input to the trained model 90 as image data 71 at regular intervals. The regular interval is not particularly limited and may be, for example, 5 minutes or 10 minutes. Alternatively, every time the X-ray diagnostic apparatus 10 captures an X-ray image with an X-ray dose higher than fluoroscopy, the trained model operation function 212 may input the X-ray image to the trained model 90.

[0065] Moreover, the plurality of non-image data 72a to 72l shown in FIG. 3 are non-image data 72 sequentially acquired by the data acquisition function 211 as the procedure progresses.

[0066] The trained model 90 estimates the time required for the procedure and the success or failure of the procedure based on, for example, image data 71 and various non-image data 72 such as the surgeon's level of proficiency represented in the surgeon information and the severity of the subject P1's condition.

[0067] In addition, the source from which the data acquisition function 211 acquires the non-image data 72 is not particularly limited, but for example, the data acquisition function 211 may acquire subject information from an external electronic medical record system, etc. In addition, the data acquisition function 211 may acquire protocols, programs, operating conditions of applications used, etc. used in the procedure from logs stored in the memory circuitry 24. In addition, the data acquisition function 211 may acquire the type of catheter, the type of approach, etc. input by the operator from the input interface 22.

[0068] 4 is a diagram showing an example of input data and output data that change with the progress of the procedure according to this embodiment. As shown in FIG. 4, the number of image data 71 and non-image data 72 input to the trained model 90 increases with the passage of time.

[0069] 4 shows an example of image data 71, non-image data 72, and output data 80 10 minutes after the start of the procedure. At this point, the trained model 90 outputs that the procedure will be successful and that the time required for the procedure, i.e., the time until the procedure is completed, is 90 minutes.

[0070] 4 shows an example of image data 71, non-image data 72, and output data 80 60 minutes after the start of the procedure. At this point, the trained model 90 outputs that the procedure will be successful and that the procedure will take 40 minutes.

[0071] 4 shows an example of image data 71, non-image data 72, and output data 80 at 100 minutes after the start of the procedure. At this point, the trained model 90 outputs that the procedure is successful and that the time required for the procedure is 0 minutes, i.e., the procedure is completed.

[0072] The trained model operation function 212 sends the output data 80 output from the trained model 90, that is, the time required for the procedure and the judgment result of the success or failure of the procedure, to the output function 213.

[0073] 1, the output function 213 outputs the time required for the procedure calculated by the trained model operation function 212. The output function 213 also outputs the success or failure of the procedure determined by the trained model operation function 212.

[0074] More specifically, the output function 213 displays the contents of the output data 80 on the displays 23a and 23b. Fig. 5 is a diagram showing an example of a display of the time required for a procedure and the determination result of the success or failure of the procedure according to this embodiment. For example, as shown in Fig. 5, the output function 213 displays the time required for the procedure as "30 minutes remaining" on the display 23a installed in the examination room R1.

[0075] For example, the output function 213 displays the time required for the procedure and the result of the judgment of the success or failure of the procedure on the display 23b installed in the operation room R2 as "success, 30 minutes remaining." The supervisor P3 or the like in the operation room R2 refers to the time required for the procedure and the result of the judgment of the success or failure of the procedure displayed on the display 23b. For example, the supervisor P3 or the like may instruct the surgeon P2 in the examination room R1 to change the approach of the procedure or the device, or may instruct a change to another surgeon, as necessary. Note that the output function 213 may display a message on the display 23b suggesting a change of approach or surgeon when the result of the judgment of the success or failure of the procedure is "unsuccessful."

[0076] 5, the output function 213 does not display the result of the judgment on the success or failure of the procedure on the display 23a. This is because there are cases where it is better not to disclose the result of the judgment on the success or failure of the procedure in the examination room R1 where the operator P2 and the subject P1 are present.

[0077] 5 is an example, and the output function 213 may change the display content according to the result of the judgment of the success or failure of the procedure, such that when the judgment result of the success or failure of the procedure is "success", the judgment result of the success or failure of the procedure is also displayed on the display 23a, and when the judgment result of the success or failure of the procedure is "unsuccessful", the judgment result of the success or failure of the procedure is not displayed on the display 23a. Also, the output function 213 may display the same content on the displays 23a and 23b.

[0078] The output function 213 also displays the contents of the output data 80 on the display 301 of the terminal device 30 provided outside the examination room R1 and the operation room R2. For example, as shown in FIG. 5, the output function 213 displays a message including the time required for the procedure, such as "The procedure will be completed in 30 minutes. Please start preparation." In the example shown in FIG. 5, the output function 213 displays a message urging the medical staff in charge of the procedure of the next patient to start preparation, such as "Please start preparation." However, the message may be changed depending on the length of time required for the procedure. For example, the output function 213 may display a message instructing the medical staff to wait if the time required for the procedure is longer than 30 minutes, and display a message urging the medical staff to start preparation if the time required for the procedure is 30 minutes or less.

[0079] In addition, the output function 213 may control a speaker (not shown) installed in the examination room R1, the operation room R2, or a waiting room for the surgeon or medical staff, etc., to output audio such as the time required for the procedure, the result of the judgment on the success or failure of the procedure, a message instructing the patient to wait, or a message encouraging the start of preparations.

[0080] 1, the reception function 214 receives an operation input by the operator via the input interface 22. For example, the reception function 214 receives an operation to start or end the imaging process of the X-ray diagnostic apparatus 10 input by the operator. When the reception function 214 receives an operation to start or end the imaging process, it sends to the imaging function 215 a notification that the operation to start or end the imaging process has been received.

[0081] The imaging function 215 controls the entire X-ray diagnostic apparatus 10 based on the operation of starting or ending the imaging process received by the reception function 214, and executes the imaging process of imaging the subject P1. For example, the imaging function 215 controls the rotation and movement of the C-arm 15 and the movement of the top board 14 by controlling the C-arm / top board mechanism control circuit 19. The imaging function 215 also controls the X-ray high voltage device 11 and the aperture control circuit 20 to control the irradiation of X-rays to the subject P1. The imaging function 215 also collects X-ray images based on electrical signals converted from X-rays by the X-ray detector 16. The imaging function 215 sends the collected X-ray images to the data acquisition function 211.

[0082] The learning data generation function 216 generates multiple pieces of learning data by extracting a set of time-series data related to the progress of a procedure at different time lengths. The learning data is data used by a learning function 217 (described later) to generate a trained model 90. The learning data includes learning image data and learning non-image data.

[0083] The generation of learning data by the learning data generation function 216 will be described with reference to FIGS.

[0084] Fig. 6 is a diagram showing an example of learning image data according to this embodiment. For example, the image data group 701 shown in Fig. 6 is a set of time-series data in which multiple X-ray images are arranged in chronological order in the order in which they were captured. The image data group 701 is image data for one case in which the time from the start to the end of a procedure (hereinafter referred to as procedure time) was 100 minutes and the result of the procedure was "successful." Each of the multiple X-ray images included in the image data group 701 is associated with the elapsed time from the start of the procedure.

[0085] The learning data generation function 216 divides the image data group 701 into a combination of a plurality of learning image data 1071. For example, the learning data generation function 216 divides the image data group 701 at regular intervals of time.

[0086] 6, the learning data generation function 216 sets the X-ray images from the start of the procedure up to 100 minutes, i.e., all X-ray images included in the image data group 701, as learning image data 1071a. The learning data generation function 216 sets the X-ray images from the start of the procedure up to 60 minutes as learning image data 1071b. The learning data generation function 216 sets the X-ray images from the start of the procedure up to 10 minutes as learning image data 1071c. Hereinafter, when there is no particular distinction between the learning image data 1071a to 1071c, they will simply be referred to as learning image data 1071.

[0087] The elapsed time that is the basis for dividing the image data group 701 by the learning data generation function 216 is not particularly limited, and may be a fixed-length time unit such as 5 minutes or 10 minutes, or a variable-length time unit.

[0088] In addition, the learning data generation function 216 generates teacher data 1080a to 1080c (hereinafter, simply referred to as teacher data 1080 when no particular distinction is made) corresponding to each of the divided learning image data 1071a to 1071c according to the elapsed time corresponding to the division points at which the image data group 701 is divided.

[0089] The teacher data 1080a-1080c are the time required for the procedure corresponding to each of the learning image data 1071a-1071c and the success or failure of the procedure. The success or failure of the procedure is the same as the result of the procedure in the image data group 701. The time required for the procedure corresponding to each of the learning image data 1071a-1071c is the procedure time in the image data group 701 minus the elapsed time for each of the teacher data 1080a-1080c.

[0090] In the example shown in Figure 6, the combination of training image data 1071a and teacher data 1080a is training dataset 1A, the combination of training image data 1071b and teacher data 1080b is training dataset 2A, and the combination of training image data 1071c and teacher data 1080c is training dataset 3A.

[0091] 7 is a diagram showing an example of learning non-image data according to the present embodiment. A non-image data group 702 shown in Fig. 7 is a set of time-series data in which non-image data generated in a case where the image data group 701 shown in Fig. 6 was captured is arranged in time series.

[0092] The learning data generation function 216 divides the non-image data group 702 according to the elapsed time corresponding to the division points into which the image data group 701 is divided. In the example shown in Fig. 7, the learning data generation function 216 generates learning non-image data 1072a-1072c corresponding to elapsed times of 100 minutes, 60 minutes, and 10 minutes from the start of the procedure, respectively, in the same manner as the image data group 701 shown in Fig. 6. Hereinafter, when there is no need to distinguish between the individual learning non-image data 1072a-1072c, they will simply be referred to as learning non-image data 1072.

[0093] Training data set 1B, which is a combination of training non-image data 1072a and training data 1080a shown in Fig. 7, corresponds to training data set 1A in Fig. 6. Training data set 2B, which is a combination of training non-image data 1072b and training data 1080b, corresponds to training data set 1B in Fig. 6. Training data set 2C, which is a combination of training non-image data 1072c and training data 1080c, corresponds to training data set 1C in Fig. 6.

[0094] Moreover, the teacher data 1080a to 1080b shown in Fig. 7 are the same as those in Fig. 6. That is, one learning image data 1071, one learning non-image data 1072, and one teacher data 1080 form one learning data set.

[0095] 6 and 7, the training data generation function 216 generates multiple training data sets from the image data group 701 and the non-image data group 702 in one case. In addition, the training data generation function 216 executes a division process as shown in FIGS. 6 and 7 on the multiple image data groups 701 and the multiple non-image data groups 702 in multiple cases, and generates multiple training data sets from each of the multiple cases.

[0096] 1, the learning function 217 learns the correspondence between the multiple pieces of learning data generated by the learning data generation function 216 and the time required for a procedure and the success or failure of the procedure, thereby generating a trained model 90. The generation process of the trained model 90 by the learning function 217 is also called the production process of the trained model 90.

[0097] The learning function 217 learns by associating a plurality of learning image data 1071, a plurality of learning non-image data 1072, and a plurality of teacher data 1080 contained in a plurality of learning data sets.

[0098] Fig. 8 is a diagram showing an example of a method for generating a trained model 90 according to this embodiment. As shown in Fig. 8, the learning function 217 executes deep learning such as RNN or LSTM based on a plurality of training image data 1071a to 1071n, a plurality of training non-image data 1072a to 1072n, and a plurality of teacher data 1080a to 1080n, to generate a trained model 90.

[0099] The learning function 217 stores the generated trained model 90 in the memory circuitry 24.

[0100] Next, a process flow for calculating the time required for a procedure executed in the X-ray diagnostic apparatus 10 of the medical information processing system S of this embodiment configured as above and for determining whether the procedure has been successful will be described.

[0101] Fig. 9 is a flowchart showing an example of a process flow for calculating the time required for a procedure and determining whether the procedure is successful or not according to the present embodiment. The process shown in Fig. 9 starts, for example, when the imaging process of the subject P1 by the imaging function 215 starts. In addition, it is assumed that the trained model 90 is generated by the learning function 217 and stored in the memory circuitry 24 before the start of this flowchart.

[0102] First, the data acquisition function 211 acquires non-image data 72 from an external device or the input interface 22, etc. (S1). The data acquisition function 211 also acquires image data 71, that is, an X-ray image of the subject P1 captured by the X-ray diagnostic device 10 (S2). The data acquisition function 211 sends the acquired image data and non-image data to the trained model operation function 212.

[0103] Next, the trained model operation function 212 reads out the trained model 90 from the memory circuitry 24, and inputs the X-ray image acquired by the data acquisition function 211 and the non-image data 72 to the trained model 90 (S3). The trained model 90 outputs the time required for the procedure corresponding to the input X-ray image and non-image data 72, and the judgment result of the success or failure of the procedure. The trained model operation function 212 sends the output result of the trained model 90 to the output function 213.

[0104] Next, the output function 213 outputs both or either of the time required for the procedure calculated by the trained model operation function 212 and the success or failure of the procedure determined by the trained model operation function 212 (S4). Specifically, the output function 213 displays the time required for the procedure on a display 23a installed in the examination room R1. The output function 213 also displays the time required for the procedure and the determination result of the success or failure of the procedure on a display 23b installed in the operation room R2. The output function 213 also displays the time required for the procedure on a display 301 of a terminal device 30 installed outside the examination room R1 and the operation room R2.

[0105] Next, the reception function 214 determines whether or not an operation to end the procedure input by the operator has been received (S5). The operation to end the procedure is, for example, an operation to end the imaging process. If the reception function 214 determines that an operation to end the procedure has not been received (S5 "No"), the process returns to S1.

[0106] Moreover, if the reception function 214 determines that an operation to end the procedure has been received (S12 "Yes"), the processing of this flowchart ends.

[0107] In the past, it was sometimes difficult to estimate the time required for procedures such as interventional therapy because the difficulty of the procedure varied greatly depending on the location and shape of the lesion. This made it difficult to schedule the procedure, and it was sometimes difficult for medical professionals to plan and prepare for the procedure for the next patient.

[0108] In contrast, the X-ray diagnostic apparatus 10 of this embodiment calculates the time required for a procedure based on data related to the progress of a plurality of procedures acquired at different times during the procedure. Therefore, the X-ray diagnostic apparatus 10 of this embodiment can support medical personnel in systematically preparing for the procedure for the next patient.

[0109] Furthermore, in the X-ray diagnostic apparatus 10 of this embodiment, the success or failure of a procedure is determined based on data related to the progress of a plurality of procedures acquired at different times during the procedure. Therefore, the X-ray diagnostic apparatus 10 of this embodiment can estimate the success or failure of a procedure while it is being performed.

[0110] For example, when the success or failure of a procedure is judged during the procedure, it becomes a criterion for the operator P2 or the supervisor P3 to decide whether or not to interrupt the procedure. In addition to the interruption of the procedure, the success or failure of the procedure can also be a criterion for deciding whether or not to replace the current operator P2 with another more skilled doctor, or to change the approach to the procedure. Therefore, the X-ray diagnostic apparatus 10 of this embodiment can support the operator P2 or the supervisor P3 in deciding whether or not to continue the procedure, whether or not to replace the operator P2, or whether or not to change the approach to the procedure.

[0111] In this embodiment, data used to calculate the time required for the procedure or to determine the success or failure of the procedure includes medical images of the subject P1 on which the procedure is performed. Therefore, the X-ray diagnostic apparatus 10 of this embodiment can estimate the time required for the procedure or the success or failure of the procedure with high accuracy based on the change in the state of the treatment target part of the subject P during the procedure.

[0112] More specifically, in this embodiment, the X-ray diagnostic device 10 calculates the time required for the procedure or determines whether the procedure is successful or not, based on the X-ray image of the subject P1. Since the X-ray image of the subject P1 depicts devices such as catheters and stents, or the state of the treatment target area of ​​the subject P1, the X-ray diagnostic device 10 can obtain information on the progress of the procedure, the speed of the procedure, the possibility of success, etc., from the X-ray image captured during the procedure, and can estimate the time required for the procedure or the success or failure of the procedure with high accuracy.

[0113] In addition, the X-ray diagnostic device 10 of this embodiment calculates the time required for the procedure or judges whether the procedure is successful or not based on the non-image data 72. More specifically, the X-ray diagnostic device 10 of this embodiment calculates the time required for the procedure or judges whether the procedure is successful or not based on the non-image data 72 including at least one of procedure information related to the procedure, subject information related to the subject P1, operator P2 information related to the operator P2 performing the procedure, measurement data measured during the procedure, or equipment information related to the equipment used in the procedure. The speed of the procedure and the possibility of success of the procedure vary depending on the skill level of the operator P2, the severity of the subject P1, etc. The X-ray diagnostic device 10 of this embodiment calculates the time required for the procedure or judges whether the procedure is successful or not based on such non-image data 72, thereby making it possible to estimate the time required for the procedure or the success or failure of the procedure with high accuracy.

[0114] Furthermore, the X-ray diagnostic apparatus 10 of this embodiment outputs the time required for the procedure, allowing the operator P2, the supervisor P3, or the medical staff in charge of the procedure for the next patient to know the time until the procedure is completed. Therefore, the X-ray diagnostic apparatus 10 of this embodiment can support the operator P2, the supervisor P3, or the medical staff in charge of the procedure for the next patient to proceed with the procedure as planned.

[0115] Furthermore, according to the X-ray diagnostic apparatus 10 of this embodiment, the success or failure of the procedure is output, which can assist the surgeon P2 or the supervisor P3, etc., in making a decision as to whether or not to continue the procedure, whether to replace the surgeon P2, or whether to change the approach of the procedure, etc.

[0116] Furthermore, the X-ray diagnostic apparatus 10 of this embodiment generates multiple pieces of learning data by extracting a set of time-series data related to the progress of a procedure at different time lengths, and learns the correspondence between the learning data and the time required for the procedure. Therefore, according to the X-ray diagnostic apparatus 10 of this embodiment, multiple pieces of learning data can be generated from one case, and the trained model 90 can be produced efficiently.

[0117] In addition, the trained model 90 in this embodiment includes a “self-learning model” that further updates the internal algorithm of the trained model 90 by obtaining user feedback on the time required for the procedure output by the X-ray diagnostic apparatus 10 and the success or failure of the procedure.

[0118] (Variation 1) In the above embodiment, the X-ray diagnostic apparatus 10 is an example of a medical information processing device, but the medical information processing device may be an information processing device other than the X-ray diagnostic apparatus 10. For example, the medical information processing device may be a PC (Personal Computer) or a server device that can communicate with the X-ray diagnostic apparatus 10 via an in-hospital network or the like.

[0119] The medical information processing device may be any of various modalities that capture still or moving images of the subject P1 during a procedure. For example, the medical information processing device may be an endoscope device. When this configuration is adopted, medical images such as in-vivo images of the subject P1 captured by the endoscope device are examples of the image data 71 and the learning image data 1071.

[0120] The medical information processing device may be a remote medical support system that performs a surgical treatment such as an operation by remote control. When this configuration is adopted, an image of a treatment target area of ​​the subject P1 captured in the remote medical support system is an example of the image data 71 and the learning image data 1071.

[0121] It should be noted that the image data 71 and the learning image data 1071 are not limited to images captured as still images, and may be frames included in a moving image.

[0122] (Variation 2) In the above embodiment, the learning data generation function 216 and the learning function 217 generate the learning data and the learned model 90, but the learned model 90 may be generated by another information processing device. When this configuration is adopted, the X-ray diagnostic apparatus 10 acquires the learned model 90 from the other information processing device and stores it in the memory circuitry 24.

[0123] (Variation 3) In the above-described embodiment, the trained model 90 is stored in the memory circuitry 24, but the trained model 90 may also be incorporated into the trained model operation function 212.

[0124] (Variation 4) In the above-described embodiment, the trained model 90 outputs both the time required for the procedure and the success or failure of the procedure, but it may also be configured to output only either the calculation of the time required for the procedure or the judgment of the success or failure of the procedure.

[0125] Furthermore, the X-ray diagnostic apparatus 10 may have a trained model for calculating the time required for a procedure and a trained model for determining whether the procedure is successful or not, as separate trained models. When this configuration is adopted, the trained model for calculating the time required for a procedure may be incorporated into the time calculation function, and the trained model for determining whether the procedure is successful or not may be incorporated into the determination function.

[0126] (Variation 5) In the above embodiment, the data related to the progress of the procedure includes both image data 71 and non-image data 72, but it may be only either image data 71 or non-image data 72.

[0127] (Variation 6) In the above embodiment, the X-ray diagnostic apparatus 10 outputs the time required for the procedure and the result of the determination of the success or failure of the procedure, but the information that is output is not limited to this.

[0128] For example, the trained model 90 of the X-ray diagnostic apparatus 10 may output the probability of the procedure being successful as a value from 0 to 100%. The output format is not limited to %, and the trained model 90 may output an evaluation value indicating the degree of success of the procedure.

[0129] The trained model 90 may also output the degree of achievement of the current procedure as a value between 0 and 100%. For example, in a procedure where three stents were planned to be inserted, when only one stent has been inserted, the trained model 90 outputs the degree of achievement as 30%. The trained model 90 may also output an estimated value of the degree of achievement at the end of the procedure as a value between 0 and 100%.

[0130] (Variation 7) In the above-described embodiment, the learning function 217 performs supervised learning using the teacher data 1080 generated by the learning data generation function 216 when generating the trained model 90, but the learning method is not limited to this, and for example, semi-supervised learning may be adopted.

[0131] (Variation 8) The trained model 90 may be constructed by an integrated circuit such as an ASIC or an FPGA. The X-ray diagnostic apparatus 10 may calculate the time required for a procedure or determine the success or failure of the procedure by using a mathematical model, a lookup table, a database, or the like, instead of the trained model 90 generated by a neural network.

[0132] At least one of the embodiments described above can assist medical personnel in planning preparations for a procedure for the next patient.

[0133] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0134] 21 Processing circuit 23,23a,23b Display 24 Memory circuit 30 Terminal Equipment 71,71a~71g Image data 72, 72a-72l Non-image data 80 Output Data 90 trained models 211 Data Acquisition Function 212 Trained model operation function 213 Output Function 214 Reception Function 215 Imaging Function 216 Learning data generation function 217 Learning Function 301 Display 701 Image data group 702 Non-image data group 1071, 1071a~1071n Learning image data 1072, 1072a~1072n Non-image data for learning 1080,1080a~1080n Teacher data P1 Subject P2 caster P3 supervisor R1 Examination Room R2 control room S Medical Information Processing System

Claims

1. a data acquisition unit that acquires data related to the progress of the procedure, including medical images, in a time series manner during the procedure; a time calculation unit that calculates a time required for the procedure each time the medical image is acquired based on a plurality of data related to the progress of the procedure acquired by the data acquisition unit at different timings during the execution of the procedure; a determination unit that determines the likelihood of success or failure of the procedure being performed based on data related to the progress of the multiple procedures acquired by the data acquisition unit at different times during the performance of the procedure; Equipped with The data related to the progress of the procedure includes at least one of information representing the proficiency of a surgeon performing the procedure and information representing the severity of the subject of the procedure. Medical information processing equipment.

2. The data related to the progress of the procedures includes medical images of a subject on which the procedures are performed. The medical information processing device according to claim 1 .

3. The medical image is an X-ray image. The medical information processing device according to claim 2 .

4. The data related to the progress of the procedure further includes at least one of procedure information regarding the procedure, measurement data measured during the procedure, or equipment information regarding equipment used in the procedure. The medical information processing device according to claim 1 .

5. An output unit that outputs the time required for the procedure calculated by the time calculation unit, The medical information processing device according to claim 1 .

6. An output unit that outputs the success or failure of the procedure determined by the determination unit, The medical information processing device according to claim 1 .

7. a data acquisition unit that acquires data including medical images related to the progress of a procedure in a time series manner during the procedure; and a determination unit that determines the possibility of success or failure of the procedure being performed each time the medical image is acquired based on data related to the progress of a plurality of procedures acquired by the data acquisition unit at different times during the procedure, The data related to the progress of the procedure includes at least one of information representing the proficiency of a surgeon performing the procedure and information representing the severity of the subject of the procedure. Medical information processing equipment.

Images