Program, information processing method, learning model generation method, learning model re-learning method, and information processing system

Abstract

To provide a program and the like which enable a catheter system to be easily used.　This program causes a computer to execute a process which acquires a tomographic image (485) generated by using a diagnostic imaging catheter inserted into a hollow organ, inputs the acquired tomographic image (485) to a training model (651), which, when receiving the tomographic image (485), outputs a point of view on the state of the hollow organ or a surrounding state of the hollow organ, and outputs the point of view output from the training model (651).

Description

[Technical field]

[0001] The present invention relates to a program, an information processing method, a method for generating a learning model, a method for relearning a learning model, and an information processing system. [Background technology]

[0002] 2. Description of the Related Art A catheter system is used in which a diagnostic imaging catheter is inserted into a hollow organ such as a blood vessel to capture a tomographic image (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 164071 Summary of the Invention [Problem to be solved by the invention]

[0004] Skill is required to interpret tomographic images taken using diagnostic imaging catheters, and therefore extensive training is required to use the catheter system.

[0005] In one aspect, an object of the present invention is to provide a program or the like that enables easy use of a catheter system. [Means for solving the problem]

[0006] The program acquires a tomographic image generated using an imaging diagnostic catheter inserted into a tubular organ, and when the tomographic image is input, Device information indicating the status of an indwelling device placed in a luminal organ The acquired tomographic image is input to a learning model that outputs Device information The computer is caused to execute a process of outputting the above. Effect of the Invention

[0007] In one aspect, a program or the like can be provided that makes it easy to use the catheter system. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram illustrating an overview of a catheter system. [Diagram 2] FIG. 1 is an explanatory diagram for explaining an overview of a catheter for diagnostic imaging. [Diagram 3] FIG. 1 is an explanatory diagram illustrating a configuration of a catheter system. [Figure 4] FIG. 2 is an explanatory diagram illustrating the configuration of a first learning model. [Diagram 5] FIG. 2 is an explanatory diagram illustrating the configuration of a second learning model. [Figure 6] 13 is an example of a screen displayed by the catheter system. [Figure 7] 13 is an example of a screen displayed by the catheter system. [Figure 8] 1 is an example of a report. [Figure 9] 11 is a flowchart illustrating a process flow of a program. [Figure 10] FIG. 11 is an explanatory diagram illustrating the configuration of a catheter system according to a second embodiment. [Figure 11] 13 is an example of a screen displayed by the catheter system of the third embodiment. [Figure 12] 13 is a flowchart illustrating the flow of processing of a program according to a third embodiment. [Figure 13] 13 is an example of a screen displayed by the catheter system of the fourth embodiment. [Figure 14] 13 is a flowchart illustrating the flow of processing of a program according to a fourth embodiment. [Figure 15] 13 is an example of a screen displayed by the catheter system of the first modified example of the fourth embodiment. [Figure 16] 13 is an example of a screen displayed by the catheter system of the second modification of the fourth embodiment. [Figure 17] FIG. 2 is an explanatory diagram illustrating the record layout of a training data DB. [Figure 18] 13 is a flowchart illustrating the flow of processing of a program according to a fifth embodiment. [Figure 19] FIG. 13 is an explanatory diagram for explaining the record layout of a correction DB. [Figure 20] 23 is an example of a screen displayed by the catheter system of the sixth embodiment. [Figure 21] FIG. 23 is a functional block diagram of a catheter system according to a seventh embodiment. [Figure 22] FIG. 13 is an explanatory diagram illustrating the configuration of a catheter system according to an eighth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] [Embodiment 1] FIG. 1 is an explanatory diagram for explaining an overview of a catheter system 10. The catheter system 10 includes a catheter 40 for diagnostic imaging, an MDU (Motor Driving Unit) 33, and an information processing device 20. The catheter 40 for diagnostic imaging is connected to the information processing device 20 via the MDU 33. A display device 31 and an input device 32 are connected to the information processing device 20. The input device 32 is, for example, a keyboard, a mouse, a trackball, or a microphone. The display device 31 and the input device 32 may be stacked together to form a touch panel. The input device 32 and the information processing device 20 may be configured as an integrated unit.

[0010] 2 is an explanatory diagram for explaining an overview of the diagnostic imaging catheter 40. The diagnostic imaging catheter 40 has a probe portion 41 and a connector portion 45 disposed at an end of the probe portion 41. The probe portion 41 is connected to the MDU 33 via the connector portion 45. In the following description, the side of the diagnostic imaging catheter 40 farther from the connector portion 45 is referred to as the tip side.

[0011] A shaft 43 is inserted inside the probe portion 41. A sensor 42 is connected to the tip side of the shaft 43. A ring-shaped tip marker 44 is fixed near the tip of the probe portion 41.

[0012] The function of the MDU 33 allows the sensor 42 and shaft 43 to advance and retreat while rotating inside the probe portion 41. By a pull-back operation in which the sensor 42 is rotated while being pulled toward the MDU 33 side at a constant speed, a plurality of transverse slice images 485 (see FIG. 4) approximately perpendicular to the probe portion 41 and centered on the probe portion 41 are continuously captured at a predetermined interval.

[0013] The sensor 42 is, for example, an ultrasonic transducer that transmits and receives ultrasonic waves, or a transmitting / receiving unit for OCT (Optical Coherence Tomography) that irradiates near-infrared light and receives reflected light. The tubular organ into which the diagnostic imaging catheter 40 is inserted and used is, for example, a blood vessel, a pancreatic duct, a bile duct, or a bronchus.

[0014] 2 shows an example of an imaging diagnostic catheter 40 for IVUS (Intravascular Ultrasound) used when capturing an ultrasonic tomographic image from inside a blood vessel. In the following description, the imaging diagnostic catheter 40 will be described as an IVUS catheter.

[0015] The diagnostic imaging catheter 40 is not limited to a mechanical scanning type that mechanically rotates and advances and retreats. The diagnostic imaging catheter 40 may be an electronic radial scanning type that uses a sensor 42 in which multiple ultrasonic transducers are arranged in a ring shape.

[0016] The diagnostic imaging catheter 40 may have a so-called linear scanning type sensor 42 in which a plurality of ultrasonic transducers are arranged in a row along the longitudinal direction. The diagnostic imaging catheter 40 may have a so-called two-dimensional array type sensor 42 in which a plurality of ultrasonic transducers are arranged in a matrix.

[0017] The diagnostic imaging catheter 40 can capture cross-sectional images including not only luminal walls such as blood vessel walls, but also reflectors present inside the luminal cavity of a luminal organ, such as red blood cells, and organs present outside the luminal organ, such as the pericardium and heart.

[0018] In the following description, a set of a plurality of transverse images 485 from which a longitudinal image can be generated is referred to as one set of transverse images 485. Similarly, obtaining a set of transverse images 485 from which a longitudinal image can be generated using the imaging diagnostic catheter 40 is referred to as one image acquisition.

[0019] One set of transverse images 485 is acquired, for example, by one pull-back operation by the MDU 33. One set of transverse images 485 may be acquired while a user manually pushes and pulls the catheter for diagnostic imaging 40. Here, the pushing and pulling operation of the catheter for diagnostic imaging 40 includes both the operation of pushing and pulling the probe portion 41 and the operation of pushing and pulling the sensor 42 inside the probe portion 41.

[0020] For example, the user pulls back or pushes in the sensor 42 at a substantially constant speed. The transverse cross-sectional images 485 acquired during the period from when the user issues an instruction to start acquisition by voice input or the like to when the user issues an instruction to end acquisition constitute one set of transverse cross-sectional images 485.

[0021] A sensor or the like may be provided to detect the amount that the user pushes or pulls the sensor 42. Images acquired while the user pulls back or pushes the sensor 42 through a predetermined range constitute a set of cross-sectional images 485.

[0022] When the position of the sensor 42 can be detected, the user may push and pull the sensor 42 at any speed and in any direction. The transverse cross-sectional images 485 rearranged in order along the longitudinal direction of the probe portion 41 constitute one set of transverse cross-sectional images 485. When the intervals between the transverse cross-sectional images 485 are not constant, position information along the longitudinal direction of the probe portion 41 is associated with each of the transverse cross-sectional images 485 and recorded. In the following description, a case where the intervals between the transverse cross-sectional images 485 are constant will be described as an example.

[0023] 3 is an explanatory diagram illustrating the configuration of the catheter system 10. As described above, the catheter system 10 has an information processing device 20, an MDU 33, and an image diagnostic catheter 40. The information processing device 20 has a control unit 21, a main storage device 22, an auxiliary storage device 23, a communication unit 24, a display unit 25, an input unit 26, a catheter control unit 271, and a bus.

[0024] The control unit 21 is an arithmetic and control device that executes the program of this embodiment. The control unit 21 uses one or more central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), multi-core CPUs, etc. The control unit 21 is connected to each hardware unit constituting the information processing device 20 via a bus.

[0025] The main memory device 22 is a storage device such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), a flash memory, etc. The main memory device 22 temporarily stores information required during processing performed by the control unit 21 and programs being executed by the control unit 21.

[0026] The auxiliary storage device 23 is a storage device such as an SRAM, a flash memory, a hard disk, or a magnetic tape. The auxiliary storage device 23 stores a program to be executed by the control unit 21, a learning model 65, and various data required for executing the program. The learning model 65 includes a first learning model 651 and a second learning model 652. The communication unit 24 is an interface that communicates between the information processing device 20 and a network.

[0027] Display unit 25 is an interface that connects display device 31 to the bus. Input unit 26 is an interface that connects input device 32 to the bus. Catheter control unit 271 controls MDU 33, controls sensor 42, and generates transverse tomographic images 485 and longitudinal tomographic images based on signals received from sensor 42. The function and configuration of catheter control unit 271 are similar to those of conventionally used ultrasound diagnostic devices, and therefore a description thereof will be omitted. Note that control unit 21 may realize the function of catheter control unit 271.

[0028] The information processing device 20 is connected to various imaging diagnostic devices 37, such as an X-ray angiography device, an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, or an ultrasound diagnostic device, via a HIS (Hospital Information System) or the like.

[0029] The information processing device 20 of this embodiment is a dedicated ultrasound diagnostic device, or a personal computer, tablet, smartphone, or the like having the functions of an ultrasound diagnostic device.

[0030] 4 is an explanatory diagram for explaining the configuration of the first learning model 651. The first learning model 651 is a model that receives one set of transverse slice images 485 and outputs findings related to the state of a hollow organ or the state around the hollow organ, such as the necessity of treatment, the presence or absence of blood flow stasis, or the presence or absence of branching. Note that "necessity of treatment" may be the necessity of IVR (Interventional Radiology) that performs treatment inside the hollow organ, or the necessity of general treatment including medication and dietary therapy.

[0031] The findings output by the first learning model 651 are probabilities for predetermined options such as "yes" and "no" for each of a plurality of items. Tables 1 to 5 show examples of items for which the first learning model 651 outputs probabilities. Each row in Tables 1 to 5 shows one item. The first learning model 651 outputs the probability of the option for each item. Tables 1 to 5 show examples of items included in the findings output by the first learning model 651.

[0032] Table 1 provides information regarding the need for treatment.

[0033] [Table 1]

[0034] Table 2 shows items related to blood flow information.

[0035] [Table 2]

[0036] Table 3 shows items related to qualitative shape information of hollow organs and their surroundings.

[0037] [Table 3]

[0038] Table 4 shows items related to property information indicating the properties of hollow organs and the surrounding area of ​​hollow organs.

[0039] [Table 4]

[0040] "In-stent stenosis" shown in Table 4 indicates the presence or absence of stenosis of a stent placed in a luminal organ several months to several years ago. When the transverse image 485 is taken immediately after the procedure of placing the stent, it indicates the presence or absence of stenosis of the placed stent. In other words, the transverse image 485 may be a tomographic image of an untreated luminal organ, a tomographic image of a luminal organ under follow-up observation after the procedure, or a tomographic image of a luminal organ taken immediately after the end of a series of procedures in the luminal organ.

[0041] Table 5 shows items related to device information that indicate the status of an indwelling device, such as a stent, placed in a luminal organ.

[0042] [Table 5]

[0043] Each item shown in Tables 1 to 5 is an example. The first learning model 651 may output the probability for some of the items shown in Tables 1 to 5. The first learning model 651 may output the probability for items other than the items shown in Tables 1 to 5.

[0044] The options for each item shown in Tables 1 to 5 are examples. For example, items shown in each table with two options, "Yes" and "No," may be shown with three or more options, such as "Large," "Small," and "No."

[0045] As described above, the first learning model 651 is a model that receives an input of a set of cross-sectional images 485 and outputs findings regarding the state of a hollow organ or the state of the surroundings of the hollow organ.

[0046] The first learning model 651 receives, for example, a set of transverse images 485 acquired by one pull-back operation by the MDU 33. The first learning model 651 may receive an input of a part of the transverse images 485, such as half or one-third, of the transverse images 485 acquired by one pull-back operation. The first learning model 651 may receive an input of a set of transverse images 485 acquired by manually moving the sensor 42 forward and backward. The first learning model 651 may be a model that receives an input of one transverse image 485 and outputs a finding regarding the state of a hollow organ or the state around the hollow organ.

[0047] The first learning model 651 includes an input layer, a neural network 659, a plurality of softmax layers 655, and an output layer. The neural network 659 is, for example, a convolutional neural network (CNN) having a set of a plurality of convolution layers and pooling layers, and a fully connected layer. One softmax layer 655 is provided for each row shown in Tables 1 to 5.

[0048] The input layer receives an image obtained by combining a set of cross-sectional images 485 in the scanning order. The image is output to the output layer via a neural network 659 and a softmax layer 655, which indicates the probability of each item shown in Tables 1 to 5.

[0049] For example, in Fig. 4, the probability that "need for treatment" is "no" is 95 percent, the probability that "blood flow stagnation" is "no" is 90 percent, and the probability that "branching" is "yes" is 90 percent. The first learning model 651 may be divided into separate models for Tables 1 to 5. The first learning model 651 may be divided into separate models for each item to be output.

[0050] A selection layer may be provided after the softmax layer 655 to select and output the option with the highest probability.

[0051] FIG. 5 is an explanatory diagram for explaining the configuration of the second learning model 652. The second learning model 652 is a model that receives the cross-sectional image 485 and outputs a quantitative finding related to the state of the hollow organ or the state around the hollow organ. The finding output by the second learning model 652 is a probability related to each value of a plurality of items. Table 6 shows examples of items included in the finding for which the second learning model 652 outputs a probability. Each row of Table 6 shows one item.

[0052] [Table 6]

[0053] Each item in Table 6 is an example of quantitative shape information. The second learning model 652 may output the probability for some of the items in Table 5. The second learning model 652 may output the probability for items other than those in Table 6, such as the length of the stenosis.

[0054] The second learning model 652 includes an input layer, a neural network 659, multiple softmax layers 655, and an output layer. The neural network 659 is, for example, a CNN having multiple convolutional layers, pooling layers, and a fully connected layer. For each row shown in Table 6, one softmax layer 655 is provided.

[0055] The input layer receives an image obtained by combining a set of cross-sectional images 485 in the scanning order. The neural network 659 and the softmax layer 655 output the probability that the value of each item in Table 6 is within a given range to the output layer.

[0056] For example, in FIG. 5, the probability that "eccentricity" is "0 percent or more and less than 10 percent" is 0 percent, and the probability that it is "10 percent or more and less than 20 percent" is 10 percent.

[0057] The second learning model 652 outputs the probability of a representative value, such as the average, maximum or minimum value, of each item in a set of cross-sectional images 485 .

[0058] The second learning model 652 may output the probability of each item shown in Table 6 for each cross-sectional image 485. In this case, one cross-sectional image 485 may be input to the input layer, or the target cross-sectional image 485 and several cross-sectional images 485 before and after it may be input.

[0059] 5 are merely examples. The second learning model 652 may output probabilities at any interval, such as every 5 percent or every 20 percent.

[0060] A selection layer that selects and outputs the option with the highest probability may be provided after the softmax layer 655. The selection layer may output the results of a calculation such as a weighted average based on the representative value and probability of each option.

[0061] Instead of using the second learning model 652, the control unit 21 may extract the inner wall of the tubular organ by image processing of the transverse cross-sectional image 485 and calculate each item shown in Table 6. The method of extracting the boundary from the image and calculating the length, etc. has been used conventionally, so detailed explanation will be omitted.

[0062] Each item shown in Table 6 may be judged by degree instead of numerical value. Table 7 shows examples of items when the same items as in Table 6 are judged by degree. When judging by degree, the probability of each item can be output using the first learning model 651 described using FIG. 4.

[0063] [Table 7]

[0064] To the first learning model 651 and the second learning model 652, data at a stage prior to the formation of the transverse layer image 485, such as sound ray data acquired by the catheter control unit 271 from the sensor 42, may be input.

[0065] Fig. 6 is an example of a screen displayed by the catheter system 10. The screen shown in Fig. 6 includes a transverse image field 51, a longitudinal image field 52, and a findings field 53. The transverse image field 51 displays the transverse image 485 described above. The longitudinal image field 52 displays the longitudinal image generated by the catheter control unit 271.

[0066] A transverse slice position marker 551 indicating the position of the transverse slice image 485 displayed in the transverse slice image field 51 is displayed on the edge of the transverse slice image field 52. A longitudinal slice position marker 552 indicating the position of the longitudinal slice image displayed in the transverse slice image field 52 is displayed near the edge of the transverse slice image field 51. The user can appropriately change the cross section to be displayed by operating the input device 32 to change the positions of the transverse slice position marker 551 and the longitudinal slice position marker 552. The control unit 21 may also accept voice input from the user.

[0067] The findings field 53 includes a qualitative evaluation field 531 and a quantitative evaluation field 532. The qualitative evaluation field 531 lists items that indicate a probability of exceeding a predetermined threshold among the qualitative findings output from the first learning model 651 described using Fig. 4. The quantitative evaluation field 532 displays each item of the quantitative findings output from the second learning model 652 described using Fig. 5.

[0068] The control unit 21 may display the transverse tomographic image field 51 and the longitudinal tomographic image field 52 at the same scale.

[0069] Figure 7 is an example of a screen displayed by the catheter system 10. When the user selects an item in the findings column 53 on the screen described using Figure 6 and instructs the user to indicate the basis, the control unit 21 displays the screen shown in Figure 7. Figure 7 shows an example of a display in which the user indicates the basis for the determination of "flap: 80 percent."

[0070] 7 includes a transverse image field 51 and a grounds display field 56. A transverse image 485 is displayed in the transverse image field 51. A grounds display field 56 displays the transverse image 485 with a grounds marker 561 superimposed thereon, the grounds marker 561 indicating the grounds region that was the grounds for the determination of "flap: 80 percent." The grounds marker 561 is an example of grounds information regarding the grounds for the findings displayed in the findings field 53.

[0071] In addition, if the part that was the basis for the ``flap'' determination is not sufficiently included in the screen described using Figure 6, the control unit 21 extracts a transverse cross-sectional image 485 that includes the part that was the basis for the ``flap'' determination, and displays it in the transverse cross-sectional image column 51 and the basis display column 56.

[0072] The control unit 21 extracts the evidence region by a model visualization method such as Gradient-weighted Class Activation Mapping (Grad-CAM) or Grad-CAM++. The evidence region is a region that strongly influenced the output of the item specified by the user among the multiple cross-sectional images 485 input to the learning model 65. The evidence marker 561 is displayed with finer hatching in a location that has a higher influence on the output.

[0073] By comparing the cross-sectional image field 51 with the grounds display field 56, the user can judge from a professional point of view whether the grounds for the determination by the control unit 21 are appropriate or not. Note that the longitudinal tomographic image may be displayed on the screen described using Fig. 7. The control unit 21 may display a grounds marker 561 superimposed on the longitudinal tomographic image.

[0074] Fig. 8 is an example of a report. The control unit 21 prints the report described with reference to Fig. 8 on paper, for example, or outputs it in a file format such as a portable document format (PDF). The report is used as a report to another doctor, etc. The report may be recorded in an electronic medical record via the HIS.

[0075] The report includes a patient information field 541, a summary field 542, and a details field 543. The patient information field 541 displays patient information such as the patient's name, patient ID, gender, and examination date. The summary field 542 records a text summary of the examination using the catheter system 10. The items to be recorded in the summary field 542 may be automatically generated by the control unit 21 based on the output of the learning model 65, or may be input by the user.

[0076] 8, a message indicating that there is a finding of dissection of a hollow organ is displayed in the upper part of the summary section 542. Below that, a longitudinal tomographic image section 52 and a transverse tomographic image section 51 are displayed, in which the tomographic images showing the finding of dissection are displayed. A transverse layer position marker 551 displayed on the edge of the longitudinal tomographic image section 52 allows the user to confirm which part of the longitudinal tomographic image the dissection has occurred.

[0077] A basis display field 56 showing the basis of the dissociation findings is displayed next to the transverse image field 51. The transverse images 485 displayed in the transverse image field 51 and the basis display field 56 are the same. A basis marker 561 superimposed on the transverse image 485 in the basis display field 56 allows the user to confirm the basis of the "dissociation findings" displayed at the top of the summary field 542.

[0078] Fig. 9 is a flow chart for explaining the flow of the program processing. The control unit 21 acquires one set of transverse images 485 from the catheter control unit 271 (step S501). The control unit 21 inputs the acquired transverse images 485 to the first learning model 651 and the second learning model 652 to acquire qualitative findings and quantitative findings (step S502). The control unit 21 displays the screen described using Fig. 6 on the display device 31 (step S503).

[0079] The control unit 21 determines whether or not an instruction to display the grounds has been received from the user (step S504). The user can input an instruction to display the grounds by, for example, double-clicking an item in the findings column 53. The control unit 21 may also receive a voice input from the user.

[0080] When it is determined that the instruction to display the grounds has been received (YES in step S504), the control unit 21 acquires an item for displaying the grounds (step S505). The control unit 21 extracts a grounds region related to the item acquired in step S505 by a model visualization method such as Grad-CAM or Grad-CAM++ (step S506).

[0081] The control unit 21 uses the screen described with reference to Fig. 7 to display the cross-sectional image 485 with the grounds marker 561 superimposed thereon in the grounds display field 56 (step S507). If it is determined that an instruction to display the grounds has not been received (NO in step S504), or after step S507 is completed, the control unit 21 ends the process.

[0082] According to this embodiment, it is possible to provide a catheter system 10 that displays various findings based on tomographic images captured using the diagnostic imaging catheter 40. It is possible to provide a catheter system 10 that can be easily used even by a user who is not sufficiently skilled in interpreting tomographic images. According to this embodiment, it is possible to provide a catheter system 10 that displays both qualitative and quantitative evaluations.

[0083] According to the present embodiment, it is possible to provide a catheter system 10 that displays an area that is the basis of a finding using a grounds marker 561. The user can determine whether or not the determination by the control unit 21 is appropriate using the grounds marker 561. By displaying a tomographic image without the grounds marker 561 superimposed and a tomographic image with the grounds marker 561 superimposed side by side, it is possible to provide a catheter system 10 in which the grounds marker 561 does not interfere with the interpretation of the tomographic image.

[0084] According to the present embodiment, it is possible to provide a catheter system 10 that automatically outputs a report. Therefore, it is possible to provide a catheter system 10 that can efficiently share information with paramedical staff such as nurses and doctors of other departments.

[0085] In addition to the cross-sectional image 485, the first learning model 651 and the second learning model 652 may also be input with medical information obtained in real time, such as images taken using an imaging diagnostic device 37, blood pressure, heart rate, or oxygen saturation.

[0086] In addition to the cross-sectional image 485, medical information acquired from an electronic medical record, such as medical history, height, weight, and images previously taken using the image diagnostic device 37, may be input to the first learning model 651 and the second learning model 652. The medical information input to the first learning model 651 and the medical information input to the second learning model 652 may be the same or different.

[0087] In this case, the first learning model 651 receives the cross-sectional image 485 and medical information, and outputs findings on the state of the hollow organ or the state around the hollow organ. Similarly, the second learning model 652 receives the cross-sectional image 485 and medical information, and outputs quantitative findings on the state of the hollow organ or the state around the hollow organ.

[0088] In addition to the transverse image 485, by including medical information acquired in real time or non-real time in the input data for the first learning model 651 or the second learning model 652, a catheter system 10 can be provided that outputs findings with high accuracy.

[0089] According to the present embodiment, a catheter system 10 can be provided that displays findings for a luminal organ that has previously been treated, such as with stent placement, an untreated luminal organ, and a luminal organ immediately after intraluminal treatment.

[0090] [Embodiment 2] This embodiment relates to a catheter system 10 in which a catheter control device 27 and an information processing device 20 are separate entities. Descriptions of parts common to the first embodiment will be omitted.

[0091] 10 is an explanatory diagram illustrating the configuration of a catheter system 10 of embodiment 2. The catheter system 10 of this embodiment has an information processing device 20, a catheter control device 27, an MDU 33, and an image diagnostic catheter 40. The information processing device 20 has a control unit 21, a main storage device 22, an auxiliary storage device 23, a communication unit 24, a display unit 25, an input unit 26, and a bus.

[0092] Catheter control device 27 is an ultrasonic diagnostic device for IVUS that controls MDU 33, controls sensor 42, and generates transverse image 485 and longitudinal image based on a signal received from sensor 42. The function and configuration of catheter control device 27 are similar to those of conventional ultrasonic diagnostic devices, and therefore a description thereof will be omitted.

[0093] Catheter control device 27 and information processing device 20 may be directly connected via a cable or wireless communication, or may be connected via a network.

[0094] The information processing device 20 of this embodiment is a general-purpose personal computer, a tablet, a smartphone, a mainframe computer, a virtual machine running on a mainframe computer, a cloud computing system, or a quantum computer. The information processing device 20 may be a plurality of personal computers performing distributed processing.

[0095] [Embodiment 3] This embodiment relates to a catheter system 10 that outputs the success or failure of an IVR treatment based on tomographic images taken before and after the IVR using a diagnostic imaging catheter 40. Explanations of parts common to the first embodiment will be omitted.

[0096] 11 is an example of a screen displayed by the catheter system 10 of embodiment 3. The screen shown in FIG. 11 includes a post-treatment longitudinal image field 522, a pre-treatment longitudinal image field 521, a transverse image field 51, a switching button 58, and a findings field 53.

[0097] The pre-treatment longitudinal tomographic image field 521 displays a longitudinal tomographic image taken using the diagnostic imaging catheter 40 before performing an IVR treatment. The post-treatment longitudinal tomographic image field 522 displays a longitudinal tomographic image taken using the diagnostic imaging catheter 40 before performing an IVR treatment.

[0098] 11, a longitudinal image of a partially stenosed hollow organ is displayed in a pre-treatment longitudinal image field 521. A longitudinal image of the hollow organ in which the stenosed portion has been dilated by balloon angioplasty or the like is displayed in a post-treatment longitudinal image field 522.

[0099] The control unit 21 displays the pre-treatment longitudinal tomographic image field 521 and the post-treatment longitudinal tomographic image field 522 by matching the left-right positions so that the longitudinal positions are aligned vertically. The matching is performed based on landmarks such as the branching of side branches or blood vessels present around the hollow organ through which the diagnostic imaging catheter 40 is inserted.

[0100] The control unit 21 may perform matching based on the insertion length of the diagnostic imaging catheter 40 inserted into the tubular organ. The control unit may perform matching based on the positions of the tip marker 44 or the sensor 42 on images taken by the diagnostic imaging device 37 before and after the treatment.

[0101] The control unit 21 may match the cross sections of the longitudinal images before and after the treatment based on the transverse image 485. The control unit 21 performs matching based on the surrounding blood vessels, pericardium, etc. included in the transverse image 485 of the position where the treatment is not performed. In this case, the longitudinal images before and after the treatment can be displayed as cross sections in the same direction.

[0102] A transverse layer position marker 551 indicating the position of the transverse layer image 485 displayed in the transverse layer image field 51 is displayed on the edge of the transverse layer position marker 551. A longitudinal layer position marker 552 indicating the position of the longitudinal layer image displayed in the longitudinal layer image field 52 is displayed on the edge of the transverse layer image field 51. The user can appropriately change the cross section to be displayed by changing the positions of the transverse layer position marker 551 and the longitudinal layer position marker 552 by operating the input device 32. The control unit 21 may also accept voice input from the user.

[0103] The findings on the success or failure of the treatment and a comparison of the state before and after the treatment are displayed in the findings column 53. The control unit 21 judges the success or failure of the treatment based on whether or not a judgment criterion set by the user before performing the treatment is achieved. The control unit 21 may judge the success or failure of the treatment based on a judgment criterion that is predetermined for each treatment.

[0104] When the user selects the switching button 58 displayed at the top of the transverse image field 51, the control unit 21 alternates between the image before the treatment and the image after the treatment as the transverse image 485 displayed in the transverse image field 51. The user can easily compare the tomographic images before and after the treatment.

[0105] When a post-treatment tomographic image is displayed in the transverse tomographic image field 51, the control unit 21 displays a transverse layer position marker 551 in the post-treatment longitudinal tomographic image field 522 as shown in Fig. 11. When a pre-treatment tomographic image is displayed in the transverse tomographic image field 51, the control unit 21 displays the transverse layer position marker 551 in the pre-treatment longitudinal tomographic image field 521.

[0106] Fig. 12 is a flow chart for explaining the processing flow of the program of the third embodiment. The control unit 21 acquires a criterion for judging the success or failure of the treatment (step S511). The criterion is, for example, the degree of stenosis of the luminal organ as shown in Fig. 11. The criterion may be the minimum value of the inner diameter of the luminal organ, or the inner diameter of the placed stent. The criterion may be, for example, a qualitative state such as the presence or absence of blood flow stagnation.

[0107] The control unit 21 accepts an input of the judgment criteria by the user via, for example, the input device 32. The control unit 21 may acquire the judgment criteria that are determined in advance for each type and part of the IVR and that are recorded in the auxiliary storage device 23.

[0108] The control unit 21 acquires one set of transverse images 485 captured before treatment and stored in the auxiliary storage device 23 (step S512). The control unit 21 may acquire the transverse images 485 captured in real time from the catheter control unit 271. The control unit 21 inputs the acquired transverse images 485 to the learning model 65 that outputs findings corresponding to the judgment criterion acquired in step S511, and acquires findings (step S513).

[0109] The control unit 21 acquires one set of tomographic images 48 captured after the end of the treatment and stored in the auxiliary storage device 23 (step S514). The control unit 21 may acquire a transverse image 485 captured in real time from the catheter control unit 271. The control unit 21 inputs the acquired transverse image 485 to the learning model 65 that outputs findings corresponding to the judgment criterion acquired in step S511, and acquires findings (step S515).

[0110] The control unit 21 judges whether the treatment has been successful or not based on the pre-treatment findings acquired in step S513, the post-treatment findings acquired in step S515, and the judgment criteria acquired in step S511 (step S516). The control unit 21 displays whether the treatment has been successful or not using the screen described with reference to Fig. 11 (step S517). Thereafter, the control unit 21 ends the process.

[0111] The success or failure of the treatment is not limited to the two options of success and failure. The control unit 21 may obtain a judgment criterion for dividing the treatment into three or more ranks in step S511. The judgment criterion may be determined by combining a plurality of findings.

[0112] The control unit 21 may determine the success or failure of the treatment using a learning model that outputs the success or failure of the treatment when the transverse cross-sectional image 485 before the treatment and the transverse cross-sectional image 485 after the treatment are input.

[0113] According to this embodiment, it is possible to provide a catheter system 10 that judges whether a treatment is successful or not. If a predetermined judgment criterion is not met, the user can promptly perform additional treatment. This can significantly reduce the physical burden on the patient compared to when additional treatment is required at a later date.

[0114] According to this embodiment, by matching and displaying the positions of the longitudinal tomographic images before and after the treatment, it is possible to provide a catheter system 10 that allows the user to easily compare the conditions before and after the treatment. According to this embodiment, by matching the cross sections of the longitudinal tomographic images before and after the treatment, it is possible to provide a catheter system 10 that allows the user to accurately compare the conditions before and after the treatment.

[0115] [Embodiment 4] This embodiment relates to a catheter system 10 that displays the position of a tomographic image captured using a diagnostic imaging catheter 40 superimposed on an image acquired from an imaging diagnostic device 37. Descriptions of parts common to the first embodiment will be omitted.

[0116] Fig. 13 is an example of a screen displayed by the catheter system 10 of the fourth embodiment. The screen shown in Fig. 13 includes a findings field 53 and an other device image field 59. The findings field 53 displays findings acquired by the control unit 21. The other device image field 59 displays medical images captured by the imaging diagnostic device 37.

[0117] A scan area 591 indicating the position of the tomographic image captured by the diagnostic imaging catheter 40 is displayed as a rectangle indicating the outline of the longitudinal tomographic image superimposed on the other device image field 59. The control unit 21 may display the longitudinal tomographic image in real time inside the scan area 591. The control unit 21 may receive a selection regarding the display form of the scan area 591 from the user.

[0118] An outline of a method for displaying the scan area 591 will be described below, taking as an example a case where the image diagnostic device 37 is an X-ray angiography device. The sensor 42 is mounted on a sensor marker that is opaque to X-rays. Since the tip marker 44 and the sensor marker are opaque to X-rays, they are clearly displayed in the medical image captured by the X-ray angiography device.

[0119] The control unit 21 detects the tip marker 44 and the sensor marker from the medical image. The detected sensor marker indicates the position of the sensor 42. For example, when a set of transverse slice images 485 is generated using a pullback operation by the MDU 33, both ends of the operating range of the sensor 42 at the time of image acquisition correspond to the positions of the short sides of the scan area 591.

[0120] The control unit 21 determines the length of the short side of the scan area 591 based on the display range of the transverse cross-sectional image 485 and the scale of the other device image. The control unit 21 superimposes and displays the rectangular scan area 591 determined from the position and length of the short side on the other device image field 59 displaying the medical image.

[0121] By the above processing, even if the tip portion of the diagnostic imaging catheter 40 is not parallel to the projection surface of the diagnostic imaging device 37, the control unit 21 can display the scan area 591 at the correct position in the other device image field 59.

[0122] Returning to Fig. 13, the explanation will be continued. A plurality of transverse slice position markers 551 are displayed within a scan area 591. Transverse slice image fields 51 displaying transverse slice images 485 corresponding to the respective transverse slice position markers 551 are displayed around the other device image field 59. The user can appropriately change the cross-sectional position of the transverse slice image 485 by moving the transverse slice position markers 551 via the input device 32. The control unit 21 may also receive voice input from the user.

[0123] The control unit 21 may switch between the object arrangement image 482 and the transverse cross-sectional image 485 based on an instruction from the user. The control unit 21 may display the object arrangement image 482 and the transverse cross-sectional image 485 side by side. The control unit 21 may display a longitudinal cross-sectional image or a longitudinal object arrangement image.

[0124] 14 is a flowchart explaining the flow of processing of the program of embodiment 4. Controller 21 acquires a transverse cross-sectional image 485 and a medical image from catheter controller 271 and image diagnostic device 37, respectively, during one image acquisition (step S531).

[0125] The control unit 21 detects the tip marker 44 and the sensor marker from the medical image (step S532). For example, when a set of transverse slice images 485 is generated using a pullback operation by the MDU 33, the control unit 21 determines the position and dimensions of the scan area 591 based on the positions of the sensor markers detected at both ends of the pullback operation. The control unit 21 determines the position of the transverse slice position marker 551 based on the scan area 591 (step S533).

[0126] It is preferable that the control unit 21 traces and displays the position corresponding to the scan area 591 on a medical image captured thereafter in real time.

[0127] The control unit 21 inputs the acquired cross-sectional image 485 into the learning model 65 to acquire findings (step S534). The control unit 21 displays the screen described using Fig. 13 on the display device 31 (step S535). Thereafter, the control unit 21 ends the process.

[0128] According to this embodiment, it is possible to provide a catheter system 10 that displays the position of a tomographic image captured using the diagnostic imaging catheter 40 superimposed on a medical image captured by the diagnostic imaging device 37. A user can easily change the position of the transverse slice image 485 to be displayed by operating the transverse slice position marker 551. As described above, it is possible to provide a catheter system 10 that allows a user to easily grasp the positional relationship between a tomographic image and the surrounding organs.

[0129] The diagnostic imaging device 37 is not limited to an X-ray angiography device. For example, an ultrasonic diagnostic device combined with an external probe or a TEE (Transesophageal Echocardiography) probe can capture a tomographic image different from that of the diagnostic imaging catheter 40 in real time.

[0130] When the diagnostic imaging catheter 40 is equipped with both an ultrasonic sensor 42 and an OCT sensor 42, a transverse cross-sectional image 485 by ultrasonic waves and a transverse cross-sectional image 485 by OCT can be captured at substantially the same cross section. For example, by superimposing findings from a transverse cross-sectional image 485 by OCT, which has excellent resolution, on a transverse cross-sectional image 485 by ultrasonic waves, which has a greater penetration depth than OCT, a catheter system 10 can be provided that displays information making use of the advantages of both.

[0131] The medical image is not limited to a medical image captured in real time. The control unit 21 may superimpose the scan area 591 on a medical image captured by any image diagnostic device, such as a CT, MRI, PET, X-ray angiography device, or ultrasonic diagnostic device, and recorded in an electronic medical record, etc. The control unit 21 determines the position of the scan area 591 based on the branching of blood vessels, the position of the heart, etc. included in each image.

[0132] The processing of this embodiment may be executed on the side of the diagnostic imaging device 37 and displayed on a display device connected to the diagnostic imaging device 37 .

[0133] [Variation 1] Fig. 15 is an example of a screen displayed by the catheter system 10 of the first modification of the fourth embodiment. The screen shown in Fig. 15 has a transverse section image field 51 and an other device schema field 592. In Fig. 15, a schema that typically shows a hollow organ is used instead of a medical image captured by the image diagnostic device 37.

[0134] The position on the schema corresponding to the tomographic image shown in the transverse image column 51 is indicated by a thick arrow-shaped catheter marker 593. After step S533 described using Fig. 14, the control unit 21 determines and displays the position of the catheter marker 593 on the schema based on the correspondence between the medical image and the schema.

[0135] According to this embodiment, a catheter system 10 can be provided that uses a schema and allows the user to more easily grasp the position of a tomographic image.

[0136] [Variation 2] 16 is an example of a screen displayed by the catheter system 10 of the second modification of the fourth embodiment. The screen shown in FIG.

[0137] The other device schema field 592 displays a three-dimensional schematic diagram of a schema and a catheter marker 593. The transverse layer schema field 595 displays a schema of the transverse layer image 485. The schema of the transverse layer image 485 may be automatically generated based on the transverse layer image 485, or may be selected and displayed from the schemas stored in the auxiliary storage device 23 for each site and type of lesion. The transverse layer image field 51 displays the transverse layer image 485 captured by the diagnostic imaging catheter 40.

[0138] Note that the longitudinal section image field 52 may be displayed instead of or together with the transverse section image field 51. When the longitudinal section image field 52 is displayed, the control unit 21 displays a catheter marker 593 indicating the longitudinal section in the other device schema field 592.

[0139] According to this embodiment, since the schema is three-dimensional, it is possible to provide a catheter system 10 that allows the user to easily grasp the position and direction of the tomographic image.

[0140] [Embodiment 5] This embodiment relates to a program for generating a first learning model 651 and a second learning model 652. Explanation of parts common to the first embodiment will be omitted.

[0141] 17 is an explanatory diagram for explaining the record layout of a training data DB (Database). The training data DB is a database in which inputs and correct labels are recorded in association with each other, and is used for training a model by machine learning. The training data DB has a tomographic image data field, a qualitative field, and a quantitative field.

[0142] The qualitative fields include fields corresponding to the qualitative findings output by the first learning model 651, such as a treatment necessity field, a blood flow stasis field, and a bifurcation field. The quantitative fields include fields corresponding to the quantitative findings output by the second learning model 652, such as an eccentricity field, a stenosis field, and an internal diameter field.

[0143] The tomographic image data field records a set of cross-sectional images 485. The necessity of treatment field displays the result of a specialist's judgment on the necessity of treatment based on the cross-sectional images 485 recorded in the tomographic image data field. Similarly, the blood flow stagnation field records the presence or absence of blood flow stagnation, and the branching field records the presence or absence of branching.

[0144] Similarly, the eccentricity field, the stenosis field, and the inner diameter field respectively record the eccentricity, the stenosis, and the inner diameter. The training data DB has one record for one set of cross-sectional images 485.

[0145] The training data DB contains a large number of combinations of sets of transverse images 485 taken using the diagnostic imaging catheter 40 and measurement results such as the need for treatment and eccentricity as determined by a specialist physician or other such person.

[0146] When generating the first learning model 651, the set of cross-sectional images 485 recorded in the tomographic image data field is used as input data, and the data recorded in each of the qualitative fields is used as the correct label.

[0147] When generating the second learning model 652, the set of cross-sectional images 485 recorded in the tomographic image data field is used as input data, and the data recorded in each of the quantitative fields is used as the correct label.

[0148] 18 is a flowchart explaining the flow of processing of the program according to embodiment 5. An example will be explained in which machine learning is performed on each of the first learning model 651 and the second learning model 652 using the information processing device 20.

[0149] 18 may be executed on hardware other than the information processing device 20, and the first learning model 651 and the second learning model 652 for which machine learning has been completed may be copied to the auxiliary storage device 23 via a network. The first learning model 651 and the second learning model 652 trained on one piece of hardware may be used by multiple information processing devices 20.

[0150] Prior to the execution of the program of FIG. 18, an untrained model such as a CNN having a convolutional layer, a pooling layer, a fully connected layer, and a softmax layer 655 is prepared. The untrained model is not limited to a CNN. Any type of model such as a decision tree or a random forest can be used. The program of FIG. 18 adjusts each parameter of the prepared model to perform machine learning.

[0151] The control unit 21 acquires training records to be used for one epoch of training from the training data DB (step S621). The number of training records to be used for one epoch of training is a so-called hyperparameter, and is appropriately determined.

[0152] The control unit 21 generates an input image from the input data included in each acquired training record (step S622). Specifically, the control unit 21 generates a single image by combining the transverse images 485 included in the tomographic image field in the scanning order. The combined tomographic images may be recorded in the tomographic image data field.

[0153] The control unit 21 adjusts the parameters of the model so that, when an input data vector is input to the input layer of the model, a correct label is output from the output layer (step S623).

[0154] The control unit 21 determines whether to end the process (step S624). For example, the control unit 21 determines to end the process when learning for a predetermined number of epochs is completed. The control unit 21 may obtain test data from the training data DB, input the test data to the model under machine learning, and determine to end the process when an output of a predetermined accuracy is obtained.

[0155] If it is determined that the process should not be terminated (NO in step S624), control unit 21 returns to step S621. If it is determined that the process should be terminated (YES in step S624), control unit 21 records the parameters of the trained model in auxiliary storage device 23 (step S625). Thereafter, control unit 21 terminates the process. Through the above process, a trained model is generated.

[0156] The training data DB may be created, for example, separately into a database for generating the first learning model 651 and a database for generating the second learning model 652.

[0157] According to this embodiment, the first learning model 651 and the second learning model 652 can be generated by machine learning.

[0158] [Embodiment 6] This embodiment relates to a catheter system 10 that allows a user to correct findings output by a learning model 65. Descriptions of parts common to the first embodiment will be omitted.

[0159] 19 is an explanatory diagram for explaining the record layout of the correction DB. The correction DB is a database that records correction information that associates findings output by the catheter system 10 with corrections made by the user.

[0160] The correction DB has a tomographic image data field, a qualitative data field, and a quantitative data field, each of which has an output field and a correction field.

[0161] The tomographic image data field records one set of transverse images 485. The output field of the qualitative data field records the qualitative findings output by the control unit 21 to the display device 31. The correction field of the qualitative data field records the qualitative findings corrected by the user.

[0162] The quantitative data field output field records the quantitative findings output by the control unit 21 to the display device 31. The quantitative data field correction field records the quantitative findings corrected by the user. The correction DB has one record for each correction made by the user to the set of cross-sectional images 485.

[0163] Fig. 20 is an example of a screen displayed by the catheter system of embodiment 6. Fig. 20 is a screen displayed by the control unit 21 on the display device 31 when the user instructs correction of the qualitative findings while the screen described using Fig. 6 is being displayed, for example.

[0164] The screen shown in Fig. 20 includes a cross-sectional image field 51, a candidate label field 571, a correct label field 572, and a free description field 573. The candidate label field 571 displays candidates for labels indicating qualitative findings. The correct label field 572 displays a label that the user judges to be correct.

[0165] The user can input the correct finding by dragging and dropping the label displayed in the candidate label column 571 into the correct label column 572. The control unit 21 may also accept input by voice input. For example, when the user utters "Correct, hematoma," the control unit 21 moves the label "hematoma" in the first line of the candidate label column 571 to the correct label column 572.

[0166] If a label that represents an appropriate finding cannot be found in the candidate label field 571, the user can input any finding using the free description field 573. Although not shown in the figure, the user can also appropriately change quantitative findings in the same way.

[0167] The control unit 21 reflects the correction result by the user in the report described with reference to Fig. 8. Even in the case where the learning model 65 cannot make an accurate judgment, an appropriate report that reflects the judgment of the doctor who is the user can be created.

[0168] The corrections entered by the user using the screen of Fig. 20 are recorded in the correction DB described using Fig. 19. The data recorded in the correction DB is used for re-learning the learning model 65 and for correcting hyperparameters used during machine learning by machine learning engineers.

[0169] [Embodiment 7] 21 is a functional block diagram of the catheter system of the seventh embodiment. The catheter system 10 includes an acquisition unit 86 and an output unit 87. The acquisition unit 86 acquires a tomographic image generated using the diagnostic imaging catheter 40 inserted into a hollow organ. The output unit 87 inputs the tomographic image acquired by the acquisition unit 86 to a learning model 65 that outputs findings related to the state of the hollow organ or the state around the hollow organ when a tomographic image is input, and outputs the findings output from the learning model 65.

[0170] [Embodiment 8] Fig. 22 is an explanatory diagram for explaining the configuration of a catheter system 10 according to an eighth embodiment. This embodiment relates to a form in which an information processing device 20 according to this embodiment is realized by combining and operating a general-purpose computer 90 and a program 97. Explanations of parts common to the second embodiment will be omitted.

[0171] The catheter system 10 of this embodiment includes a computer 90. The computer 90 includes a control unit 21, a main memory device 22, an auxiliary memory device 23, a communication unit 24, a display unit 25, an input unit 26, a reading unit 29, and a bus. The computer 90 is an information device such as a general-purpose personal computer, a tablet, a smartphone, or a server computer.

[0172] The program 97 is recorded in a portable recording medium 96. The control unit 21 reads the program 97 via the reading unit 29 and stores it in the auxiliary storage device 23. The control unit 21 may also read out the program 97 stored in a semiconductor memory 98 such as a flash memory implemented in the computer 90. Furthermore, the control unit 21 may download the program 97 from another server computer (not shown) connected via the communication unit 24 and a network (not shown), and store it in the auxiliary storage device 23.

[0173] The program 97 is installed as a control program for the computer 90, and is loaded into and executed by the main storage device 22. This causes the computer 90 to function as the information processing device 20 described above.

[0174] The technical features (constitutive elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims. [Explanation of symbols]

[0175] 10 Catheter System 20 Information processing device 21 Control section 22 Main storage 23 Auxiliary storage device 24 Communications Department 25 Display section 26 Input section 27 Catheter Control Device 271 Catheter control section 29 Reading unit 31 Display device 32 Input Devices 33 MDU 37 Diagnostic Imaging Devices 40 Diagnostic Imaging Catheter 41 Probe section 42 Sensors 43 Shaft 44 Tip Marker 45 Connector part 48 Tomographic image 485 Cross-sectional image (tomogram) 51 Cross section image column 52 Longitudinal Section 521 Pre-treatment longitudinal section 522 Post-treatment longitudinal section 53 Findings column 531 Qualitative Evaluation Column 532 Quantitative evaluation section 542 Summary column 543 Details column 551 Cross-Section Marker 552 Longitudinal fault position marker 56 Basis display column 561 Grounds Marker 571 Candidate label field 572 Correct answer label field 573 Free text section 58 Switch button 59 Other device image column 591 Scan Area 592 Other device schema column 593 Catheter Marker 595 Cross-Sectional Schematic 65 Learning Model 651 First Learning Model (Learning Model) 652 Second Learning Model 655 Softmax Layer 659 Neural Networks 86 Acquisition Department 87 Output section 90 Computer 96 Portable recording media 97 Program 98 Semiconductor Memory

Claims

1. A tomographic image is obtained using an imaging catheter inserted into the tubular organ; The acquired tomographic image is input to a learning model that outputs device information indicating the state of an indwelling device placed in a tubular organ when the tomographic image is input, and the device information output from the learning model is output. A program that causes a computer to carry out processing.

2. The learning model outputs device information when medical information different from the acquired tomographic image is further input. The program according to claim 1.

3. The device information is information indicating a state of a stent placed in a luminal organ. The program according to claim 1 or 2.

4. the tomographic image is a post-treatment tomographic image generated using an imaging diagnostic catheter inserted into a hollow organ after a treatment has been performed on the hollow organ, The learning model outputs device information indicating the state of an indwelling device placed in a tubular organ when a post-treatment tomographic image is input. The program according to claim 1 or 2.

5. the tomographic image is a transverse image showing a cross section of the hollow organ and the periphery of the hollow organ cut in a direction intersecting with a longitudinal direction of the hollow organ, The learning model outputs the device information when a plurality of transverse cross-sectional images generated along the longitudinal direction of a hollow organ are input. The program according to any one of claims 1 to 4.

6. A longitudinal section image of the hollow organ and its surroundings, which is generated based on the plurality of transverse section images, and the device information are simultaneously output. The program according to claim 5.

7. outputting the longitudinal tomographic image and a selected tomographic image selected from the plurality of transverse tomographic images at the same time; A marker indicating the position of the selected tomographic image is superimposed on the longitudinal tomographic image. The program according to claim 6.

8. A marker indicating the position of the acquired tomographic image is superimposed on a medical image or a schema different from the acquired tomographic image. The program according to any one of claims 1 to 7.

9. Outputting basis information regarding the basis of the device information together with the device information. The program according to any one of claims 1 to 8.

10. Accepts modifications to the device information The program according to any one of claims 1 to 9.

11. Recording correction information that associates the device information with the received correction. The program according to claim 10.

12. Re-learning the learning model based on the correction information. The program according to claim 11.

13. Outputting a report including the tomographic image generated using the imaging diagnostic catheter and text showing device information. A program according to any one of claims 1 to 12.

14. A tomographic image is obtained using an imaging catheter inserted into the tubular organ; The acquired tomographic image is input to a learning model that outputs device information indicating the state of an indwelling device placed in a tubular organ when the tomographic image is input, and the device information output from the learning model is output. An information processing method for causing a computer to execute processing.

15. Reading out from a database a plurality of sets of recorded training data in which a tomographic image generated using a diagnostic imaging catheter inserted into a luminal organ is associated with device information showing the state of an indwelling device placed in the luminal organ; Using the training data, a learning model is generated that outputs device information indicating the state of an indwelling device placed in a tubular organ when a tomographic image is input. A method for generating a learning model in which processing is performed by a computer.

16. A tomographic image is obtained using an imaging catheter inserted into the tubular organ; inputting the acquired tomographic image into a learning model that outputs device information indicating the state of an indwelling device placed in a tubular organ when the tomographic image is input, and outputting the device information output from the learning model; receiving correction information for the output device information; Re-learning the learning model using the received correction information. A method for retraining a learning model in which processing is performed by a computer.

17. an acquisition unit that acquires a tomographic image generated using a diagnostic imaging catheter inserted into a hollow organ; an output unit that inputs the tomographic image acquired by the acquisition unit into a learning model that outputs device information indicating a state of an indwelling device placed in a tubular organ when the tomographic image is input, and outputs device information output from the learning model; An information processing system comprising:

Images