Computer program, information processing method, and information processing apparatus

Abstract

This computer program causes a computer to execute processing of: acquiring, on the basis of a signal detected by a catheter inserted into a hollow organ, a plurality of first medical images generated along an axial direction of the hollow organ; extracting, from the acquired plurality of first medical images, a plurality of second medical images which are at a prescribed interval along the axial direction of the hollow organ; identifying a type and a region of an object included in the extracted plurality of second medical images, by inputting the plurality of second medical images into a learning model which outputs, in response to input of a medical image, information related to an object included in the medical image; determining a reference image from among the plurality of second medical images on the basis of the type and the region of the object; and inputting, to the learning model, the first medical images within a prescribed range in the axial direction of the hollow organ from the reference image.

Description

Technical Field

[0001] The present invention relates to a computer program, an information processing method, and an information processing apparatus.

Background Art

[0002] Medical images including intravascular ultrasound (IVUS) images of blood vessels are generated by the intravascular ultrasound method using a catheter, and intravascular ultrasound examinations are performed. On the other hand, for the purpose of assisting doctors' diagnoses, technologies for adding information to medical images by image processing and machine learning have been developed (for example, Patent Document 1). The feature detection method in the blood vessel image described in Patent Document 1 detects a lumen wall, a stent, etc. included in the blood vessel image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the detection method of Patent Document 1, since features such as lumen walls are uniformly detected for all generated blood vessel images, there is a problem that it is difficult to efficiently detect such features.

[0005] An object of the present disclosure is to provide a computer program or the like that can efficiently detect an object based on a medical image obtained by scanning a lumen organ with a catheter.

Means for Solving the Problems

[0006] The computer program according to this embodiment causes the computer to acquire a plurality of first medical images generated along the axial direction of a tubular organ based on signals detected by a catheter inserted into the tubular organ, extract a plurality of second medical images at predetermined intervals along the axial direction of the tubular organ from the acquired plurality of first medical images, input the extracted plurality of second medical images into a learning model that outputs information about objects contained in a medical image when a medical image is input, thereby identifying the type and region of objects contained in the extracted plurality of second medical images, determine a reference image from the plurality of second medical images based on the identified type and region of objects, and input a first medical image within a predetermined range along the axial direction of the tubular organ into the learning model from the determined reference image.

[0007] The information processing method according to this embodiment involves a computer acquiring a plurality of first medical images generated along the axial direction of a tubular organ based on signals detected by a catheter inserted into the tubular organ; extracting a plurality of second medical images at predetermined intervals along the axial direction of the tubular organ from the acquired plurality of first medical images; inputting the extracted plurality of second medical images into a learning model that outputs information about objects contained in a medical image when a medical image is input, thereby identifying the type and region of objects contained in the extracted plurality of second medical images; determining a reference image from the plurality of second medical images based on the identified type and region of objects; and inputting a first medical image within a predetermined range along the axial direction of the tubular organ into the learning model from the determined reference image.

[0008] The information processing device according to this embodiment includes: an acquisition unit that acquires a plurality of first medical images generated along the axial direction of a tubular organ based on signals detected by a catheter inserted into the tubular organ; an extraction unit that extracts a plurality of second medical images at predetermined intervals in the axial direction of the tubular organ from the acquired plurality of first medical images; an identification unit that identifies the type and region of objects contained in the extracted plurality of second medical images by inputting the extracted plurality of second medical images into a learning model that outputs information about objects contained in a medical image when a medical image is input; a determination unit that determines a reference image from the plurality of second medical images based on the identified type and region of the objects; and an input unit that inputs a first medical image within a predetermined range in the axial direction of the tubular organ into the learning model from the determined reference image. [Effects of the Invention]

[0009] According to this disclosure, it is possible to provide a computer program, etc., that efficiently detects objects based on medical images obtained by scanning tubular organs with a catheter. [Brief explanation of the drawing]

[0010] [Figure 1] This is an explanatory diagram showing an example of the configuration of an imaging diagnostic device. [Figure 2] This is an explanatory diagram illustrating the overview of a diagnostic imaging catheter. [Figure 3] This is an explanatory diagram showing a cross-section of a blood vessel through which the sensor part has been inserted. [Figure 4] This is an explanatory diagram illustrating tomographic images. [Figure 5] This is a block diagram showing an example of the configuration of an image processing device. [Figure 6] This is an explanatory diagram showing an example of a learning model. [Figure 7] This is an explanatory diagram regarding the extraction of the second medical image. [Figure 8] This is an explanatory diagram regarding the representative value (plaque bathenness) of the second medical image. [Figure 9]This is an explanatory diagram regarding the range determined based on the reference image. [Figure 10] This flowchart shows the information processing procedure performed by the control unit. [Figure 11] This flowchart shows the procedure for selecting the stent size. [Figure 12] This is an explanatory diagram relating to the range determined based on the reference image in Embodiment 2. [Modes for carrying out the invention]

[0011] The image processing method, image processing apparatus, and program of this disclosure will be described in detail below with reference to the drawings illustrating their embodiments. In the following embodiments, cardiac catheterization, which is an endovascular treatment, will be described as an example, but the tubular organ targeted by catheterization is not limited to blood vessels, and may be other tubular organs such as the bile duct, pancreatic duct, bronchi, or intestines.

[0012] (Embodiment 1) Figure 1 is an explanatory diagram showing an example configuration of the diagnostic imaging device 100. In this embodiment, a diagnostic imaging device using a dual-type catheter equipped with both intravascular ultrasound (IVUS) and optical coherence tomography (OCT) functions will be described. The dual-type catheter is provided with three modes: one in which ultrasound tomography images are acquired using IVUS only, one in which optical coherence tomography images are acquired using OCT only, and one in which both ultrasound and OCT tomography images are acquired using IVUS and OCT. These modes can be switched between. Hereinafter, ultrasound tomography images and optical coherence tomography images will be referred to as IVUS images and OCT images, respectively. Furthermore, IVUS images and OCT images will be collectively referred to as tomography images and will correspond to medical images.

[0013] The imaging diagnostic apparatus 100 of the present embodiment includes an intravascular inspection apparatus 101, an angiography apparatus 102, an image processing apparatus 3, a display apparatus 4, and an input apparatus 5. The intravascular inspection apparatus 101 includes an imaging diagnostic catheter 1 and an MDU (Motor Drive Unit) 2. The imaging diagnostic catheter 1 is connected to the image processing apparatus 3 via the MDU 2. The display apparatus 4 and the input apparatus 5 are connected to the image processing apparatus 3. The display apparatus 4 is, for example, a liquid crystal display or an organic EL display, etc., and the input apparatus 5 is, for example, a keyboard, a mouse, a trackball, or a microphone, etc. The display apparatus 4 and the input apparatus 5 may be integrally laminated to form a touch panel. Also, the input apparatus 5 and the image processing apparatus 3 may be integrally configured. Furthermore, the input apparatus 5 may be a sensor that accepts gesture input or gaze input, etc.

[0014] The angiography apparatus 102 is connected to the image processing apparatus 3. The angiography apparatus 102 is an angiography device for obtaining an angiogram, which is a fluoroscopic image of a patient's blood vessel, by imaging the blood vessel using X-rays from outside the patient's body while injecting a contrast agent into the patient's blood vessel. The angiography apparatus 102 includes an X-ray source and an X-ray sensor, and the X-ray sensor receives the X-rays irradiated from the X-ray source to image a fluoroscopic image of the patient. Note that the imaging diagnostic catheter 1 is provided with a marker that does not transmit X-rays, and the position of the imaging diagnostic catheter 1 (marker) is visualized in the angiogram. The angiography apparatus 102 outputs the obtained angiogram to the image processing apparatus 3, and causes the angiogram to be displayed on the display apparatus 4 via the image processing apparatus 3. Note that the display apparatus 4 displays the angiogram and a tomographic image captured using the imaging diagnostic catheter 1.

[0015] Figure 2 is an explanatory diagram illustrating the overview of the imaging diagnostic catheter 1. Note that the upper dashed-dotted area in Figure 2 is an enlargement of the lower dashed-dotted area. The imaging diagnostic catheter 1 has a probe 11 and a connector portion 15 located at the end of the probe 11. The probe 11 is connected to the MDU 2 via the connector portion 15. In the following description, the side of the imaging diagnostic catheter 1 furthest from the connector portion 15 will be referred to as the tip side, and the side with the connector portion 15 will be referred to as the proximal end side. The probe 11 is equipped with a catheter sheath 11a, and its tip is provided with a guidewire insertion portion 14 through which a guidewire can be inserted. The guidewire insertion portion 14 constitutes a guidewire lumen and is used to receive a guidewire that has been previously inserted into the blood vessel and to guide the probe 11 to the affected area by the guidewire. The catheter sheath 11a forms a continuous tube from the connection portion with the guidewire insertion portion 14 to the connection portion with the connector portion 15. A shaft 13 is inserted inside the catheter sheath 11a, and a sensor unit 12 is connected to the tip of the shaft 13.

[0016] The sensor unit 12 has a housing 12d, and the tip side of the housing 12d is formed in a hemispherical shape to suppress friction and catching with the inner surface of the catheter sheath 11a. Inside the housing 12d, there are arranged an ultrasonic transmitting and receiving unit 12a (hereinafter referred to as the IVUS sensor 12a) that transmits ultrasonic waves into the blood vessel and receives reflected waves from inside the blood vessel, and an optical transmitting and receiving unit 12b (hereinafter referred to as the OCT sensor 12b) that transmits near-infrared light into the blood vessel and receives reflected light from inside the blood vessel. In the example shown in FIG. 2, the IVUS sensor 12a is provided on the tip side of the probe 11, and the OCT sensor 12b is provided on the base end side, and they are arranged at a distance x along the axial direction on the central axis of the shaft 13 (on the two-dot chain line in FIG. 2). In the catheter 1 for image diagnosis, the IVUS sensor 12a and the OCT sensor 12b are attached with the direction substantially perpendicular to the axial direction of the shaft 13 (the radial direction of the shaft 13) as the transmitting and receiving direction of ultrasonic waves or near-infrared light. It is desirable that the IVUS sensor 12a and the OCT sensor 12b are attached slightly offset from the radial direction so as not to receive reflected waves or reflected light on the inner surface of the catheter sheath 11a. In the present embodiment, for example, as shown by the arrow in FIG. 2, the IVUS sensor 12a is attached with the direction inclined toward the base end side with respect to the radial direction as the irradiation direction of ultrasonic waves, and the OCT sensor 12b is attached with the direction inclined toward the tip side with respect to the radial direction as the irradiation direction of near-infrared light.

[0017] An electrical signal cable (not shown) connected to the IVUS sensor 12a and an optical fiber cable (not shown) connected to the OCT sensor 12b are inserted into the shaft 13. The probe 11 is inserted into the blood vessel from the tip side. The sensor unit 12 and the shaft 13 can move forward and backward inside the catheter sheath 11a, and can also rotate in the circumferential direction. The sensor unit 12 and the shaft 13 rotate about the central axis of the shaft 13 as the rotation axis. In the image diagnostic apparatus 100, by using the imaging core constituted by the sensor unit 12 and the shaft 13, the state inside the blood vessel is measured by an ultrasonic tomographic image (IVUS image) taken from the inside of the blood vessel or an optical coherence tomographic image (OCT image) taken from the inside of the blood vessel.

[0018] The MDU2 is a drive device to which the probe 11 (imaging catheter 1) is detachably attached by a connector 15. It controls the operation of the imaging catheter 1 inserted into a blood vessel by driving a built-in motor in response to the operation of a medical professional. For example, the MDU2 performs a pullback operation, rotating the sensor unit 12 and shaft 13 inserted into the probe 11 circumferentially while pulling them toward the MDU2 at a constant speed. The sensor unit 12 rotates while moving from the tip to the proximal end due to the pullback operation, and continuously scans the inside of the blood vessel at predetermined time intervals, thereby continuously acquiring multiple transverse images approximately perpendicular to the probe 11 at predetermined intervals. The MDU2 outputs the reflected ultrasound wave data received by the IVUS sensor 12a and the reflected light data received by the OCT sensor 12b to the image processing device 3.

[0019] The image processing device 3 acquires a signal dataset, which is the reflected ultrasound wave data received by the IVUS sensor 12a via the MDU 2, and a signal dataset, which is the reflected light data received by the OCT sensor 12b. The image processing device 3 generates ultrasound line data from the ultrasound signal dataset and constructs an ultrasound tomographic image (IVUS image) of the transverse layer of the blood vessel based on the generated ultrasound line data. The image processing device 3 also generates optical line data from the reflected light signal dataset and constructs an optical tomographic image (OCT image) of the transverse layer of the blood vessel based on the generated optical line data. Here, the signal datasets acquired by the IVUS sensor 12a and the OCT sensor 12b, and the tomographic images constructed from the signal datasets will be explained. Figure 3 is an explanatory diagram showing a cross-section of a blood vessel through which the sensor unit 12 is inserted, and Figure 4 is an explanatory diagram explaining the tomographic image.

[0020] First, using Figure 3, we will explain the operation of the IVUS sensor 12a and OCT sensor 12b within a blood vessel, and the signal dataset (ultrasound line data and optical line data) acquired by the IVUS sensor 12a and OCT sensor 12b. When tomographic imaging is started with the imaging core inserted into the blood vessel, the imaging core rotates around the central axis of the shaft 13 in the direction indicated by the arrow. At this time, the IVUS sensor 12a transmits and receives ultrasound at each rotation angle. Lines 1, 2, ... 512 indicate the direction of ultrasound transmission and reception at each rotation angle. In this embodiment, the IVUS sensor 12a intermittently transmits and receives ultrasound 512 times while rotating 360 degrees (1 rotation) within the blood vessel. Since the IVUS sensor 12a acquires data for one line in the transmission and reception direction with each ultrasound transmission and reception, 512 ultrasound line data lines extending radially from the center of rotation can be obtained during one rotation. The 512 ultrasound line data are dense near the rotation center, but become sparser as they move away from the rotation center. Therefore, the image processing device 3 can generate a two-dimensional ultrasound tomography image (IVUS image) as shown in Figure 4A by generating pixels in the empty spaces of each line using a well-known interpolation process.

[0021] Similarly, the OCT sensor 12b also transmits and receives measurement light at each rotation angle. Since the OCT sensor 12b also transmits and receives measurement light 512 times while rotating 360 degrees within the blood vessel, 512 optical line data points extending radially from the center of rotation can be obtained during one rotation. With respect to the optical line data, the image processing device 3 can generate a two-dimensional optical coherence tomography (OCT) image similar to the IVUS image shown in Figure 4A by generating pixels in the empty spaces of each line using a well-known interpolation process. That is, the image processing device 3 generates optical line data based on interference light generated by interfering reflected light with, for example, reference light obtained by separating light from a light source within the image processing device 3, and constructs an optical tomography (OCT) image of the transverse layer of the blood vessel based on the generated optical line data.

[0022] The two-dimensional tomographic image generated from these 512 line data points is called one frame of an IVUS image or OCT image. Since the sensor unit 12 scans while moving within the blood vessel, one frame of an IVUS image or OCT image is acquired at each position after one rotation within the movement range. That is, one frame of an IVUS image or OCT image is acquired at each position from the tip to the proximal end of the probe 11 within the movement range, so as shown in Figure 4B, multiple frames of IVUS images or OCT images are acquired within the movement range.

[0023] The diagnostic imaging catheter 1 has X-ray-insensitive markers to confirm the positional relationship between the IVUS image obtained by the IVUS sensor 12a or the OCT image obtained by the OCT sensor 12b and the angiography image obtained by the angiography device 102. In the example shown in Figure 2, marker 14a is provided at the tip of the catheter sheath 11a, for example, at the guidewire insertion section 14, and marker 12c is provided on the shaft 13 side of the sensor section 12. When the diagnostic imaging catheter 1 configured in this way is imaged with X-rays, an angiography image is obtained in which markers 14a and 12c are visualized. The positions of markers 14a and 12c are just examples; marker 12c may be provided on the shaft 13 instead of the sensor section 12, and marker 14a may be provided at a location other than the tip of the catheter sheath 11a.

[0024] Figure 5 is a block diagram showing an example configuration of the image processing device 3. The image processing device 3 is a computer (information processing device) and includes a control unit 31, a main memory unit 32, an input / output interface 33, an auxiliary memory unit 34, and a reading unit 35.

[0025] The control unit 31 is configured using one or more computing devices such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-purpose computing on graphics processing units), and TPU (Tensor Processing Unit). The control unit 31 is connected to each hardware component of the image processing device 3 via a bus.

[0026] The main memory unit 32 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, and temporarily stores the data necessary for the control unit 31 to perform calculation processing.

[0027] The input / output interface 33 is an interface to which the endovascular examination device 101, the angiography device 102, the display device 4, and the input device 5 are connected. The control unit 31 acquires IVUS images and OCT images from the endovascular examination device 101 and angiography images from the angiography device 102 via the input / output interface 33. The control unit 31 also outputs medical image signals of the IVUS images, OCT images, or angiography images to the display device 4 via the input / output interface 33, thereby displaying the medical images on the display device 4. Furthermore, the control unit 31 receives information input to the input device 5 via the input / output interface 33.

[0028] A communication unit consisting of, for example, a wireless slave device such as 4G, 5G, or WiFi is connected to the input / output I / F 33, and the image processing device 3 may be connected to an external server such as a cloud server connected to an external network such as the Internet via the communication unit. The control unit 31 may access the external server via the communication unit and the external network, refer to medical data, research paper information, etc. stored (stored) in the storage device included in the external server, and perform processing related to information provision (processing to provide support information). Alternatively, the control unit 31 may perform the processing in this embodiment in cooperation with the external server, for example, by performing inter-process communication.

[0029] The auxiliary storage unit 34 is a storage device such as a hard disk, EEPROM (Electrically Erasable Programmable ROM), or flash memory. The auxiliary storage unit 34 stores the computer program P (program product) executed by the control unit 31 and various data necessary for the processing of the control unit 31. The auxiliary storage unit 34 may also be an external storage device connected to the image processing device 3. The computer program P (program product) may be written to the auxiliary storage unit 34 during the manufacturing stage of the image processing device 3, or it may be distributed by a remote server device and acquired by the image processing device 3 via communication and stored in the auxiliary storage unit 34. The computer program P (program product) may also be recorded in a readable form on a recording medium 30 such as a magnetic disk, optical disk, or semiconductor memory, and the reading unit 35 may read it from the recording medium 30 and store it in the auxiliary storage unit 34.

[0030] The image processing device 3 may be a multicomputer comprising multiple computers. Alternatively, the image processing device 3 may be a server-client system, a cloud server, or a virtual machine virtually constructed by software. In the following description, the image processing device 3 will be described as a single computer. In this embodiment, an angiography device 102 that acquires two-dimensional angiography images is connected to the image processing device 3, but it is not limited to the angiography device 102 as long as it is a device that images the patient's tubular organs and the diagnostic imaging catheter 1 from multiple directions outside the body.

[0031] In the image processing apparatus 3 of this embodiment, the control unit 31 reads and executes a computer program P stored in the auxiliary storage unit 34 to construct an IVUS image based on the signal dataset received from the IVUS sensor 12a and an OCT image based on the signal dataset received from the OCT sensor 12b. As will be described later, the IVUS sensor 12a and the OCT sensor 12b have a shift in their observation positions at the same imaging timing, so the control unit 31 performs a process to correct the shift in observation positions in the IVUS image and the OCT image. Therefore, the image processing apparatus 3 of this embodiment provides images that are easy to interpret by providing IVUS images and OCT images with aligned observation positions.

[0032] In this embodiment, the imaging catheter is described as a dual-type catheter equipped with both intravascular ultrasound (IVUS) and optical coherence tomography (OCT) functions, but is not limited to this. The imaging catheter may also be a single-type catheter equipped with either intravascular ultrasound (IVUS) or optical coherence tomography (OCT) functions. Hereafter, in this embodiment, the imaging catheter is assumed to have intravascular ultrasound (IVUS) functionality, and the description will be based on IVUS images generated by this IVUS function. However, in the description of this embodiment, the medical image is not limited to IVUS images, and OCT images may be used as medical images to perform the processing of this embodiment.

[0033] Figure 6 is an explanatory diagram showing an example of the learning model 341. The learning model 341 is a neural network that performs, for example, object detection, semantic segmentation, or instance segmentation. Based on each IVUS image in the input IVUS image set, the learning model 341 outputs whether or not an object such as a stent or plaque is contained in the IVUS image (presence or absence), and if an object is contained (presence or absence), the type (class) of the object, the region in the IVUS image, and the estimation accuracy (score).

[0034] The learning model 341 is composed of, for example, a pre-trained convolutional neural network (CNN) using deep learning. The learning model 341 has, for example, an input layer 341a into which a medical image such as an IVUS image is input, an intermediate layer 341b that extracts image features, and an output layer 341c that outputs information indicating the position and type of objects contained in the medical image. The input layer 341a of the learning model 341 has multiple neurons that accept input of the pixel value of each pixel contained in the medical image, and passes the input pixel value to the intermediate layer 341b. The intermediate layer 341b has a configuration in which a convolutional layer that convolves the pixel value of each pixel input to the input layer 341a and a pooling layer that maps the pixel value convolved in the convolutional layer are alternately connected, and extracts image features while compressing the pixel information of the medical image. The intermediate layer 341b passes the extracted features to the output layer 341c. The output layer 341c has one or more neurons that output the location, range, and type of image regions of objects contained in the image, for example, as a label image. The label image is, for example, an image where pixels corresponding to plaque regions are class "1" and pixels corresponding to other images are class "0". The learning model 341 is assumed to be a CNN, but the configuration of the learning model 341 is not limited to a CNN. The learning model 341 may be a pre-trained model with a configuration such as a neural network other than a CNN, an FCN (fully convolutional network) such as U-net, SegNet, SSD, SPPnet, SVM (Support Vector Machine), Bayesian network, or regression tree. Alternatively, the learning model 341 may perform object recognition by inputting image features output from the intermediate layer into an SVM (support vector machine). In this way, the learning model 341 has segmentation, object detection, classification, or regression functions, and outputs (estimates) information about objects contained in the medical image by performing segmentation, object detection, classification, or regression. Information about the object includes whether or not it exists, its type, its region, or its estimated accuracy (score).

[0035] The learning model 341 can be generated by preparing training data that associates medical images containing objects such as stents, plaques, calcified areas, and dissections with labels (label images) indicating the location (region) and type of each object, and then using this training data to machine-learn an untrained neural network. With the learning model 341 configured in this way, by inputting a medical image such as an IVUS image into the learning model 341, information indicating the location and type of objects contained in the medical image can be obtained. If the medical image does not contain objects, the information indicating the location and type is not output from the learning model 341. Therefore, by using the learning model 341, the control unit 31 can obtain whether or not an object is contained in the medical image input to the learning model 341, and if so, the type (class), location (region in the medical image), and estimation accuracy (score) of the object. In other words, by using the learning model 341 trained in this way, as shown in the figure in this embodiment, a label image indicating the region of the plaque on a pixel-by-pixel basis can be obtained by inputting an IVUS image into the learning model 341. In IVUS images, the region extracted as plaque (plaque region) is the region encompassing both the plaque itself (caused by cholesterol, etc.) and the tunica media. In other words, in this embodiment, plaque in IVUS images represents the region encompassing both the plaque and the tunica media.

[0036] The control unit 31 may input each IVUS image (frame image) to the learning model 341 one by one for processing, or it may input multiple consecutive frame images simultaneously and detect plaque regions from multiple frame images simultaneously. For example, the control unit 31 uses a 3D-CNN (e.g., 3D U-net) as the learning model 341 to handle 3D input data. The control unit 31 then treats the 2D frame images as 3D data with the coordinates of the 2D frame images as the 2 axes and the time (generation time) t when each frame image was acquired as the 1 axis. The control unit 31 inputs multiple frame images (e.g., 16 frames) for a predetermined unit time as a set to the learning model 341 and simultaneously outputs images with labels indicating the plaque regions for each of the multiple frame images. This makes it possible to detect plaque regions by considering consecutive frame images before and after in the time series, thereby improving detection accuracy.

[0037] Based on this information obtained from the learning model 341, the control unit 31 derives object information relating to the types and regions of objects included in the IVUS image. Alternatively, the control unit 31 may derive the object information by using the information obtained from the learning model 341 itself as object information.

[0038] Figure 7 is an explanatory diagram regarding the extraction of the second medical image. Figure 8 is an explanatory diagram regarding the representative value (plaque bathenness) of the second medical image. Figure 9 is an explanatory diagram regarding the range determined based on the reference image. These figures in this embodiment will be used to explain the axial position of the blood vessels (luminal organs) corresponding to each of the generated IVUS images (medical images).

[0039] During the pullback operation, the sensor unit 12 is pulled back by the MDU2 at a constant speed (e.g., 0.5 mm / sec or 1.0 mm / sec) from the distal to the proximal side. The spatial resolution of the imaging core in IVUS image generation is, for example, 100 to 200 μm. For example, if the pullback speed is 1.0 mm / sec and the spatial resolution is 100 μm, 10 IVUS images will be generated per second. In this case, the length of the blood vessels included in these 10 IVUS images will be 1 mm. That is, these 10 IVUS images will show cross-sectional views (longitudinal views) of blood vessels at distances of 0.1 mm (separation distance). Furthermore, these 10 IVUS images will be generated in units of 0.1 seconds. Thus, the time at which the IVUS image is generated (generation time) corresponds to the axial position of the blood vessel shown in the IVUS image (axial position).

[0040] When the generated IVUS images are aligned along the axial direction of the blood vessel (luminal organ) to show the position of the vessel, these multiple IVUS images will be positioned according to the spacing distance determined based on the pullback velocity and spatial resolution. This allows the axial position of each of the multiple IVUS images to be identified. In other words, the position of the generated IVUS image indicates the axial position of the blood vessel contained within that IVUS image.

[0041] If the pullback length is 10 cm (100 mm), 1000 IVUS images will be generated during a 10-second pullback time. All IVUS images (medical images) acquired in a single pullback scan are designated as the first IVUS image (first medical image). For example, in the first IVUS images, which are arranged along the axis of the blood vessel (luminal organ) with a separation distance of 0.1 mm, multiple IVUS images are extracted at predetermined intervals of, for example, 1 mm, and these extracted IVUS images are designated as the second IVUS image. In other words, the first IVUS image is composed of the IVUS images extracted as the second IVUS image and the remaining IVUS images that were not extracted as the second IVUS image (remaining IVUS images).

[0042] From the perspective of the time of generation, the second IVUS images extracted at predetermined intervals of 1 mm are extracted at time intervals of 1 second. In other words, the IVUS images extracted from the first IVUS images (all generated IVUS images) at time intervals of 1 second (sampling period of 1 second) correspond to the second IVUS images. In this way, by extracting the second IVUS images from all IVUS images (first IVUS images) at predetermined distance intervals such as 1 mm (predetermined time intervals such as 1 second), it is possible to efficiently extract the second IVUS images over the entire length of the blood vessel in which the pullback operation is performed. Since the distance interval for extraction is set to approximately 1 mm, even if stent varieties with stent lengths in 3 mm increments are available, it is possible to select the appropriate stent variety.

[0043] As described above, multiple second IVUS images correspond to a portion of all IVUS images (first IVUS images) generated in a single pullback. For example, by extracting at predetermined intervals of 1 mm (at a spatial resolution of 100 μm), the number of second IVUS images becomes one-tenth (1 / 10) of the number of first IVUS images (all generated IVUS images).

[0044] Figure 7 shows secondary IVUS images extracted at, for example, 1 mm intervals from the blood vessel targeted by the pullback operation. Between two adjacent secondary IVUS images, nine primary IVUS images (primary IVUS images that were not extracted as secondary IVUS images in all generated IVUS images) are located. Each of the extracted secondary IVUS images is input into the learning model 341 for segmentation processing, and the regions of objects such as plaques in each of the secondary IVUS images are identified.

[0045] The control unit 31 calculates (derives) the plaque bathen (plaque cross-sectional area ratio), which is the ratio of the area (cross-sectional area) of the plaque region to the cross-sectional area of ​​the vessel, for each of the second IVUS images. In the IVUS image, the vessel represents the region surrounded by the external elastic plate. The control unit 31 may use the plaque bathen as a representative value for each of the second IVUS images, generate a graph as shown in the figure of this embodiment, and display the generated graph on the display device 4.

[0046] The control unit 31 identifies the largest plaque bathden (representative value) in each second IVUS image and identifies the second IVUS image containing the largest plaque bathden as the reference image. Since the reference image is the IVUS image containing the largest plaque bathden, it is an IVUS image that is highly likely to be used by physicians and others in making decisions when analyzing lesions. By performing this processing, the identified reference image and its position (reference point) can be provided to physicians and others at an early stage.

[0047] The control unit 31 identifies the distal image and the proximal image from the second IVUS images located distally and proximal, respectively, using the position of the reference image as a reference point. The control unit 31 may, for example, identify the second IVUS images with the smallest plaque bathen (representative value) within a predetermined range centered on the position of the reference image (reference point) as the distal image and the proximal image.

[0048] A reference image is located between the identified distal image and the proximal image. The control unit 31 may define the range from the distal image to the proximal image as a priority processing range (high priority range) and define the range other than the range from the distal image to the proximal image as a low priority range. In this embodiment, the low priority range includes two ranges: a range that is more proximal to the proximal image and a range that is more proximal to the distal image.

[0049] The control unit 31 identifies a plurality of first IVUS images located within the range from the distal image to the proximal image, which is the priority processing range (high priority range), and inputs these identified first IVUS images into the learning model 341. The IVUS images located within the range from the distal image to the proximal image include second IVUS images that have already been input into the learning model 341 and first IVUS images that were not extracted as second IVUS images. Therefore, the control unit 31 may input only the first IVUS images located within the range from the distal image to the proximal image that were not extracted as second IVUS images into the learning model 341.

[0050] When the control unit 31 inputs first IVUS images located in the range from the distal image to the proximal image to the learning model 341, it may input the first IVUS images to the learning model 341 in order of proximity to the reference image. In this case, either the distal or proximal first IVUS image adjacent to the reference image is input to the learning model 341 first, and either the distal or proximal first IVUS image adjacent to the distal image is input to the learning model 341 last. Among multiple first IVUS images located in the range from the distal image to the proximal image, it is assumed that the first IVUS images closer to the reference image contain more information about the lesion. Therefore, by sequentially inputting the first IVUS images to the learning model 341 in order of proximity to the reference image, support information can be efficiently provided to physicians and other medical professionals.

[0051] The control unit 31 may output information about objects such as plaques included in the first IVUS image located in the range from the distal image to the proximal image by acquiring estimation results from the learning model 341, and display this information on the display device 4. The information about objects may include, for example, an image showing the area of ​​plaque included in each first IVUS image, plaque barden, vessel diameter or area (Vessel), lumen diameter or area (Lumen), and stenosis rate.

[0052] The control unit 31 may perform processing such as inputting the first IVUS image located in the range from the distal image to the proximal image (high priority range) into the learning model 341, and then perform processing such as inputting the first IVUS image located in a range other than the range from the distal image to the proximal image (low priority range) into the learning model 341. In this embodiment, plaque bathen was used as a representative value in the IVUS image (second IVUS image), but the control unit 31 is not limited to this. For example, the control unit 31 may identify a reference screen etc. by using measured values ​​derived based on the type of object, such as the ratio of diameter or area of ​​vessel (Vessel) to lumen diameter (Lumen / Vessel), stenosis rate, stent, dissection, calcification, etc., included in the second IVUS image, as a representative value. When these measured values ​​etc. are used as representative values, the reference screen may be identified based on the minimum value.

[0053] Figure 10 is a flowchart showing the information processing procedure by the control unit 31. The control unit 31 of the image processing device 3 performs the following processing based on the input data output from the input device 5 in response to the operation of the operator of the diagnostic imaging catheter, such as a physician.

[0054] The control unit 31 acquires multiple first IVUS images generated along the axial direction of a tubular organ such as a blood vessel (S101). The control unit 31 acquires a medical image consisting of these IVUS images by reading all the IVUS images (first IVUS images) obtained by pullback.

[0055] The control unit 31 extracts multiple second IVUS images from multiple first IVUS images at predetermined intervals in the axial direction of the tubular organ (S102). The control unit 31 extracts multiple IVUS images from multiple first IVUS images (all IVUS images) at predetermined intervals of, for example, 1 mm in the axial direction of the blood vessel (tubular organ), and sets the extracted multiple IVUS images as second IVUS images. If the spatial resolution of the imaging core in the generation of IVUS images is, for example, 100 μm (0.1 mm), then one-tenth (1 / 10) the number of second IVUS images (number) of the number of first IVUS images (all IVUS images) will be extracted. In this embodiment, the predetermined interval is set to 1 mm (the time interval is 1 second), but it is not limited to this. The image processing device 3 may also be equipped with a selection unit that accepts selection by a physician or the like within a predetermined range, such as 0.5 mm to 1.5 mm (0.5 seconds to 1.5 seconds), thereby enabling the selection of the predetermined interval. The value of the predetermined interval for extracting the second IVUS image may be stored in the auxiliary storage unit 34 of the image processing device 3.

[0056] The control unit 31 inputs the extracted secondary IVUS images into the learning model 341 (S103). The control unit 31 inputs the extracted secondary IVUS images, for example, at predetermined intervals of 1 mm, into the learning model 341 which has the function of a segmentation network.

[0057] The control unit 31 identifies the type and region of objects contained in each of the second IVUS images based on the estimation results output from the learning model 341 (S104). The learning model 341 outputs information regarding the type (class), region, and class accuracy (estimation accuracy) of objects such as plaque contained in the input IVUS images as estimation results. Alternatively, the learning model 341 may identify the external elastic lamina (Vessel) and lumen (Lumen) of blood vessels contained in the IVUS images as object types, and identify plaque interposed between them based on the external elastic lamina (Vessel) and lumen (Lumen) of the blood vessels. For example, if the object type is plaque, the control unit 31 identifies the region of plaque contained in each of the second IVUS images based on the estimation results output from the learning model 341.

[0058] The control unit 31 derives representative values ​​for each of the multiple second IVUS images based on the type and region of the identified object (S105). The control unit 31 derives plaque bathen (cross-sectional area ratio of plaque) calculated based on the identified plaque, etc., as a representative value for each of the multiple second IVUS images. In this embodiment, plaque bathen was used as a representative value for the IVUS image (second IVUS image), but it is not limited to this, and for example, measured values ​​derived based on the type of object such as the ratio of diameter or area between the vessel (Vessel) and the lumen diameter (Lumen) (Lumen / Vessel), stenosis rate, stent, dissection, calcified area, etc., may be used as representative values.

[0059] The control unit 31 determines a reference image from a plurality of second IVUS images based on the derived representative values ​​(S106). Based on the representative values ​​(plaque bathen) derived from each of the plurality of second IVUS images, the control unit 31 determines, for example, the second IVUS image with the largest representative value (plaque bathen) as the reference image.

[0060] The control unit 31 determines the distal and proximal images based on the derived representative values ​​and reference images (S107). The control unit 31 uses the axial position of the blood vessel (luminal organ) indicated by the reference image as a reference point, and determines the second IVUS image with the smallest representative value (plaque barden) among the second IVUS images for the distal and proximal sides as the distal and proximal images, respectively. When determining the distal and proximal images, the control unit 31 may determine the distal and proximal images within a predetermined range centered on the reference point determined by the reference image.

[0061] The control unit 31 inputs the first IVUS images located between the distal image and the proximal image to the learning model 341 (S108). The area between the distal image and the proximal image contains multiple first IVUS images that were not extracted as second IVUS images. Since the reference image is included in the range from the distal image to the proximal image, this range corresponds to a high-priority range that is processed preferentially. The control unit 31 may input only the first IVUS images located in the range from the distal image to the proximal image that were not extracted as second IVUS images to the learning model 341.

[0062] Based on the estimation results output from the learning model 341, the control unit 31 identifies the type and region of objects contained in each of the first IVUS images located between the distal image and the proximal image (S109). Based on the estimation results output from the learning model 341, the control unit 31 identifies the region of plaque contained in each of the first IVUS images located within the range from the distal image to the proximal image (high priority range).

[0063] The control unit 31 outputs information (such as representative values) regarding the type and region of the identified object (S110). The control unit 31 outputs information regarding the type and region of objects such as plaque identified in each of the first IVUS images located in the high-priority range, including the reference image, to the display device 4, and displays the information on the display device 4. When outputting the information, the control unit 31 may include in the information measured values ​​derived based on the type of object, such as plaque barden, the ratio of diameter or area between the vessel (Vessel) and the lumen diameter (Lumen / Vessel), stenosis rate, stent, dissection, and calcification, in each of the first IVUS images located in the high-priority range. After processing the first IVUS images located in the range from the distal image to the proximal image (high-priority range) by inputting them into the learning model 341, the control unit 31 may also perform the same processing as for the first IVUS images located in a range other than the range from the distal image to the proximal image (low-priority range), such as inputting them into the learning model 341.

[0064] In the flowchart of this embodiment, the control unit 31 is described as performing the processes from S101 to S104 sequentially, but it is not limited to this. The control unit 31 may, in parallel with the process of generating and acquiring the first IVUS image by performing pullback, that is, during the pullback operation, extract second IVUS images at predetermined intervals from a plurality of already generated first IVUS images, and perform processing such as segmentation by inputting the second IVUS images into the learning model 341 to identify the type and region of the object. In other words, the control unit 31 may extract second IVUS images at predetermined intervals while performing pullback and input them into the learning model 341 to proceed with segmentation. The control unit 31 may also perform the processes from S101 to S105 in parallel with the pullback.

[0065] Figure 11 is a flowchart showing the procedure for selecting the stent size. The control unit 31 of the image processing device 3 performs the following processing based on the input data output from the input device 5 in response to the operation of the operator of the diagnostic imaging catheter, such as a physician. Based on the information (representative values, etc.) regarding the type and region of objects included in the first IVUS image derived in the processes S101 to S110 described above, the control unit 31 performs the following processing related to the selection of the stent size.

[0066] The control unit 31 acquires information (such as representative values) regarding the type and region of the identified object (T101). The control unit 31 uses the output result of processing S110 in the flow described above to acquire information (such as representative values) regarding the type and region of the identified object in the first IVUS image located in the range from the distal image to the proximal image (high priority range). The control unit 31 may also use the acquired information as input factors to launch an application (stent selection app) for selecting a stent size.

[0067] The control unit 31 determines whether an internal organ is abnormal based on representative values ​​of the reference image, etc. (T102). If the representative value is plaque bathden, the control unit 31 determines whether an internal organ shown in the IVUS image (reference image) is abnormal based on whether the plaque bathden is above a predetermined value, such as 50% or more. Alternatively, the control unit 31 may determine whether an internal organ is abnormal in the internal organ shown in the reference image, depending on the vascular characteristics, presence or absence of complication risk findings, plaque distribution, calcification, side branches, or remodeling.

[0068] If there is an abnormality in the internal organ (T102: YES), the control unit 31 determines the stent placement site based on the position of the reference image, etc. (T103). The control unit 31 determines a candidate stent length (T104). The control unit 31 determines the stent placement site (landing zone) based on information such as representative values ​​of each IVUS image (first IVUS image), including the reference image. The control unit 31 determines a candidate stent length based on the determined stent placement site.

[0069] The control unit 31 determines the range of the reference vessel (T105). The control unit 31 determines the range of the reference vessel to be within 1 cm before and after the lesion (distal and proximal to the reference point) included in the reference image, for example.

[0070] The control unit 31 determines the reference vessel diameter (T106). Within the range of the determined reference vessel, the control unit 31 determines the maximum value of the vessel diameter as the reference vessel diameter.

[0071] The control unit 31 determines a candidate stent diameter (T107). For example, the control unit 31 determines a candidate stent diameter of 0.9 times the determined reference vessel diameter. Alternatively, the control unit 31 may determine a candidate stent diameter based on a representative value calculated from the distal reference lumen diameter or the lumen diameter or vessel diameter of the mid-wall, etc.

[0072] If there are no abnormalities in the internal organ (T102:NO), or after the execution of T104 and T107, the control unit 31 determines a candidate stent type (T108). The control unit 31 determines a candidate stent type with an appropriate size by selecting from pre-prepared stent types according to the determined candidate stent diameter and stent length.

[0073] According to this embodiment, the image processing device 3 extracts multiple second medical images (second IVUS images) that are part of the first medical image from a plurality of acquired first medical images (first IVUS images), and inputs each of the extracted second medical images into the learning model 341. The image processing device 3 may, for example, extract multiple second medical images at predetermined intervals of 1 mm or so in the axial direction of the tubular organ. By extracting in this way, the number of second medical images can be made less than the number of first medical images. Therefore, by inputting a number of second medical images that is less than the number of first medical images into the learning model 341, a reference image can be determined, reducing the computational load when using the learning model 341 and reducing the time required to determine the reference image.

[0074] The reference image is determined based on the type and region of objects identified by the estimation results of the learning model 341, and therefore includes internal body parts used for decision-making when physicians and other operators of diagnostic imaging catheters perform lesion analysis. The image processing device 3 inputs first medical images within a predetermined range in the axial direction of the tubular organ from the reference image to the learning model 341. Therefore, it prioritizes inputting first medical images within a predetermined range including the reference image, such as those near the reference image, into the learning model 341, and can efficiently detect information about objects contained in these first medical images. By performing this processing, the computational load can be reduced compared to, for example, inputting all medical images generated using a diagnostic imaging catheter uniformly into the learning model 341, and support information for physicians and other operators can be provided early by prioritizing the input of some medical images within a predetermined range including the reference image into the learning model 341.

[0075] In this embodiment, the input to the learning model 341 is not limited to only a portion of medical images that fall within a predetermined range, including the reference image. After inputting the portion of medical images into the learning model 341, the remaining medical images (medical images other than the portion of medical images) may be input to the learning model 341 to identify the types and regions of objects in these remaining medical images. In other words, the processing priority for these remaining medical images may be set lower than the processing priority for the portion of medical images that fall within a predetermined range, including the reference image.

[0076] According to this embodiment, the image processing device 3 identifies (segments) the external elastic lamina (Vessel) and lumen (Lumen) of a blood vessel as object types, and calculates the plaque area ratio (plaque bathen) based on the segmented regions of the external elastic lamina (Vessel) and lumen (Lumen) of the blood vessel. The image processing device 3 uses the plaque bathen as a representative value for each of the second medical images, and determines a reference image based on this representative value, thereby enabling efficient determination of a reference image including internal body parts necessary for decision-making when physicians perform lesion analysis, etc. In this embodiment, plaque bathen is used as a representative value, but the device is not limited to this, and for example, a stenosis rate using a predetermined reference value may be used as a representative value.

[0077] According to this embodiment, the second medical image with the largest representative value of plaque bathen is determined as the reference image, allowing physicians and others to efficiently determine a reference image that includes internal body parts necessary for decision-making when performing lesion analysis, etc. With respect to the reference image, the second medical image with the smallest representative value among the second medical images located distally and proximal to each other is determined as the distal and proximal images, allowing for efficient identification of the range including the reference image.

[0078] According to this embodiment, the necessity of a stent is determined based on a representative value indicating plaque bathenness. For example, if the plaque bathenness (representative value) is 50% or more, the image processing device 3 identifies candidate stent lengths based on the distance from the distal image to the proximal image, that is, the distance from the site where the distal image was generated to the site where the proximal image was generated. Furthermore, the image processing device 3 identifies candidate stent diameters based on the smaller of the lumen diameters of the distal and proximal images. The candidate stent diameter may be identified, for example, as 90% (0.9 times) of the smaller lumen diameter. In this way, the image processing device 3 can efficiently identify candidate stent sizes based on representative values ​​and the distal and proximal images, and can output information regarding the identified candidate stent sizes to support physicians and others in analyzing lesions.

[0079] (Embodiment 2) Figure 12 is an explanatory diagram relating to the range determined based on the reference image in Embodiment 2. In Embodiment 2, the control unit 31 identifies the reference image based on the rate of change of the plaque baden (typical value) of each second IVUS image.

[0080] The control unit 31 may derive an approximate curve equation for the plaque baden (representative value) identified according to the position of each second IVUS image, and identify a reference image based on the position where the derivative value in the approximate curve equation becomes 0 (the point of change where the small change changes from positive to negative). In other words, the control unit 31 may determine areas with large fluctuations in neighboring values ​​from the derivative value, etc., and specify the detection range in detail and the range to be detected coarsely.

[0081] If the control unit 31 identifies multiple reference images based on the calculated differential values, it may determine the priority of these reference images (reference points) based on the magnitude of the plaque baden (representative value). In this embodiment, for example, two points are included where the differential value is 0 (a point of change where the small change changes from positive to negative). In this case, the control unit 31 may determine the reference image with a large plaque baden (representative value) as the reference point for determining the high-priority range, and the reference image with a small plaque baden (representative value) as the reference point for determining the medium-priority range.

[0082] The control unit 31 may perform processing on each reference image (reference point) according to its priority if there are multiple reference images (reference points) with different priorities. That is, the control unit 31 may input a first IVUS image of the range based on the highest priority (high priority) reference image (reference point) to the learning model 341, and then input a first IVUS image of the range based on the next highest priority (medium priority) reference image (reference point) to the learning model 341.

[0083] If a stent has been placed, the control unit 31 may perform stent detection and scan only the area containing the stent in detail using segmentation by the learning model 341. In processing findings related to the risk of complications, vascular dissections, etc., may be missed at wide intervals. Therefore, if findings such as dissections are found in the IVUS image (second IVUS image), the vicinity may be scanned in detail at intervals of 0.1 mm or similar. Furthermore, if areas with a risk of complications are known in advance from the angiography image, the range determined automatically or manually (acceptance of operations by the operator) may be scanned in detail using segmentation by the learning model 341.

[0084] According to this embodiment, by determining the areas with large fluctuations in neighboring values ​​for the representative values ​​of each of the second IVUS images from the derivative values, etc., the range to be preferentially input into the learning model 341 and scanned using segmentation, etc., can be determined, thereby efficiently identifying the IVUS images (first IVUS images) to be processed preferentially.

[0085] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The technical features described in each embodiment can be combined with each other, and the scope of the invention is intended to include all modifications within the claims and equivalents thereof. [Explanation of Symbols]

[0086] 1. Diagnostic imaging catheter 11 probes 11a Catheter Sheath 12 Sensor section 12a Ultrasonic Transceiver Unit 12a IVUS sensor 12b Optical Transceiver Unit 12b OCT sensor 12c marker 12d Housing 13 shafts 14 Guide wire insertion section 14a Marker 15 Connector section 2 MDU 3. Image Processing Device (Information Processing Device) 30 Recording media 31 Control Unit 32 Main memory 33 Input / Output Interfaces 34 Auxiliary storage section 341 Learning Models 35 Reading section 4 Display device 5 Input devices 100 Imaging diagnostic equipment 101 Intravascular Examination Device 102 Angiography equipment

Claims

1. On the computer, Based on signals detected by a catheter inserted into a tubular organ, multiple first medical images are acquired along the axial direction of the tubular organ. From the acquired plurality of first medical images, a plurality of second medical images are extracted that are spaced at predetermined intervals in the axial direction of the tubular organ. By inputting the extracted plurality of second medical images into a learning model that outputs information about objects contained in a medical image when a medical image is input, the types and regions of objects contained in the extracted plurality of second medical images are identified. Based on the identified object type and region, a reference image is determined from the plurality of second medical images. From the determined reference image, a first medical image within a predetermined range in the axial direction of the tubular organ is input to the learning model. Based on the identified object type and region, representative values ​​are derived for each of the plurality of second medical images. Based on the representative values ​​of the plurality of second medical images, one of the plurality of second medical images is determined to be the reference image. In the axial direction of the tubular organ, among the plurality of second medical images located distal to the reference image, the second medical image with the smallest representative value is determined to be the distal image. Among the multiple second medical images located proximal to the aforementioned reference image, the second medical image with the smallest representative value is determined to be the proximal image. A first medical image located between the distal image and the proximal image is input to the learning model. A computer program designed to execute a process.

2. The aforementioned tubular organ is a blood vessel, The identified type of object includes plaque interposed between the external elastic lamina and the lumen of the blood vessel. The aforementioned representative value is a plaque bathen, which is identified based on the area of ​​the plaque. The computer program according to claim 1.

3. Among the plurality of second medical images, the second medical image with the largest representative value is determined to be the reference image. The computer program according to claim 1 or claim 2.

4. Based on the aforementioned representative values, the distal image, and the proximal image, candidate stent sizes to be inserted into the tubular organ are identified. Output information regarding the identified candidate stent sizes. A computer program according to any one of claims 1 to 3.

5. Based on signals detected by a catheter inserted into a tubular organ, a plurality of first medical images generated along the axial direction of the tubular organ are acquired and stored in a storage unit. From the plurality of first medical images stored in the memory unit, a plurality of second medical images are extracted that are spaced at predetermined intervals in the axial direction of the tubular organ. By inputting the extracted plurality of second medical images into a learning model that outputs information about objects contained in a medical image when a medical image is input, the types and regions of objects contained in the extracted plurality of second medical images are identified. Based on the identified object type and region, a reference image is determined from the plurality of second medical images. From the determined reference image, a first medical image within a predetermined range in the axial direction of the tubular organ is input to the learning model. Based on the identified object type and region, representative values ​​are derived for each of the plurality of second medical images. Based on the representative values ​​of the plurality of second medical images, one of the plurality of second medical images is determined to be the reference image. In the axial direction of the tubular organ, among the plurality of second medical images located distal to the reference image, the second medical image with the smallest representative value is determined to be the distal image. Among the multiple second medical images located proximal to the aforementioned reference image, the second medical image with the smallest representative value is determined to be the proximal image. A first medical image located between the distal image and the proximal image is input to the learning model. An information processing method in which a computer performs the processing.

6. An acquisition unit acquires multiple first medical images generated along the axial direction of a tubular organ based on signals detected by a catheter inserted into the tubular organ, An extraction unit extracts a plurality of second medical images from the acquired plurality of first medical images, which are spaced at predetermined intervals in the axial direction of the tubular organ. The extracted plurality of second medical images are input to a learning model that outputs information about objects contained in a medical image when a medical image is input, thereby identifying the type and region of objects contained in the extracted plurality of second medical images, A determination unit that determines a reference image from the plurality of second medical images based on the type and region of the identified object, The system includes an input unit that inputs a first medical image within a predetermined range in the axial direction of the tubular organ from the determined reference image into the learning model. Based on the identified object type and region, representative values ​​are derived for each of the plurality of second medical images. Based on the representative values ​​of the plurality of second medical images, one of the plurality of second medical images is determined to be the reference image. In the axial direction of the tubular organ, among the plurality of second medical images located distal to the reference image, the second medical image with the smallest representative value is determined to be the distal image. Among the multiple second medical images located proximal to the aforementioned reference image, the second medical image with the smallest representative value is determined to be the proximal image. A first medical image located between the distal image and the proximal image is input to the learning model. Information processing device.

Images