Information processing device, method of operating the information processing device, and program

Abstract

An information processing device according to the present invention comprises: an acquisition unit that acquires a polar coordinate image, which is a medical image of polar coordinate expression obtained by imaging a biological lumen by using a device inserted into the biological lumen, and which has a first axis as an angle and a second axis intersecting the first axis as the distance from the device; an output unit that, when the polar coordinate image has been inputted, inputs a polar coordinate image of a prescribed angle portion exceeding 360° into a trained model such that first segment data obtained by dividing an image region corresponding to a specific object and another image will be outputted, and outputs the first segment data of the prescribed angle portion; an extraction unit that extracts first segment data of a 360° portion from the first segment data of the prescribed angle portion; and a conversion unit that converts the extracted first segment data into second segment data of an orthogonal coordinate expression.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a method of operating the information processing apparatus. and Progra Mu Thereof.

Background Art

[0002] With the recent progress of artificial intelligence technology, techniques for processing medical images using artificial intelligence have been proposed. For example, in Patent Document 1, a system for predicting a boundary position such as a blood vessel lumen from a medical image of a coronary artery using a CNN (Convolution Neural Network) that performs segmentation of an input image is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, there is also image data expressed in a polar coordinate system like an IVUS (Intravascular Ultrasound) image in medical images. When displaying such a medical image, it is displayed after being converted into a rectangular coordinate system (XY coordinate system) corresponding to the real space. In this case, there is a problem that discontinuous portions may occur in the image expressed in rectangular coordinates due to the coordinate system conversion.

[0005] The invention according to Patent Document 1 performs segmentation after converting an image expressed in polar coordinates into an image expressed in rectangular coordinates. However, due to the above problem, there is a risk that the boundary position cannot be accurately predicted.

[0006] One aspect of this invention is to provide an information processing device that can suitably predict image regions corresponding to specific objects from medical images. [Means for solving the problem]

[0007] An information processing device relating to one aspect includes an acquisition unit that acquires a polar coordinate medical image of a biological lumen, which is an image of the biological lumen taken using a device inserted into the biological lumen, where the first axis is the angle and the second axis intersecting the first axis is the distance from the device; an output unit that, when the polar coordinate image is input to a model trained to output first segment data that separates an image region corresponding to a specific object from other image regions, inputs the polar coordinate image for a predetermined angle exceeding 360 degrees and outputs first segment data for the predetermined angle; and first segment data for the predetermined angle Both ends from By removing the first segment data for angles exceeding 360 degrees, The system includes an extraction unit that extracts first segment data for 360 degrees, and a conversion unit that converts the extracted first segment data into second segment data in orthogonal coordinate representation. [Effects of the Invention]

[0008] In one respect, it is possible to suitably predict the image region corresponding to a specific object from a medical image. [Brief explanation of the drawing]

[0009] [Figure 1] This is an explanatory diagram showing an example configuration of an image diagnostic system. [Figure 2] This is a block diagram showing an example server configuration. [Figure 3] This is a block diagram showing an example of the configuration of a medical imaging device. [Figure 4] This is an explanatory diagram illustrating the relationship between polar coordinate images and vascular tomographic images. [Figure 5A] This is an explanatory diagram regarding image discontinuities caused by coordinate system transformations. [Figure 5B] This is an explanatory diagram regarding image discontinuities caused by coordinate system transformations. [Figure 6] This is an explanatory diagram regarding object identification processing. [Figure 7] This is an explanatory diagram regarding the display processing of vascular tomographic images. [Figure 8] This is an explanatory diagram regarding the process of generating a discriminative model. [Figure 9] This is a flowchart showing the steps involved in generating the identification model. [Figure 10] This is a flowchart showing the procedure for object identification processing. [Figure 11] This is a flowchart showing the procedure for object identification processing according to Embodiment 2. [Figure 12] This is a flowchart showing the procedure for object identification processing according to Embodiment 3. [Modes for carrying out the invention]

[0010] The present invention will be described in detail below with reference to the drawings illustrating its embodiments. (Embodiment 1) Figure 1 is an explanatory diagram showing an example configuration of an image diagnostic system. This embodiment describes an image diagnostic system that predicts an image region (hereinafter referred to as the "object region") corresponding to a specific object within a medical image, obtained by imaging a blood vessel (biological lumen). The image diagnostic system includes a server 1 and an image diagnostic device (information processing device) 2. Each device is connected via a network N such as the Internet.

[0011] In this embodiment, blood vessels are given as an example of a biological lumen, but the biological lumen to be studied is not limited to blood vessels, and may be other biological lumen such as the bile duct, pancreatic duct, bronchi, or intestines.

[0012] Server 1 is a server computer capable of various information processing and information transmission and reception. Note that the device corresponding to Server 1 may be a device such as a personal computer. Server 1 performs machine learning to learn predetermined training data and functions as a generation device that generates an identification model 50 (see FIG. 8) for identifying an object region with a medical image obtained by imaging blood vessels as an input. As will be described later, the identification model 50 is a machine learning model that tasks image segmentation (semantic classification of image regions) and outputs segment data that classifies an object region in a medical image and other image regions.

[0013] The image diagnostic apparatus 2 is an imaging device that images a medical image of a patient's blood vessels, and is, for example, an IVUS device that performs an ultrasonic examination using a catheter 201. The catheter 201 is a medical instrument inserted into a patient's blood vessel, and an ultrasonic probe for transmitting and receiving ultrasonic signals is attached to the tip. For example, the ultrasonic probe is configured to be rotatable in the circumferential direction of the catheter 201 and movable in the axial direction of the blood vessel. The image diagnostic apparatus 2 transmits an ultrasonic signal from the ultrasonic probe, receives a reflected wave, and generates and displays an ultrasonic tomographic image based on the received reflected wave.

[0014] In the present embodiment, the image diagnostic apparatus 2 is described as an IVUS device, but it may be an optical imaging device such as an OCT (Optical Coherence Tomography) device.

[0015] The data of the identification model 50 generated by the server 1 is installed in the image diagnostic apparatus 2. The image diagnostic apparatus 2 inputs an image captured using the catheter 201 into the identification model 50 and identifies an object region corresponding to a specific object (EEM (External Elastic Membrane), lumen, etc.). Then, the image diagnostic apparatus 2 displays a blood vessel tomographic image in which the region can be identified.

[0016] In this case, the diagnostic imaging device 2 identifies the object region from a polar coordinate image in which the first axis is the rotation angle of the ultrasound probe (device) and the second axis is the distance from the ultrasound probe. As will be described in detail later, the image data that the diagnostic imaging device 2 primarily obtains by transmitting and receiving ultrasound signals is not the tomographic image in the final Cartesian coordinate system (so-called B-mode image), but an image (or numerical array) represented in polar coordinates (see Figure 4). The diagnostic imaging device 2 inputs the polar coordinate image into the identification model 50 and obtains segment data that divides the object region as output. The diagnostic imaging device 2 converts the segment data obtained from the identification model 50 into segment data represented in Cartesian coordinates and identifies the object region in the final tomographic image.

[0017] In this embodiment, the image diagnostic device 2 identifies object regions using the identification model 50, but the server 1 on the cloud may identify the object regions. Alternatively, a general-purpose computer (such as a personal computer) connected to the image diagnostic device 2 may perform the processing. Thus, the processing entity that performs the series of processes is not particularly limited.

[0018] Figure 2 is a block diagram showing an example configuration of Server 1. Server 1 comprises a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary memory unit 14. The control unit 11 has one or more arithmetic processing units such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), and GPU (Graphics Processing Unit), and performs various information processing, control processing, etc. by reading and executing the program P1 stored in the auxiliary storage unit 14. The main memory unit 12 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, and temporarily stores the data necessary for the control unit 11 to perform arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing, and sends and receives information with the outside. The auxiliary storage unit 14 is a non-volatile storage area such as a large-capacity memory or hard disk, and stores the program P1 and other data necessary for the control unit 11 to perform processing.

[0019] Server 1 may be a multi-computer system consisting of multiple computers, or it may be a virtual machine created virtually by software.

[0020] Furthermore, in this embodiment, the server 1 is not limited to the above configuration and may include, for example, an input unit for receiving operation input, a display unit for displaying images, etc. The server 1 may also include a read unit for reading portable storage media 1a such as a CD (Compact Disk)-ROM or DVD (Digital Versatile Disc)-ROM, and may read and execute the program P1 from the portable storage media 1a. Alternatively, the server 1 may read the program P1 from a semiconductor memory 1b.

[0021] Figure 3 is a block diagram showing an example configuration of the medical imaging device 2. The medical imaging device 2 comprises a control unit 21, a main memory unit 22, a communication unit 23, a display unit 24, an input unit 25, an image processing unit 26, and an auxiliary memory unit 27. The control unit 21 is one or more arithmetic processing units such as CPUs, MPUs, and GPUs, and performs various information processing and control processing by reading and executing the program P2 stored in the auxiliary storage unit 27. The main memory unit 22 is a temporary storage area such as RAM, and temporarily stores data necessary for the control unit 21 to perform arithmetic processing. The communication unit 23 is a communication module for performing communication-related processing, and transmits and receives information with the outside. The display unit 24 is a display screen such as a liquid crystal display, and displays images. The input unit 25 is an operation interface such as a keyboard or mouse, and accepts operation input from the user. The image processing unit 26 is an image processing module that processes signals transmitted and received via the catheter 201 and generates images.

[0022] The auxiliary storage unit 27 is a non-volatile storage area such as a hard disk or large-capacity memory, and stores the program P2 and other data necessary for the control unit 21 to execute processing. The auxiliary storage unit 27 also stores the identification model 50. The identification model 50 is a machine learning model generated by learning predetermined training data, and is a trained model that takes polar coordinate images of blood vessels (biological tubular lumen) as input and outputs segment data that separates the object region from other image regions. The identification model 50 is intended to be used as a program module that constitutes part of artificial intelligence software.

[0023] The medical imaging device 2 may also be equipped with a reading unit that reads a portable storage medium 2a such as a CD-ROM, and may read and execute the program P2 from the portable storage medium 2a. Alternatively, the medical imaging device 2 may read the program P2 from the semiconductor memory 2b.

[0024] Figure 4 is an explanatory diagram illustrating the relationship between polar coordinate images and vascular tomographic images. Figure 4 illustrates how tomographic images of multiple frames in orthogonal coordinate representation are generated from polar coordinate images obtained as primary image data.

[0025] As already mentioned, the diagnostic imaging device 2 acquires polar coordinate images as primary image data. Polar coordinate images are image data in which the first axis (horizontal axis in Figure 4) is the rotation angle of the ultrasound probe and the second axis (vertical axis in Figure 4) is the distance from the ultrasound probe. In Figure 4, the first and second axes are orthogonal, but the intersection angle between them may be acute or obtuse. When imaging blood vessels using catheter 201, images are acquired while rotating the ultrasound probe and moving it along the axis of the blood vessel, so an image expressed in polar coordinates is obtained first. As shown in Figure 4, the diagnostic imaging device 2 extracts 360-degree (one rotation) of images corresponding to each frame from the polar coordinate image, converts them to a Cartesian coordinate system, and generates a vascular tomographic image.

[0026] Note that some literature refers to an image where the first axis is angle and the second axis is distance (upper part of Figure 4) as "Cartesian coordinates," and the image obtained by transforming this image (lower part of Figure 4) as "polar coordinates." However, in this specification, the former is defined as an image in polar coordinates, and the latter as an image in Cartesian coordinates.

[0027] Figures 5A and 5B illustrate the discontinuity in images caused by coordinate system transformations. Figure 5A shows a vascular tomographic image, and Figure 5B shows the segmented data (mask image) obtained by performing segmentation on the same tomographic image. When a polar coordinate image is converted to a tomographic image represented in Cartesian coordinates, a portion of the polar coordinate image that is continuous along the first axis is extracted and converted into tomographic images for each frame. Therefore, there is a risk that the parts corresponding to the start and end points of the rotation angle (0 degrees and 360 degrees) will be discontinuous. In Figures 5A and 5B, the part at the 9 o'clock position corresponds to this discontinuity. If segmentation is performed on a tomographic image in this state, the discontinuous parts will be recognized as they are, and there is a risk that distorted results will be presented.

[0028] Therefore, in this embodiment, the object region is identified by processing the original image data, i.e., the polar coordinate image. Figure 6 is an explanatory diagram of the object identification process. Figure 6 conceptually illustrates how the object region is identified from the polar coordinate image.

[0029] First, the imaging diagnostic device 2 extracts polar coordinate images corresponding to each frame of the final vascular tomographic image from the polar coordinate images acquired using the catheter 201. In this case, the imaging diagnostic device 2 extracts polar coordinate images for a predetermined angle exceeding 360 degrees, rather than one frame, i.e., 360 degrees, for each frame of the final tomographic image. For example, along the first axis, the imaging diagnostic device 2 extracts polar coordinate images for 390 degrees, adding an excess of 15 degrees at each end of each frame (360 degrees), and extracts polar coordinate images to identify object regions in the tomographic image of each frame.

[0030] The excess angle is an arbitrary design element, and it is sufficient if it exceeds by at least one pixel along the first axis. Furthermore, the excess angle may be set arbitrarily by the user.

[0031] The image diagnostic device 2 inputs the extracted polar coordinate image into the identification model 50 to identify the object region. The identification model 50 is a machine learning model generated by learning from predetermined training data, and is, for example, a semantic segmentation model, which is a type of CNN.

[0032] A semantic segmentation model is a neural network that identifies objects in an image on a pixel-by-pixel basis. It comprises a convolutional layer (encoder) that convolves the input image and a deconvolutional layer (decoder) that maps the convolved features to the original image size. The deconvolutional layer identifies which objects are located where in the image based on the features generated by the convolutional layer, and generates binarized data indicating which object each pixel corresponds to.

[0033] In this embodiment, a semantic segmentation model is given as an example of the discrimination model 50, but other neural networks or models based on other learning algorithms such as GANs (Generative Adversarial Networks) may also be used.

[0034] Server 1 performs training using training data in which segment data, which distinguishes object regions from other image regions, is attached as ground truth data to training vascular images. As a result, Server 1 generates a discriminative model 50 that outputs segment data distinguishing object regions from other image regions when a polar coordinate image is input. The training process related to the discriminative model 50 will be described in detail later.

[0035] The object regions to be identified include, for example, the EEM region of a blood vessel, the lumen region, and the region between the lumen boundary and the EEM boundary (so-called plaque). The identification model 50 according to this embodiment identifies the image region corresponding to the EEM as the object region. Note that the EEM and lumen are just examples of objects, and for example, a predetermined device visible in the image (such as a guidewire that guides the catheter 201, or a stent placed in a blood vessel) may also be identified. Furthermore, the identification model 50 may be capable of identifying multiple types of objects simultaneously.

[0036] The diagnostic imaging device 2 inputs the polar coordinate image of a predetermined angle extracted above into the identification model 50 and outputs segment data of a predetermined angle that separates the object region from other image regions. The segment data is data in which the object region and other image regions have been binarized, and each pixel in the image is assigned a class label that represents the type of region to which each pixel belongs. Figure 6 shows a mask image in which the object region is shown in white and other image regions are shown in black as an example of segment data.

[0037] The image diagnostic device 2 extracts 360-degree segment data, equivalent to a tomographic image of one frame, from the segment data of a predetermined angle output from the identification model 50. Specifically, as shown in Figure 6, the image diagnostic device 2 removes the excess segment data exceeding 360 degrees from both ends of the segment data of the predetermined angle on the first axis, and extracts 360-degree segment data. As a result, segmentation is performed considering the information before and after the start and end points of the rotation angle (both ends of the excess), allowing for a favorable prediction of the object region.

[0038] Figure 7 is an explanatory diagram regarding the display processing of vascular tomographic images. Based on the segment data extracted above, the image diagnostic device 2 identifies object regions in the tomographic image represented in orthogonal coordinates. The image diagnostic device 2 then generates and displays (outputs) a tomographic image in which the object regions can be identified.

[0039] Specifically, the diagnostic imaging device 2 converts segment data expressed in polar coordinates into segment data expressed in orthogonal coordinates. The diagnostic imaging device 2 also extracts one frame, or 360 degrees, of polar coordinate images from the polar coordinate images input to the identification model 50, i.e., polar coordinate images for a predetermined angle exceeding 360 degrees. The diagnostic imaging device 2 then converts the extracted polar coordinate images into orthogonal coordinates to generate a tomographic image.

[0040] For convenience, in the following explanation, segment data in polar coordinate representation (segment data output from the identification model 50) will be referred to as "first segment data," segment data in orthogonal coordinate representation will be referred to as "second segment data," and both will be referred to collectively as "segment data."

[0041] The medical imaging device 2 generates a tomographic image for output (display) that allows for the identification of object regions based on the second segment data obtained by converting the first segment data. For convenience, in the following description, the tomographic image for output will be referred to as the "output image". The display method of the output image is not particularly limited, but for example, as shown in Figure 7, the medical imaging device 2 generates an output image in which a predetermined display object (for example, a color-displayed ring-shaped object) is superimposed on the boundary between the object region (EEM region) and other image regions. In Figure 7, for illustrative purposes, the display object is shown with a thick line. The medical imaging device 2 displays the generated output image on the display unit 24.

[0042] For example, the diagnostic imaging device 2 may output the output image to an external display device (such as a monitor installed in the catheterization lab). Alternatively, the diagnostic imaging device 2 may output the output image to a printer or the like for printing.

[0043] Figure 8 is an explanatory diagram regarding the generation process of the discrimination model 50. The following describes the machine learning process for generating the discrimination model 50.

[0044] Server 1 performs training using training data in which ground truth segment data is attached to training vascular images. Here, the training vascular images are, for example, tomographic images in orthogonal coordinate representation. For example, Server 1 receives setting inputs (for example, drawing inputs to draw the EEM boundary of the blood vessel) from a designated operator (for example, the developer of this system) to attach second segment data that separates the object region from other image regions to each of the multiple frames of tomographic images acquired in response to the pullback movement of the catheter 201, using an actual patient as the subject. Server 1 uses the second segment data set by the operator as ground truth data to generate the discrimination model 50.

[0045] Since the image and segment data that serve as input and output for the discrimination model 50 are in polar coordinates, Server 1 first performs preprocessing for generating training data by converting the training tomographic images and second segment data from a Cartesian coordinate system to a polar coordinate system. Specifically, as shown in Figure 8, Server 1 converts the tomographic images and second segment data of each frame into polar coordinate images and first segment data, and concatenates them along the first axis to generate polar coordinate images and first segment data corresponding to multiple frames. From the generated polar coordinate images and first segment data, Server 1 extracts pairs of polar coordinate images and first segment data for a predetermined angle while shifting the region to be extracted along the first axis. Server 1 uses the extracted polar coordinate images and first segment data as the training input images and ground truth data, respectively.

[0046] Server 1 provides the polar coordinate image and first segment data extracted above to the discrimination model 50 and performs training. Specifically, Server 1 inputs the polar coordinate image into the discrimination model 50, outputs the first segment data, compares it with the correct first segment data, and updates parameters such as the weights between neurons so that the two approximate each other. Server 1 sequentially provides multiple pairs of polar coordinate image and first segment data and performs training to optimize the parameters. In this way, Server 1 generates the discrimination model 50.

[0047] The above learning method is merely an example, and this embodiment is not limited thereto. For example, the preprocessing to convert the training data in Cartesian coordinate representation (tomographic image and second segment data) to a polar coordinate system may be omitted, and the training data in polar coordinate representation (polar coordinate image and first segment data) may be provided from the beginning.

[0048] Figure 9 is a flowchart showing the procedure for generating the discrimination model 50. Based on Figure 9, the process for generating the discrimination model 50 using machine learning will be explained. The control unit 11 of server 1 acquires training data for generating the identification model 50 (step S11). The training data is data to which second segment data is added to the training vascular tomography, which separates the object region from other image regions. The training vascular tomography is a vascular image represented in a Cartesian coordinate system, and consists of multiple frames of tomography images acquired in accordance with the pullback movement of the catheter 201. The second segment data is data in which the object region and other image regions have been binarized, and is segment data represented in a Cartesian coordinate system.

[0049] The control unit 11 converts the training tomographic image and second segment data into polar coordinate images, where the first axis is the rotation angle of the ultrasound probe (device) and the second axis is the distance from the ultrasound probe, and into first segment data that is coaxial with the said polar coordinate image (step S12). Specifically, the control unit 11 converts multiple frames of tomographic images into polar coordinate systems and concatenates them along the first axis to generate polar coordinate images corresponding to multiple frames in a tomographic image represented in orthogonal coordinates. Similarly, the control unit 11 converts the first segment data corresponding to the tomographic image of each frame into polar coordinate systems and concatenates them along the first axis to generate first segment data represented in polar coordinates. From the converted polar coordinate images and first segment data, the control unit 11 extracts polar coordinate images and first segment data for a predetermined angle exceeding 360 degrees (step S13).

[0050] The control unit 11 generates a discrimination model 50 based on the polar coordinate image for a predetermined angle and the first segment data extracted in step S13, which outputs first segment data that separates the object region from other image regions when a polar coordinate image for a predetermined angle is input (step S14). Specifically, as described above, the control unit 11 generates a CNN related to semantic segmentation. The control unit 11 inputs the polar coordinate image into the discrimination model 50, outputs the first segment data, and compares it with the correct first segment data. The control unit 11 optimizes parameters such as the weights between neurons so that the two approximate each other, and generates the discrimination model 50. The control unit 11 then completes the series of processes.

[0051] Figure 10 is a flowchart showing the procedure for object identification processing. Based on Figure 10, the processing performed by the image diagnostic device 2 will be explained. The control unit 21 of the diagnostic imaging device 2 acquires a polar coordinate medical image of a blood vessel (biological lumen) captured using a catheter 201, where the first axis is the rotation angle of the ultrasound probe (device) and the second axis is the distance from the ultrasound probe (step S31). The control unit 21 extracts polar coordinate images for a predetermined angle exceeding 360 degrees from the acquired polar coordinate image (step S32).

[0052] The control unit 21 inputs the extracted polar coordinate image to the identification model 50 and outputs first segment data that separates the object region from other image regions (step S33). The control unit 21 extracts 360 degrees of first segment data from the output first segment data (step S34). The control unit 21 also extracts 360 degrees of polar coordinate image from a predetermined angle of polar coordinate image input to the identification model 50 (step S35).

[0053] The control unit 21 converts the first segment data and polar coordinate image extracted in steps S34 and S35 into second segment data and a tomographic image in orthogonal coordinate representation (step S36). Based on the converted second segment data, the control unit 21 generates an output image (tomographic image) that can identify the object region (step S37). For example, the control unit 21 generates a tomographic image as the output image in which the display object is superimposed on the boundary between the object region (EEM region) and other image regions. The control unit 21 displays (outputs) the generated output image (step S38) and terminates the series of processes.

[0054] Based on the above, according to this embodiment 1, it is possible to suitably predict the object region from a medical image represented in polar coordinates.

[0055] Furthermore, according to this embodiment 1, the object region predicted from the polar coordinate image can be presented in an identifiable manner in the tomographic image (output image) represented in orthogonal coordinates.

[0056] Furthermore, according to this embodiment 1, objects that serve as criteria for vascular imaging diagnosis, such as EEM and lumen, can be suitably identified.

[0057] Furthermore, according to this embodiment 1, data obtained by adding second segment data to a normally observed tomographic image can be converted to a polar coordinate system and used as training data. This allows for efficient creation of training data (annotation).

[0058] (Embodiment 2) In Embodiment 1, the original image data was described as being a polar coordinate image. However, if the original image data is a tomographic image represented in Cartesian coordinates, it may be processed after being inversely transformed into polar coordinates. In this embodiment, we describe a method of inversely transforming a vascular tomographic image into a polar coordinate image and identifying object regions. Note that components that overlap with those in Embodiment 1 are denoted by the same reference numerals and their descriptions are omitted.

[0059] Figure 11 is a flowchart showing the procedure for object identification processing according to Embodiment 2. Based on Figure 11, the processing content performed by the image diagnostic device 2 according to Embodiment 2 will be explained.

[0060] The control unit 21 of the diagnostic imaging device 2 acquires a tomographic image in orthogonal coordinate representation of a medical image of a blood vessel (biological lumen) (step S201). This tomographic image is, for example, a previously acquired vascular tomographic image and is an image file stored in a format such as DICOM (Digital Imaging and Communications in Medicine). For example, with respect to a patient who has undergone vascular treatment in the past, the control unit 21 acquires multiple frames of tomographic images of the patient's blood vessels in response to the pullback movement of the catheter 201.

[0061] Furthermore, the tomographic images to be processed are not limited to previously acquired image files, but may also be images acquired in real time. Also, the file format of the tomographic images to be processed is not limited to DICOM; any tomographic image represented in orthogonal coordinates is acceptable.

[0062] The control unit 21 converts the acquired tomographic images into polar coordinate images (step S202). Specifically, the control unit 21 converts the tomographic images of each frame into polar coordinate images with the first axis representing angle and the second axis representing distance, and then concatenates them along the first axis to generate polar coordinate images for multiple frames. The control unit 21 then proceeds to step S32.

[0063] After extracting 360 degrees of first segment data from a predetermined angle of first segment data output from the identification model 50 (step S34), the control unit 21 converts the first segment data into second segment data in orthogonal coordinate representation (step S203). The control unit 21 also selects the tomographic image of the frame corresponding to the second segment data from the tomographic images of multiple frames acquired in step S201 (step S204). Based on the second segment data converted in step S203, the control unit 21 generates an output image (tomographic image) capable of identifying the object region (step S205). The control unit 21 then proceeds to step S38.

[0064] Based on the above, according to this second embodiment, the object region can be suitably predicted even from a tomographic image represented in orthogonal coordinates.

[0065] (Embodiment 3) In this embodiment, we will describe a method in which the identification model 50 is retrained (updated) based on the prediction results of object regions based on the identification model 50.

[0066] Figure 12 is a flowchart showing the procedure for object identification processing according to Embodiment 3. After displaying (outputting) the output image (step S38), the image diagnostic device 2 performs the following processing. The control unit 21 of the image diagnostic device 2 receives a correction input to modify the object region shown in the output image (step S301). For example, for each of the multiple frames of output images, the control unit 21 receives an operation input from the user (medical professional) observing the image to redraw the display object (EEM boundary) presented as the object region.

[0067] The control unit 21 converts the second segment data representing the corrected object region into first segment data in polar coordinates (step S302). Specifically, the control unit 21 inversely transforms the second segment data of each frame into polar coordinates and concatenates them along the first axis to generate first segment data for multiple frames. The control unit 21 updates the discrimination model 50 based on the polar coordinate image corresponding to the output image (tomographic image) and the first segment data converted in step S302 (step S303). Specifically, the control unit 21 provides the discrimination model 50 with the polar coordinate image for a predetermined angle input to the discrimination model 50 in step S33 and the first segment data for a predetermined angle corresponding to the polar coordinate image from the first segment data generated in step S302 as training data for retraining, and updates parameters such as weights between neurons. The control unit 21 then completes the series of processes.

[0068] In the above description, the image diagnostic device 2 is assumed to perform the update (retraining) process in step S303, but the server 1 may also perform the process.

[0069] Based on the above, according to this embodiment 3, the prediction accuracy can be improved by retraining based on the prediction results of the object region based on the identification model 50.

[0070] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the claims and not in the sense described above, and all modifications within the sense and scope equivalent to the claims are intended to be included. [Explanation of Symbols]

[0071] 1 server 11 Control Unit 12 Main memory 13 Communications Department 14 Auxiliary storage P1 Program 2. Diagnostic imaging equipment 21 Control Unit 22 Main memory 23 Communications Department 24 Display 25 Input section 26 Image Processing Unit 27 Auxiliary storage P2 Program 50 Discrimination Models

Claims

1. An acquisition unit that acquires a polar coordinate medical image of a biological lumen, obtained by imaging the biological lumen using a device inserted into the biological lumen, wherein the first axis is the angle and the second axis intersecting the first axis is the distance from the device; An output unit that takes the aforementioned polar coordinate image as input and outputs first segment data that separates the image region corresponding to a specific object from other image regions, and outputs first segment data for the predetermined angle portion when the aforementioned polar coordinate image is input, An extraction unit extracts 360 degrees of first segment data by removing the first segment data for angles exceeding 360 degrees from both ends of the first segment data for the predetermined angle, A conversion unit that converts the extracted first segment data into second segment data in orthogonal coordinate representation. An information processing device equipped with the following features.

2. The system includes a second output unit that outputs a tomographic image obtained by converting the polar coordinate image to a Cartesian coordinate system based on the second segment data, the tomographic image being capable of identifying the image region corresponding to the object. The information processing apparatus according to claim 1.

3. After outputting the tomographic image, a receiving unit receives a correction input to correct the image region corresponding to the object, An inverse conversion unit converts second segment data representing the image region corresponding to the modified object into the first segment data, The system includes an update unit that updates the model based on the polar coordinate image and the converted first segment data. The information processing apparatus according to claim 2.

4. The aforementioned polar coordinate image is an image of the blood vessel taken using a catheter inserted into the blood vessel. The output unit outputs the first segment data, which divides the image region corresponding to the external elastic lamina or lumen of the blood vessel. The information processing apparatus according to any one of claims 1 to 3.

5. The acquisition unit acquires a tomographic image of the biological lumen in orthogonal coordinates and converts it into a polar coordinate image. The output unit inputs the converted polar coordinate image to the model and outputs the first segment data. The information processing apparatus according to any one of claims 1 to 4.

6. Information processing device, A medical image in polar coordinates obtained by imaging a biological lumen using a device inserted into the biological lumen, wherein the first axis is the angle and the second axis intersecting the first axis is the distance from the device, and a polar coordinate image is obtained. When the aforementioned polar coordinate image is input to a model that has been trained to output first segment data that separates the image region corresponding to a specific object from other image regions, the aforementioned polar coordinate image for a predetermined angle exceeding 360 degrees is input, and the first segment data for the predetermined angle is output. From both ends of the first segment data for the predetermined angle, the first segment data for the angle exceeding 360 degrees is removed, and the first segment data for 360 degrees is extracted. Convert the extracted first segment data into second segment data in orthogonal coordinate representation. The method of operation of an information processing device that performs processing.

7. A medical image in polar coordinates obtained by imaging a biological lumen using a device inserted into the biological lumen, wherein the first axis is the angle and the second axis intersecting the first axis is the distance from the device, and a polar coordinate image is obtained. When the aforementioned polar coordinate image is input to a model that has been trained to output first segment data that separates the image region corresponding to a specific object from other image regions, the aforementioned polar coordinate image for a predetermined angle exceeding 360 degrees is input, and the first segment data for the predetermined angle is output. From both ends of the first segment data for the predetermined angle, the first segment data for the angle exceeding 360 degrees is removed, and the first segment data for 360 degrees is extracted. Convert the extracted first segment data into second segment data in orthogonal coordinate representation. A program that instructs a computer to perform a process.

Images