Program, information processing device and information processing method

Abstract

Provided are a program and the like that enable suitable detection of an object region in a medical image.　The program causes a computer to execute processes for: acquiring a medical image generated on the basis of a signal detected by a catheter inserted to a hollow organ; detecting an object region included in the medical image by inputting the acquired medical image to a model trained so as to detect the object region included in the medical image when the medical image is inputted; calculating respective reliabilities for a plurality of boundary points located on a boundary between the detected object region and another image region in the medical image; and performing interpolation using the boundary points for which the calculated reliabilities satisfy a predetermined criterion, thereby correcting the boundary based on a result of detection of the object region by the model.

Description

[Technical Field]

[0001] The present technology relates to a program, an information processing device, and an information processing method. [Background technology]

[0002] Generally, medical imaging diagnostic devices that generate images of the inside of the human body, such as ultrasound diagnostic devices and X-ray imaging devices, are known, and medical images obtained by these types of medical imaging diagnostic devices are widely used for diagnosis and treatment.

[0003] On the other hand, technologies for adding information to medical images using image processing and machine learning are being developed to assist doctors in making diagnoses. For example, Patent Document 1 discloses a method for detecting objects of interest in vascular images based on a set of co-registered medical image data obtained from multiple imaging modalities. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special publication 2016-525893 Summary of the Invention [Problem to be solved by the invention]

[0005] Artifacts may appear in medical images. Artifacts are unintended or virtual images that do not actually exist, and are images that are formed due to the equipment or imaging conditions used to capture medical images. In such cases, there is a problem in that object detection from medical images cannot be performed properly.

[0006] An object of the present disclosure is to provide a program or the like that can suitably detect an object region in a medical image. [Means for solving the problem]

[0007] A program according to one aspect of the present disclosure causes a computer to execute a process of acquiring a medical image generated based on a signal detected by a catheter inserted into a tubular organ, inputting the acquired medical image into a model that has been trained to detect object areas included in the medical image when the medical image is input, thereby detecting object areas included in the medical image, calculating reliability for each of a plurality of boundary points located on the boundary between the detected object area and other image areas in the medical image, and correcting the boundary based on the detection result of the object area by the model by interpolating using boundary points whose calculated reliability satisfies a predetermined standard. [Effects of the Invention]

[0008] According to the present disclosure, an object region in a medical image can be suitably detected. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of the configuration of an image diagnostic system. [Figure 2] FIG. 1 is a block diagram illustrating an example of the configuration of an information processing device. [Figure 3] FIG. 1 is a schematic diagram of a first learning model. [Figure 4] FIG. 1 is a conceptual diagram showing an example of an artifact that occurs in a medical image. [Figure 5] 10 is a flowchart illustrating an example of a processing procedure executed by an information processing device. [Figure 6] FIG. 1 is a schematic diagram of a second learning model. [Figure 7] 10 is a flowchart illustrating the procedure for generating a second learning model. [Figure 8] 10 is a flowchart illustrating an example of a processing procedure executed by an information processing device according to a second embodiment. [Figure 9] FIG. 10 is an overview of the third learning model. [Figure 10]11 is a flowchart illustrating an example of a processing procedure executed by an information processing device according to a third embodiment. [Figure 11] FIG. 10 is an explanatory diagram regarding correction of an object region. [Figure 12] FIG. 2 is an explanatory diagram showing an example of a display screen of the imaging diagnostic apparatus. [Figure 13] 10 is a flowchart illustrating an example of a processing procedure executed by an information processing device according to a fourth embodiment. [Figure 14] 13 is a flowchart illustrating an example of a processing procedure executed by an information processing device according to a fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] The present invention will be specifically described with reference to the drawings showing embodiments thereof.

[0011] (First embodiment) 1 is an explanatory diagram showing an example of the configuration of an image diagnostic system. The image diagnostic system includes an information processing device 1 and an image diagnostic device 2. The information processing device 1 and the image diagnostic device 2 are communicatively connected via a network N such as a LAN (Local Area Network) or the Internet.

[0012] The diagnostic imaging device 2 is a device unit for imaging the hollow organs of a subject. The diagnostic imaging device 2 is a device unit for generating medical images including ultrasonic tomographic images of the subject's blood vessels by, for example, an intravascular ultrasound (IVUS) method using a catheter 21, and for performing intravascular ultrasound examination and diagnosis. The diagnostic imaging device 2 includes the catheter 21, an MDU (Motor Drive Unit) 22, an image processing device 23, and a display device 24.

[0013] The catheter 21 is an imaging diagnostic catheter for obtaining an ultrasonic tomographic image of a blood vessel by the IVUS method. The ultrasonic tomographic image is an example of a catheter image generated using the catheter 21. The catheter 21 has a probe portion 211 and a connector portion 212 disposed at the end of the probe portion 211. The probe portion 211 is connected to the MDU 22 via the connector portion 212. A shaft 213 is inserted inside the probe portion 211. A sensor 214 is connected to the tip side of the shaft 213.

[0014] The sensor 214 is an ultrasonic transducer that transmits ultrasonic waves based on a pulse signal within the blood vessel and receives waves reflected by the biological tissue of the blood vessel or by a medical device. The shaft 213 and the sensor 214 are configured to be movable within the probe unit 211 in the longitudinal direction of the blood vessel while rotating in the circumferential direction of the blood vessel.

[0015] The MDU 22 is a drive device to which the catheter 21 is detachably attached, and controls the operation of the catheter 21 inserted into the blood vessel by driving a built-in motor in response to operation by the user. The MDU 22 rotates the shaft 213 and the sensor 214 in the circumferential direction while moving them longitudinally from the distal end to the proximal end. The sensor 214 continuously scans the inside of the blood vessel at predetermined time intervals and outputs reflected wave data of the detected ultrasound to the imaging diagnostic device 2.

[0016] The image processing device 23 is a processing device that generates ultrasonic tomographic images (medical images) of blood vessels based on reflected wave data output from the ultrasonic probe of the catheter 21. The image processing device 23 generates one image per rotation of the sensor 214. The generated image is a transverse image centered on the probe portion 211 and approximately perpendicular to the probe portion 211. The image processing device 23 successively generates multiple transverse images at predetermined intervals by a pullback operation that rotates the sensor 214 while pulling it toward the MDU 22 at a constant speed. The image processing device 23 displays the generated ultrasonic tomographic images on the display device 24 and also includes an input interface for receiving input of various setting values ​​when performing an examination.

[0017] The display device 24 is a liquid crystal display panel, an organic EL display panel, etc. The display device 24 displays the medical image generated by the image processing device 23, the estimation results received from the information processing device 1, etc.

[0018] In this embodiment, intravascular examination is described as an example, but the luminal organ to be examined is not limited to a blood vessel and may be, for example, an internal organ such as the intestine. Furthermore, the catheter 21 may be a catheter for generating optical tomographic images, such as for OCT (Optical Coherence Tomography) or OFDI (Optical Frequency Domain Imaging), which generate optical tomographic images using near-infrared light. In this case, the sensor 214 is a transceiver that emits near-infrared light and receives reflected light. The catheter 21 may have both an ultrasonic transducer and an OCT or OFDI transceiver 214, and may be used to generate catheter images including both ultrasonic tomographic images and optical tomographic images.

[0019] The information processing device 1 is an information processing device capable of various information processing and transmitting and receiving information, such as a server computer or a personal computer. The information processing device 1 may be a local server installed in the same facility (hospital, etc.) as the image diagnostic device 2, or may be a cloud server communicatively connected to the image diagnostic device 2 via the Internet, etc. The information processing device 1 functions as a detection device that detects an object region, such as an intraluminal region, from a medical image generated by the image diagnostic device 2 using a first learning model 141 (see FIG. 2 ), and provides the detection result to the image diagnostic device 2. In particular, the information processing device 1 according to this embodiment provides a detection result in which an object region is suitably detected by performing preprocessing, which will be described later, on the medical image input to the first learning model 141.

[0020] 2 is a block diagram showing an example of the configuration of the information processing device 1. The information processing device 1 includes a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary memory unit 14. The information processing device 1 may be a multi-computer consisting of multiple computers, or may be a virtual machine virtually constructed by software.

[0021] The control unit 11 has one or more arithmetic processing devices such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), etc., and performs various information processing, control processing, etc. by reading and executing a program P stored in the auxiliary storage unit 14. The main storage unit 12 is a temporary storage area such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), or a flash memory, and temporarily stores data necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing, and transmits and receives information to and from the outside.

[0022] The auxiliary memory unit 14 is a non-volatile storage area such as a large-capacity memory or a hard disk. The auxiliary memory unit 14 stores programs and data referenced by the control unit 11, including the program P. The auxiliary memory unit 14 also stores a first learning model 141. The auxiliary memory unit 14 may also store a second learning model 142, a third learning model 143, etc. Learning models other than the first learning model 141 will be described in detail in other embodiments. The auxiliary memory unit 14 may also be an external storage device connected to the information processing device 1.

[0023] The program P may be written to the auxiliary storage unit 14 during the manufacturing stage of the information processing device 1, or may be distributed by a remote server device and acquired by the information processing device 1 via communication and stored in the auxiliary storage unit 14. The program P may be readably recorded on a recording medium 1a such as a magnetic disk, an optical disk, or a semiconductor memory.

[0024] In this embodiment, the information processing device 1 is not limited to the above configuration, and may include, for example, an input unit that accepts operation input, a display unit that displays images, and the like.

[0025] FIG. 3 is a schematic diagram of the first learning model 141. The first learning model 141 is a machine learning model configured to, when a medical image is input, detect an object region within the medical image and output a detection result. Examples of objects include a blood vessel lumen, a vascular membrane, a blood vessel wall, a stent (a medical device present within a blood vessel), a guidewire, and a calcified portion within a blood vessel. FIG. 3 illustrates an example of the first learning model 141 that detects a blood vessel lumen region as an object region. The first learning model 141 is defined by its definition information. The definition information of the first learning model 141 includes, for example, structural information and layer information of the first learning model 141, information on the nodes included in each layer, and parameters such as weights and biases between nodes. The auxiliary storage unit 14 stores definition information related to the first learning model 141. The first learning model 141 is expected to be used as a program module constituting part of artificial intelligence software.

[0026] The first learning model 141 is, for example, a convolutional neural network (CNN) that has been trained by deep learning. The first learning model 141 recognizes an object region on a pixel-by-pixel basis by an image recognition technique using so-called semantic segmentation.

[0027] The first learning model 141 includes an input layer 141a to which a medical image is input, an intermediate layer 141b that extracts and restores image features, and an output layer 141c that outputs a label image that indicates an object region included in the medical image in pixel units. The first learning model 141 is, for example, a U-Net.

[0028] The input layer 141a of the first learning model 141 has multiple nodes that accept input of pixel values ​​of each pixel included in a medical image, and passes the input pixel values ​​to the intermediate layer 141b. The intermediate layer 141b has multiple nodes that extract features of the input data, and passes the features extracted using various parameters to the output layer. The intermediate layer 141b has a convolutional layer (CONV layer) and a deconvolutional layer (DECONV layer). The convolutional layer is a layer that compresses the dimensions of image data. Features of object regions are extracted through dimensional compression. The deconvolutional layer performs deconvolution processing to restore the original dimensions. The restoration processing in the deconvolutional layer generates a binarized label image that indicates whether each pixel in the image is an object region or not.

[0029] The output layer 141c has one or more nodes that output a labeled image. The labeled image is, for example, a binary image in which pixels corresponding to the lumen region of a blood vessel are assigned class "1" and pixels corresponding to other images are assigned class "0." The output layer 141c converts the output value of the class classification for each pixel into a probability using an activation function 141d, such as a softmax function, based on the feature values ​​input from the intermediate layer 141b. Based on the converted probability, the output layer 141c outputs a labeled image in which each pixel is classified into a class. The activation function 141d is not limited to a softmax function, and other functions such as a sigmoid function and a ReLU function may also be used.

[0030] In this embodiment, the first learning model 141 receives as input a plurality of frames of medical images that are successive in time series, and detects an object region from the medical image of each frame. Specifically, the first learning model 141 receives as input a plurality of frames of medical images that are successive along the longitudinal direction of a blood vessel in accordance with the scanning of the catheter 21. The first learning model 141 detects an object region from the medical image of each successive frame along the time axis t.

[0031] The first learning model 141 is generated and trained in advance in the information processing device 1 or an external device. The control unit 11 of the information processing device 1 trains the first learning model 141 by collecting in advance a group of information in which a large number of images collected in the past are labeled with known object regions as training data. The object regions may be labeled, for example, based on judgments made by a doctor with specialized knowledge. The control unit 11 classifies the collected training data, using some as test data and the rest as training data.

[0032] The control unit 11 inputs medical images of learning data (training data) extracted from the information group as input data to the first learning model 141. The control unit 11 calculates the error between the output value output from the first learning model 141 and the correct label of the training data extracted from the information group using a loss function (error function). As the loss function, for example, the mean square error E shown in the following (Equation 1) can be used.

[0033]

number

[0034] where y k is the output value output from the first learning model 141, t k represents the training data, and k represents the number of data (number of dimensions).

[0035] The control unit 11 performs learning by repeatedly updating various parameters and weights constituting the first learning model 141 using, for example, an error backpropagation method so as to minimize the calculated error. The various parameters and weights are optimized, and the first learning model 141 outputs an object region when a medical image is input.

[0036] The control unit 11 evaluates whether the machine learning of the first learning model 141 has been performed properly. Specifically, medical images of test data (training data) are input to the first learning model 141, and the error of the output value output from the first learning model 141 is calculated. The control unit 11 determines whether the error of the output value is less than a threshold. If the error is less than the threshold, it determines that the learning has been performed properly, and terminates the learning of the first learning model 141. As a result, when a medical image is input, the first learning model 141 is constructed, which has been trained to be able to properly detect an object region.

[0037] In this embodiment, the first learning model 141 is a CNN and semantic segmentation model, but the configuration of the model is not limited. The first learning model 141 may be any model that can identify the position and shape of an object in a medical image. The first learning model 141 may be a model based on other learning algorithms, such as a recurrent neural network (RNN), a support vector machine (SVM), or a regression tree.

[0038] Here, artifacts in medical images will be described. FIG. 4 is a conceptual diagram showing an example of an artifact that occurs in a medical image. Artifacts may occur in medical images generated by the image diagnostic device 2. An artifact is an image of a part that is not the target of the examination, or a virtual image that does not actually exist, and is an image that is generated due to the device, imaging conditions, the method of operating the catheter 21, etc. An example of an artifact that occurs in a medical image will be described using FIG. 4. It should be noted that the artifact is not limited to the example of FIG. 4.

[0039] For example, if calcified tissue 41 is present in a blood vessel, artifacts 42 are formed at positions spaced apart at equal intervals as the distance between the calcified tissue 41 and the catheter 21. If calcified tissue 41 is present, the ultrasound transmitted from the catheter 21 is reflected multiple times within the body lumen, forming artifacts 42, which are bright white images. This phenomenon is called multiple reflection.

[0040] In areas where a guidewire is present, a roughly fan-shaped artifact 44 is formed radially outward of the catheter 21 relative to the guidewire image 43. When a guidewire is present, most of the ultrasound is reflected by the guidewire, which is a strong reflector, and ultrasound that penetrates radially outward of the catheter 21 relative to the guidewire is greatly attenuated, resulting in the formation of an artifact 44 in which part of the image is blackened. This phenomenon is called acoustic shadowing. Similar artifacts are also formed in areas where a stent is placed, on the outside of heavily calcified areas, and highly attenuating plaque.

[0041] When a false lumen exists between the intima and media of a blood vessel, an artifact 45 corresponding to the false lumen region is formed. If the false lumen region is displayed, there is a risk that information about the lumen region, which is the actual target of detection, will be erroneously extracted. In this embodiment, images of parts that are not the target of the examination, including such false lumen regions, are also included as one type of artifact.

[0042] If air bubbles remain in the air trap at the tip of the catheter 21, an artifact 46 will appear in part of the image. The air bubbles attenuate the ultrasound, causing part of the image to turn black, resulting in the formation of the artifact 46.

[0043] When the above-described artifacts occur in a medical image, information about the object region to be detected is lost. Furthermore, the characteristics of the medical image change due to the effects of the artifacts. Therefore, when an object region is detected from a medical image containing artifacts using the first learning model 141, there is a risk of the detection accuracy decreasing. In this embodiment, when the information processing device 1 acquires a medical image from the image diagnostic device 2, the information processing device 1 performs preprocessing to estimate the detection accuracy for the medical image. As a result of the preprocessing, object regions are not detected from medical images estimated to have low detection accuracy, and object regions are detected only from medical images estimated to have high detection accuracy.

[0044] The preprocessing method in this embodiment will be specifically described. The preprocessing performed by the information processing device 1 includes a process of deriving determination information for determining whether or not to detect an object region from a medical image, and a determination process of determining whether or not to detect an object region based on the determination information.

[0045] The determination information includes information regarding the accuracy of detecting an object region in a medical image. In this embodiment, the determination information includes the output of the activation function 141d included in the first learning model 141. As described above, the output layer 141c of the first learning model 141 uses a softmax function in the activation function 141d to output an output value indicating the class classification for each pixel in the medical image. The softmax function is a function that converts the output values ​​corresponding to each class so that the sum of the output values ​​becomes 1.0. In other words, the output value of the softmax function is the probability that each pixel in the medical image is classified into class "1" (vascular lumen region). The output value of the softmax function indicates the likelihood that the pixel is in class "1" and indicates the accuracy of the object region detection result. The output value corresponding to each class is output as a numerical value ranging from 0.0 to 1.0. The information processing device 1 acquires an output value for each pixel in the medical image using the first learning model 141.

[0046] The information processing device 1 determines whether to perform object region detection processing on the medical image based on the output value of the acquired activation function 141d. For example, if the output value of the lumen region for a pixel in the medical image is close to 1, the probability that it is a lumen region is high, indicating a high accuracy of the detection result. If the output value of the lumen region for the pixel is close to 0, the probability that it is not a lumen region is high, indicating a high accuracy of the detection result. On the other hand, if the output value of the lumen region for the pixel is around 0.5, the probability that it is a lumen region and the probability that it is not a lumen region are both low, indicating a low accuracy of the detection result. From this perspective, the information processing device 1 determines the accuracy of the medical image based on the output value of the activation function 141d, thereby determining whether to perform object region detection processing.

[0047] Specifically, the number of pixels whose output value of the activation function 141d is within a predetermined range (for example, 0.4 to 0.6) is obtained, and the proportion of pixels whose output value is within the predetermined range is calculated relative to all pixels in the medical image. If the proportion of pixels whose output value is within the predetermined range is less than a threshold, it is estimated that the accuracy of detecting an object region in the medical image is high, and it is determined that object region detection should be performed on the medical image. If the proportion of pixels whose output value is within the predetermined range is equal to or greater than a threshold, it is estimated that the accuracy of detecting an object region in the medical image is low, and it is determined that object region detection should not be performed on the medical image. The determination of whether or not to perform detection is not limited to using the number of pixels, and the variance of the output values ​​for each pixel may also be used.

[0048] 5 is a flowchart showing an example of a processing procedure executed by the information processing device 1. When the MDU 22 executes a pullback operation and a medical image is output from the image diagnostic device 2, the control unit 11 of the information processing device 1 executes the following processing in accordance with the program P.

[0049] The control unit 11 of the information processing device 1 acquires medical images of the subject from the image diagnostic device 2 (step S11). The acquired medical images are multiple frames of tomographic images that are consecutive in time series. The control unit 11 then proceeds to derive determination information.

[0050] The control unit 11 inputs the acquired medical image into the first learning model 141 (step S12), and acquires the output value of the activation function 141d included in the first learning model 141 for each pixel (step S13), thereby deriving judgment information.

[0051] Next, the control unit 11 determines whether or not to perform object region detection processing on the medical image based on the output value of the derived activation function 141d (step S14). Specifically, the control unit 11 determines whether or not to perform detection processing by determining whether or not the ratio of the number of pixels whose output values ​​are within a predetermined range to the total number of pixels in the medical image is equal to or greater than a threshold value.

[0052] If it is determined that an object region is to be detected for the acquired medical image because the proportion of pixels whose output values ​​are within a predetermined range is less than the threshold (step S14: YES), the control unit 11 inputs the medical image to the first learning model 141 (step S15). The control unit 11 detects an object region in the medical image by acquiring the output value output from the first learning model 141 (step S16).

[0053] The control unit 11 generates image information that displays the detected object region (step S17). The control unit 11 outputs the generated image information to the image diagnostic device 2 (step S18). The image displayed based on the image information displays the object region included in the medical image in a identifiable display mode. The image is, for example, an image in which a label image output from the first learning model 141 is superimposed on the original medical image. The control unit 11 processes, for example, the label image output from the first learning model 141 into a semi-transparent mask, and generates image information to be displayed by superimposing it on the original medical image.

[0054] On the other hand, when it is determined that object region detection will not be performed on the acquired medical image because the proportion of pixels whose output values ​​are within the predetermined range is equal to or greater than the threshold (step S14: NO), the control unit 11 generates warning information (step S19). The warning information includes warning screen information indicating, by text or the like, that object region detection will not be performed. The warning information may also include audio data indicating by audio that object region detection will not be performed. The warning information may also include information indicating the proportion of pixels whose output values ​​are within the predetermined range. The control unit 11 outputs the generated warning information to the image diagnostic apparatus 2 (step S20), and ends the series of processes. The control unit 11 may also perform loop processing that returns the process to step S11.

[0055] Doctors and other medical personnel can recognize the detection status of the object in the medical image based on the warning information displayed via the image diagnostic device 2. Based on the warning information, doctors and other medical personnel can take appropriate measures, such as acquiring the medical image again.

[0056] In this embodiment, the image information, warning information, etc. are output to the image diagnostic device 2, but it goes without saying that the image information, warning information, etc. may be output to a device (e.g., a personal computer) other than the image diagnostic device 2 from which the medical image was obtained.

[0057] According to this embodiment, by performing preprocessing before detecting an object region, it is possible to preferably detect an object region using only medical images that are estimated to have high object region detection accuracy. This reduces the influence of artifacts in medical images, and improves the accuracy of the detection results obtained by the first learning model 141. A warning is displayed for medical images that are estimated to have low object region detection accuracy, so it is possible to reliably notify doctors and others that an object region will not be detected.

[0058] (Second embodiment) In the image diagnostic system of the second embodiment, the content of the determination information is different from that of the first embodiment, so the following mainly describes the above differences. Since the other configurations are the same as those of the first embodiment, the same reference numerals are used for the common configurations, and detailed descriptions thereof will be omitted.

[0059] The determination information of the second embodiment includes an evaluation index related to the detection accuracy of an object region included in a medical image. The information processing device 1 derives the determination information using a second learning model 142. The second learning model 142 is a machine learning model configured to output an evaluation index related to the detection accuracy of an object region included in a medical image when the medical image is input. The auxiliary storage unit 14 of the information processing device 1 stores definition information related to the second learning model 142. The evaluation index related to the detection accuracy of an object region is information indicating the detection accuracy estimated when an object region is detected from a medical image. For example, the evaluation index may be the value of a loss function for the output value of a first learning model 141 that detects an object region included in a medical image. As described above, the loss function indicates the error between the output value of the first learning model 141 and the correct label of the training data, and serves as an index of the accuracy of the first learning model 141.

[0060] 6 is a schematic diagram of the second learning model 142. The second learning model 142 is a neural network model, such as a CNN, generated by deep learning, for example. The second learning model 142 has an input layer 142a to which a medical image is input, an intermediate layer 142b that extracts image features, and an output layer 142c that outputs output data indicating an evaluation index for the medical image.

[0061] The input layer 142a of the second learning model 142 has multiple nodes that accept input of pixel values ​​of each pixel included in a medical image and passes the input pixel values ​​to the intermediate layer 142b. The intermediate layer 142b has multiple nodes that extract features of the input data and passes the features extracted using various parameters to the output layer. The output layer 142c outputs continuous values ​​indicating an evaluation index. Note that the output layer 142c is not limited to outputting continuous values ​​through regression, and may output discrete values ​​indicating an evaluation index through classification.

[0062] In this embodiment, the second learning model 142 is a CNN, but the configuration of the model is not limited thereto. The second learning model 142 may be a model based on other learning algorithms, such as an RNN, an SVM, or a regression tree.

[0063] The information processing device 1 generates the second learning model 142 in advance and uses it to derive the judgment information. FIG.

[0064] The control unit 11 of the information processing device 1 acquires training data for training (learning) the second learning model 142 (step S31). The training data includes a medical image and label data indicating an evaluation index related to the detection accuracy of an object region included in the medical image. The evaluation index uses the value of a loss function obtained based on the first learning model 141. The first learning model 141 is a semantic segmentation model, and the value of the loss function includes the total value of the loss function for each pixel of the medical image for the entire image size.

[0065] The control unit 11 acquires, as training data, a plurality of information groups that associate a loss function calculated during the evaluation of learning in the generation stage of the above-mentioned first learning model 141 with a medical image corresponding to the loss function. That is, the control unit 11 sets, as training data, medical images of test data (training data) for the first learning model 141 and the value of the loss function for the object region output by the first learning model 141 when the medical image is input.

[0066] The control unit 11 inputs medical images of training data extracted from the information group as input data to the second learning model 142 (step S32). The control unit 11 acquires an evaluation index (loss function) output from the second learning model 142 (step S33). The control unit 11 calculates the error between the acquired evaluation index (output value) and the correct label of the training data extracted from the information group using a predetermined loss function. The control unit 11 adjusts various parameters, weights, etc., using, for example, backpropagation, so as to optimize (minimize or maximize) the loss function (step S34). Before learning begins, it is assumed that initial settings are assigned to the definition information describing the second learning model 142.

[0067] The control unit 11 determines whether to end the learning (step S35). For example, the control unit 11 acquires test data from the information group and inputs it into the second learning model 142, and determines to end the learning when the calculated error satisfies a predetermined standard. The control unit 11 may also determine to end the learning when the number of learning iterations meets a predetermined standard.

[0068] If it is determined not to end the learning (step S35: NO), the control unit 11 returns the process to step S31. If it is determined to end the learning (step S35: YES), the control unit 11 stores definition information related to the second learning model 142 in the auxiliary storage unit 14 as the trained second learning model 142 (step S36), and ends the process according to this flowchart. Through the above process, it is possible to construct a second learning model 142 that has been trained to be able to appropriately estimate an evaluation index related to the detection accuracy of an object region included in a medical image.

[0069] The information processing device 1 uses the evaluation index obtained by the second learning model 142 as judgment information to determine whether to detect an object region from a medical image. For medical images with a large evaluation index, the detection accuracy is estimated to be low, so object region detection is not performed. For medical images with a small evaluation index, the detection accuracy is estimated to be high, so object region detection is performed.

[0070] In the above, an example has been described in which the value of a loss function obtained based on the first learning model 141 is used as the evaluation index, but the evaluation index is not limited to a loss function and may be any information indicating the detection accuracy of an object region included in a medical image. For example, the accuracy rate, precision, recall, etc. of the first learning model 141 may be used as the evaluation index. For example, a value determined by a doctor or the like with specialized knowledge may be used as the evaluation index.

[0071] 8 is a flowchart showing an example of a processing procedure executed by the information processing device 1 in the second embodiment. The same step numbers are assigned to the processes common to those in FIG. 5 of the first embodiment, and detailed descriptions thereof will be omitted.

[0072] The control unit 11 of the information processing device 1 acquires a medical image of a subject from the image diagnostic device 2 (step S11). The control unit 11 derives determination information based on the acquired medical image. In this embodiment, the control unit 11 derives an evaluation index related to the detection accuracy of an object region included in the medical image as the determination information. More specifically, the control unit 11 derives a value of a loss function.

[0073] The control unit 11 inputs the acquired medical image into the second learning model 142 (step S41) and acquires the value of the evaluation index to be output (step S42), thereby deriving the judgment information.

[0074] The control unit 11 determines whether or not to perform object region detection processing on the medical image based on the acquired evaluation index (step S14). Specifically, the control unit 11 determines whether or not to perform detection processing by determining whether or not the evaluation index on the medical image is equal to or greater than a threshold value.

[0075] If the evaluation index is less than the threshold, the control unit 11 determines to detect an object region in the medical image. If the evaluation index is equal to or greater than the threshold, the control unit 11 determines not to detect an object region in the medical image. Thereafter, the control unit 11 executes the processes of steps S15 to S20 shown in FIG. 8 according to the determination result.

[0076] According to this embodiment, by pre-processing using the second learning model 142, medical images that are estimated to have a large loss function of the first learning model 141 are extracted in advance, and object areas can be accurately detected using other medical images.

[0077] The control unit 11 of the information processing device 1 may re-learn the second learning model 142 after detecting the object region. The control unit 11 detects the object region from a newly acquired medical image using the first learning model 141, and then acquires the value of a loss function for the detection result. The control unit 11 performs re-learning using the acquired loss function value and the corresponding medical image as training data. The control unit 11 optimizes the weights, etc. of the second learning model 142, and updates the second learning model 142. According to the above-mentioned processing, the second learning model 142 can be further optimized through the operation of this image diagnostic system.

[0078] (Third embodiment) In the diagnostic imaging system of the third embodiment, the content of the determination information is different from that of the first and second embodiments, so the following mainly describes the above differences. Since the other configurations are the same as those of the first and second embodiments, the same reference numerals are used for the common configurations, and detailed descriptions thereof will be omitted.

[0079] The determination information in the third embodiment includes information regarding the presence or absence of artifacts in the medical image. In the third embodiment, the detection accuracy for the medical image is estimated based on the presence or absence of artifacts. A medical image containing artifacts has significantly different feature amounts compared to a medical image without artifacts, and therefore is estimated to have low detection accuracy. A medical image without artifacts has only a small change in feature amounts, and therefore is estimated to have high detection accuracy.

[0080] The information processing device 1 derives judgment information using a third learning model 143. The third learning model 143 is a machine learning model configured to output information indicating the presence or absence of artifacts contained in a medical image when the medical image is input. The auxiliary storage unit 14 of the information processing device 1 stores definition information related to the third learning model 143. Similar to the second learning model 142 of the second embodiment, the information processing device 1 learns training data to generate the third learning model 143 in advance. Then, when the information processing device 1 acquires a medical image from the image diagnostic device 2, it inputs the image to the third learning model 143 and estimates the presence or absence of artifacts.

[0081] 9 is a schematic diagram of the third learning model 143. The third learning model 143 includes, for example, a CNN 143a and a classifier 143b. The CNN 143a is a deep learning model that receives input of a medical image, extracts features of the image, and outputs the extracted features. The CNN 143a may use a trained model that has been trained in advance, for example, by transfer learning.

[0082] The classifier 143b is, for example, a one-class SVM. The classifier 143b outputs a classification result in which the medical image is classified as "normal" or "abnormal" based on the feature amount extracted by the CNN 143a. More specifically, the classifier 143b outputs a binary value indicating whether the medical image is free of artifacts, which is considered to be normal data.

[0083] The information processing device 1 performs unsupervised learning using only a large number of artifact-free medical images collected in the past, with the artifact-free medical images considered normal data. The classifier 143b is trained to identify "outliers" from the normal data, with the artifact-free medical images considered normal data. The classifier 143b identifies medical images other than artifact-free ones, i.e., medical images containing artifacts, as abnormal values. This constructs a third learning model 143 that has been trained to be able to appropriately output information indicating the presence or absence of artifacts when a medical image is input.

[0084] The configuration of the third learning model 143 is not limited to the above example. The third learning model 143 may be any model as long as it can identify the presence or absence of artifacts in a medical image. The third learning model 143 may output a classification result of the presence or absence of artifacts in a medical image through supervised learning using training data including both medical images with and without artifacts. The third learning model 143 may also be a model based on other learning algorithms, such as an RNN or a GAN (Generative Adversarial Network).

[0085] The information processing device 1 determines whether to detect an object region from a medical image using the presence or absence of artifacts obtained by the above-described third learning model 143 as determination information. For medical images with artifacts, detection accuracy is estimated to be low, so object region detection is not performed. For medical images without artifacts, detection accuracy is estimated to be high, so object region detection is performed.

[0086] 10 is a flowchart showing an example of a processing procedure executed by the information processing device 1 in the third embodiment. The same step numbers are used for the processing common to FIG. 5 in the first embodiment, and detailed descriptions thereof will be omitted.

[0087] The control unit 11 of the information processing device 1 acquires a medical image of a subject from the image diagnostic device 2 (step S11). The control unit 11 derives determination information based on the acquired medical image. In this embodiment, the presence or absence of an artifact in the medical image is derived as the determination information.

[0088] The control unit 11 inputs the acquired medical image into the third learning model 143 (step S51), and acquires the presence or absence of an artifact to be output (step S52), thereby deriving determination information.

[0089] The control unit 11 determines whether or not to perform object region detection processing on the medical image based on the presence or absence of the acquired artifacts (step S14). If there are no artifacts in the medical image, the control unit 11 determines to perform object region detection on the medical image. If there are artifacts in the medical image, the control unit 11 determines not to perform object region detection on the medical image. Thereafter, the control unit 11 executes the processing of steps S15 to S20 shown in FIG. 10 according to the determination result.

[0090] According to this embodiment, the presence or absence of artifacts is accurately estimated using the third learning model 143. By extracting medical images with artifacts, i.e., medical images estimated to have low detection accuracy, in advance, it is possible to accurately detect object regions using other medical images.

[0091] The information processing device 1 may use multiple of the three types of determination information described in the first to third embodiments to determine whether to perform object region detection. For example, the information processing device 1 may execute preprocessing in parallel, obtain determination results based on each of the three types of determination information, and determine whether to perform detection processing by comprehensively evaluating the determination results. Alternatively, the information processing device 1 may execute multiple preprocessing processes sequentially, for example, after executing the preprocessing of the first embodiment and determining not to perform detection processing, and then execute the preprocessing of the second embodiment and make a determination again.

[0092] (Fourth embodiment) The image diagnostic system of the fourth embodiment differs from the first embodiment in that post-processing is performed on the detection results of the first learning model 141, and therefore the following mainly describes this difference. Since the other configurations are the same as those of the first embodiment, the same reference numerals are used for the common configurations, and detailed descriptions thereof will be omitted.

[0093] The information processing device 1 of the fourth embodiment performs post-processing on the object region detection result obtained by the first learning model 141 to correct the boundary of the object region when the accuracy of the object region detection result is estimated to be low.

[0094] 11 is an explanatory diagram relating to correction of an object region. Using FIG. 11, the post-processing and object region correction method performed by the control unit 11 of the information processing device 1 will be specifically described.

[0095] An object region is detected from a medical image using a first learning model 141. The first learning model 141 is, for example, a semantic segmentation model, and generates a label image in which the object region is displayed in black and the region other than the object region is displayed in white, as shown in the upper right of Fig. 11. The control unit 11 sets multiple boundary points at predetermined intervals on the boundary line between the object region and other regions adjacent to the object region, as shown in the center right of Fig. 11.

[0096] Next, the output value of the activation function 141d included in the first learning model 141 is obtained. The output of the activation function 141d is obtained for each pixel of the medical image. Pixels whose output values ​​are within a predetermined range (e.g., 0.4 to 0.6) indicate a low probability of the detection result, while pixels whose output values ​​are outside the predetermined range indicate a high probability of the detection result. The concept of the output value for each pixel is shown in the center left of Figure 11. In Figure 11, pixels whose output values ​​are within the predetermined range are hatched downward to the right, and pixels whose output values ​​are outside the predetermined range are hatched downward to the left. In the example of Figure 11, the lower right of the medical image contains many pixels whose output values ​​are within the predetermined range, i.e., pixels with low probability.

[0097] The control unit 11 also divides the medical image into multiple regions. As described above, the medical image (tomographic image) obtained by the IVUS method is an image obtained by rotating the sensor 214, and therefore is a circular image centered on the rotation axis. The medical image is divided into multiple regions in the circumferential direction based on the center of the circle. Each region is, for example, a plurality of sector-shaped regions with the same central angle. Note that the method for dividing the medical image and the shape of the regions are not limited, but it is preferable to divide the medical image into regions by dividing the boundaries within the medical image in a substantially perpendicular direction. For example, the medical image may be divided into strip-like rectangular regions extending circumferentially from the center of the circle. Each region may simply separate the boundary points of the medical image, and some of the regions may overlap. While the above description uses a polar coordinate image as an example, the medical image is not limited to a polar coordinate image and may be a Cartesian coordinate image with the circumferential direction θ of the blood vessel and the radial direction r as its axes. In this case, the medical image is divided into multiple regions by equally dividing the medical image at predetermined intervals in the radial direction r.

[0098] The information processing device 1 calculates the reliability of the boundary points included in each region. The reliability is the degree of likelihood of the detection result. The reliability is calculated based on the output value of the activation function 141d of the pixels included in the region including the boundary point. The information processing device 1 obtains the reliability of the boundary points in the region by, for example, calculating the ratio of the number of pixels whose output values ​​are outside a predetermined range to all pixels in the region. The reliability may also be calculated based on the variance of the output values ​​of each pixel. If the reliability of all boundary points is high, the object region obtained by the first learning model 141 is output to the image diagnostic device 2 without correction. If there is a boundary point with low reliability, the detection result of the first learning model 141 is corrected. The horizontal hatching on the center left of Figure 11 indicates one region including a boundary point with high reliability, and the vertical hatching indicates one region including a boundary point with low reliability.

[0099] If there is a low-reliability boundary point, the object region is corrected by generating a new boundary by interpolation in place of the boundary including the low-reliability boundary point. The following describes the process of generating a new boundary by spline interpolation as an example. The information processing device 1 removes low-reliability boundary points by extracting high-reliability boundary points from all boundary points set on the boundary. Using multiple high-reliability boundary points, a new boundary connecting the boundary points is generated by connecting the high-reliability boundary points set on both sides of the removed boundary point with a spline curve by spline interpolation. If there are multiple consecutive low-reliability boundary points, it is advisable to interpolate between the high-reliability boundary points set on both sides of the consecutive low-reliability boundary points.

[0100] In this way, as shown in the lower part of FIG. 11, a new boundary line is generated in the portion where the detection accuracy of the object region is estimated to be low. A circular, i.e., closed curve, boundary line that complements the edge of the object region is generated using the new boundary line and the boundary line based on the detection result by the first learning model 141, and a new object region is formed by this boundary line. Note that the method of interpolating the boundary line is not limited. For example, circular interpolation may be used, or interpolation may be performed using the boundaries of previous and next frames. A machine learning model that generates an image of the boundary line according to the boundary points may also be used. The information processing device 1 outputs the new object region obtained by the above correction to the image diagnostic device 2.

[0101] FIG. 12 is an explanatory diagram illustrating an example of a display screen of the image diagnostic device 2. The image diagnostic device 2 displays a display screen 240 based on the detection result information received from the information processing device 1 on the display device 24. The display screen 240 includes a medical image field 241 and an object image field 242. The medical image field 241 displays medical images generated by the image processing device 23 in real time. The object image field 242 includes an object image that displays an object region included in the medical image in a distinguishable display manner. The object image is, for example, an image in which a label image indicating an object region 243 is superimposed on the original medical image. When a boundary line is interpolated, the control unit 11 corrects the object region of the label image according to the interpolated boundary. The control unit 11 processes the label image into a semi-transparent mask and generates image information to be superimposed on the original medical image. In this case, the control unit 11 may display the object region based on the detection result of the first learning model 141 and the interpolated object region in a distinguishable display manner by, for example, setting different transparency, colors, etc.

[0102] The object image may further include boundaries 244, 245 of the object region. The control unit 11 superimposes and displays the boundary 244 based on the detection result of the first learning model 141, or a boundary formed by the boundary 244 based on the detection result of the first learning model 141 and a new boundary 245 obtained by interpolation, on the original medical image. In this case, the control unit 11 may display the boundary 244 based on the detection result of the first learning model 141 and the new boundary 245 obtained by interpolation in a distinguishable display manner, for example, by setting different colors, line types, etc.

[0103] 13 is a flowchart showing an example of a processing procedure executed by the information processing device 1 in the fourth embodiment. When the MDU 22 executes a pullback operation and a medical image is output from the image diagnostic device 2, the control unit 11 of the information processing device 1 executes the following processing in accordance with the program P.

[0104] The control unit 11 of the information processing device 1 acquires medical images of a subject from the image diagnostic device 2 (step S61). The acquired medical images are images of multiple frames of tomographic images that are consecutive in time series. The control unit 11 inputs the acquired medical images into the first learning model 141 (step S62) to detect an object region (step S63). The control unit 11 acquires a label image indicating the object region.

[0105] The control unit 11 acquires the output value of the activation function 141d in the first learning model 141 for each pixel (step S64).

[0106] Based on the acquired output values, the control unit 11 calculates the reliability of each boundary point set on the boundary between the object region and other regions (step S65). Specifically, the control unit 11 divides the medical image into multiple fan-shaped regions and calculates the number of pixels included in each region whose output values ​​are outside a predetermined range. The control unit 11 calculates the ratio of pixels whose output values ​​are outside the predetermined range to all pixels in the region, thereby obtaining the reliability of the boundary points included in the region. The control unit 11 performs the above-mentioned process for all regions and calculates the reliability of all boundary points set on the boundaries.

[0107] The control unit 11 determines whether the calculated reliability of each boundary point is greater than or equal to a preset threshold value (e.g., 0.5), and extracts boundary points from all boundary points whose calculated reliability is greater than or equal to the threshold value (step S66).

[0108] The control unit 11 performs spline interpolation using the extracted boundary points (step S67). The control unit 11 corrects the boundary lines by generating new boundary lines (spline curves) that connect boundary points whose reliability is equal to or greater than the threshold and are set on both sides of the boundary points determined to be less than the threshold (step S68). A new object region is formed by the new boundary lines and the boundary lines based on the detection results by the first learning model 141. If the reliability of all boundary points is equal to or greater than the threshold, the above-mentioned correction process may be omitted.

[0109] The control unit 11 generates detection result information (screen information) for displaying the detected object region, etc. (step S69). The control unit 11 outputs the generated detection result information to the image diagnostic device 2 (step S70), and ends the series of processes.

[0110] According to this embodiment, by performing post-processing to correct the boundary, it is possible to reduce the influence of artifacts in the medical image and to suitably detect the object region.

[0111] In the above-described process, the control unit 11 of the information processing device 1 may be configured to accept user corrections to the interpolated boundary or the corrected object region. For example, when it is determined that the interpolated boundary should be corrected, the control unit 11 acquires correction data for the interpolated boundary by accepting user input, and stores the data as a new boundary.

[0112] The control unit 11 may further re-learn the first learning model 141 after correcting or modifying the object region. The control unit 11 acquires correction data or modification data for the object region for the medical image. The control unit 11 performs re-learning using the acquired correction data or modification data and the corresponding medical image as training data. The control unit 11 optimizes the weights, etc., of the first learning model 141 and updates the first learning model 141. According to the above-mentioned process, the first learning model 141 is optimized through the operation of this image diagnostic system, and the object region can be detected more effectively.

[0113] (Fifth embodiment) The image diagnostic system of the fifth embodiment differs from the fourth embodiment in that reliability is calculated using elements other than the output value of the activation function 141d, and therefore the following mainly describes this difference.

[0114] The information processing device 1 of the fifth embodiment calculates reliability using, for example, artifact detection results in medical images as a factor other than the output value of the activation function 141d. For example, if an artifact is included in a frame chronologically preceding a frame (hereinafter referred to as a target frame) for which reliability is to be calculated among multiple frames included in a medical image, there is a high possibility that an artifact will also occur in the target frame. From this perspective, the information processing device 1 calculates reliability based on the presence or absence of an artifact in the previous frame. If an artifact is present in the previous frame, the reliability for all boundary points in the target frame is set to be low. When acquiring the position of an artifact along with the presence or absence of an artifact, the reliability for boundary points within the region including the position of the target frame corresponding to the detected position of the artifact in the previous frame may be set to be low. Note that the information processing device 1 may calculate reliability using multiple frames subsequent to the previous frame.

[0115] Furthermore, the artifact detection result may include the type of artifact in addition to the presence or absence of the artifact. For example, if the type of artifact is a guidewire, there is a high possibility that boundary information of the lumen region to be detected is missing along the shape of the guidewire. In this way, based on the position and type of artifact, the reliability of boundary points within a region including a shape portion corresponding to the type of artifact is set to be low. The information processing device 1 pre-stores the type of artifact and the shape portion predicted to have a decrease in accuracy according to the type of artifact, in association with each other. Note that the information processing device 1 is not limited to using the artifact detection result in the previous frame, but may also use the artifact detection result in the target frame or a frame following the target frame.

[0116] 14 is a flowchart showing an example of a processing procedure executed by the information processing device 1 in the fifth embodiment. The same step numbers are assigned to the processes common to those in FIG. 13 of the fourth embodiment, and detailed descriptions thereof will be omitted.

[0117] The control unit 11 of the information processing device 1 acquires medical images of a subject from the image diagnostic device 2 (step S61). The acquired medical images are multiple frames of tomographic images that are consecutive in time series. The control unit 11 inputs the acquired medical images to the first learning model 141 (step S62) to detect an object region (step S63). The control unit 11 acquires an output value of the activation function 141d in the first learning model 141 for each pixel (step S64).

[0118] Furthermore, the control unit 11 acquires the artifact detection result in the frame preceding the frame for which the reliability is to be calculated (step S81). The method for acquiring the artifact detection result is not limited, but for example, the artifact region may be acquired by inputting the acquired medical image into a learning model that detects the artifact region from the medical image.

[0119] Based on the acquired output values ​​and artifact detection results, the control unit 11 calculates the reliability of each boundary point set on the boundary between the object region and other regions (step S65). Thereafter, the control unit 11 executes the processes of steps S66 to S70 shown in FIG.

[0120] According to this embodiment, by determining whether or not the boundary needs to be corrected using information other than the output value, it is possible to more appropriately detect the object area.

[0121] The examples shown in each of the above embodiments can be combined in whole or in part to realize other embodiments. The sequences shown in each embodiment are not limited to the above, and each processing step may be executed in a different order, or multiple processes may be executed in parallel.

[0122] The embodiments disclosed herein are illustrative in all respects and are not restrictive. The scope of the present invention is defined by the claims, and includes all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0123] 1. Information processing equipment 11 Control section 12 Main memory 13 Communications Department 14 Auxiliary storage P Program 141 First Learning Model 142 Second Learning Model 143 Third Learning Model 2. Diagnostic imaging equipment 21 Catheter 211 Probe section 212 Connector part 213 Shaft 214 Sensors 22 MDU 23 Image processing device 24 Display device

Claims

1. A medical image is acquired based on a signal detected by a catheter inserted into a hollow organ. Detecting an object area included in a medical image by inputting the acquired medical image into a model that has been trained to detect an object area included in the medical image when the medical image is input; Calculating a reliability for each of a plurality of boundary points located on the boundary between the detected object region and another image region in the medical image; correcting the boundary based on the detection result of the object region by the model by interpolating using boundary points whose calculated reliability satisfies a predetermined standard; The model detects the object region pixel by pixel using semantic segmentation, Using the model, an output of an activation function included in the model is obtained for each pixel included in a predetermined region of the medical image; Calculating the reliability of the boundary points included in a predetermined region of the medical image based on the ratio of pixels for which the output of the activation function is within a predetermined range to all pixels included in the predetermined region of the medical image A program that causes a computer to execute a process.

2. generating screen information that displays, in different display modes, a boundary based on the detection result of the object region by the model and a new boundary generated by correcting the boundary based on the detection result of the object region by the model; Output the generated screen information The program according to claim 1.

3. A new boundary is generated by performing spline interpolation using boundary points whose reliability satisfies a predetermined standard to correct the boundary based on the detection result of the object region. The program according to claim 1 or 2.

4. Obtain the output of an activation function included in the model; Calculating the reliability based on the obtained output of the activation function The program according to any one of claims 1 to 3.

5. The reliability is calculated based on the presence or absence of artifacts in the medical image. The program according to any one of claims 1 to 4.

6. an acquisition unit that acquires a medical image generated based on a signal detected by a catheter inserted into a hollow organ; a detection unit that detects an object region included in a medical image by inputting the acquired medical image into a model that has been trained to detect an object region included in the medical image when the medical image is input; a calculation unit that calculates a reliability for each of a plurality of boundary points located on the boundary between the detected object region and another image region in the medical image; a correction unit that corrects the boundary based on the detection result of the object region by the model by interpolating using boundary points whose calculated reliability satisfies a predetermined standard, The model detects the object region pixel by pixel using semantic segmentation, The calculation unit uses the model to obtain an output of an activation function included in the model for each pixel included in a predetermined region of the medical image; Calculating the reliability of the boundary points included in a predetermined region of the medical image based on the ratio of pixels for which the output of the activation function is within a predetermined range to all pixels included in the predetermined region of the medical image Information processing device.

7. A medical image generated based on a signal detected by a catheter inserted into a hollow organ is input into a model that has been trained to detect an object area included in the medical image when the medical image is input, thereby detecting an object area included in the medical image; Calculating a reliability for each of a plurality of boundary points located on the boundary between the detected object region and another image region in the medical image; correcting the boundary based on the detection result of the object region by the model by interpolating using boundary points whose calculated reliability satisfies a predetermined standard; The model detects the object region pixel by pixel using semantic segmentation, Using the model, an output of an activation function included in the model is obtained for each pixel included in a predetermined region of the medical image; Calculating the reliability of the boundary points included in a predetermined region of the medical image based on the ratio of pixels for which the output of the activation function is within a predetermined range to all pixels included in the predetermined region of the medical image Information processing methods.

Images