Computer program, information processing method, information processing apparatus, and model generation method

Abstract

The computer program of the present invention causes a computer to perform the following processing: obtaining a plurality of medical images through a catheter inserted into a luminal organ, the luminal organ having a main trunk, side branches branching from the main trunk, and branch portions of the main trunk and the side branches, the plurality of medical images being generated based on signals detected by a sensor while moving along the long side direction of the luminal organ; and identifying the cross-sections of the main trunk, the side branches, and the branch portions by inputting the obtained medical images into a learning model that identifies the cross-sections of the main trunk, the side branches, and the branch portions.

Description

Technical Field

[0001] The present invention relates to a computer program, an information processing method, an information processing apparatus, and a model generation method. Background Art

[0002] The following examination is being carried out: An intravascular ultrasound (IVUS) method using a catheter is used to generate a medical image including an ultrasonic tomographic image of a blood vessel, and an intravascular ultrasound examination is performed. In the intravascular ultrasound method, an intravascular diagnostic catheter moves an ultrasonic sensor from the distal end to the proximal end of the blood vessel, and scans the blood vessel and its surrounding tissues.

[0003] On the other hand, for the purpose of assisting a doctor's diagnosis, a technique for adding information to a medical image through image processing and machine learning is being developed.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2019-165970 Summary of the Invention

[0007] However, there is a problem that it is difficult to distinguish the main trunk and branches of a blood vessel by performing image recognition on the lumen boundary of a branch of a blood vessel in a single tomographic image, and useful information cannot be added to the medical image.

[0008] An object of the present invention is to provide a computer program, an information processing method, an information processing apparatus, and a model generation method that can analyze a medical image obtained by scanning a luminal organ to identify the main trunk and branches of a blood vessel.

[0009] The computer program of the present invention causes a computer to execute the following processing: obtaining a plurality of medical images by a catheter inserted into a luminal organ, the luminal organ having a main trunk, branches bifurcating from the main trunk, and a branch portion of the main trunk and the branches, the plurality of medical images being generated based on signals detected while a sensor moves along the long side direction of the luminal organ; and identifying the main trunk cross section, the branch cross section, and the branch portion cross section by inputting the obtained medical images into a learning model that identifies the main trunk cross section, the branch cross section, and the branch portion cross section.

[0010] The information processing method of the present invention is used to cause a computer to perform the following processing: obtaining a plurality of medical images through a catheter inserted into a luminal organ, the luminal organ having a main trunk, side branches branching from the main trunk, and branch portions of the main trunk and the side branches, the plurality of medical images being generated based on signals detected by a sensor while moving along the long side direction of the luminal organ; and identifying the main trunk cross-section, the side branch cross-section, and the branch portion cross-section by inputting the obtained medical images into a learning model that identifies the main trunk cross-section, the side branch cross-section, and the branch portion cross-section.

[0011] The information processing apparatus of the present invention includes: an obtaining unit that obtains a plurality of medical images through a catheter inserted into a luminal organ, the luminal organ having a main trunk, side branches branching from the main trunk, and branch portions of the main trunk and the side branches, the plurality of medical images being generated based on signals detected by a sensor while moving along the long side direction of the luminal organ; and a learning model that, when the obtained medical images are input, identifies the main trunk cross-section, the side branch cross-section, and the branch portion cross-section, and outputs information indicating the main trunk cross-section, the side branch cross-section, and the branch portion cross-section.

[0012] The model generation method of the present invention is used to cause a computer to perform the following processing: generating training data in which data indicating that they are luminal cross-sections is attached to a plurality of medical images, the plurality of medical images being generated by inserting a catheter into a luminal organ having a main trunk, side branches branching from the main trunk, and branch portions of the main trunk and the side branches, and being generated based on signals detected by a sensor while moving along the long side direction of the luminal organ, and being respectively a plurality of medical images including the main trunk cross-section, a plurality of medical images including the main trunk cross-section and the side branch cross-section, and a plurality of medical images including the branch portion cross-section; and generating a learning model that identifies the main trunk cross-section, the side branch cross-section, and the branch portion cross-section when a medical image is input based on the generated training data.

[0013] Advantages of the Invention

[0014] According to the present invention, it is possible to analyze medical images obtained by scanning a luminal organ to identify the main trunk and side branches of blood vessels Description of the Drawings

[0015] Figure 1 It is an explanatory diagram showing a configuration example of an image diagnostic apparatus.

[0016] Figure 2 It is a block diagram showing a configuration example of an image processing apparatus.

[0017] Figure 3 It is an explanatory diagram showing an image recognition method using a learning model.

[0018] Figure 4 It is an explanatory diagram showing bifurcated blood vessels.

[0019] Figure 5 It is an explanatory diagram showing a tomographic image of a blood vessel obtained by scanning a bifurcated blood vessel.

[0020] Figure 6 It is a flowchart showing the sequence of the information processing method of Embodiment 1.

[0021] Figure 7 It is a side view of a blood vessel showing a specific method of the branching structure of the blood vessel.

[0022] Figure 8 It is a cross-sectional view of a blood vessel showing a specific method of the branching structure of the blood vessel.

[0023] Figure 9 It is an explanatory diagram showing a display example of a guidance image.

[0024] Figure 10 It is a flowchart showing the sequence of the information processing method of Embodiment 2.

[0025] Figure 11A It is an explanatory diagram showing a model image of a main trunk and side branches.

[0026] Figure 11B It is an explanatory diagram showing a model image of a main trunk and side branches.

[0027] Figure 11C It is an explanatory diagram showing a model image of a main trunk and side branches.

[0028] Figure 12 It is an explanatory diagram showing a configuration example of an image diagnosis system.

[0029] Figure 13 It is a block diagram showing a configuration example of an information processing device.

[0030] Figure 14 It is a flowchart showing a method for generating a learning model. Detailed Embodiments

[0031] Hereinafter, specific examples of a computer program, an information processing method, an information processing device, and a model generation method according to embodiments of the present invention will be described with reference to the accompanying drawings.

[0032] (Embodiment 1)

[0033] Figure 1It is an explanatory diagram showing a configuration example of the image diagnostic apparatus 100. The image diagnostic apparatus 100 is a medical apparatus that generates a medical image including an ultrasonic tomographic image of a blood vessel (a luminal organ) by the intravascular ultrasound (IVUS) method, and performs intravascular ultrasonic examination and diagnosis. In particular, the image diagnostic apparatus 100 of Embodiment 1 analyzes the branch structure of a blood vessel, discriminates the main trunk A and the branch B that bifurcates, and overlays and displays the guiding images G1, G2 showing the branch structure (refer to Figure 9 ) on the medical image.

[0034] The image diagnostic apparatus 100 includes a catheter 1, an MDU (Motor Drive Unit) 2, an image processing apparatus (information processing apparatus) 3, a display apparatus 4, and an input apparatus 5. The image diagnostic apparatus 100 generates a medical image including an ultrasonic tomographic image of a blood vessel by the IVUS method using the catheter 1, and performs intravascular ultrasonic examination.

[0035] The catheter 1 is an image diagnostic catheter for obtaining an ultrasonic tomographic image of a blood vessel by the IVUS method. The catheter 1 has an ultrasonic probe at its distal end for obtaining an ultrasonic tomographic image of a blood vessel. The ultrasonic probe has an ultrasonic vibrator that emits ultrasonic waves in the blood vessel, and an ultrasonic sensor that receives reflected waves (ultrasonic echoes) reflected by the living tissue or medical device of the blood vessel. The ultrasonic probe is configured to be able to rotate around the circumference of the blood vessel and move forward and backward in the longitudinal direction of the blood vessel.

[0036] The MDU 2 is a driving device to which the catheter 1 can be detachably attached, and drives an in-built motor according to the operation of a medical practitioner, thereby controlling the movement of the catheter 1 inserted into the blood vessel. The MDU 2 rotates the ultrasonic probe of the catheter 1 in the circumferential direction while moving it from the distal (tip) side to the proximal (base) side (refer to Figure 3 ). The ultrasonic probe continuously scans the blood vessel at a prescribed time interval, and outputs the detected reflected wave data of the ultrasonic waves to the image processing apparatus 3.

[0037] The image processing apparatus 3 generates a plurality of medical images arranged in time series including a tomographic image of a blood vessel based on the reflected wave data output from the ultrasonic probe of the catheter 1 (refer to Figure 3 ). The ultrasonic probe scans the inside of the blood vessel while moving from the distal (tip) side to the proximal (base) side, and thus the plurality of medical images arranged in time series are tomographic images of the blood vessel observed at a plurality of positions within the range from the distal to the proximal.

[0038] The display apparatus 4 is a liquid crystal display panel, an organic EL display panel, etc., and displays the medical image generated by the image processing apparatus 3.

[0039] The input device 5 is an input interface such as a keyboard or a mouse that accepts the input of various set values during inspections and the operations of the image processing device 3. The input device 5 can be a touch panel, soft keys, hard keys, etc. provided on the display device 4.

[0040] Figure 2 It is a block diagram showing a configuration example of the image processing device 3. The image processing device 3 is a computer and includes a control unit 31, a main storage unit 32, an input / output I / F 33, and an auxiliary storage unit 34.

[0041] The control unit 31 is composed of one or more arithmetic processing devices such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-purpose computing on graphics processing units), and TPU (Tensor Processing Unit).

[0042] The main storage unit 32 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, and temporarily stores the data necessary for the control unit 31 to perform arithmetic processing.

[0043] The input / output I / F 33 is an interface to which the catheter 1, the display device 4, and the input device 5 are connected. The control unit 31 obtains the reflected wave data output from the ultrasonic probe via the input / output I / F 33. In addition, the control unit 31 outputs a medical image signal to the display device 4 via the input / output I / F 33. Moreover, the control unit 31 accepts the information input to the input device 5 via the input / output I / F 33.

[0044] The auxiliary storage unit 34 is a storage device such as a hard disk, EEPROM (Electrically Erasable Programmable ROM), or flash memory. The auxiliary storage unit 34 stores the computer program P to be executed by the control unit 31, various data necessary for the processing of the control unit 31. In addition, the auxiliary storage unit 34 stores the learning model 341.

[0045] The learning model 341 is a model for identifying a specified target contained in a medical image. For example, the learning model 341 can classify the target in pixel units by utilizing an image recognition technique using semantic segmentation, and can identify the lumen boundary of a blood vessel contained in a medical image.

[0046] In addition, the auxiliary storage unit 34 may also be an external storage device connected to the image processing device 3. The computer program P may be written into the auxiliary storage unit 34 at the manufacturing stage of the image processing device 3, or the image processing device 3 may obtain a program distributed from a remote server device through communication and store it in the auxiliary storage unit 34. The form of the computer program P may be recordable on a recording medium 3a such as a magnetic disk, an optical disk, or a semiconductor memory in a readable manner.

[0047] The control unit 31 performs the following processing: reads and executes the computer program P stored in the auxiliary storage unit 34, thereby obtaining a medical image generated by the image diagnostic device 100, and detecting the lumen boundary of a blood vessel contained in the medical image. Specifically, the control unit 31 uses the learning model 341 to detect the image region of the lumen boundary in the medical image. In particular, the control unit 31 of Embodiment 1 has a function of distinguishing the main trunk A and the branch B in the branch of the blood vessel. Further, the image processing device 3 outputs the recognition result of the lumen boundary to the image diagnostic device 100, and displays guidance images G1 and G2 (see Figure 9 ) representing the positions and regions of the main trunk A and the branch B in such a manner that it is easy for a medical practitioner to recognize the lumen boundaries of the main trunk A and the branch B as a result.

[0048] Figure 3 It is an explanatory diagram showing an image recognition method using the learning model 341. The learning model 341 learns in such a way that it can recognize the region of the lumen boundary of a blood vessel in a medical image in pixel units.

[0049] The learning model 341 is, for example, a convolutional neural network (CNN: Convolutional neural network) after learning based on deep learning. The learning model 341 performs recognition on the target in pixel units by using an image recognition technique called semantic segmentation.

[0050] The learning model 341 has an input layer 341a into which a medical image is input, an intermediate layer 341b that extracts and restores the feature amount of the image, and an output layer 341c that outputs a label image representing the target contained in the medical image in pixel units. The learning model 341 is, for example, a U-Net.

[0051] The input layer 341a of the learning model 341 has a plurality of neurons that receive the pixel values of each pixel contained in the medical image, and transmits the input pixel values to the intermediate layer 341b. The intermediate layer 341b has a convolutional layer (CONV layer) and a deconvolutional layer (DECONV layer). The convolutional layer is a layer that reduces the dimension of the image data. By reducing the dimension, the feature amount of the target is extracted. The deconvolutional layer performs deconvolution processing to restore to the original dimension. Through the restoration processing in the deconvolutional layer, a binarized label image is generated, which indicates whether each pixel in the image is the target. The output layer 341c has one or more neurons that output the label image. The label image is, for example, an image in which pixels corresponding to the lumen boundary of a blood vessel are of class "1" and pixels corresponding to other images are of class "0".

[0052] The learning model 341 can be generated by preparing training data and using this training data to perform machine learning on an unlearned neural network. The training data includes a medical image including the main trunk A, a label image representing the lumen boundary pixels and the center position of the main trunk A in the medical image, a medical image including the completely separated main trunk A and the side branch B, a label image representing the lumen boundary pixels and the center position of the main trunk A and the side branch B in the medical image, a medical image including the branch portion of the main trunk A and the side branch B, and a label image representing the lumen boundary pixels of the branch portion in the medical image. In the medical image including the branch portion, the main trunk cross-section and the side branch cross-section are combined.

[0053] According to the learning model 341 learned in this way, as Figure 3 shown, by inputting the medical image including the main trunk A into the learning model 341, a label image representing the region of the lumen boundary of the main trunk A in pixel units can be obtained. In addition, for the learning model 341, by inputting the medical image including both the main trunk A and the side branch B into the learning model 341, a label image representing the regions of the main trunk A and the side branch B in pixel units can be obtained. Moreover, for the learning model 341, by inputting the medical image including the branch portion of the main trunk A and the side branch B into the learning model 341, a label image representing the region of the branch portion in pixel units can be obtained. In the label image obtained here, the main trunk A and the side branch B are output in an indistinguishable manner.

[0054] Figure 4 is an explanatory diagram showing a bifurcated blood vessel, Figure 5 is an explanatory diagram showing a tomographic image of a blood vessel obtained by scanning a bifurcated blood vessel.

[0055] Figure 4 Among them, the tomographic images in the planes indicated by the reference numerals a to f are Figure 5As shown, in the case where the main trunk A and the branch B branching off from the main trunk A are completely separated (the cross-section indicated by reference numeral f), the main trunk A and the branch B in the medical image can be easily understood. However, in the branching portion of the blood vessel (the cross-sections indicated by reference numerals c, d, and e, etc.), the structures of the main trunk A and the branch B are not easily understood.

[0056] The image processing apparatus 3 according to the first embodiment executes the following processing: identifying the structures of the main trunk A and the branch B of such a branching portion, and displaying the guidance images G1 and G2 that support the identification of the branching structure performed by a medical practitioner.

[0057] Figure 6 is a flowchart showing the order of the information processing method, Figure 7 is a side view of a blood vessel showing a specific method of the branching structure of the blood vessel, Figure 8 is a cross-sectional view of a blood vessel showing a specific method of the branching structure of the blood vessel.

[0058] The control unit 31 obtains a plurality of medical images arranged in time series from the image diagnostic apparatus 100 (step S11). The plurality of medical images arranged in time series obtained here are, for example, tomographic images observed in the range from the distal end to the proximal end of the blood vessel.

[0059] The control unit 31 selects a plurality of medical images in the distal portion (step S12). The control unit 31 only needs to select at least two medical images including the completely separated main trunk A and branch B. For example, as Figure 7 shown, the control unit 31 only needs to select the medical image in the cross-section f and the medical image in the cross-section g.

[0060] And, the control unit 31 inputs the medical images selected in step S12 into the learning model 341, and detects the lumen boundaries and the central positions of the main trunk A and the branch B included in the medical images (step S13).

[0061] Next, as Figure 8 shown, the control unit 31 specifies the branching structure of the blood vessel based on the detection result of step S13 (step S14). The time difference between the shooting times of the two images selected in step S12 corresponds to the length in the long side direction of the main trunk A. Specifically, the product of the moving speed of the ultrasonic probe during blood vessel scanning and the time difference corresponds to the length of the portion captured by the two medical images. Figure 8 The straight line (dashed line) passing through the center of the main trunk A included in the two medical images in

[0062] When the central coordinates of the main trunk A captured at the scanning time point t1 are (x1, y1), and the central coordinates of the main trunk A captured at the scanning time point t2 = t + Δt are (x2, y2), the central coordinates (xN, yN) of the main trunk A in the branch captured at the scanning time point tN = t + NΔt are expressed by the following mathematical formulas (1) and (2).

[0063] xN = x1 + (x2 - x1) × N…(1)

[0064] yN = y1 + (y2 - y1) × N…(2)

[0065] When the diameter of the main trunk A captured at the scanning time point t1 is r1, and the diameter of the main trunk A captured at the scanning time point t2 = t + Δt is r2, the central coordinate rN of the main trunk A in the branch captured at the scanning time point tN = t + NΔt is expressed by the following mathematical formula (3).

[0066] rN = r1 + (r2 - r1) × N…(3)

[0067] The above mathematical formula (3) describes one diameter, but the major axis and minor axis of the main trunk A can be calculated in the same way.

[0068] In addition, the above mathematical formulas (1) to (3) illustrate an example of estimating the central position and diameter of blood vessels in the branch by linear interpolation based on the central position and diameter of the main trunk obtained from two medical images. However, it can also be the case that the central position and diameter of blood vessels in other medical images are calculated by polynomial interpolation based on the central positions of blood vessels in three or more medical images.

[0069] Similarly, the diameter of the inner cavity boundary of the main trunk A can be estimated. In addition, the central position and diameter in the medical image of the collateral branch B can also be calculated by the same method.

[0070] The control unit 31 specifies the area where the straight line representing the inner cavity boundary of the main trunk A intersects the straight line representing the inner cavity boundary of the collateral branch B as the branch.

[0071] After completing the processing of step S14, the control unit 31 selects the medical image in the branch (step S15) and detects the inner cavity boundaries of the main trunk A and the collateral branch B contained in the medical image (step S16). The area of the inner cavity boundary of the main trunk A and the area of the inner cavity boundary of the collateral branch B in the branch are recognized as an area such as local connection.

[0072] Further, the control unit 31 specifies the main trunk A portion and the side branch B portion (step S17) in the region of the inner cavity boundary detected in step S16 based on the information representing the branch structure specified in step S14. From the information on the branch structure specified in step S14, the central position and diameter of the main trunk A in the medical image of the branch can be obtained. That is, the control unit 31 can obtain an elliptical line representing the main trunk A in the medical image. The control unit 31 can identify, as the inner cavity boundary of the main trunk A, the region near the elliptical line of the main trunk A obtained from the information on the branch structure among the inner cavity boundaries detected in step S16.

[0073] Similarly, the control unit 31 can identify, as the inner cavity boundary of the main trunk A, the region near the elliptical line of the side branch B obtained from the information on the branch structure among the inner cavity boundaries detected in step S16.

[0074] Next, the control unit 31 selects the medical image in the proximal portion compared to the branch (step S18). If there is one branch, the medical image selected in step S18 includes only the main trunk A. The control unit 31 detects the inner cavity boundary and the central position of the main trunk A included in the medical image in the proximal portion (step S19).

[0075] Further, the control unit 31 overlays and displays the guiding images G1 and G2 representing the main trunk A portion and the side branch B portion in the medical image on the medical image (step S20).

[0076] Figure 9 FIG. is an explanatory diagram showing a display example of the guiding images G1 and G2. As Figure 9 shown, in the medical image of the branch, the cross-sections of the main trunk A and the side branch B are partially combined, and it is difficult to clearly distinguish the inner cavity boundary of the main trunk A portion and the inner cavity boundary of the side branch B portion. Therefore, the control unit 31 overlays and displays the guiding image G1 corresponding to the region of the inner cavity boundary of the main trunk A in the medical image on the medical image. The guiding image G1 is an image having a shape substantially the same as the region of the inner cavity boundary of the main trunk A.

[0077] In addition, the control unit 31 overlays and displays the guiding image G2 corresponding to the region of the inner cavity boundary of the side branch B in the medical image on the medical image. The guiding image G2 is an image having a shape substantially the same as the region of the inner cavity boundary of the side branch B.

[0078] In addition, the control unit 31 may display the guide images G1 and G2 in different forms. For example, the guide images G1 and G2 may be displayed in different line types and colors. In addition, the original image without overlapping the guide images G1 and G2 and the image with overlapping the guide images G1 and G2 on the medical image may be displayed side by side. Furthermore, the image with overlapping the guide images G1 and G2 and the original image without overlapping the guide images G1 and G2 may be selectively switched to be displayed.

[0079] Alternatively, the control unit 31 may display the guide images G1 and G2 in a manner that complements the inner cavity boundary of the trunk A or the side branch B that is not displayed in the medical image. Figure 9 In the example shown, a portion of the lumen boundary that should be annular is notched, but the lumen boundary of the trunk A and the lumen boundary of the side branch B may be displayed by inferring the notched portion.

[0080] According to the computer program P, the image processing device 3, and the information processing method configured in this way, a medical image obtained by scanning a blood vessel can be analyzed to identify the trunk A and the side branches B of the blood vessel.

[0081] In addition, guide images G1 and G2 showing the inner cavity boundaries of the trunk A and the side branches B of the blood vessel can be displayed to assist medical professionals in recognizing the inner cavity boundaries of the trunk A and the side branches B.

[0082] It should be noted that the image processing device 3 , the computer program P, and the information processing method described in the first embodiment are merely examples and are not limited to the configuration of the first embodiment.

[0083] For example, in the first embodiment, blood vessels are taken as an example of the observation or diagnosis object, but the present invention can also be applied when observing lumen organs such as intestines other than blood vessels.

[0084] Furthermore, although an ultrasonic image is described as an example of a medical image, the medical image is not limited to an ultrasonic image and may be, for example, an OCT (Optical Coherence Tomography) image.

[0085] (Implementation Method 2)

[0086] The image processing device of the second embodiment is different from the first embodiment in that the lumen boundary and plaque of the blood vessel are recognized, the diameter, cross-sectional area, volume, etc. of the trunk A and the side branch B can be calculated and displayed, and the model image that reproduces the branch structure can be generated and displayed. Therefore, the above differences are mainly described below. The other configurations and effects are the same as those of the first embodiment, and therefore, the same reference numerals are marked at corresponding places and detailed descriptions are omitted.

[0087] The learning model 341 of Embodiment 2 is a model for identifying the lumen boundary and plaque of blood vessels contained in, for example, medical images.

[0088] Figure 10 It is a flowchart showing the sequence of the information processing method of Embodiment 2. The control unit 31 of Embodiment 2 performs the same processing as steps S11 to S20 of Embodiment 1, but in steps S33, S36, and S39, it detects the lumen boundary and plaque of blood vessels.

[0089] Next, the control unit 31 overlays and displays a guidance image representing the plaque on the medical image (step S40). In addition, the medical practitioner can use the input device 5 to select whether a guidance image representing the plaque is needed. When the control unit 31 receives content indicating that the guidance image is not needed, it does not display the guidance image, and when it receives content indicating that the guidance image is needed, it displays the guidance image.

[0090] Next, the control unit 31 calculates the cross-sectional diameter, cross-sectional area, or volume per unit length of the lumen of the main trunk A, and overlays and displays the calculated cross-sectional diameter, cross-sectional area, and volume of the lumen of the main trunk A on the display device 4 (step S41).

[0091] Similarly, the control unit 31 calculates the cross-sectional diameter, cross-sectional area, or volume per unit length of the lumen of the branch B based on the branch structure specified in step S34, and displays the calculated cross-sectional diameter, cross-sectional area, and volume of the lumen of the branch B on the display device 4 (step S42). However, as Figure 8 shown, the tomographic image of the branch B scanned by the catheter 1 inserted into the main trunk A is a tomographic image as if the branch B is cut obliquely (hereinafter referred to as an oblique tomographic image). Therefore, the control unit 31 converts it into a tomographic image in which the branch B is cut substantially vertically, that is, a tomographic image cut by a plane substantially perpendicular to the center line of the branch B (hereinafter referred to as an axial tomographic image) for the cross-sectional diameter, cross-sectional area, or volume of the lumen. For example, calculate the angle θ at which the center line of the main trunk A intersects the center line of the branch B, use the angle θ to convert the oblique tomographic image into an axial image, and calculate the cross-sectional diameter, cross-sectional area, or volume per unit length of the lumen of the branch B.

[0092] Next, the control unit 31 reproduces a model image representing the branch structure of the blood vessel based on the branch structure calculated in step S34 and the recognition results of the lumen boundary and plaque detected from the medical image, and displays it on the display device 4 (step S43).

[0093] Figures 11A to 11C It is an explanatory diagram showing the model images of the main trunk A and the branch B. For example, Figures 11A to 11CAs shown in the upper figure above, the control unit 31 can generate a cross-section of a blood vessel as a model image. This model image represents the vessel wall portion, the lumen boundary, and the plaque portion of the blood vessel.

[0094] In addition, as Figures 11A to 11C shown in the lower figure below, the control unit 31 can generate a cross-sectional image of the main trunk A and the branch B as a model image. This model image also shows the vessel wall portion, the lumen boundary, and the plaque portion of the blood vessel. Moreover, when generating the cross-sectional image of the branch B, the control unit 31 can generate a model image converted into an axial image.

[0095] According to the computer program P, the image processing apparatus 3, and the information processing method configured as described above, it is possible to display a guidance image representing a plaque formed on the vessel wall, and it is possible to support the identification of the plaque portion by a medical practitioner.

[0096] In addition, it is possible to calculate and display the cross-sectional diameter, area, and volume of the lumens of the main trunk A and the branch B.

[0097] Moreover, it is possible to generate and display a model image representing a branch structure, and it is possible to support the identification of the branch portion of the blood vessel by a medical practitioner. In particular, by displaying a model image of the cross-section showing the lumen boundary and the plaque of the main trunk A and the branch B, it is possible to more easily identify the plaque formed on the blood vessel.

[0098] Furthermore, by displaying the axial cross-sections of the main trunk A and the branch B, it is possible to more easily identify the plaque formed on the blood vessel.

[0099] (Embodiment 3)

[0100] Figure 12 FIG. is an explanatory diagram showing a configuration example of an image diagnosis system. The difference in the image diagnosis system of Embodiment 3 is that the information processing apparatus 6 as a server executes analysis processing of medical images. Therefore, the above difference will be mainly described below. Other configurations and effects are the same as those of Embodiment 1 or 2. Therefore, the same reference numerals are used in corresponding parts and detailed descriptions are omitted.

[0101] The image diagnosis system of Embodiment 3 includes an information processing apparatus 6 and an image diagnosis apparatus 200. The information processing apparatus 6 and the image diagnosis apparatus 200 are communicatively connected via a network N such as a LAN (Local Area Network) or the Internet. The information processing apparatus 6 and the image diagnosis apparatus 200 are communicatively connected via a network N such as a LAN (Local Area Network) or the Internet.

[0102] Figure 13FIG. 0 is a block diagram showing a configuration example of the information processing apparatus 6. The information processing apparatus 6 is a computer, and includes a control unit 61, a main storage unit 62, a communication unit 63, and an auxiliary storage unit 64. The communication unit 63 is a communication circuit for exchanging data with the image processing apparatus 3 via the network N. The hardware configurations of the control unit 61, the main storage unit 62, the communication unit 63, and the auxiliary storage unit 64 are the same as those of the image processing apparatus 3 described in the first embodiment. The computer program P, the learning model 641, and the recording medium 6a stored in the auxiliary storage unit 64 are also the same as the various programs and models in the first or second embodiment.

[0103] In addition, the information processing apparatus 6 may be a multi-computer composed of multiple computers, or a virtual machine virtually constructed by software. The information processing apparatus 6 may be a local server installed in the same facility (such as a hospital) as the image diagnosis apparatus 200, or a cloud server communicatively connected to the image diagnosis apparatus 200 via the Internet or the like.

[0104] Figure 14 FIG. 7 is a flowchart showing a method for generating the learning model 641. The control unit 61 collects a plurality of medical images including the main trunk cross-section (step S51). For example, the control unit 61 collects medical images from the image diagnosis apparatus 200.

[0105] Next, the control unit 61 collects a plurality of medical images including the main trunk cross-section and the side branch cross-section (step S52). Similarly, the control unit 61 collects a plurality of medical images including the branch cross-section (step S53).

[0106] In addition, the control unit 61 collects a plurality of medical images including the main trunk cross-section with plaques formed thereon (step S54). In addition, the control unit 61 collects a plurality of medical images including the main trunk cross-section and the side branch cross-section with plaques formed thereon (step S55). Similarly, the control unit 61 collects a plurality of medical images including the branch cross-section with plaques formed thereon (step S56).

[0107] Next, the control unit 61 generates training data in which the label image is associated with the medical images collected in steps S51 to S56 (step S57). The label image of the medical image including the main trunk cross-section is an image of pixels representing the inner cavity boundary of the main trunk A and the plaque. The label image of the medical image including the main trunk cross-section and the branch cross-section is an image of pixels representing the inner cavity boundaries of the main trunk A and the branch B and the plaque. The label image of the medical image including the branch cross-section is an image of pixels representing the inner cavity boundary of the branch of the main trunk A and the branch B and the plaque. The label image of the medical image including the main trunk cross-section with a plaque formed thereon is an image of pixels representing the plaque together with the pixels of the inner cavity boundary of the main trunk A and the plaque. The label image of the medical image including the main trunk cross-section with a plaque formed thereon and the branch cross-section is an image of pixels representing the plaque together with the pixels of the inner cavity boundaries of the main trunk A and the branch B and the plaque. The label image of the medical image including the branch cross-section with a plaque formed thereon is an image of pixels representing the plaque together with the pixels of the inner cavity boundary of the branch of the main trunk A and the branch B and the plaque.

[0108] Furthermore, the control unit 61 uses the generated training data to perform machine learning on the unlearned neural network, thereby generating a learning model 641 (step S58).

[0109] Based on the learning model 641 learned in this way, by inputting the medical image including the main trunk A into the learning model 641, a label image representing the region of the inner cavity boundary of the main trunk A and the region of the plaque part in pixel units can be obtained. In addition, for the learning model 641, by inputting the medical image including both the main trunk A and the branch B into the learning model 641, a label image representing the regions of the main trunk A and the branch B and the region of the plaque part in pixel units can be obtained. By inputting the medical image including the branch of the main trunk A and the branch B into the learning model 641, a label image representing the region of the inner cavity boundary of the branch and the region of the plaque part in pixel units can be obtained.

[0110] The information processing device 6 configured in this way obtains a medical image from the image processing device 3 via the network N, performs the same processing as the image processing device 3 of the first embodiment based on the obtained medical image, and sends the recognition result of the target to the image device. The image processing device 3 obtains the recognition result of the target sent from the information processing device 6 and, as Figure 9 shown, overlaps the guide images G1 and G2 representing the regions of the main trunk A and the branch B of the blood vessel on the medical image and displays them on the display device 4.

[0111] In the information processing device 6, the computer program P, and the information processing method of the third embodiment as well, similar to the first embodiment, it is possible to analyze the medical image obtained by scanning the blood vessel and identify the main trunk A and the branch B of the blood vessel.

[0112] In the embodiments of the present invention, all points are examples and should not be considered as restrictive content. The scope of the present invention is not limited to the above meaning, but also includes all changes within the meaning and scope equivalent to the technical solution shown by the technical solution.

[0113] Description of Reference Numerals

[0114] 1 catheter

[0115] 2 MDU

[0116] 3 image processing device

[0117] 3a recording medium

[0118] 4 display device

[0119] 5 input / output device

[0120] 6 information processing device

[0121] 6a recording medium

[0122] 31 control unit

[0123] 32 main storage unit

[0124] 33 input / output I / F

[0125] 34 auxiliary storage unit

[0126] 61 control unit

[0127] 62 main storage unit

[0128] 63 communication unit

[0129] 64 auxiliary storage unit

[0130] 341 learning model

[0131] P computer program

[0132] A main trunk

[0133] B branch.

Claims

1. A computer program, characterized in that, it is used to cause a computer to perform the following processing: obtaining a plurality of medical images through a catheter inserted into a luminal organ, the luminal organ having a main trunk, side branches branching from the main trunk, and branch portions of the main trunk and the side branches, the plurality of medical images being generated based on signals detected while a sensor moves along the long side direction of the luminal organ; identifying the main trunk cross-section, the side branch cross-section, and the branch portion cross-section by inputting the obtained medical images into a learning model that identifies the main trunk cross-section, the side branch cross-section, and the branch portion cross-section; outputting, through the learning model, a label image representing at least one region of the main trunk cross-section, the side branch cross-section, and the branch portion cross-section in pixel units.

2. The computer program according to claim 1, characterized in that, it is used to cause a computer to perform the following processing: identifying at least the main trunk cross-section or the side branch cross-section by inputting the obtained first medical image into the learning model; identifying the branch portion cross-section by inputting the obtained second medical image into the learning model; distinguishing the main trunk portion and the side branch portion constituting the branch portion cross-section based on the recognition result obtained from the first medical image.

3. The computer program according to claim 2, characterized in that, it is used to cause a computer to perform the following processing: specifying the branch structure of the main trunk and the side branches based on the recognition results obtained from a plurality of first medical images; specifying the main trunk portion and the side branch portion included in the branch portion cross-section based on the specified branch structure.

4. The computer program according to claim 3, characterized in that, it is used to cause a computer to perform the following processing: the learning model recognizes plaques formed in the branch portion; identifying the branch portion cross-section and the plaques by inputting the obtained second medical image into the learning model; specifying the plaque portion included in the branch portion cross-section based on the branch structure.

5. The computer program according to any one of claims 2 to 4, characterized in that, it is used to cause a computer to perform the following processing: specifying the branch structure of the main trunk and the side branches based on the recognition results obtained from a plurality of first medical images, and calculating the cross-sectional diameter, cross-sectional area, or volume per unit length of the side branches.

6. The computer program according to any one of claims 2 to 5, characterized in that, it is used to cause a computer to perform the following processing: specifying the branch structure of the main trunk and the side branches based on the recognition results obtained from a plurality of first medical images, and generating a model image of the main trunk and the side branches.

7. The computer program according to any one of claims 1 to 6, characterized in that, it is used to cause a computer to perform the following processing: overlaying an image representing the main trunk portion or the side branch portion on a medical image including the branch portion cross-section.

8. The computer program according to any one of claims 1 to 7, characterized in that, it is used to cause a computer to perform the following processing: the luminal organ is a blood vessel, and medical images of the blood vessel generated based on detected signals are obtained through the catheter.

9. An information processing method, characterized in that, it is used to cause a computer to perform the following processing: A plurality of medical images are obtained through a catheter inserted into a luminal organ, the luminal organ having a main trunk, side branches branching off from the main trunk, and branch portions of the main trunk and the side branches, and the plurality of medical images being generated based on signals detected while a sensor moves along the long side direction of the luminal organ; The main trunk cross section, the side branch cross section, and the branch portion cross section are identified by inputting the obtained medical images into a learning model that identifies the main trunk cross section, the side branch cross section, and the branch portion cross section; Through the learning model, a label image representing at least one region of the main trunk cross section, the side branch cross section, and the branch portion cross section in pixel units is output.

10. An information processing apparatus, characterized in that, it has: An acquisition unit that obtains a plurality of medical images through a catheter inserted into a luminal organ, the luminal organ having a main trunk, side branches branching off from the main trunk, and branch portions of the main trunk and the side branches, and the plurality of medical images being generated based on signals detected while a sensor moves along the long side direction of the luminal organ; and A learning model that, when the obtained medical images are input, identifies the main trunk cross section, the side branch cross section, and the branch portion cross section, and outputs information representing the main trunk cross section, the side branch cross section, and the branch portion cross section, Through the learning model, a label image representing at least one region of the main trunk cross section, the side branch cross section, and the branch portion cross section in pixel units is output.

11. A model generation method, characterized in that, it causes a computer to execute the following processing: Generate training data with data indicating that they are luminal cross sections attached to a plurality of medical images, the plurality of medical images being generated by inserting a catheter into a luminal organ having a main trunk, side branches branching off from the main trunk, and branch portions of the main trunk and the side branches, and being based on signals detected while a sensor moves along the long side direction of the luminal organ, and being respectively a plurality of medical images including the main trunk cross section, a plurality of medical images including the main trunk cross section and the side branch cross section, and a plurality of medical images including the branch portion cross section; And generate a learning model that identifies the main trunk cross section, the side branch cross section, and the branch portion cross section when medical images are input, based on the generated training data, Through the learning model, a label image representing at least one region of the main trunk cross section, the side branch cross section, and the branch portion cross section in pixel units is output.

Images