Information processing method, program and image diagnosis device

Abstract

This information processing method causes a computer to execute processing for: obtaining a cross-sectional image of a blood vessel and device information pertaining to a treatment device used for intravascular treatment; detecting an object including a guide wire from the cross-sectional image; and determining whether or not the treatment device affects a prescribed object when the treatment device is inserted into the blood vessel, on the basis of the guide wire detection results and the device information.

Description

Information processing method, program, and image diagnostic apparatus

[0001] The present invention relates to an information processing method, a program, and an image diagnostic apparatus.

[0002] There are systems that support endovascular treatment such as PCI (Percutaneous Coronary Intervention). For example, Patent Document 1 discloses a program or the like that outputs perforation information related to vascular perforation by inputting a plurality of contrast images captured when an endovascular treatment device is inserted into a blood vessel into a machine learning model.

[0003] Japanese Unexamined Patent Publication No. 2024-142135

[0004] In one aspect, an object of the present invention is to provide an information processing method or the like that can suitably evaluate the influence exerted by a treatment device.

[0005] (1) An information processing method, wherein a computer executes processing including: acquiring a cross-sectional image of a blood vessel and device information relating to a treatment device used for endovascular treatment; detecting an object including a guide wire from the cross-sectional image; and determining whether the treatment device affects a predetermined object when the treatment device is inserted into the blood vessel based on a detection result of the guide wire and the device information.

[0006] (2) In the information processing method according to (1) above, a center position of the treatment device when the treatment device is inserted into the blood vessel is specified based on the detection result of the guide wire, and whether the treatment device affects the object is determined based on the center position and the device information.

[0007] (3) In the information processing method according to (2) above, an assumed region that the treatment device is expected to affect when the treatment device is inserted into the blood vessel is specified based on the center position and the device information, and whether the treatment device affects the object is determined by determining whether the assumed region overlaps the object.

[0008] (4) The information processing method described in (2) above calculates the distance between the center position and the object, and determines whether the therapeutic device affects the object by determining whether the calculated distance is less than or equal to a threshold determined according to the device information.

[0009] (5) Any of the information processing methods described in (2) to (4) above acquires the cross-sectional images of multiple frames captured along the longitudinal direction of the blood vessel, detects the object including the guide wire from each frame, identifies the trajectory of the guide wire based on the detection result of the guide wire in each frame, and determines the center position based on the trajectory of the guide wire.

[0010] (6) The information processing method of (5) above identifies a straight trajectory in a frame a predetermined number of frames prior to the target frame that is in contact with the trajectory of the guide wire and whose inclination and direction coincide with the trajectory of the guide wire, using the cross-sectional image of each frame as the target frame, and identifies a point on the straight line connecting the position of the guide wire in the target frame and the straight trajectory as the center position.

[0011] (7) The information processing method described in (5) or (6) above generates a three-dimensional model of the blood vessel and guidewire based on the cross-sectional image of each frame and the detection result of the guidewire in each frame, and identifies the trajectory of the guidewire based on the three-dimensional model.

[0012] (8) The information processing method of (7) above further acquires sensor position information relating to the position of the sensor used to capture the cross-sectional image of each frame on the extravascular image of the blood vessel captured from outside the blood vessel, and information on the distance traveled by the sensor between each frame, generates a three-dimensional model of the blood vessel and guide wire based on the cross-sectional image of each frame and the detection result of the guide wire in each frame, determines the bending state of the blood vessel based on the sensor position information and the distance traveled information, and deforms the three-dimensional model according to the determination result of the bending state.

[0013] (9) If any of the information processing methods described in (1) to (8) above determines that the therapeutic device is affecting the object, it outputs a warning.

[0014] (10) The information processing method described in (9) above changes the output mode of the warning according to the degree to which the therapeutic device has an effect on the object.

[0015] (11) The information processing method of (9) or (10) above identifies the affected area of ​​the object that is affected by the therapeutic device based on the detection result of the guide wire and the device information, identifies the direction in which a predetermined organ is located in the cross-sectional image based on the detection result of the object, and changes the output mode of the warning depending on whether the affected area is located on the organ side within the object.

[0016] (12) Any of the information processing methods described in (1) to (11) above identifies the affected area of ​​the object that is affected by the therapeutic device based on the detection result of the guide wire and the device information, and outputs the cross-sectional image which displays the affected area in a different display manner from other areas.

[0017] (13) The information processing method of (12) above outputs a cross-sectional image which displays the region obtained by extending the affected region by a predetermined distance toward the object side in a different display manner from the other regions.

[0018] (14) In any of the information processing methods described in (1) to (13) above, the device information includes at least one of the type, dimensions, and operating parameters of the therapeutic device.

[0019] (15) Any of the information processing methods described in (1) to (14) above detects the object by inputting the acquired cross-sectional image into a model that has been trained to detect the object when the cross-sectional image is input.

[0020] (16) The program obtains a cross-sectional image of a blood vessel and device information relating to a therapeutic device used for endovascular treatment, detects an object including a guidewire from the cross-sectional image, and causes the computer to perform a process to determine whether the therapeutic device will affect a predetermined object when the therapeutic device is inserted into the blood vessel, based on the detection result of the guidewire and the device information.

[0021] (17) The diagnostic imaging device is a diagnostic imaging device comprising a control unit, wherein the control unit acquires a cross-sectional image of a blood vessel and device information relating to a therapeutic device used for endovascular treatment, detects an object including a guidewire from the cross-sectional image, and determines, based on the detection result of the guidewire and the device information, whether or not the therapeutic device will affect a predetermined object when the therapeutic device is inserted into the blood vessel.

[0022] In one respect, it allows for a favorable evaluation of the effects of therapeutic devices.

[0023] This is a diagram showing an example of the configuration of an image diagnostic system. This is a block diagram showing an example of the configuration of an image diagnostic device. This is an explanatory diagram regarding object detection processing. This is an explanatory diagram regarding the generation of a 3D model. This is a diagram showing an example of the display of a cross-sectional image. This is an explanatory diagram regarding the process of identifying the assumed region. This is a flowchart showing an example of the processing procedure executed by the image diagnostic device. This is an explanatory diagram regarding the deformation processing of a 3D model of a blood vessel. This is an explanatory diagram regarding the process of determining the bending state. This is a flowchart showing an example of the processing procedure executed by the image diagnostic device according to Embodiment 2. This is a diagram showing an overview of Embodiment 3. This is a flowchart showing an example of the processing procedure executed by the image diagnostic device according to Embodiment 3.

[0024] The present invention will be described in detail below based on the drawings illustrating its embodiments. (Embodiment 1) Figure 1 is a diagram showing an example of the configuration of an image diagnostic system. In this embodiment, an image diagnostic system that determines whether or not a therapeutic device inserted into a blood vessel affects objects such as plaque and the blood vessel wall will be described. The image diagnostic system includes an image diagnostic device 1 and a fluoroscopic image acquisition device 2. Each device is connected to each other via a network such as a LAN (Local Area Network).

[0025] The diagnostic imaging device 1 is a device that generates cross-sectional images of a patient's blood vessels, and is, for example, a diagnostic imaging device that uses a diagnostic imaging catheter with IVUS (Intravascular Ultrasound) functionality. The diagnostic imaging device 1 is connected to the catheter 101 via an MDU (Motor Drive Unit) 102.

[0026] In this embodiment, the catheter 101 is described as a catheter equipped with IVUS functionality, but the catheter 101 may also be a catheter equipped with OCT (Optical Coherence Tomography) functionality, or a dual-type catheter equipped with both IVUS and OCT functionality.

[0027] The catheter 101 is a medical device inserted into the patient's blood vessel and is equipped with a sensor that transmits and receives ultrasound waves. The sensor emits ultrasound waves radially based on a control signal output by the MDU 102. The diagnostic imaging device 1 acquires line data indicating the intensity with respect to distance from the sensor based on the reflected ultrasound waves and generates a cross-sectional image of the blood vessel.

[0028] The MDU 102 is a drive unit to which the catheter 101 is detachably attached. By driving a built-in motor in response to user operation, it controls the longitudinal and rotational movement of the imaging core (sensor and shaft) of the catheter 101 inserted into the blood vessel.

[0029] The fluoroscopic image acquisition device 2 is a device for acquiring fluoroscopic images (extravascular images) of the inside of a patient's body, and is, for example, an angiography device used for angiography. The fluoroscopic image acquisition device 2 is equipped with an X-ray source and an X-ray sensor, and generates an X-ray fluoroscopic image of the patient by receiving X-rays irradiated from the X-ray source with the X-ray sensor. An X-ray opaque marker is attached to the tip of the catheter 101, and the position of the catheter 101 is visualized in the fluoroscopic image.

[0030] In this embodiment, the "extravascular image" obtained by imaging a blood vessel from outside the blood vessel is described as a fluoroscopic image related to angiography, but the "extravascular image" may be an image from another modality such as CT (Computed Tomography) or MRI (Magnetic Resonance Imaging).

[0031] This embodiment describes a case where PCI treatment is performed to remove lesional tissue within a blood vessel. Specifically, it considers a case where a relatively hard lesion, such as a calcified lesion, is removed using a treatment device. The "treatment device" in this case is, for example, a rotablator, diamondback, or excimer laser, but is not particularly limited.

[0032] Because these therapeutic devices are highly invasive, contact with healthy blood vessel walls, lipid-rich plaques, etc., can lead to adverse events such as distal occlusion and cardiac tamponade. Therefore, there is a demand to confirm in advance whether the therapeutic device can remove only the target lesion.

[0033] Therefore, in this embodiment, based on a cross-sectional image of the blood vessel captured by the catheter 101, the system performs a process to determine in advance whether a therapeutic device inserted into the blood vessel will affect objects such as the blood vessel wall and plaque, and outputs the determination result. Specifically, the diagnostic imaging device 1 detects various objects, including the guidewire, from the cross-sectional image, and identifies the expected area affected by the therapeutic device (for example, the contact range of the therapeutic device) based on the detection result of the guidewire, etc. The diagnostic imaging device 1 determines whether the identified expected area overlaps with a predetermined object other than the guidewire (such as lipid plaque), and if it determines that there is an overlap, it outputs a warning or the like to the user (healthcare professional). In this case, the "predetermined object" is, for example, lipid plaque, blood vessel wall, etc., but is not particularly limited, and may include, for example, stable plaque.

[0034] Figure 2 is a block diagram showing an example configuration of the medical imaging device 1. The medical imaging device 1 comprises a control unit 11, a main memory unit 12, a communication unit 13, a display unit 14, an input unit 15, and an auxiliary storage unit 16. The control unit 11 is a processor such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), or GPUs (Graphics Processing Units), and performs various information processing by reading and executing programs P stored in the auxiliary storage unit 16. The main memory unit 12 is a temporary storage area such as an SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory), and temporarily stores data necessary for the control unit 11 to perform calculation processing. The communication unit 13 is a communication module for performing communication-related processing, and transmits and receives information with the outside. The display unit 14 is a display screen such as a liquid crystal display, and displays images. The input unit 15 is an operation interface such as a keyboard or mouse, and accepts operation input from the user.

[0035] The auxiliary storage unit 16 is a non-volatile storage area such as a hard disk, and stores the program P (program product) and other data necessary for the control unit 11 to execute processing. The auxiliary storage unit 16 also stores the detection model 40. The detection model 40 is a machine learning model that has been trained with predetermined training data, and is a model that detects objects including guide wires when a cross-sectional image of a blood vessel is input.

[0036] The diagnostic imaging device 1 may also be equipped with a reading unit that reads portable storage media 1a such as a CD (Compact Disk)-ROM or DVD (Digital Versatile Disc)-ROM, and may read and execute a program P from the portable storage media 1a.

[0037] Figure 3 is an explanatory diagram regarding the object detection process. The outline of this embodiment will be described below.

[0038] Figure 3 illustrates how objects such as guidewires, plaques, luminal boundaries, and medial boundaries (e.g., EEM (external elastic membrane)) are detected from a cross-sectional image of a blood vessel when input to the detection model 40. The detection model 40 is a machine learning model that has been trained on predetermined training data, and is a semantic segmentation model (e.g., U-net), which is a type of CNN (Convolutional Neural Network). The detection model 40 detects objects in the input image on a pixel-by-pixel basis and outputs a detection result indicating which object each pixel corresponds to.

[0039] The detection model 40 may be a neural network other than a CNN, or a machine learning model other than a neural network. In this embodiment, object detection is performed using a machine learning model (detection model 40), but object detection may also be performed by rule-based pattern matching.

[0040] The image diagnostic apparatus 1 detects various objects including a guidewire by inputting a cross-sectional image captured by a catheter 101 into the detection model 40. Specifically, in addition to the guidewire, the image diagnostic apparatus 1 detects objects such as plaques, calcified lesions, lumen boundaries, media boundaries, side branches, and epicardium.

[0041] The image diagnostic apparatus 1 sequentially inputs cross-sectional images at each position (each frame) of a blood vessel captured in accordance with a pullback operation of the catheter 101 into the detection model 40, thereby detecting objects such as a guidewire from each frame. Note that, for example, the detection model 40 may be configured as a 3D-CNN to enable processing of cross-sectional images of a plurality of frames at one time.

[0042] As described above, the image diagnostic apparatus 1 detects various objects including a guidewire from cross-sectional images of each frame captured along the longitudinal direction of a blood vessel. Next, the image diagnostic apparatus 1 generates a three-dimensional model of the blood vessel and the guidewire based on the cross-sectional image of each frame and the detection result of objects such as the guidewire in each frame.

[0043] FIG. 4 is an explanatory diagram regarding a three-dimensional model generation process. FIG. 4 illustrates how a three-dimensional model of a blood vessel and a guidewire is generated from cross-sectional images of each frame continuously captured along the longitudinal direction of the blood vessel.

[0044] As shown in FIG. 4, the image diagnostic apparatus 1 generates a three-dimensional model of a blood vessel by arranging cross-sectional images of each frame along one direction in the order of imaging. Further, the image diagnostic apparatus 1 renders the guidewire on the three-dimensional model according to the position of the guidewire in each frame, thereby generating a three-dimensional model that includes not only the blood vessel but also the guidewire.

[0045] As described above, the image diagnostic apparatus 1 generates a three-dimensional model of a blood vessel and a guidewire. In the present embodiment, the image diagnostic apparatus 1 identifies the trajectory of the guidewire from the three-dimensional model, and identifies an assumed region affected by a therapeutic device (such as a rotablator) from the trajectory of the guidewire. Then, the image diagnostic apparatus 1 determines whether the identified assumed region overlaps with a predetermined object other than the guidewire (such as lipid plaque, blood vessel wall, etc.), and if it determines that they overlap, it presents the result to the user and issues a warning.

[0046] FIG. 5 is a diagram showing a display example of a cross-sectional image. FIG. 5 illustrates a display example of a cross-sectional image displayed when the assumed region 51 of the therapeutic device overlaps with plaque 52.

[0047] The image diagnostic apparatus 1 superimposes and displays objects such as plaque 52 and guidewire 53 on the cross-sectional image based on the detection results of various objects obtained by the detection model 40. Then, the image diagnostic apparatus 1 identifies the assumed region 51 affected by the therapeutic device, and superimposes and displays it on the cross-sectional image.

[0048] The assumed region 51 is a region that is assumed to be affected (abraded) when an object such as plaque 52 is present in the region, and is rendered as a circular or elliptical region, for example, as shown in FIG. 5. For example, when the therapeutic device is a rotablator, a Diamondback 360, or the like, the assumed region 51 is a region predicted to be passed through by the device. Also, for example, when the therapeutic device is an excimer laser, the assumed region 51 is a region predicted to be affected (ablated) by the laser light.

[0049] The image diagnostic apparatus 1 identifies the assumed region 51 of the therapeutic device from the trajectory or the like of the guidewire identified above. FIG. 6 is an explanatory diagram related to the identification process of the assumed region. In FIG. 6, the trajectory 61 of the guidewire is schematically illustrated.

[0050] First, the diagnostic imaging device 1 uses the cross-sectional image of each frame as the target frame and identifies a straight trajectory 62 that is in substantially contact with the guide wire trajectory 61 and whose inclination and direction are substantially the same as the guide wire trajectory 61 in a frame a predetermined number of frames prior to the target frame. Specifically, the diagnostic imaging device 1 identifies the straight trajectory 62 as a straight line connecting the position 63 of the guide wire M frames prior to the target frame and the position 64 of the guide wire N frames prior (M and N are integers of 1 or more, and M ≠ N).

[0051] Next, the diagnostic imaging device 1 identifies a plane perpendicular to the guide wire trajectory 61 at the guide wire position 65 of the target frame. On that perpendicular plane, the diagnostic imaging device 1 identifies a straight line 66 connecting the guide wire position 65 of the target frame and the straight trajectory 62.

[0052] The diagnostic imaging device 1 then identifies a point 67 (e.g., the midpoint) on the identified line 66 as the center position of the therapeutic device when it is inserted into a blood vessel. The diagnostic imaging device 1 refers to device information relating to the therapeutic device and depicts (identifies) the area within a predetermined distance range from the center position as the assumed area 51. The device information is necessary to determine the range of the assumed area 51 and includes at least one of the following: the type of therapeutic device, its dimensions, and its operating parameters (e.g., rotation speed). The assumed area 51 does not necessarily have to be the same as the device information; for example, it may be displayed as an area extended by a predetermined distance (e.g., a few pixels). By extending the assumed area 51 by a predetermined distance, a safety margin can be secured against the area that may be removed by the therapeutic device. The device information is pre-set and input by the user. The diagnostic imaging device 1 repeats the above process for each frame to identify the assumed area 51.

[0053] Let's return to Figure 5 and continue the explanation. As described above, the diagnostic imaging device 1 identifies the center position of the therapeutic device based on the trajectory of the guidewire and the device information, and identifies the area within a predetermined distance range from that center position as the assumed area 51. In addition to objects such as the plaque 52 and the guidewire 53, the diagnostic imaging device 1 displays the assumed area 51 by color coding, etc. In Figure 5, the color coding is illustrated using hatching, etc.

[0054] In this embodiment, the assumed region 51 of the therapeutic device was identified by referring to the three-dimensional trajectory of the guidewire, but this embodiment is not limited to this. For example, the diagnostic imaging device 1 may identify the assumed region 51 by simply drawing a circular or elliptical region with the position of the guidewire 53 as the center position of the therapeutic device.

[0055] The diagnostic imaging device 1 determines whether the assumed region 51 overlaps with a predetermined object such as plaque 52 (e.g., lipid plaque). If it determines that the assumed region 51 overlaps with an object, the diagnostic imaging device 1 outputs a warning. Specifically, the diagnostic imaging device 1 identifies an overlapping region 54 where the assumed region 51 overlaps with the object (plaque 52) as an affected area of ​​the object affected by the therapeutic device. The diagnostic imaging device 1 displays the overlapping region 54 in a different display manner (e.g., a different display color) than other regions and outputs a predetermined warning in text, voice, etc.

[0056] In this embodiment, as shown in Figure 5, the diagnostic imaging device 1 displays the overlapping region 54 as an area extended by a predetermined distance (for example, several pixels) toward the plaque 52. The portion extended toward the plaque 52 indicates that the influence (contact) of the treatment device may extend to that portion. By extending the overlapping region 54 by a predetermined distance, the area that may be abraded by the treatment device can be suitably presented. Note that this distance may be changed according to the operating parameters (rotation speed, etc.) indicated by the device information.

[0057] Furthermore, it is preferable for the diagnostic imaging device 1 to change the warning output mode (level) according to the degree of overlap between the assumed area 51 and the object, in other words, the degree of influence the therapeutic device has on the object. For example, if it is determined that the assumed area 51 overlaps with the blood vessel wall, the diagnostic imaging device 1 calculates the area, width, etc. of the overlapping area 54 as the degree of influence (degree of overlap), and changes the warning level according to the calculated degree of influence. For example, the diagnostic imaging device 1 outputs a warning at a "low level" if the assumed area 51 only slightly overlaps with the blood vessel wall (the therapeutic device touches the blood vessel wall), at a "medium level" if the assumed area 51 reaches the center of the blood vessel wall, and at a "high level" if the assumed area 51 extends beyond the blood vessel wall (the therapeutic device penetrates the blood vessel wall). This makes it possible to present the degree of influence of the therapeutic device to the user.

[0058] Furthermore, the diagnostic imaging device 1 may change the warning output mode (level) depending on whether the overlapping region 54 is located on the heart (organ) side when viewed from the center of the cross-sectional image. Specifically, the diagnostic imaging device 1 identifies the direction in which the heart is located on the cross-sectional image based on the detection results of objects such as the epicardium and side branches. The diagnostic imaging device 1 then determines whether the overlapping region 54, in which the assumed region 51 overlaps with an object such as a blood vessel wall, is located on the heart side. If the overlapping region 54 is located on the heart side, that is, inside the heart, the diagnostic imaging device 1 outputs a "low level" warning. Conversely, if the overlapping region 54 is not located on the heart side, that is, outside the heart, the risk of cardiac tamponade, etc., is high, so the diagnostic imaging device 1 outputs a "high level" warning. This allows the user to be appropriately presented with the expected effects of the therapeutic device.

[0059] As described above, the diagnostic imaging device 1 identifies the expected area 51 that will be affected by the therapeutic device, determines whether the expected area 51 overlaps with objects such as plaque 52 or blood vessel walls, and outputs a warning if there is an overlap. This effectively supports the prior determination of the success or failure of treatment by the therapeutic device.

[0060] In addition, the above describes a case where it is determined whether or not the device overlaps with an object other than the target lesion (e.g., a calcified lesion). However, it is also possible to determine whether or not the assumed area 51 of the treatment device appropriately overlaps with the target lesion. In other words, this system may be configured as a system for determining whether or not the target lesion can be successfully removed.

[0061] Figure 7 is a flowchart showing an example of a processing procedure performed by the medical imaging device 1. Based on Figure 7, the processing performed by the medical imaging device 1 will be explained. The control unit 11 of the medical imaging device 1 acquires cross-sectional images of multiple frames captured along the longitudinal direction of the blood vessel and device information relating to a therapeutic device inserted into the blood vessel (step S11). The device information includes at least one of the following: type of therapeutic device, dimensions, and operating parameters. The control unit 11 detects various objects, including guide wires, by inputting the cross-sectional images of each frame into the detection model 40 (step S12).

[0062] The control unit 11 generates a three-dimensional model of the blood vessel and guidewire based on the cross-sectional image of each frame and the guidewire detection result in each frame (step S13). Based on the generated three-dimensional model, the control unit 11 identifies the trajectory of the guidewire (step S14).

[0063] The control unit 11 identifies the central position of the therapeutic device when it is inserted into a blood vessel, based on the identified trajectory of the guidewire (step S15). Specifically, as illustrated in Figure 6, the control unit 11 uses the cross-sectional images of each frame as target frames and identifies a straight trajectory in a predetermined number of frames prior to the target frame that is in substantially contact with the trajectory of the guidewire and whose inclination and direction are substantially the same as the trajectory of the guidewire. The control unit 11 identifies a plane perpendicular to the trajectory of the guidewire in the target frame and identifies a straight line connecting the position of the guidewire and the straight trajectory on that plane. The control unit 11 then identifies a point on that straight line (for example, the midpoint between the guidewire and the straight trajectory) as the central position of the therapeutic device.

[0064] The control unit 11 identifies the expected area affected by the therapeutic device when it is inserted into a blood vessel, based on the identified central position of the therapeutic device and the device information (step S16). Specifically, the control unit 11 identifies the expected area of ​​the therapeutic device by drawing a circular, elliptical, or other shape of region centered on the position identified in step S15.

[0065] The control unit 11 determines whether the identified assumed region overlaps with a predetermined object other than the guidewire (e.g., lipid plaque, blood vessel wall, etc.) (step S17). If it determines that there is an overlap (S17: YES), the control unit 11 outputs a warning that the assumed region overlaps with the object (step S18). Specifically, the control unit 11 displays a cross-sectional image showing the overlapping region (affected region) where the assumed region overlaps with the object in a different display manner from other regions, and outputs a warning in text, voice, etc. After executing the process in step S18, or if the answer in step S17 is NO, the control unit 11 terminates the series of processes.

[0066] Based on the above, this embodiment 1 allows for a suitable evaluation of the effects of the therapeutic device.

[0067] (Embodiment 2) This embodiment describes a method for refining the three-dimensional model of a blood vessel by referring not only to cross-sectional images (intravascular images) but also to extravascular images, and for suitably identifying the trajectory of a guidewire. Note that components that overlap with those in Embodiment 1 are denoted by the same reference numerals and their descriptions are omitted.

[0068] Figure 8 is an explanatory diagram regarding the deformation process of a three-dimensional model of a blood vessel. Figure 8 illustrates how a three-dimensional model 82 of a blood vessel, generated from a cross-sectional image, is deformed to match the bending state of the central axis of the blood vessel (the sensor's movement axis) as determined from an extravascular image. Based on Figure 8, the outline of this embodiment will be explained.

[0069] Similar to Embodiment 1, the diagnostic imaging device 1 acquires cross-sectional images at various positions along the longitudinal direction of the blood vessel, detects objects such as guidewires, and generates a three-dimensional model 82 of the blood vessel and guidewire. Since this three-dimensional model 82 is generated based solely on the cross-sectional images, the central axis 821 is linear, as shown in Figure 8.

[0070] Each frame's cross-sectional image is assigned a timestamp indicating the time of acquisition. The imaging diagnostic device 1 also acquires information on the distance traveled by the catheter 101 when each frame's cross-sectional image was acquired. This distance traveled information indicates the distance the catheter 101 traveled between each frame and is acquired from the MDU 102 that drives the catheter 101.

[0071] Furthermore, the diagnostic imaging device 1 acquires extravascular images (e.g., fluoroscopic images) from the fluoroscopic imaging device 2 at the time when the cross-sectional images of each frame were captured. The diagnostic imaging device 1 detects the position of the sensor 81 by detecting radiopaque markers from the acquired extravascular images at each time point. As a result, the diagnostic imaging device 1 acquires sensor position information indicating the position of the sensor 81 at each imaging time point.

[0072] Alternatively, the detection of sensor 81 can be processed by the fluoroscopic image acquisition device 2, and the processing result (sensor position information) can simply be passed to the image diagnostic device 1.

[0073] As shown in Figure 8, the diagnostic imaging device 1 associates the position on the central axis 821 of the three-dimensional model 82 (the frame of the cross-sectional image) with the extravascular image based on the timestamp when each image was captured. As described above, the diagnostic imaging device 1 can identify the position of the sensor 81 at each point in time, i.e., the trajectory of the blood vessel, by detecting an opaque marker from each extravascular image. The diagnostic imaging device 1 can generate a new three-dimensional model 83 by bending the central axis 821 of the three-dimensional model 82 according to the position of the sensor 81 at each point in time.

[0074] However, since extravascular images are two-dimensional images obtained by irradiating with X-rays from a specific direction, changes in the position of the sensor 81 within the image plane can be identified, but changes in the position of the sensor 81 in a direction perpendicular to the extravascular image cannot be identified. Therefore, in this embodiment, by combining the travel distance information of the catheter 101 with the sensor position information on the extravascular image, the curvature of the blood vessel, including in the direction perpendicular to the extravascular image, is determined and a three-dimensional model 83 is generated.

[0075] Figure 9 is an explanatory diagram relating to the process for determining the bending state. In Figure 9, plane 92 represents a plane parallel to the extravascular image, and reference numeral 91 schematically represents a blood vessel between cross-sectional image frames. It should be assumed that the blood vessel 91 between frames can be sufficiently approximated as a straight line.

[0076] The diagnostic imaging device 1 refers to the distance information of the catheter 101's movement during the acquisition of a cross-sectional image to determine the distance ΔL between position (frame) P and position A on the blood vessel 91. The diagnostic imaging device 1 also refers to the sensor position information indicating the position of the sensor 81 detected from the extravascular image to calculate the distance ΔP between the position P of the sensor 81 during the acquisition of the cross-sectional image frame at position P and the position B of the sensor 81 during the acquisition of the cross-sectional image frame at position A.

[0077] As shown in Figure 9, if the angle between the blood vessel 91 and the plane 92 (extravascular image) is denoted as φ, the following relationship (1) holds between ΔL and ΔP.

[0078] ΔP=ΔL×cosφ…(1)

[0079] According to formula (1), the angle φ can be calculated from ΔL and ΔP. The diagnostic imaging device 1 repeatedly calculates the angle φ for each frame based on the distance traveled by the catheter 101 between cross-sectional image frames (ΔL) and the distance traveled by the sensor 81 on the corresponding extravascular image (ΔP), and determines the curvature of the blood vessel 91. The diagnostic imaging device 1 identifies the trajectory of the central axis of the blood vessel 91 according to the determined curvature and generates a three-dimensional model 83 as illustrated in Figure 8.

[0080] The diagnostic imaging device 1 identifies the trajectory of the guidewire from the generated three-dimensional model 83, identifies the area that is expected to be affected by the therapeutic device, and determines whether or not it overlaps with an object such as plaque. The subsequent processing is the same as in Embodiment 1, so the explanation is omitted in this embodiment.

[0081] Figure 10 is a flowchart showing an example of a processing procedure performed by the image diagnostic device 1 according to Embodiment 2. Based on Figure 10, the processing content according to this embodiment will be explained. The control unit 11 of the image diagnostic device 1 acquires information on the distance traveled by the sensor between frames when the cross-sectional image of each frame is captured, in addition to the cross-sectional image of the blood vessel and device information (step S201). The control unit 11 also acquires sensor position information by detecting the position of the sensor of the catheter 101 from the extravascular image (for example, a fluoroscopic image such as angiography) corresponding to each frame (step S202).

[0082] The control unit 11 detects objects including guidewires by inputting the cross-sectional images of each frame into the detection model 40 (step S203). Based on the cross-sectional images of each frame and the guidewire detection results in each frame, the control unit 11 generates a three-dimensional model of the blood vessels and guidewires (step S204).

[0083] The control unit 11 determines the curvature of the blood vessel (trajectory of the central axis) based on the distance the sensor moves between frames as indicated by the movement distance information and the position of the sensor at the time of imaging in each frame as indicated by the sensor position information (step S205). Specifically, as illustrated in Figure 9, the control unit 11 identifies the trajectory of the central axis of the blood vessel while calculating the angle between the extravascular image and the blood vessel in each frame. The control unit 11 deforms the 3D model according to the result of the curvature determination (step S206). The control unit 11 identifies the trajectory of the guidewire based on the deformed 3D model (step S207) and proceeds to step S15.

[0084] Based on the above, according to this embodiment 2, the three-dimensional model of the blood vessel can be refined, and the trajectory of the guidewire can be identified more favorably.

[0085] (Embodiment 3) In Embodiment 1, a method was described in which the therapeutic device determines whether or not it affects an object by identifying the expected area affected by the therapeutic device and determining whether or not that expected area overlaps with an object such as plaque. In this embodiment, a method is described in which the therapeutic device determines whether or not it affects an object by determining whether or not the distance between the therapeutic device and the object is less than or equal to a threshold (for example, the diameter of the therapeutic device).

[0086] Figure 11 is a diagram illustrating the overview of Embodiment 3. Figure 11 shows how the distance between the center position 55 of the therapeutic device and the plaque 52 (object) is measured (calculated). Based on Figure 11, the overview of this embodiment will be explained.

[0087] The diagnostic imaging device 1 identifies the center position 55 of the therapeutic device from the trajectory of the guide wire, in the same manner as in Embodiment 1. In this embodiment, the diagnostic imaging device 1 determines whether or not the therapeutic device affects the object (for example, whether or not it makes contact) based on the distance between the center position 55 and the object (plaque 52, etc.).

[0088] Specifically, the diagnostic imaging device 1 calculates the distance between the center position 55 of the treatment device and each pixel within the object, and lists the distances for each pixel. For example, the diagnostic imaging device 1 may calculate the distance only for points (pixels) on the object's frame. As shown by the thick arrows in Figure 11, the diagnostic imaging device 1 calculates the distance to the pixel where the calculated distance is smallest as the distance between the center position of the treatment device and the object.

[0089] The diagnostic imaging device 1 determines whether the calculated distance is below a threshold determined according to the device information (for example, the diameter of the therapeutic device). If it determines that the distance is below the threshold, the diagnostic imaging device 1 outputs a warning indicating that the therapeutic device is affecting the object.

[0090] The method of outputting the warning is the same as in Embodiment 1. For example, the diagnostic imaging device 1 identifies the affected area of ​​an object affected by the therapeutic device (the overlapping area where the therapeutic device overlaps with the object) and displays a cross-sectional image showing the affected area in a different display manner from other areas. The diagnostic imaging device 1 only needs to identify the affected area as a set of pixels whose distance from the center position 55 of the therapeutic device is below a threshold. The diagnostic imaging device 1 presents the affected area to the user and outputs a warning in the form of text, voice, etc.

[0091] Figure 12 is a flowchart showing an example of a processing procedure performed by the image diagnostic device 1 according to Embodiment 3. After identifying the center position of the therapeutic device from the trajectory of the guidewire (step S15), the image diagnostic device 1 according to this embodiment performs the following processing. The control unit 11 of the image diagnostic device 1 calculates the distance between the center position of the therapeutic device and an object such as plaque (step S301). For example, the control unit 11 calculates the distance between each pixel in the object and the center position of the therapeutic device, and calculates the distance of the pixel that takes the minimum value as the distance between the center position of the therapeutic device and the object.

[0092] The control unit 11 determines whether the calculated distance is less than or equal to a threshold determined according to the device information (for example, the diameter of the therapeutic device) (step S302). If it is determined that the distance is not less than or equal to the threshold (S302: NO), the control unit 11 terminates the series of processes.

[0093] If the control unit determines that the threshold is below the threshold (S302: YES), it outputs a warning that the therapeutic device is affecting (for example, making contact with) an object (step S303). For example, the control unit 11 identifies a set of pixels whose distance from the center position of the therapeutic device is below the threshold as the affected area (overlapping area), displays a cross-sectional image showing the affected area in a different display manner from other areas, and outputs a warning in text, voice, etc. After outputting the warning, the control unit 11 terminates the series of processes.

[0094] Based on the above, according to this embodiment 3, it is possible to appropriately determine whether or not a therapeutic device affects an object without having to specify the expected area affected by the therapeutic device.

[0095] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the claims and not in the sense described above, and all modifications within the sense and scope equivalent to the claims are intended to be included.

[0096] The matters described in each embodiment can be combined with each other. Furthermore, the independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. In addition, the claims use a form in which claims referencing two or more other claims (multi-claim form), but are not limited to this. A form in which multi-claims referencing at least one multi-claim (multi-multi-claim) may also be used.

[0097] 1. Image diagnostic device (information processing device) 11. Control unit 12. Main memory unit 13. Communication unit 14. Display unit 15. Input unit 16. Auxiliary memory unit P. Program 40. Detection model

Claims

1. An information processing method in which a computer performs a process to determine whether the therapeutic device will affect a predetermined object when the therapeutic device is inserted into the blood vessel, based on the detection result of the guide wire and the device information.

2. The information processing method according to claim 1, which identifies the central position of the therapeutic device when the therapeutic device is inserted into the blood vessel based on the detection result of the guidewire, and determines whether the therapeutic device affects the object based on the central position and the device information.

3. The information processing method according to claim 2, which determines whether the therapeutic device affects an object by identifying an assumed region that the therapeutic device would affect when the therapeutic device is inserted into the blood vessel, based on the central position and the device information, and determining whether the assumed region overlaps with the object.

4. The information processing method according to claim 2, which determines whether the therapeutic device affects the object by calculating the distance between the center position and the object, and determining whether the calculated distance is less than or equal to a threshold determined according to the device information.

5. The information processing method according to claim 2, comprising: acquiring cross-sectional images of multiple frames captured along the longitudinal direction of the blood vessel; detecting the object including the guide wire from each frame; identifying the trajectory of the guide wire based on the detection result of the guide wire in each frame; and determining the center position based on the trajectory of the guide wire.

6. The information processing method according to claim 5, wherein the cross-sectional image of each frame is used as the target frame, a straight trajectory is identified in a frame a predetermined number of times prior to the target frame that is in contact with the trajectory of the guide wire and whose inclination and direction coincide with the trajectory of the guide wire, and a point on the straight line connecting the position of the guide wire in the target frame and the straight trajectory is identified as the center position.

7. The information processing method according to claim 5, comprising generating a three-dimensional model of the blood vessel and guidewire based on the cross-sectional image of each frame and the detection result of the guidewire in each frame, and identifying the trajectory of the guidewire based on the three-dimensional model.

8. The information processing method according to claim 7, further acquiring sensor position information relating to the position of the sensor used to capture the cross-sectional image of each frame on an extravascular image of the blood vessel captured from outside the blood vessel, and information on the distance traveled by the sensor between each frame; generating a three-dimensional model of the blood vessel and guidewire based on the cross-sectional image of each frame and the detection result of the guidewire in each frame; determining the bending state of the blood vessel based on the sensor position information and the distance traveled information; and deforming the three-dimensional model according to the determination result of the bending state.

9. The information processing method according to claim 1, wherein a warning is output when it is determined that the therapeutic device is affecting the object.

10. The information processing method according to claim 9, wherein the output mode of the warning is changed according to the degree of influence the therapeutic device has on the object.

11. The information processing method according to claim 9, which involves identifying an affected area of ​​an object that is affected by the therapeutic device based on the detection result of the guidewire and the device information, identifying the direction in which a predetermined organ is located in the cross-sectional image based on the detection result of the object, and changing the output mode of the warning depending on whether the affected area is located on the organ side within the object.

12. The information processing method according to claim 1, which identifies the affected area of ​​the object affected by the therapeutic device based on the detection result of the guide wire and the device information, and outputs the cross-sectional image which displays the affected area in a different display manner from other areas.

13. The information processing method according to claim 12, which outputs a cross-sectional image in which the affected area is extended by a predetermined distance toward the object, and the area is displayed in a different manner from the other areas.

14. The information processing method according to claim 1, wherein the device information includes at least one of the type, dimensions, and operating parameters of the therapeutic device.

15. The information processing method according to claim 1, which detects an object by inputting the acquired cross-sectional image into a model that has been trained to detect the object when the cross-sectional image is input.

16. A program that causes a computer to perform a process to acquire a cross-sectional image of a blood vessel and device information relating to a therapeutic device used for endovascular treatment, to detect an object including a guidewire from the cross-sectional image, and to determine whether the therapeutic device will affect a predetermined object when the therapeutic device is inserted into the blood vessel, based on the guidewire detection result and the device information.

17. An image diagnostic apparatus comprising a control unit, wherein the control unit acquires a cross-sectional image of a blood vessel and device information relating to a therapeutic device used for endovascular treatment; detects an object including a guidewire from the cross-sectional image; and determines, based on the guidewire detection result and the device information, whether the therapeutic device will affect a predetermined object when the therapeutic device is inserted into the blood vessel.

Images