Medical device and image generation method

Abstract

A medical device (1) includes: an image information acquisition unit (511) for acquiring image information including an MRI image or CT image of an organ in a living body; a biomagnetism information acquisition unit (512) for acquiring biomagnetism information obtained from biomagnetism generated by the organ; a model image generation unit (521) for generating, using the image information, an organ model image representing the organ in three dimensions or two dimensions; an electrocardiographic current image generation unit (523) for generating an electrocardiographic current image that represents, using change in color attribute, change over time of a current that flows through each position of the organ, said change being obtained from the biomagnetism information; and a composite image generation unit (52) for generating a composite image that superimposes the organ model image and the electrocardiographic current image.

Description

[Technical Field]

[0001] The present invention relates to a medical device and an image generating method. [Background technology]

[0002] There are known techniques for visually displaying the state of organs in a living body. For example, Patent Documents 1 and 2 disclose techniques for estimating current vectors flowing through the heart from cardiac measurement results, and displaying arrows and colors (color map display) representing the current vectors on a cardiac model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4597329 [Patent Document 2] Patent No. 4027867 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in arrhythmia treatment, there is a demand for improved technology that allows for visual confirmation of the state of the heart, including the lesion (arrhythmic site in the case of arrhythmia treatment). However, with the technology described in Patent Documents 1 and 2, the arrow representing the current vector obstructs the surgeon's view, potentially hindering the surgeon's confirmation of the state of the heart. Thus, even with the above-mentioned prior art, there is still room for improvement in the technology for displaying the state of the heart, including the lesion. Note that this issue is not limited to the heart, but is common to the treatment or diagnosis of any organ through which bioelectric current flows, such as the brain or kidneys.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to improve the technology for displaying the state of an organ including a lesion. [Means for solving the problem]

[0006] The present invention has been made to solve at least part of the above-mentioned problems, and can be realized in the following aspects.

[0007] (1) According to one aspect of the present invention, there is provided a medical device comprising: an image information acquisition unit that acquires image information including MRI images or CT images of an organ in a living body; a biomagnetic field information acquisition unit that acquires biomagnetic field information obtained from the biomagnetic field generated by the organ; a model image generation unit that uses the image information to generate an organ model image that represents the organ in three or two dimensions; a cardiac current image generation unit that generates a cardiac current image that represents, by changes in color attributes, changes over time in current flowing through each position of the organ, the changes being obtained from the biomagnetic field information; and a composite image generation unit that generates a composite image by superimposing the organ model image and the cardiac current image.

[0008] According to this configuration, the medical device generates a composite image by superimposing an organ model image, which represents the organ in three or two dimensions, on a cardiac current image, which represents the change over time in the current flowing at each position in the organ. Therefore, the surgeon can use the composite image to intuitively recognize the change in the current flowing at each position in the organ. Furthermore, the cardiac current image in the composite image represents the change over time in the current flowing at each position in the organ through changes in color attributes. Therefore, compared to conventional arrow displays representing current vectors, there is no risk that the arrow representing the current vector will obstruct the surgeon's view and hinder the surgeon's confirmation of the organ's condition. As a result, the time required to find a lesion (such as an arrhythmia site) can be shortened, and the efficiency and safety of the procedure can be improved.

[0009] (2) In the medical device of the above aspect, the cardiac current image generating unit may represent the change in the color attribute by changing any one of hue, saturation, brightness, or a combination thereof. According to this configuration, the cardiac current image generating unit generates a cardiac current image that expresses changes in color attributes by changing any of hue, saturation, brightness, or a combination thereof, allowing the surgeon to more intuitively recognize changes in the current flowing through each position in the organ.

[0010] (3) In the medical device of the above form, the cardiac current image generating unit may generate the cardiac current image in which at least one of the hue, saturation, and brightness of the part of the organ at a specified time corresponding to the part through which a relatively high current is flowing is higher than that of other parts at the same time. According to this configuration, the cardiac current image generating unit generates a cardiac current image in which at least one of hue, saturation, and brightness of the portion corresponding to the portion where a relatively high current flows is made higher than that of the other portion, which allows the surgeon to more intuitively recognize the change in the current flowing through each position of the organ.

[0011] (4) In the medical device of the above form, the cardiac current image generating unit may generate the cardiac current image in which the color attribute is changed in a first pattern when the current value increases over time at a certain position of the organ, and may generate the cardiac current image in which the color attribute is changed in a second pattern different from the first pattern when the current value decreases over time. According to this configuration, the cardiac current image generating unit generates a cardiac current image in which the color attribute is changed in a first pattern when the current value at a certain position of an organ increases over time, and the color attribute is changed in a second pattern when the current value decreases over time. Therefore, the surgeon can intuitively recognize whether the current value at a certain position of an organ is increasing or decreasing based on the pattern of change in the color attribute.

[0012] (5) In the medical device of the above form, the biomagnetic field information includes information regarding the magnetic field strength distribution of the biomagnetic field generated by the organ, and further includes a magnetic field strength distribution image generation unit that uses the biomagnetic field information to generate a magnetic field strength distribution image representing the strength of the biomagnetic field at each position of the organ, and the composite image generation unit may generate a composite image by superimposing the magnetic field strength distribution image in addition to the organ model image and the cardiac current image. According to this configuration, the composite image generator generates a composite image by superimposing the magnetic field strength distribution image in addition to the organ model image and cardiac current image. This allows the surgeon to recognize the strength of the biomagnetic field at each position of the organ using the magnetic field strength distribution image of the composite image, thereby further improving the efficiency and safety of the procedure.

[0013] (6) According to one aspect of the present invention, there is provided an image generation method comprising the steps of acquiring image information including MRI images or CT images of an organ in a living body, acquiring biomagnetic field information obtained from the biomagnetic field generated by the organ, generating an organ model image representing the organ in three or two dimensions using the image information, generating a cardiac current image representing, by changes in color attributes, changes over time in current flowing through each position in the organ, the changes being obtained from the biomagnetic field information, and generating a composite image by superimposing the organ model image and the cardiac current image.

[0014] The present invention can be realized in various forms, such as a medical device (image generating device) that generates images for display, an image generating method, a medical system including a medical device, a manufacturing method for these devices and systems, and a computer program that realizes the functions of these devices and systems. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is an explanatory diagram illustrating a configuration of a medical device. [Figure 2] FIG. 2 is a functional block diagram of a main control unit and a composite image generation unit. [Figure 3] FIG. 2 is an explanatory diagram of a three-dimensional organ model and an organ model image. [Figure 4] 1A and 1B are diagrams illustrating a method for acquiring biomagnetic field information using a magnetic sensor array. [Figure 5] 10A and 10B are diagrams illustrating a method for generating a magnetic field intensity distribution image. [Figure 6] 10A to 10C are explanatory diagrams illustrating examples of magnetic field strength distribution images in different imaginary planes of the heart. [Figure 7] FIG. 2 is a diagram schematically showing a method for generating a three-dimensional magnetic field intensity distribution model. [Figure 8] FIG. 10 is a diagram showing a method for acquiring changes over time in the current flowing through the heart. [Figure 9] FIG. 10 is a diagram schematically illustrating a method for generating a cardiac current image. [Figure 10] FIG. 10 is a diagram illustrating an example of a composite image. [Figure 11] FIG. 10 is a diagram showing another example of a composite image. [Figure 12] FIG. 10 is a diagram illustrating an example of a change in color attribute. [Figure 13] FIG. 10 is a diagram illustrating another example of a change in color attribute. [Figure 14] FIG. 10 is a functional block diagram of a main control unit and a composite image generation unit according to a second embodiment. [Figure 15] FIG. 10 is a diagram illustrating an example of a composite image according to a second embodiment. [Figure 16] FIG. 10 is a diagram showing another example of a composite image according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] First Embodiment FIG. 1 is an explanatory diagram illustrating the configuration of a medical device 1. The medical device 1 is a device used for treating or diagnosing a living body (here, a human body) 90, and can generate and display a cardiac current image that represents the change over time in the current flowing through each position of the organs of the human body 90 by changing color attributes. The medical device 1 includes a magnetic sensor array 10, a CT device 40, a computer 50, a monitor 60, and an operation unit 70. In the following example, a medical device 1 used for treating arrhythmia will be illustrated.

[0017] The magnetic sensor array 10 is a device that detects information (hereinafter also referred to as "biomagnetic field information") related to the biomagnetic field generated by a human body 90 that is the subject of treatment or diagnosis. The biomagnetic field information includes the strength and direction of the biomagnetic field. The magnetic sensor array 10 has a plurality of magnetic sensors 11 arranged in a matrix, both vertically and horizontally. The magnetic sensors 11 are detection elements that detect the biomagnetic field information, and may be, for example, GSR (GHz-Spin-Rotation Sensor) sensors, magnetoresistance elements (MR), magnetoimpedance elements (MI), or superconducting quantum interference devices (SUQUID).

[0018] The magnetic sensor array 10 is disposed near the center of a bed 95 on which a human body 90 lies. The magnetic sensor array 10 may be configured to be attached to the human body 90 during treatment or diagnosis. The magnetic sensor array 10 may also be configured to be attached to the human body 90 during treatment. For example, the magnetic sensor array 10 may be configured in a belt shape and wrapped around the human body 90, or in the shape of clothing or a hat. In these cases, the magnetic sensors 11 may be arranged along the shape of the human body 90. The magnetic sensor array 10 may also be configured in the shape of two or more plates and three-dimensionally arranged on one or both of the front and back surfaces of the human body, and one or both of the side surfaces. Hereinafter, an example will be described in which the magnetic sensor array 10 detects cardiac magnetic field information (such as the strength and direction of the cardiac magnetic field) generated by a heart 91, which is one of the organs of the human body 90.

[0019] The CT (Computed Tomography) device 40 emits X-rays inside a gantry (mounting stand). The medical device 1 is equipped with a tube and an arc-shaped detector for detecting X-rays, and generates a CT image showing the shape of a heart 91 by rotating the tube 360° around a human body 90 lying on a bed 95, and outputs image information including the CT image to a computer 50. Note that the medical device 1 may be an MRI (Magnetic Resonance Imaging) device instead of a CT device as a device for generating an image showing the shape of the internal organs of the human body 90. That is, the medical device 1 may acquire image information including MRI images instead of image information including CT images.

[0020] The computer 50 is a device that controls the entire medical device 1, and is electrically connected to the magnetic sensor array 10, the CT device 40, the monitor 60, and the operation unit 70. The computer 50 includes a CPU, a ROM, and a RAM (not shown), and implements the functions of the main control unit 51 and the composite image generation unit 52 by expanding a computer program stored in the ROM into the RAM and implementing it with the CPU.

[0021] The main control unit 51 transmits and receives information to and from the magnetic sensor array 10, the CT device 40, the monitor 60, and the operation unit 70, and controls the entire medical device 1. The main control unit 51 includes an image information acquisition unit 511 and a biomagnetic field information acquisition unit 512. The image information acquisition unit 511 controls the CT device 40 to acquire information including a CT image of the human body 90 (hereinafter also referred to as "image information"). The biomagnetic field information acquisition unit 512 controls the magnetic sensor array 10 to acquire information (biomagnetic field information) related to the biomagnetic field generated by the human body 90. In other words, the main control unit 51 functions as a so-called console for the CT device 40 and the magnetic sensor array 10. The image information acquisition unit 511 and the biomagnetic field information acquisition unit 512 will be described in detail later.

[0022] The composite image generating unit 52 generates an organ model image, a cardiac current image, and a composite image obtained by combining these, and displays the generated composite image on the monitor 60. The composite image generating unit 52 includes a model image generating unit 521, a magnetic field strength distribution image generating unit 522, and a cardiac current image generating unit 523. Details of each of these functional units will be described later.

[0023] The monitor 60 is a display unit having a display screen 61, and is configured with a liquid crystal display or the like. The medical device 1 may also have a display unit other than the monitor 60. For example, the medical device 1 may have smart glasses having a display screen, or may have a projector that projects images. The operation unit 70 is configured with any means, such as a keyboard, operation buttons, a touch panel, a foot switch, or a voice recognition device. The operation unit 70 is operated when the surgeon switches the display content of the display screen 61.

[0024] FIG. 2 is a functional block diagram of the main controller 51 and the composite image generator 52. FIG. 3 is an explanatory diagram of a three-dimensional organ model OM and an organ model image SI. FIG. 3(A) shows an example of the three-dimensional organ model OM, and FIG. 3(B) shows an example of the organ model image SI. The image information acquirer 511 of the main controller 51 controls the CT device 40 to acquire image information including CT images from the CT device 40 and store the image information in the storage device of the computer 50. Specifically, the image information acquirer 511 captures cross sections of the entire heart 91 over time and acquires image information including cross sections of the entire heart 91 at each time point. The image information acquirer 511 may acquire image information directly by controlling the CT device 40, or may acquire image information from a storage medium in which previously acquired image information is stored.

[0025] The model image generation unit 521 of the main control unit 51 generates the three-dimensional organ model OM shown in FIG. 3(A) from the image information acquired by the image information acquisition unit 511. Various existing technologies can be applied to the specific method by which the biomagnetic field information acquisition unit 512 generates the three-dimensional organ model OM from image information including CT images. In the illustrated example, the three-dimensional organ model OM is stereoscopic image data representing the external and internal shapes of the heart 91. Here, the model image generation unit 521 integrates cross-sectional images (multiple consecutive CT images) of the entire heart 91 at a certain time acquired by the image information acquisition unit 511 to generate the three-dimensional organ model OM of the heart 91 at that time. Thereafter, the model image generation unit 521 integrates the three-dimensional organ model OM of the heart 91 at different times to generate a dynamic three-dimensional organ model OM that changes over time.

[0026] The model image generation unit 521 of the composite image generation unit 52 captures this dynamic three-dimensional organ model OM on a virtual plane VP set at an arbitrary position, thereby generating an organ model image SI that represents the heart 91 in three dimensions. The position and orientation of the virtual plane VP are set at the surgeon's arbitrary position and orientation by operating the operation unit 70. For example, if the virtual plane VP intersects with the three-dimensional organ model OM, an organ model image SI representing a cross section of the three-dimensional organ model OM is generated, as shown in FIG. 3(B). If the set virtual plane VP does not intersect with the three-dimensional organ model OM, an organ model image SI representing the appearance (outer surface) of the three-dimensional organ model OM as seen from the virtual plane VP is generated.

[0027] The model image generating unit 521 may generate an organ model image SI that represents the heart 91 in two dimensions. The two-dimensional organ model image SI is an image that represents only the surface of the portion of the three-dimensional organ model OM that intersects with the virtual plane VP. The three-dimensional organ model image SI is an image that represents not only the portion that intersects with the virtual plane VP but also the portion of the three-dimensional organ model OM in the depth direction that is visible from the virtual plane VP. The dimensions (two-dimensional / three-dimensional / both) of the image generated by the model image generating unit 521 can be set arbitrarily by the surgeon by operating the operating unit 70.

[0028] The three-dimensional organ model OM includes information related to the coordinate positions of parts corresponding to specific parts of the heart 91. "Information related to the coordinate positions of specific parts of the heart 91" refers to, for example, information such as the position of the sinus node, the position of the atrioventricular node, the orientation of the His bundle, and the position of the Purkinje fibers. The information related to the coordinate positions of specific parts of the heart 91 can be acquired, for example, by fitting a contour image showing the general positional relationship between these specific parts (sinus node, atrioventricular node, His bundle, Purkinje fibers, etc.) to the three-dimensional organ model OM generated by the model image generation unit 521.

[0029] FIG. 4 is a diagram schematically illustrating a method for acquiring biomagnetic field information using the magnetic sensor array 10. The biomagnetic field information acquiring unit 512 (FIG. 2) of the main control unit 51 controls the magnetic sensor array 10 to acquire biomagnetic field information and stores it in the memory unit of the computer 50. As described above, the biomagnetic field information includes the strength and direction of the biomagnetic field MFh generated by the internal organs of the human body 90. As shown in FIG. 4, in the heart 91, an electrical signal CD is generated from the sinus node to contract the atrium and ventricle. The magnetic sensor array 10 detects the strength and direction of the biomagnetic field MFh generated by this electrical signal CD, and the biomagnetic field information acquiring unit 512 acquires the strength and direction of the biomagnetic field MFh as biomagnetic field information.

[0030] If an organ has a lesion (for example, an arrhythmic area in the heart 91), the strength and direction of the biomagnetic field MFh in the biomagnetic field information will be affected by the lesion. In other words, the strength and direction of the biomagnetic field MFh of an organ with a lesion will differ from the strength and direction of the biomagnetic field MFh of a healthy organ without a lesion. For this reason, the location of the lesion in the organ can be identified using the biomagnetic field information (by comparing it with the biomagnetic field MFh of a healthy organ). Therefore, it can be said that the biomagnetic field information acquired by the biomagnetic field information acquisition unit 512 includes information about the lesion in the organ.

[0031] FIG. 5 is a diagram schematically illustrating a method for generating a magnetic field intensity distribution image MI. FIG. 5(A) is a diagram illustrating an example of the strength (detection value Vd) of the biomagnetic field MFh detected by each magnetic sensor 11 of the magnetic sensor array 10. FIG. 5(B) is a diagram illustrating an example of the magnetic field intensity distribution image MI. The magnetic sensors 11 in the magnetic sensor array 10 are arranged in a matrix on a two-dimensional plane (XY plane). Therefore, as shown in FIG. 5(A), the magnetic sensor array 10 can detect the strength (detection value Vd) of the biomagnetic field MFh at each position on the two-dimensional plane. FIG. 5(A) illustrates the time-series change in the strength of the biomagnetic field MFh at each position on the two-dimensional plane (XY plane). The magnetic sensor array 10 can detect the direction of the biomagnetic field MFh in the two-dimensional plane from the time-series change in the strength of the biomagnetic field MFh at each position on the two-dimensional plane. Furthermore, the magnetic sensor 11 is configured to be able to detect the change in the strength of the biomagnetic field MFh in the normal direction (Z direction) of the two-dimensional plane. Here, each magnetic sensor 11 includes a plurality of (for example, two) elements aligned in the normal direction of a two-dimensional plane, and can detect the strength of the biomagnetic field MFh at a position relatively close to the heart 91 and the strength of the biomagnetic field MFh at a position relatively far from the heart 91 in the normal direction (Z direction). With this configuration, the magnetic sensor array 10 can detect the strength and direction of the biomagnetic field MFh on any imaginary plane VP (XY plane) intersecting the heart 91. The magnetic sensor array 10 outputs biomagnetic field information including the strength of the biomagnetic field MFh detected by each magnetic sensor 11 to the biomagnetic field information acquisition unit 512.

[0032] FIG. 6 is an explanatory diagram illustrating magnetic field intensity distribution images MI1 to MI3 on ​​different imaginary planes VP1 to VP3 of the heart 91. The magnetic field intensity distribution image generating unit 522 of the composite image generating unit 52 generates the magnetic field intensity distribution image MI shown in FIG. 5(B) from the biomagnetic field information acquired by the biomagnetic field information acquiring unit 512. FIG. 5(B) illustrates, as an example of the magnetic field intensity distribution image MI, a magnetic field intensity distribution image MI in which the strength of the biomagnetic field MFh at each position on a two-dimensional plane (XY plane) is represented as contour lines. Note that the strength of the biomagnetic field MFh may be represented by a method other than contour lines, such as color gradation. The magnetic field intensity distribution image generating unit 522 can generate a magnetic field intensity distribution image MI on any imaginary plane VP intersecting with the heart 91 using the biomagnetic field information at a certain time t1. FIG. 6 illustrates, as an example, magnetic field intensity distribution images MI1, MI2, and MI3 corresponding to three imaginary planes VP1, VP2, and VP3, respectively, at time t1.

[0033] FIG. 7 is a diagram schematically illustrating a method for generating a three-dimensional magnetic field intensity distribution model DM. FIG. 7(A) is a diagram illustrating an example of magnetic field intensity distribution images MI1 to MI5 obtained from five different virtual planes. FIG. 7(B) is a diagram illustrating an example of a three-dimensional magnetic field intensity distribution model DM obtained from the magnetic field intensity distribution images MI1 to MI5. FIG. 7(C) is a diagram illustrating an example of a dynamic three-dimensional magnetic field intensity distribution model DM that changes over time. The magnetic field intensity distribution image generation unit 522 further integrates the magnetic field intensity distribution images MI1 to MI5 (plurality of consecutive magnetic field intensity distribution images MI) generated as described above at a certain time t1, as shown in FIG. 7(A), to generate a three-dimensional magnetic field intensity distribution model DM at a certain time t1, as shown in FIG. 7(B). Next, the magnetic field intensity distribution image generation unit 522 integrates the three-dimensional magnetic field intensity distribution models DM of the heart 91 at different times t1 to tn (n is a natural number), as shown in FIG. 7(C), to generate a dynamic three-dimensional magnetic field intensity distribution model DM that changes over time.

[0034] The magnetic field intensity distribution image generating unit 522 may generate a two-dimensional magnetic field intensity distribution image MI. The two-dimensional magnetic field intensity distribution image MI is an image that represents only the magnetic field intensity distribution of a portion of the three-dimensional magnetic field intensity distribution model DM that intersects with the virtual plane VP. The three-dimensional magnetic field intensity distribution image MI is an image that represents the magnetic field intensity distribution of the entire three-dimensional magnetic field intensity distribution model DM that can be seen from the virtual plane VP, or a portion of the three-dimensional magnetic field intensity distribution model DM that is included in an arbitrary spatial region. Therefore, the three-dimensional magnetic field intensity distribution image MI also represents the depth direction of the magnetic field intensity distribution. The dimensions (two-dimensional / three-dimensional / both) of the image generated by the magnetic field intensity distribution image generating unit 522 can be arbitrarily set by the surgeon by operating the operation unit 70.

[0035] The three-dimensional magnetic field strength distribution model DM includes information on the direction and strength of the biomagnetic field MFh as well as information related to the coordinate positions of parts corresponding to specific parts of the heart 91. As with the three-dimensional organ model OM, "information related to the coordinate positions of specific parts of the heart 91" can be information such as the position of the sinus node, the position of the atrioventricular node, the orientation of the His bundle, and the position of the Purkinje fibers. The information related to the coordinate positions of specific parts of the heart 91 can be identified, for example, from changes in the magnetic field caused by the electrical signal CD. For example, the sinus node is the part that is the origin of the electrical signal CD, and the atrioventricular node is the part that is the relay point for the electrical signal CD, so it can be identified from the generation position of the electrical signal CD, the direction of the electrical signal flow, etc.

[0036] Fig. 8 is a diagram schematically showing a method for acquiring changes over time in the current EV flowing through the heart 91. Fig. 8(A) shows an example of the electrical signal CD flowing through the heart 91 from time t1 to t10, and the current EV(t1-t10) at that time. Fig. 8(B) shows an example of the electrical signal CD flowing through the heart 91 from the next time t11 to t20, and the current EV(t11-t20) at that time. Fig. 8(C) shows an example of the electrical signal CD flowing through the heart 91 from the next time t21 to t30, and the current EV(t21-t30) at that time.

[0037] The cardiac current image generation unit 523 of the composite image generation unit 52 acquires the three-dimensional magnetic field intensity distribution model DM (a dynamic three-dimensional magnetic field intensity distribution model DM that changes over time) generated by the method of Fig. 7 from the magnetic field intensity distribution image generation unit 522. Generally, in the heart 91, as the heart 91 moves (beats) over time, a local current is generated from the sinus node SN to the atrioventricular junction AN (an electrical signal CD is generated), as shown in the left diagram of Fig. 8(A). The current then travels from the atrioventricular junction AN to the His bundle HB (an electrical signal CD is generated), as shown in the left diagram of Fig. 8(B). The current then travels to the Purkinje fibers PF (an electrical signal CD is generated), as shown in the left diagram of Fig. 8(C). As explained in Fig. 4, the three-dimensional magnetic field intensity distribution model DM is generated based on information on the strength and direction of the biomagnetic field MFh generated by this electrical signal CD (biomagnetic field information). For this reason, the cardiac current image generating unit 523 can use a three-dimensional magnetic field strength distribution model DM (a dynamic three-dimensional magnetic field strength distribution model DM that changes over time) to determine the change over time of the electrical signal CD at each position of the heart 91, in other words, the change over time EV (t1-t30) of the current EV flowing through each position of the heart 91, in accordance with well-known physical laws such as Maxwell's equations (right diagrams of FIGS. 8(A) to (C)). Note that while FIG. 8 lists times t1 to t30 for convenience, the following explanation will use times t1 to tn (n is a natural number) as examples.

[0038] In this way, the cardiac current image generating unit 523 can obtain the change over time EV(t1-tn) of the current EV flowing at each position of the heart 91 from the biomagnetic field information (indirectly via the three-dimensional magnetic field intensity distribution model DM). Note that the cardiac current image generating unit 523 may obtain the change over time EV(t1-tn) of the current EV flowing at each position of the heart 91 by other methods. For example, the cardiac current image generating unit 523 may obtain the change over time EV(t1-tn) of the current EV flowing at each position of the heart 91 directly from the detection value of the magnetic sensor array 10.

[0039] Fig. 9 is a diagram showing a schematic diagram of a method for generating a cardiac current image VI(t1-tn). Next, the cardiac current image generating unit 523 generates a cardiac current image VI(t1-tn) that represents the change over time EV(t1-tn) of the current EV flowing through each position of the heart 91 by changes in color attributes. Specifically, the cardiac current image generating unit 523 acquires from the model image generating unit 521 the three-dimensional organ model OM (stereoscopic image data representing the external and internal shapes of the heart 91) generated by the method of Fig. 3. Here, the three-dimensional organ model OM acquired by the cardiac current image generating unit 523 means a dynamic three-dimensional organ model OM that changes over time, and will hereinafter be referred to as the "three-dimensional organ model OM(t1-tn)" for convenience.

[0040] Next, the cardiac current image generating unit 523 divides the three-dimensional organ model OM(t1-tn) into voxels B0 representing predetermined unit volume elements. Thereafter, the cardiac current image generating unit 523 aligns the three-dimensional organ model OM(t1-tn) with the change over time EV(t1-tn) of the current EV flowing through each position of the heart 91, which was obtained using the method of FIG. 8. This alignment can be achieved, for example, by aligning the current EV(t21-t30) flowing through the Purkinje fibers PF, as shown in the left diagram of FIG. 8(C), with the contour shape of the three-dimensional organ model OM(t1-tn). This alignment allows the cardiac current image generating unit 523 to obtain the change over time EV(t1-tn) of the current EV flowing through each voxel B0, in other words, the change over time EV(t1-tn) of the current EV flowing through each position of the heart 91. The lower part of Fig. 9 shows the time-dependent changes EV(t1-tn) of the current EV flowing through the voxels BO11, BO13, BO15, and BO17 at different positions. As shown by the dashed lines in the lower part of Fig. 9, the current EV has different time-dependent changes EV(t1-tn) depending on the position of the voxel BO.

[0041] The cardiac current image generating unit 523 generates a cardiac current image VI(t1-tn) in which the hue and saturation of each voxel BO are set to arbitrary values, and the brightness is set to a value corresponding to the change over time EV(t1-tn) in the current EV flowing through the voxel BO. In the example in the lower part of Fig. 9, the cardiac current image generating unit 523 generates a cardiac current image VI(t1-tn) in which the brightness of a certain voxel BO is increased in proportion to the value of the current EV flowing through the voxel BO. In this way, the cardiac current image generating unit 523 can generate a cardiac current image VI(t1-tn) in which voxels BO with relatively high brightness (in other words, relatively high brightness parts) appear to move over time in accordance with the change over time EV(t1-tn) in the current EV. In other words, the cardiac current image generation unit 523 of this embodiment generates a cardiac current image VI (t1-tn) in which the brightness of voxels BO corresponding to parts of the heart 91 at a specified time (e.g., time t10) through which a relatively high current EV flows is higher than that of other parts at the same time (e.g., time t10).

[0042] FIG. 10 is a diagram showing an example of a composite image CI. The composite image generator 52 generates a composite image CI by superimposing a cardiac current image VI(t1-tn) on a virtual plane VP on an organ model image SI(t1-tn) generated by capturing a three-dimensional organ model OM(t1-tn) on the same virtual plane VP. The composite image generator 52 displays the generated composite image CI and a first window FW1 on the display screen 61 of the monitor 60. The first window FW1 displays an image representing a heart 91 and an image representing its positional relationship with the virtual plane VP. The surgeon can change the position of the virtual plane VP in the composite image CI by manipulating the virtual plane VP displayed in the first window FW1 to change its positional relationship with the image representing the heart 91. In the example of FIG. 10, the virtual plane VP does not intersect with the image representing the heart 91 in the first window FW1. Therefore, the composite image CI is an image obtained by superimposing an organ model image SI(t1-tn) representing the appearance (external surface) of the heart 91 and a cardiac current image VI(t1-tn) representing the change over time EV(t1-tn) of the current EV flowing through the same external surface of the heart 91.

[0043] In FIG. 10, the cardiac current image VI(t1-tn) of the composite image CI is displayed in white. In this cardiac current image VI(t1-tn), a relatively bright portion BP moves over time in accordance with the time-dependent change EV(t1-tn) of the current EV flowing through each position of the heart 91. For example, a relatively bright portion BP(t21) at a certain time t21 and a relatively bright portion BP(t22) at the next time t22 are located in different positions. By referring to such a composite image CI, the surgeon can intuitively recognize the time-dependent change EV(t1-tn) of the current EV flowing through each position of the heart 91. Furthermore, a portion LE in the heart 91 where the change in the current EV is abnormal (e.g., an arrhythmic portion of the heart 91) exhibits a different change in brightness from other normal portions (in the illustrated example, the brightness changes in a spiral pattern). This allows the surgeon to intuitively recognize the location of a lesion LE where an abnormal change in the current EV occurs.

[0044] In addition, even in two-dimensional processing using a two-dimensional organ model OM and a two-dimensional magnetic field intensity distribution model DM, the cardiac current image generation unit 523 and the composite image generation unit 52 can divide the two-dimensional organ model OM into pixels representing a predetermined unit area element and then perform processing similar to that described in Figures 8 to 10.

[0045] FIG. 11 is a diagram showing another example of the composite image CI. In the example of FIG. 11, the virtual plane VP intersects with the image representing the heart 91 in the first window FW1. Therefore, the composite image CI is an image obtained by superimposing an organ model image SI(t1-tn) representing a cross section of the heart 91 on the virtual plane VP and a cardiac current image VI(t1-tn) representing a change over time EV(t1-tn) in the current EV flowing through the same cross section of the heart 91. In the composite image CI of FIG. 11, as in FIG. 10, the relatively bright portions BP of the cardiac current image VI(t1-tn) move over time in accordance with the change over time EV(t1-tn) in the current EV flowing through each position of the heart 91. For example, a relatively bright portion BP(t13) at a certain time t13 and a relatively bright portion BP(t14) at the next time t14 are located at different positions.

[0046] Figure 12 shows the change of color attributes. An example 12A and 12B are diagrams illustrating examples in which the saturation is changed. FIG. 12A shows an example in which the saturation is changed, and FIG. 12B shows an example in which the hue is changed. As shown in FIG. 12A, when generating a cardiac current image VI(t1-tn), the cardiac current image generating unit 523 may set the hue and brightness to arbitrary values ​​for each voxel B0, and may set the saturation to a value corresponding to the change over time EV(t1-tn) of the current EV flowing through the voxel B0. This also achieves the same effect as the examples in FIGS. 9 to 11 in which the brightness is changed. As shown in FIG. 12B, when generating a cardiac current image VI(t1-tn), the cardiac current image generating unit 523 may set the brightness and saturation to arbitrary values ​​for each voxel B0, and may set the hue to an RGB value corresponding to the change over time EV(t1-tn) of the current EV flowing through the voxel B0. This also achieves the same effect as the examples in FIGS. 9 to 11 in which the brightness is changed. Note that Fig. 9 illustrates an example of changing the brightness, Fig. 12(A) illustrates an example of changing the saturation, and Fig. 12(B) illustrates an example of changing the hue. However, the cardiac current image generating unit 523 may generate cardiac current images VI(t1-tn) in which any of the hue, saturation, brightness, or a combination thereof is changed.

[0047] FIG. 13 is a diagram showing another example of changes in color attributes. FIG. 13 shows an example of changing the pattern of color attributes. As shown in FIG. 13, when generating a cardiac current image VI (t1-tn), the cardiac current image generating unit 523 changes the color attributes of a certain voxel BO in a first pattern if the current EV increases over time (FIG. 13: from time t1 until the current EV reaches the apex CP). In the example of FIG. 13, the first pattern changes the hue from red to yellow among warm colors and increases the brightness as the current EV increases. Furthermore, when generating a cardiac current image VI (t1-tn), the cardiac current image generating unit 523 changes the color attributes of the same voxel BO in a second pattern different from the first pattern if the current EV decreases over time (FIG. 13: from the time the current EV reaches the apex CP until time tn). 13, the second pattern changes the color attribute from yellow to blue among cool colors and lowers the brightness as the current EV decreases. The cardiac current image generating unit 523 performs similar processing on each voxel BO of the three-dimensional organ model OM(t1-tn) to generate a cardiac current image VI(t1-tn).

[0048] 13, the cardiac current image generating unit 523 generates a cardiac current image VI (t1-tn) in which the color attribute is changed in a first pattern when the current value EV at the position of an organ (voxel BO where heart 91 is located) increases over time, and the color attribute is changed in a second pattern when the current value EV decreases over time. Therefore, the surgeon can intuitively recognize whether the current value EV at the position of an organ (voxel BO where heart 91 is located) is increasing or decreasing based on the pattern of change in the color attribute.

[0049] As described above, the medical device 1 of the first embodiment generates a composite image CI by superimposing an organ model image SI(t1-tn) that represents the heart 91 (organ) in three or two dimensions on a cardiac current image VI(t1-tn) that represents the change over time EV(t1-tn) of the current EV flowing through each position of the heart 91. Therefore, using the composite image CI, the surgeon can intuitively recognize the change in the current EV flowing through each position of the heart 91. Furthermore, the cardiac current image VI(t1-tn) of the composite image CI represents the change over time EV(t1-tn) of the current EV flowing through each position of the heart 91 by changing color attributes. Therefore, compared to a conventional arrow display representing a current vector, there is no risk that the arrow representing the current vector will obstruct the surgeon's view and hinder the surgeon from checking the condition of the heart 91. As a result, the time required to find a lesion (such as an arrhythmia site) can be shortened, and the efficiency and safety of the procedure can be improved.

[0050] Furthermore, according to the medical device 1 of the first embodiment, the cardiac current image generating unit 523 generates cardiac current images VI(t1-tn) that represent changes in color attributes by changing any of the hue, saturation, brightness, or a combination thereof. This allows the surgeon to more intuitively recognize how the current EV flowing through each position of the heart 91 (organ) changes. Furthermore, the cardiac current image generating unit 523 generates cardiac current images VI(t1-tn) in which at least one of the hue, saturation, and brightness of portions (voxels BO) corresponding to portions through which a relatively high current EV flows is made higher than that of other portions (voxels BO). This allows the surgeon to more intuitively recognize how the current EV flowing through each position of the heart 91 changes.

[0051] Second Embodiment FIG. 14 is a functional block diagram of a main controller 51 and a composite image generator 52A according to the second embodiment. The medical device 1A according to the second embodiment includes a composite image generator 52A instead of the composite image generator 52 described in the first embodiment. The composite image generator 52A generates a composite image CIA by superimposing a magnetic field intensity distribution image MI(t1-tn) on the same virtual plane VP in addition to the organ model image SI(t1-tn) and cardiac current image VI(t1-tn) described in the first embodiment. The magnetic field intensity distribution image MI(t1-tn) can be generated by capturing, on the virtual plane VP, the three-dimensional magnetic field intensity distribution model DM generated by the method described in FIG. 7. The composite image generator 52A displays the generated composite image CIA, the first window FW1, and the second window FW2 on the display screen 61 of the monitor 60.

[0052] FIG. 15 is a diagram showing an example of a composite image CIA of the second embodiment. The first window FW1 is as described in the first embodiment. The second window FW2 displays images indicating whether or not the cardiac current image VI is displayed (ON / OFF) and whether or not the magnetic field intensity distribution image MI is displayed (ON / OFF). The surgeon can switch the type of image superimposed on the organ model image SI(t1-tn) of the composite image CIA by switching the ON / OFF display selection displayed in the second window FW2. In the example of FIG. 15, in the second window FW2, the display of the cardiac current image VI is set to OFF (not displayed), and the display of the magnetic field intensity distribution image MI is set to ON (displayed). Therefore, the composite image CIA in which the magnetic field intensity distribution image MI(t1-tn) is superimposed on the organ model image SI(t1-tn) is displayed at the top.

[0053] Fig. 16 is a diagram showing another example of the composite image CIA of the second embodiment. In the example of Fig. 16, in the second window FW2, the display / non-display of the cardiac current image VI is set to ON (display), and the display / non-display of the magnetic field intensity distribution image MI is set to ON (display). Therefore, a composite image CIA in which both the cardiac current image VI(t1-tn) and the magnetic field intensity distribution image MI(t1-tn) are superimposed on the organ model image SI(t1-tn) is displayed at the top.

[0054] As described above, the medical device 1A can be modified in various ways. The composite image generator 52A may generate and display a composite image CIA including an image different from the organ model image SI(t1-tn) and the cardiac current image VI(t1-tn). In the above example, the magnetic field intensity distribution image MI(t1-tn) is superimposed on the composite image CIA. However, other images may be superimposed. Examples of other images that may be used include an image showing the position of a lesion, an image showing the position of a medical device (e.g., a catheter) inserted into the heart 91, and various other images. Furthermore, the composite image generator 52A may display a second window FW2 for specifying the type of image to be displayed as the composite image CIA, in addition to the first window FW1 for specifying the virtual plane VP. Using the second window FW2 further improves the ease of use of the medical device 1A for the surgeon.

[0055] The medical device 1A of the second embodiment can also achieve the same effects as those of the first embodiment. Furthermore, according to the medical device 1A of the second embodiment, the composite image generator 52A generates a composite image CIA by superimposing a magnetic field intensity distribution image MI(t1-tn) in addition to the organ model image SI(t1-tn) and the cardiac current image VI(t1-tn). Therefore, the surgeon can recognize the strength of the biomagnetic field MFh at each position of the heart 91 (organ) using the magnetic field intensity distribution image MI(t1-tn) of the composite image CIA, thereby further improving the efficiency and safety of the procedure.

[0056] <Modification of this embodiment> In the above-described embodiments, a part of the configuration realized by hardware may be replaced by software, and conversely, a part of the configuration realized by software may be replaced by hardware. Furthermore, the present invention is not limited to the above-described embodiments, and can be embodied in various forms without departing from the spirit of the present invention, and for example, the following modifications are also possible.

[0057] [Variation 1] In the first and second embodiments, the configurations of the medical devices 1 and 1A are exemplified. However, the configuration of the medical device 1 can be modified in various ways. For example, in the medical device 1, at least some of the magnetic sensor array 10, the CT device 40, the computer 50, the display unit 60, and the operation unit 70 may be configured as an integrated device. For example, the medical device 1 may include other devices such as an MRI device, an electrocardiograph, an X-ray imaging device, and an ultrasound probe. If the medical device 1 includes an electrocardiograph, the medical device 1 may compare the actual measurement value of the electrocardiograph with the change over time EV(t1-tn) of the current EV calculated by the cardiac current image generating unit 523, thereby confirming, re-measuring, correcting, etc. the current value calculated by the cardiac current image generating unit 523.

[0058] For example, the cardiac current image generating unit 523 may use the following method a1 or a2, which are different from the above-mentioned methods, to determine the change over time EV(t1-tn) of the current EV flowing through each position in the heart 91. In methods a1 and a2, the position of a medical device (catheter, etc.) within the heart 91 can be identified, for example, as follows. The medical device is provided with a magnetic field generating unit consisting of an electromagnet. The main control unit 51 of the medical apparatus 1 acquires second magnetic field information output from the magnetic sensor array 10 while the magnetic field generating unit (electromagnet) is energized. The second magnetic field information is magnetic field information that combines the biomagnetic field MFh and the device magnetic field generated by the magnetic field generating unit of the medical device (hereinafter also referred to as the "bio-device mixed magnetic field"). The second magnetic field information includes position information of the medical device. Therefore, the main control unit 51 can identify the position of the magnetic field generating unit of the medical device by comparing the biomagnetic field information described in the first embodiment with the second magnetic field information. The medical device is provided with a magnetic field generating unit made of a permanent magnet. The main control unit 51 of the medical apparatus 1 can identify the position of the magnetic field generating unit of the medical device from the X-ray image obtained by the X-ray imaging device. This is because, when the magnetic field generating unit (magnetic source) provided in the medical device is a permanent magnet, the measured magnetic strength value does not change due to factors (mainly time) other than the relative distance between the magnetic field generating unit on the X-ray image and the magnetic sensor array 10, and therefore the magnetic field generating unit can be identified as a magnetic source that moves in conjunction with the operation of the medical device (in other words, the movement of the medical device) rather than the beating of the heart 91.

[0059] (a1) Electrodes may be provided on a medical device inserted inside the heart 91, and the time-dependent change EV(t1-tn) in the current EV flowing through each position in the heart 91 may be determined by measuring the potential using the medical device. In this case, potential change measurement is performed for a given time T at a specific point inside the heart 91, and then the measurement point is moved to obtain time-dependent change data on the potential in the desired cardiac region. The time T may be a value corresponding to the previously acquired pulsation cycle of the heart 91, and is preferably m (m is a natural number) times the pulsation cycle of the heart 91. The cardiac current image generating unit 523 converts the potential change thus obtained into the time-dependent change EV(t1-tn) in the current EV. The conversion can be performed, in principle, by unit conversion only. The cardiac current image generating unit 523 may also perform signal correction, such as noise reduction, along with the conversion. If the medical device inserted into the heart 91 is a catheter with a basket structure, it is also possible to convert the potential to current while performing measurements in real time.

[0060] (a2) A magnetic sensor may be provided in the medical device, and time-dependent changes EV(t1-tn) in the current EV flowing through various positions in the heart 91 may be determined by magnetic measurements using the medical device inserted inside the heart 91. In this case, magnetic change measurements are performed for an arbitrary time T at a specific point inside the heart 91, and then the measurement point is moved to obtain time-dependent magnetic change data in the desired cardiac region. The time T is the same as in method a1. The cardiac current image generating unit 523 converts the magnetic changes obtained in this manner into time-dependent changes EV(t1-tn) in the current EV in accordance with well-known physical laws such as Maxwell's equations. Note that if the medical device inserted into the heart 91 is a catheter with a basket structure, it is also possible to convert between magnetism and current while performing measurements in real time.

[0061] As is well known, there is a correlation between time changes in magnetism, potential, and current. Therefore, in the method described in the first embodiment and the above methods a1 and a2, the cardiac current image generating unit 523 may obtain the time change EV(t1-tn) using the myocardial impedance data of the heart 91 together with Maxwell's equations.

[0062] [Variation 2] In the first and second embodiments, examples of images generated by the composite image generating units 52 and 52A and displayed on the display screen 61 have been described. However, the image displayed on the display screen 61 can be modified in various ways. For example, at least one of the first window FW1 and the second window FW2 may be omitted. For example, the display screen 61 may display a third window for selecting a method of changing the color attributes of the cardiac current image VI (t1-tn) (the methods described in FIGS. 9, 12, and 13). For example, the display screen 61 may display various images, such as MRI images, images displaying measurement data obtained by an electrocardiograph, images displaying pulses, and images explaining the procedure. For example, an image representing the position of a medical device or an image representing the position of a lesion may be further superimposed on the composite image CI. For example, an image representing a specific region of the heart 91 (such as the sinus node, atrioventricular node, His bundle, or Purkinje fibers) may be further superimposed on the composite image CI.

[0063] [Variation 3] The configurations of the medical devices 1 and 1A of the first and second embodiments and the configurations of the modifications 1 and 2 may be appropriately combined. For example, in the configuration described in the second embodiment, variations in the color attributes of the cardiac current images VI (t1-tn) described in Figures 9, 12, and 13 may be adopted.

[0064] This aspect has been described above based on embodiments and modifications. However, the above-described embodiments are intended to facilitate understanding of this aspect and are not intended to limit this aspect. This aspect may be modified or improved without departing from the spirit and scope of the claims, and equivalents thereof are included in this aspect. Furthermore, if a technical feature is not described as essential in this specification, it may be deleted as appropriate. [Explanation of symbols]

[0065] 1,1A...Medical devices 10...Magnetic sensor array 11...Magnetic sensor 40...CT device 50...Computer 51...Main control unit 52, 52A...synthetic image generation unit 60...Monitor 61…Display screen 70...Operation unit 90…Human body 91...Heart 95...Bed 511...Image information acquisition unit 512...Biomagnetic field information acquisition unit 521...Model image generation unit 522...magnetic field intensity distribution image generation unit 523...cardiac current image generation unit VP...Virtual Plane OM: 3D organ model DM: 3D magnetic field strength distribution model SI…Organ model image VI...Cardiac current image MI...Magnetic field strength distribution image CI, CIA... composite image FW1...1st window FW2...Second window HB...Hiss Bundle LE: Lesion PF: Purkinje fibers SN…sinus node

Claims

1. A medical device comprising: an image information acquisition unit that acquires image information including MRI images or CT images of organs in a living body; a biomagnetic field information acquiring unit for acquiring biomagnetic field information obtained from the biomagnetic field generated by the organ; a model image generating unit that generates an organ model image that represents the organ in three dimensions or two dimensions using the image information; a cardiac current image generating unit that generates a cardiac current image that represents a change over time in a current flowing through each position of the organ, the change over time being obtained from the biomagnetic field information, by a change in color attribute; a composite image generating unit that generates a composite image by superimposing the organ model image and the cardiac current image; Equipped with the cardiac current image generating unit expresses the change in the color attribute by changing at least one of hue, saturation, and brightness; The cardiac current image generating unit generates different cardiac current images at a certain position of the organ when the current value rises to a predetermined value and when the current value falls to the predetermined value.

2. 10. The medical device of claim 1, The model image generation unit generating a three-dimensional organ model from the image information; generating an organ model image by capturing the generated three-dimensional organ model on a virtual plane set at an arbitrary position; The composite image generation unit generating the composite image by superimposing the cardiac current image on the virtual plane on the organ model image; A medical device that displays the composite image and a first window that represents the positional relationship between the organ and the virtual surface.

3. 3. The medical device according to claim 1 or claim 2, The cardiac current image generating unit generates the cardiac current image by changing the hue of parts of each position of the organ at a specified time that correspond to parts through which relatively high currents are flowing from other parts at the same time, and by increasing at least one of the saturation and brightness compared to other parts at the same time.

4. The medical device according to any one of claims 1 to 3, The cardiac current image generating unit, at a position of the organ, generating the cardiac current image in which the color attribute is changed in a first pattern when the current value increases over time; When the current value decreases over time, the medical device generates the cardiac current image in which the color attribute is changed in a second pattern different from the first pattern.

5. 5. The medical device according to claim 1, The biomagnetic field information includes information about a magnetic field intensity distribution of the biomagnetic field generated by the organ, a magnetic field intensity distribution image generating unit that generates a magnetic field intensity distribution image representing the strength of the biomagnetic field at each position of the organ using the biomagnetic field information; The medical device, wherein the composite image generating unit generates the composite image by superimposing the magnetic field strength distribution image in addition to the organ model image and the cardiac current image.

6. 1. A method for generating an image, comprising: a step of generating a composite image by superimposing an organ model image generated using image information including an MRI image or a CT image of an organ in a living body, the organ model image being a three-dimensional or two-dimensional representation of the organ, and a cardiac current image showing the time-varying change in current flowing through each position of the organ, the time-varying change being obtained from biomagnetic field information obtained from the biomagnetic field generated by the organ, using a change in color attribute; In the cardiac current image, the change in the color attribute is expressed by changing at least one of hue, saturation, and brightness; An image generating method in which the cardiac current image when the current value rises to a predetermined value at a certain position of the organ is different from the cardiac current image when the current value falls to the predetermined value.

Images