Image processing apparatus, image processing system, image display method, and image processing program

Abstract

This image processing device, on the basis of tomographic data acquired by a sensor that acquires tomographic data of biotissue while moving through a lumen of the biotissue, generates and updates three-dimensional data representing the biotissue, and causes a display to display the three-dimensional data as a three-dimensional image. The image processing device comprises a control unit that colors, in the three-dimensional image, at least a voxel representing the inner surface of the biotissue, or a voxel adjacent to the voxel representing the inner surface and representing the lumen, from among a first voxel group corresponding to the cross-section indicated by tomographic data newly acquired by the sensor, thereby creating a distinction from a second voxel group corresponding to another cross-section of the biotissue.

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus, an image processing system, an image display method, and an image processing program.

Background Art

[0002] Patent Documents 1 to 3 describe techniques for generating three-dimensional images of the heart cavity or blood vessels using a US image system. "US" is an abbreviation for ultrasound.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] Treatment using IVUS is widely performed in the heart cavity, cardiovascular system, lower limb artery region, etc. "IVUS" is an abbreviation for intravascular ultrasound. IVUS is a device or method that provides two-dimensional images of a plane perpendicular to the catheter long axis.

[0005] Currently, the operator needs to stack two-dimensional images of IVUS in their head and perform the procedure while reconstructing the three-dimensional structure, which is a barrier especially for young doctors or doctors with little experience. In order to remove such a barrier, it is conceivable to automatically generate a three-dimensional image representing the structure of a biological tissue such as the heart cavity or blood vessel from the two-dimensional images of IVUS and display the generated three-dimensional image for the operator.

[0006] However, IVUS has a limitation in that only the information of one cross-section can be obtained. Even if the structure of a biological tissue is represented by a three-dimensional image, the true real-time information is only the information of one cross-section. Therefore, it is required that the operator can know which part of the three-dimensional image the information currently obtained by IVUS, that is, the latest information, corresponds to.

[0007] An object of the present disclosure is to indicate which part of a three-dimensional image a cross-section of a biological tissue indicated by newly acquired tomographic data by a sensor corresponds to.

Means for Solving the Problem

[0008] An image processing apparatus according to an aspect of the present disclosure is an image processing apparatus that generates and updates three-dimensional data representing a biological tissue based on tomographic data acquired by a sensor that acquires tomographic data of the biological tissue while moving through the lumen of the biological tissue, and displays the three-dimensional data as a three-dimensional image on a display. In the three-dimensional image, among a first voxel group corresponding to a cross-section indicated by the tomographic data newly acquired by the sensor, at least a voxel representing the inner surface of the biological tissue or a voxel representing the lumen adjacent to the voxel representing the inner surface is colored differently from a second voxel group corresponding to another cross-section of the biological tissue. The apparatus includes a control unit.

[0009] As an embodiment, the control unit colors all the voxels representing the biological tissue among the first voxel group differently from the second voxel group.

[0010] As an embodiment, the control unit colors not only the first voxel group but also at least a voxel representing the inner surface or a voxel representing the lumen adjacent to the voxel representing the inner surface among a voxel group corresponding to a cross-section adjacent to the cross-section corresponding to the first voxel group differently from a voxel group corresponding to another cross-section of the biological tissue.

[0011] As one embodiment, the control unit sets the color of at least the voxels representing the inner surface among the first voxel group, or the voxels representing the inner cavity adjacent to the voxels representing the inner surface, to a color different from any color of the second voxel group, thereby coloring at least the voxels representing the inner surface among the first voxel group, or the voxels representing the inner cavity adjacent to the voxels representing the inner surface, to distinguish them from the second voxel group.

[0012] As one embodiment, the control unit causes the display to display a two-dimensional image representing a cross-section indicated by the tomographic data newly acquired by the sensor together with the three-dimensional image.

[0013] As one embodiment, the control unit causes the display to display a first graphic element representing the movement range of the sensor and a second graphic element representing the position of the sensor in combination with the three-dimensional image.

[0014] As one embodiment, the control unit causes the display to display the first graphic element in a direction in which the longitudinal direction of the inner cavity in the three-dimensional image is parallel to the major axis direction of the first graphic element.

[0015] An image processing system as one aspect of the present disclosure includes the image processing apparatus and a probe having the sensor.

[0016] As one embodiment, the image processing system further includes the display.

[0017] An image display method according to an aspect of the present disclosure is an image display method that generates and updates three-dimensional data representing a living tissue based on tomographic data acquired by a sensor that acquires tomographic data of the living tissue while moving through the lumen of the living tissue, and displays the three-dimensional data as a three-dimensional image on a display. In the three-dimensional image, a computer colors at least a voxel representing the inner surface of the living tissue, or a voxel representing the lumen adjacent to the voxel representing the inner surface, in the first voxel group corresponding to the cross section indicated by the tomographic data newly acquired by the sensor, distinguishing it from the second voxel group corresponding to other cross sections of the living tissue.

[0018] An image processing program according to an aspect of the present disclosure causes a computer that generates and updates three-dimensional data representing a living tissue based on tomographic data acquired by a sensor that acquires tomographic data of the living tissue while moving through the lumen of the living tissue, and displays the three-dimensional data as a three-dimensional image on a display, to execute a process of coloring at least a voxel representing the inner surface of the living tissue, or a voxel representing the lumen adjacent to the voxel representing the inner surface, in the first voxel group corresponding to the cross section indicated by the tomographic data newly acquired by the sensor, in the three-dimensional image, distinguishing it from the second voxel group corresponding to other cross sections of the living tissue.

Advantages of the Invention

[0019] According to the present disclosure, it is possible to indicate which part of the three-dimensional image the cross section of the living tissue indicated by the tomographic data newly acquired by the sensor corresponds to.

Brief Description of the Drawings

[0020]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0021] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0022] In each figure, the same or corresponding parts are denoted by the same reference numerals. In the description of the present embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0023] Referring to FIGS. 1 to 4 and FIG. 6, the outline of this embodiment will be described.

[0024] The image processing apparatus 11 according to this embodiment is a computer that causes a display 16 to display three-dimensional data 52 representing a biological tissue 60 as a three-dimensional image 53. As shown in FIG. 4, the image processing apparatus 11 forms a cutting region 62 that exposes the lumen 63 of the biological tissue 60 in the three-dimensional data 52 in the three-dimensional image 53. As shown in FIG. 2, the image processing apparatus 11 causes the display 16 to display a two-dimensional image 56 representing a cross-section 64 of the biological tissue 60 and a region 65 corresponding to the cutting region 62 in the cross-section 64 together with the three-dimensional image 53.

[0025] According to this embodiment, it is possible to show how a part of the structure of the biological tissue 60 has been cut out. Therefore, the user can grasp from the two-dimensional image 56 what structure the uncut and displayed part of the biological tissue 60 has in the three-dimensional image 53. For example, if the user is a surgeon, it becomes easier to perform an operation on the inside of the biological tissue 60.

[0026] The image processing apparatus 11 generates and updates the three-dimensional data 52 based on the tomographic data 51 acquired by a sensor that acquires the tomographic data 51 of the biological tissue 60 while moving the lumen 63 of the biological tissue 60. As shown in FIG. 2, in the three-dimensional image 53, the image processing apparatus 11 colors at least the voxels representing the inner surface 61 of the biological tissue 60, or the voxels representing the lumen 63 adjacent to the voxels representing the inner surface 61, in the first voxel group 54 corresponding to the cross-section 64 indicated by the newly acquired tomographic data 51 by the sensor, separately from the second voxel group 55 corresponding to other cross-sections of the biological tissue 60.

[0027] According to this embodiment, it is possible to indicate which part of the three-dimensional image 53 the cross-section 64 of the biological tissue 60 shown by the tomographic data 51 newly acquired by the sensor corresponds to. Therefore, it becomes easy for the user observing the lumen 63 of the biological tissue 60 using the three-dimensional image 53 to understand which part of the three-dimensional image 53 the information currently obtained by the sensor, that is, the latest information, corresponds to.

[0028] As a modification of this embodiment, among the voxel groups corresponding to the cross-sections adjacent to the cross-section 64 corresponding to the first voxel group 54, not only the first voxel group 54, but also at least the voxels representing the inner surface 61, or the voxels representing the lumen 63 adjacent to the voxels representing the inner surface 61 may be colored differently from the voxel groups corresponding to other cross-sections of the biological tissue 60. According to this modification, the width in the moving direction of the sensor of the voxel group colored differently from the voxel groups corresponding to other cross-sections becomes wider, and it becomes easier for the user to recognize the voxel group in the three-dimensional image 53.

[0029] As a modification of this embodiment, as shown in FIG. 14, all the voxels representing the biological tissue 60 among the first voxel group 54 may be colored differently from the second voxel group 55. According to this modification, since the first voxel group 54 is colored differently from the second voxel group 55 even on the cross-section of the biological tissue 60 formed to observe the lumen 63 of the biological tissue 60, it becomes easier for the user to understand which part of the three-dimensional image 53 the latest information corresponds to.

[0030] In this embodiment, the image processing apparatus 11 causes the display 16 to display the two-dimensional image 56 representing the cross-section 64 together with the three-dimensional image 53 in which at least the voxels representing the inner surface 61 of the biological tissue 60, or the voxels representing the lumen 63 adjacent to the voxels representing the inner surface 61 are colored differently from the second voxel group 55 corresponding to other cross-sections. Therefore, the relationship between the two-dimensional image 56 and the three-dimensional image 53 can be shown.

[0031] The biological tissue 60 includes, for example, blood vessels or organs such as the heart. The biological tissue 60 is not limited to a single anatomical organ or a part thereof, but also includes tissues having a lumen across multiple organs. As an example of such a tissue, specifically, a part of the vascular tissue from the upper part of the inferior vena cava through the right atrium to the lower part of the superior vena cava is mentioned. In the examples from FIG. 2 to FIG. 4, the biological tissue 60 is a blood vessel.

[0032] In FIG. 2, on the screen 80, an operation panel 81, a two-dimensional image 56, a three-dimensional image 53, a first graphic element 86, and a second graphic element 87 are displayed.

[0033] The operation panel 81 is a GUI component for setting the cutting region 62. "GUI" is an abbreviation for graphical user interface. The operation panel 81 is provided with a checkbox 82 for selecting whether to activate the setting of the cutting region 62, a slider 83 for setting the base angle, a slider 84 for setting the opening angle, and a checkbox 85 for selecting whether to use the center of gravity.

[0034] The base angle is the rotation angle of one of the two straight lines L1 and L2 extending from a point M in the cross-sectional image representing the cross-section 64 of the biological tissue 60. Therefore, setting the base angle corresponds to setting the direction of the straight line L1. The opening angle is the angle between the two straight lines L1 and L2. Therefore, setting the opening angle corresponds to setting the angle formed by the two straight lines L1 and L2. The point M is the center of gravity of the cross-section 64. The point M may be set to a point other than the center of gravity on the cross-section 64 when it is selected not to use the center of gravity.

[0035] The two-dimensional image 56 is an image obtained by processing the cross-sectional image. In the two-dimensional image 56, the color of the region 65 corresponding to the cutting region 62 is changed to indicate which part of the cross-section 64 is cut off.

[0036] In this embodiment, the viewpoint when the three-dimensional image 53 is displayed on the screen 80 is adjusted according to the position of the cutting region 62. The viewpoint refers to the position of a virtual camera 71 arranged in the three-dimensional space. In the two-dimensional image 56, the position of the camera 71 with respect to the cross-section 64 is displayed.

[0037] In this embodiment, the cutting region 62 can be determined using the two-dimensional image 56. Specifically, as shown in FIG. 3, by adjusting the base angle or the opening angle and setting the position or size of the region 65 delimited by the two straight lines L1 and L2 in the two-dimensional image 56, the position or size of the cutting region 62 can be set. For example, when the base angle is changed so that the straight line L1 rotates counterclockwise by approximately 90 degrees, in the two-dimensional image 56a, a region 65a that has moved according to the change in the base angle is obtained. Then, according to the position of the region 65a, the position of the cutting region 62 is adjusted. Alternatively, when the opening angle is changed so that the angle between the two straight lines L1 and L2 becomes larger, in the two-dimensional image 56b, a region 65b that has expanded according to the change in the opening angle is obtained. Then, according to the size of the region 65b, the size of the cutting region 62 is adjusted. By adjusting both the base angle and the opening angle and setting both the position and size of the region 65 in the two-dimensional image 56, both the position and size of the cutting region 62 can also be set. The position of the camera 71 may be appropriately adjusted according to the position or size of the cutting region 62.

[0038] In this embodiment, an image corresponding to the current position of the sensor, that is, the latest image, is always displayed as the two-dimensional image 56. However, as a modification of this embodiment, after the cutting region 62 is determined, an image corresponding to a position other than the current position of the sensor may be displayed as the two-dimensional image 56.

[0039] As a modification of this embodiment, instead of setting the base angle by operating the slider 83, it may be set by dragging the straight line L1 or by inputting a numerical value. Similarly, instead of setting the opening angle by operating the slider 84, it may be set by dragging the straight line L2 or by inputting a numerical value.

[0040] In the three-dimensional image 53, the cutting region 62 determined using the two-dimensional image 56 is hidden or transparent. Also, in the three-dimensional image 53, in order to represent the position where the sensor currently exists and is being updated in real time in the longitudinal direction of the lumen 63, the color of the first voxel group 54 corresponding to the current position of the sensor is changed.

[0041] In this embodiment, as shown in FIG. 2, among the first voxel group 54, the voxels representing the inner surface 61 of the biological tissue 60 are set to a color different from that of the second voxel group 55, and thus are colored separately from the second voxel group 55. As a modification of this embodiment, as shown in FIG. 14, all the voxels representing the biological tissue 60 among the first voxel group 54 may be set to different colors. As a further modification, instead of setting the first voxel group 54 and the second voxel group 55 to different colors, the first voxel group 54 may be colored separately from the second voxel group 55 by adjusting the contrast between the first voxel group 54 and the second voxel group 55.

[0042] The first graphic element 86 is a graphic element representing the movement range of the sensor. The second graphic element 87 is a graphic element representing the position of the sensor. In this embodiment, the combination of the first graphic element 86 and the second graphic element 87 is configured as a slider. The first graphic element 86 and the second graphic element 87 may be displayed at any position, but in this embodiment, they are displayed on the right side of the three-dimensional image 53.

[0043] In FIG. 4, the X direction and the Y direction orthogonal to the X direction respectively correspond to the short-side direction of the lumen 63 of the biological tissue 60. The Z direction orthogonal to the X direction and the Y direction corresponds to the longitudinal direction of the lumen 63 of the biological tissue 60.

[0044] In the example of FIG. 4, it is assumed that the check box 85 of the operation panel 81 is in a checked state, that is, using the center of gravity is selected. The image processing apparatus 11 calculates the positions of the centers of gravity B1, B2, B3, and B4 of the cross-sections C1, C2, C3, and C4 of the biological tissue 60 using the three-dimensional data 52. The image processing apparatus 11 sets two planes that intersect with a single line Lb passing through the positions of the centers of gravity B1, B2, B3, and B4 and that respectively include two straight lines L1 and L2 as the cutting planes P1 and P2. For example, if the point M shown in FIG. 2 is the point B3, the straight line L1 is the intersection line of the cross-section C3 and the cutting plane P1, and the straight line L2 is the intersection line of the cross-section C3 and the cutting plane P2. The image processing apparatus 11 forms in the three-dimensional data 52 a cutting region 62 that is sandwiched between the cutting planes P1 and P2 in the three-dimensional image 53 and exposes the lumen 63 of the biological tissue 60.

[0045] In the case of a three-dimensional model of a blood vessel bent as shown in FIG. 4, if the three-dimensional model is cut with a single plane to display the lumen 63, there are cases where the inside of the blood vessel cannot be correctly displayed. In the present embodiment, as shown in FIG. 4, by continuously capturing the center of gravity of the blood vessel, it becomes possible to cut the three-dimensional model so that the inside of the blood vessel can be surely displayed.

[0046] In FIG. 4, for convenience, four cross-sections C1, C2, C3, and C4 are shown as a plurality of cross-sections in the short-side direction of the lumen 63 of the biological tissue 60, but the number of cross-sections for which the center-of-gravity position is calculated is not limited to four, and preferably is the same as the number of cross-sectional images acquired by IVUS.

[0047] As another example different from FIG. 4, assume that the checkbox 85 on the operation panel 81 is not checked, that is, it is selected not to use the center of gravity. In such an example, the image processing apparatus 11 sets two planes including the two straight lines L1 and L2 as cutting planes P1 and P2 that intersect with an arbitrary single line passing through the point M, such as a straight line extending in the Z direction through the point M.

[0048] Referring to FIG. 1, the configuration of the image processing system 10 according to the present embodiment will be described.

[0049] The image processing system 10 includes an image processing apparatus 11, a cable 12, a drive unit 13, a keyboard 14, a mouse 15, and a display 16.

[0050] The image processing apparatus 11 is a dedicated computer specialized for image diagnosis in the present embodiment, but may also be a general-purpose computer such as a PC. "PC" is an abbreviation for personal computer.

[0051] The cable 12 is used to connect the image processing apparatus 11 and the drive unit 13.

[0052] The drive unit 13 is connected to and used with the probe 20 shown in FIG. 5, and is a device that drives the probe 20. The drive unit 13 is also called an MDU. "MDU" is an abbreviation for motor drive unit. The probe 20 is applied to IVUS. The probe 20 is also called an IVUS catheter or an image diagnosis catheter.

[0053] The keyboard 14, the mouse 15, and the display 16 are connected to the image processing apparatus 11 via an arbitrary cable or wirelessly. The display 16 is, for example, an LCD, an organic EL display, or an HMD. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electro luminescence. "HMD" is an abbreviation for head-mounted display.

[0054] The image processing system 10 further includes a connection terminal 17 and a cart unit 18 as options.

[0055] The connection terminal 17 is used to connect the image processing apparatus 11 and an external device. The connection terminal 17 is, for example, a USB terminal. "USB" is an abbreviation for Universal Serial Bus. The external device is, for example, a recording medium such as a magnetic disk drive, a magneto-optical disk drive, or an optical disk drive.

[0056] The cart unit 18 is a cart with casters for movement. The image processing apparatus 11, the cable 12, and the drive unit 13 are installed on the cart body of the cart unit 18. A keyboard 14, a mouse 15, and a display 16 are installed on the top table of the cart unit 18.

[0057] With reference to FIG. 5, the configuration of the probe 20 and the drive unit 13 according to the present embodiment will be described.

[0058] The probe 20 includes a drive shaft 21, a hub 22, a sheath 23, an outer tube 24, an ultrasonic vibrator 25, and a relay connector 26.

[0059] The drive shaft 21 passes through the sheath 23 inserted into the body cavity of a living body and the outer tube 24 connected to the proximal end of the sheath 23, and extends to the inside of the hub 22 provided at the proximal end of the probe 20. The drive shaft 21 has an ultrasonic vibrator 25 for transmitting and receiving signals at its tip and is rotatably provided in the sheath 23 and the outer tube 24. The relay connector 26 connects the sheath 23 and the outer tube 24.

[0060] The hub 22, the drive shaft 21, and the ultrasonic vibrator 25 are connected to each other so that each can move axially forward and backward integrally. Therefore, for example, when an operation is performed to push the hub 22 toward the distal end side, the drive shaft 21 and the ultrasonic vibrator 25 move toward the distal end side inside the sheath 23. For example, when an operation is performed to pull the hub 22 toward the proximal end side, the drive shaft 21 and the ultrasonic vibrator 25 move toward the proximal end side inside the sheath 23 as indicated by the arrow.

[0061] The drive unit 13 includes a scanner unit 31, a slide unit 32, and a bottom cover 33.

[0062] The scanner unit 31 is connected to the image processing apparatus 11 via a cable 12. The scanner unit 31 includes a probe connection portion 34 that connects to the probe 20 and a scanner motor 35 that is a drive source for rotating the drive shaft 21.

[0063] The probe connection portion 34 is detachably connected to the probe 20 via an insertion port 36 of the hub 22 provided at the proximal end of the probe 20. Inside the hub 22, the proximal end of the drive shaft 21 is rotatably supported, and the rotational force of the scanner motor 35 is transmitted to the drive shaft 21. Also, signals are transmitted and received between the drive shaft 21 and the image processing apparatus 11 via the cable 12. In the image processing apparatus 11, generation of a tomographic image of the living body lumen and image processing are performed based on the signal transmitted from the drive shaft 21.

[0064] The slide unit 32 has the scanner unit 31 mounted thereon so as to be movable forward and backward, and is mechanically and electrically connected to the scanner unit 31. The slide unit 32 includes a probe clamp portion 37, a slide motor 38, and a switch group 39.

[0065] The probe clamp portion 37 is coaxially arranged on the distal end side of the probe connection portion 34 and supports the probe 20 connected to the probe connection portion 34.

[0066] The slide motor 38 is a drive source that generates an axial driving force. When the slide motor 38 is driven, the scanner unit 31 moves forward and backward, and accordingly, the drive shaft 21 moves forward and backward in the axial direction. The slide motor 38 is, for example, a servo motor.

[0067] The switch group 39 includes, for example, a forward switch and a pull-back switch that are pressed during the forward and backward operations of the scanner unit 31, and a scan switch that is pressed at the start and end of image drawing. Without being limited to this example, various switches are included in the switch group 39 as necessary.

[0068] When the forward switch is pressed, the slide motor 38 rotates forward and the scanner unit 31 moves forward. On the other hand, when the pull-back switch is pressed, the slide motor 38 rotates in reverse and the scanner unit 31 moves backward.

[0069] When the scan switch is pressed, image drawing is started, the scanner motor 35 is driven, and the slide motor 38 is driven to move the scanner unit 31 backward. A user such as an operator connects the probe 20 to the scanner unit 31 in advance so that the drive shaft 21 rotates and moves toward the proximal end side in the axial direction when image drawing starts. The scanner motor 35 and the slide motor 38 stop when the scan switch is pressed again, and image drawing ends.

[0070] The bottom cover 33 covers the bottom surface and the entire circumference of the side surface on the bottom surface side of the slide unit 32, and can approach and separate from the bottom surface of the slide unit 32 freely.

[0071] Referring to FIG. 6, the configuration of the image processing apparatus 11 will be described.

[0072] The image processing apparatus 11 includes a control unit 41, a storage unit 42, a communication unit 43, an input unit 44, and an output unit 45.

[0073] The control unit 41 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or a GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. The programmable circuit is, for example, an FPGA. "FPGA" is an abbreviation for field-programmable gate array. The dedicated circuit is, for example, an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. While controlling each part of the image processing system 10 including the image processing apparatus 11, the control unit 41 executes processing related to the operation of the image processing apparatus 11.

[0074] The storage unit 42 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM or a ROM. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. The RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. The ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. The storage unit 42 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. Data used for the operation of the image processing apparatus 11, such as tomographic data 51, and data obtained by the operation of the image processing apparatus 11, such as three-dimensional data 52 and three-dimensional images 53, are stored in the storage unit 42.

[0075] The communication unit 43 includes at least one communication interface. The communication interface is, for example, a wired LAN interface, a wireless LAN interface, or an image diagnosis interface that receives an IVUS signal and performs A / D conversion. "LAN" is an abbreviation for local area network. "A / D" is an abbreviation for analog to digital. The communication unit 43 receives data used for the operation of the image processing apparatus 11 and transmits data obtained by the operation of the image processing apparatus 11. In the present embodiment, the drive unit 13 is connected to the image diagnosis interface included in the communication unit 43.

[0076] The input unit 44 includes at least one input interface. The input interface is, for example, a USB interface, an HDMI (registered trademark) interface, or an interface corresponding to a short-range wireless communication standard such as Bluetooth (registered trademark). "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface. The input unit 44 receives a user operation such as an operation of inputting data used for the operation of the image processing apparatus 11. In the present embodiment, the keyboard 14 and the mouse 15 are connected to the USB interface included in the input unit 44 or an interface corresponding to short-range wireless communication. When the touch screen is provided integrally with the display 16, the display 16 may be connected to the USB interface or the HDMI (registered trademark) interface included in the input unit 44.

[0077] The output unit 45 includes at least one output interface. The output interface is, for example, a USB interface, an HDMI (registered trademark) interface, or an interface corresponding to a short-range wireless communication standard such as Bluetooth (registered trademark). The output unit 45 outputs data obtained by the operation of the image processing apparatus 11. In the present embodiment, the display 16 is connected to the USB interface or the HDMI (registered trademark) interface included in the output unit 45.

[0078] The functions of the image processing apparatus 11 are realized by executing the image processing program according to the present embodiment with a processor as the control unit 41. That is, the functions of the image processing apparatus 11 are realized by software. The image processing program causes a computer to execute the operations of the image processing apparatus 11, thereby causing the computer to function as the image processing apparatus 11. That is, the computer functions as the image processing apparatus 11 by executing the operations of the image processing apparatus 11 according to the image processing program.

[0079] The program can be stored in a non-transitory computer-readable medium. Examples of the non-transitory computer-readable medium include a flash memory, a magnetic recording device, an optical disk, a magneto-optical recording medium, or a ROM. The program can be distributed, for example, by selling, transferring, or lending a portable medium such as an SD card storing the program, a DVD, or a CD-ROM. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program may be stored in the storage of a server and transferred from the server to another computer to distribute the program. The program may be provided as a program product.

[0080] The computer stores, once, a program stored in a portable medium or a program transferred from a server in a main memory device. Then, the computer reads the program stored in the main memory device with a processor and executes processing according to the read program with the processor. The computer may directly read a program from a portable medium and execute processing according to the program. The computer may sequentially execute processing according to the received program each time a program is transferred from a server to the computer. The processing may be executed by a so-called ASP type service that realizes functions only by execution instructions and result acquisition without transferring a program from a server to a computer. "ASP" is an abbreviation of application service provider. A program includes information for use in processing by an electronic computer and those conforming to the program. For example, data that is not a direct instruction to a computer but has a nature of defining the processing of the computer corresponds to "those conforming to the program".

[0081] Some or all functions of the image processing apparatus 11 may be realized by a programmable circuit or a dedicated circuit as the control unit 41. That is, some or all functions of the image processing apparatus 11 may be realized by hardware.

[0082] With reference to FIGS. 7 and 8, the operation of the image processing system 10 according to the present embodiment will be described. The operation of the image processing system 10 corresponds to the image display method according to the present embodiment.

[0083] Before the start of the flow of FIG. 7, the probe 20 is primed by the user. Thereafter, the probe 20 is fitted into the probe connection portion 34 and the probe clamp portion 37 of the drive unit 13 and connected and fixed to the drive unit 13. Then, the probe 20 is inserted to a target site in a living tissue 60 such as a blood vessel or the heart.

[0084] In step S101, a so-called pullback operation is performed by pressing the scan switch included in the switch group 39 and further pressing the pullback switch included in the switch group 39. The probe 20 transmits ultrasonic waves by the ultrasonic vibrator 25 that retracts axially by the pullback operation inside the biological tissue 60. The ultrasonic vibrator 25 transmits ultrasonic waves radially while moving inside the biological tissue 60. The ultrasonic vibrator 25 receives the reflected waves of the transmitted ultrasonic waves. The probe 20 inputs the signal of the reflected waves received by the ultrasonic vibrator 25 to the image processing apparatus 11. The control unit 41 of the image processing apparatus 11 processes the input signal and sequentially generates cross-sectional images of the biological tissue 60, thereby obtaining tomographic data 51 including a plurality of cross-sectional images.

[0085] Specifically, the probe 20 transmits ultrasonic waves in a plurality of directions from the rotation center toward the outside by the ultrasonic vibrator 25 while rotating the ultrasonic vibrator 25 in the circumferential direction and moving it in the axial direction inside the biological tissue 60. The probe 20 receives the reflected waves from the reflectors existing in each of the plurality of directions inside the biological tissue 60 by the ultrasonic vibrator 25. The probe 20 transmits the signal of the received reflected waves to the image processing apparatus 11 via the drive unit 13 and the cable 12. The communication unit 43 of the image processing apparatus 11 receives the signal transmitted from the probe 20. The communication unit 43 performs A / D conversion on the received signal. The communication unit 43 inputs the A / D-converted signal to the control unit 41. The control unit 41 processes the input signal and calculates the intensity value distribution of the reflected waves from the reflectors existing in the transmission direction of the ultrasonic waves of the ultrasonic vibrator 25. The control unit 41 sequentially generates a two-dimensional image having a luminance value distribution corresponding to the calculated intensity value distribution as a cross-sectional image of the biological tissue 60, thereby obtaining tomographic data 51 that is a data set of cross-sectional images. The control unit 41 stores the obtained tomographic data 51 in the storage unit 42.

[0086] In the present embodiment, the signal of the reflected waves received by the ultrasonic vibrator 25 corresponds to the raw data of the tomographic data 51, and the cross-sectional image generated by the image processing apparatus 11 processing the signal of the reflected waves corresponds to the processed data of the tomographic data 51.

[0087] As a modification example of the present embodiment, the control unit 41 of the image processing apparatus 11 may store the signal input from the probe 20 as the tomographic data 51 in the storage unit 42 as it is. Alternatively, the control unit 41 may store, in the storage unit 42 as the tomographic data 51, data indicating the intensity value distribution of the reflected wave calculated by processing the signal input from the probe 20. That is, the tomographic data 51 is not limited to the data set of the cross-sectional image of the living tissue 60, and may be data representing the cross-section of the living tissue 60 at each moving position of the ultrasonic vibrator 25 in any form.

[0088] As a modification example of the present embodiment, instead of the ultrasonic vibrator 25 that transmits ultrasonic waves in a plurality of directions while rotating in the circumferential direction, an ultrasonic vibrator that transmits ultrasonic waves in a plurality of directions without rotating may be used.

[0089] As a modification example of the present embodiment, instead of being acquired using IVUS, the tomographic data 51 may be acquired using OFDI or OCT. "OFDI" is an abbreviation for optical frequency domain imaging. "OCT" is an abbreviation for optical coherence tomography. When OFDI or OCT is used, as a sensor for acquiring the tomographic data 51 while moving the lumen 63 of the living tissue 60, instead of the ultrasonic vibrator 25 that transmits ultrasonic waves in the lumen 63 of the living tissue 60 to acquire the tomographic data 51, a sensor that emits light in the lumen 63 of the living tissue 60 to acquire the tomographic data 51 is used.

[0090] As a modification example of the present embodiment, instead of the image processing apparatus 11 generating a data set of the cross-sectional image of the living tissue 60, another apparatus may generate a similar data set, and the image processing apparatus 11 may acquire the data set from the other apparatus. That is, instead of the control unit 41 of the image processing apparatus 11 processing the IVUS signal to generate a cross-sectional image of the living tissue 60, another apparatus may process the IVUS signal to generate a cross-sectional image of the living tissue 60 and input the generated cross-sectional image to the image processing apparatus 11.

[0091] In step S102, the control unit 41 of the image processing apparatus 11 generates three-dimensional data 52 of the biological tissue 60 based on the tomographic data 51 acquired in step S101. That is, the control unit 41 generates the three-dimensional data 52 based on the tomographic data 51 acquired by the sensor. Here, when there is already generated three-dimensional data 52, it is preferable not to generate all the three-dimensional data 52 from scratch, but to update only the data at the location corresponding to the updated tomographic data 51. In that case, the data processing amount when generating the three-dimensional data 52 can be reduced, and the real-time performance of the three-dimensional image 53 in the subsequent step S103 can be improved.

[0092] Specifically, the control unit 41 of the image processing apparatus 11 generates three-dimensional data 52 of the biological tissue 60 by stacking cross-sectional images of the biological tissue 60 included in the tomographic data 51 stored in the storage unit 42 to make it three-dimensional. As the three-dimensionalization method, any method among rendering methods such as surface rendering or volume rendering, and various processes such as texture mapping including environment mapping and bump mapping associated therewith can be used. The control unit 41 stores the generated three-dimensional data 52 in the storage unit 42.

[0093] In step S103, the control unit 41 of the image processing apparatus 11 causes the display 16 to display the three-dimensional data 52 generated in step S102 as a three-dimensional image 53. At this point, the control unit 41 may set the angle at which the three-dimensional image 53 is displayed to an arbitrary angle. The control unit 41 causes the display 16 to display the latest cross-sectional image included in the tomographic data 51 acquired in step S101 together with the three-dimensional image 53.

[0094] Specifically, the control unit 41 of the image processing apparatus 11 generates a three-dimensional image 53 from the three-dimensional data 52 stored in the storage unit 42. The control unit 41 causes the display 16 to display, via the output unit 45, the latest cross-sectional image among the cross-sectional images of the biological tissue 60 included in the tomographic data 51 stored in the storage unit 42 and the generated three-dimensional image 53.

[0095] In this embodiment, the control unit 41 of the image processing apparatus 11 colors, in the three-dimensional image 53, the voxels representing the inner surface 61 of the living tissue 60 among the first voxel group 54 corresponding to the cross section 64 indicated by the tomographic data 51 newly acquired by the sensor, so as to distinguish them from the second voxel group 55 corresponding to other cross sections of the living tissue 60. Specifically, as shown in FIG. 2, the control unit 41 sets the color of the voxels representing the inner surface 61 of the living tissue 60 among the first voxel group 54 to a color different from any color of the second voxel group 55, thereby coloring the voxels representing the inner surface 61 of the living tissue 60 among the first voxel group 54 so as to distinguish them from the second voxel group 55.

[0096] As a modification of this embodiment, the control unit 41 of the image processing apparatus 11 may color, as shown in FIG. 14, all the voxels representing the living tissue 60 among the first voxel group 54 so as to distinguish them from the second voxel group 55. Specifically, the control unit 41 sets the colors of all the voxels representing the living tissue 60 among the first voxel group 54 to a color different from any color of the second voxel group 55, thereby coloring all the voxels representing the living tissue 60 among the first voxel group 54 so as to distinguish them from the second voxel group 55.

[0097] In this embodiment, the control unit 41 of the image processing apparatus 11 causes the first graphic element 86 and the second graphic element 87 to be combined and displayed on the display 16 together with the three-dimensional image 53. Specifically, as shown in FIG. 2, the control unit 41 causes a slider configured by combining the first graphic element 86 and the second graphic element 87 to be displayed on the right side of the three-dimensional image 53 via the output unit 45.

[0098] In this embodiment, the control unit 41 of the image processing apparatus 11 causes the display 16 to display the first graphic element 86 in a direction in which the longitudinal direction of the lumen 63 in the three-dimensional image 53 is parallel to the major axis direction of the first graphic element 86. Specifically, as shown in FIG. 2, the control unit 41 causes the moving range of the sensor indicated by the first graphic element 86 and the display range of the three-dimensional image 53 to coincide with each other in the vertical direction of the screen 80, and causes the position of the sensor indicated by the second graphic element 87 and the position of the first voxel group 54 to coincide with each other.

[0099] In step S104, if there is an operation for setting the angle at which the three-dimensional image 53 is to be displayed as a user change operation, the process of step S105 is performed. If there is no user change operation, the process of step S106 is performed.

[0100] In step S105, the control unit 41 of the image processing apparatus 11 receives, via the input unit 44, an operation for setting the angle at which the three-dimensional image 53 is to be displayed. The control unit 41 adjusts the angle at which the three-dimensional image 53 is to be displayed to the set angle. Then, in step S103, the control unit 41 causes the display 16 to display the three-dimensional image 53 at the angle set in step S105.

[0101] Specifically, the control unit 41 of the image processing apparatus 11 receives, via the input unit 44, an operation for rotating the three-dimensional image 53 displayed on the display 16 using the keyboard 14, the mouse 15, or the touch screen provided integrally with the display 16 by the user. The control unit 41 interactively adjusts the angle at which the three-dimensional image 53 is to be displayed on the display 16 according to the user operation. Alternatively, the control unit 41 receives, via the input unit 44, an operation for the user to input a numerical value of the angle at which the three-dimensional image 53 is to be displayed using the keyboard 14, the mouse 15, or the touch screen provided integrally with the display 16. The control unit 41 adjusts the angle at which the three-dimensional image 53 is to be displayed on the display 16 according to the input numerical value.

[0102] In step S106, if there is an update to the tomographic data 51, the processes of steps S107 and S108 are performed. If there is no update to the tomographic data 51, in step S104, the presence or absence of a user's change operation is confirmed again.

[0103] In step S107, the control unit 41 of the image processing apparatus 11 processes the signal input from the probe 20 to newly generate a cross-sectional image of the biological tissue 60, similar to the process of step S101, thereby obtaining tomographic data 51 including at least one new cross-sectional image.

[0104] In step S108, the control unit 41 of the image processing apparatus 11 updates the three-dimensional data 52 of the biological tissue 60 based on the tomographic data 51 acquired in step S107. That is, the control unit 41 updates the three-dimensional data 52 based on the tomographic data 51 acquired by the sensor. Then, in step S103, the control unit 41 causes the three-dimensional data 52 updated in step S108 to be displayed on the display 16 as a three-dimensional image 53. The control unit 41 causes the latest cross-sectional image included in the tomographic data 51 acquired in step S107 to be displayed on the display 16 together with the three-dimensional image 53. In step S108, it is preferable to update only the data at the location corresponding to the updated tomographic data 51. In that case, the amount of data processing when generating the three-dimensional data 52 can be reduced, and in step S108, the real-time performance of the three-dimensional image 53 can be improved.

[0105] In step S111, if there is an operation to set the cutting region 62 as a user setting operation, the process of step S112 is performed.

[0106] In step S112, the control unit 41 of the image processing apparatus 11 receives the operation to set the cutting region 62 via the input unit 44.

[0107] Specifically, the control unit 41 of the image processing apparatus 11 receives, via the input unit 44, an operation of setting an area 65 corresponding to the cutting area 62 for the cross-sectional image displayed on the display 16 in step S103. In the present embodiment, the control unit 41 receives, as an operation of setting the area 65 corresponding to the cutting area 62, an operation of setting two straight lines L1 and L2 extending from one point M in the cross-sectional image.

[0108] More specifically, the control unit 41 of the image processing apparatus 11 receives, via the input unit 44, an operation in which the user designates the base angle and the opening angle on the operation panel 81 as shown in FIG. 2 using the keyboard 14, the mouse 15, or the touch screen provided integrally with the display 16. That is, the control unit 41 receives, as an operation of setting the two straight lines L1 and L2, an operation of designating the direction of one of the two straight lines L1 and L2 and the angle formed by the two straight lines L1 and L2. Here, it is assumed that the check box 85 on the operation panel 81 is in a checked state, that is, using the center of gravity is selected.

[0109] As a modification of the present embodiment, the control unit 41 of the image processing apparatus 11 may receive, via the input unit 44, an operation in which the user draws the two straight lines L1 and L2 on the cross-sectional image displayed on the display 16 using the keyboard 14, the mouse 15, or the touch screen provided integrally with the display 16. That is, the control unit 41 may receive, as an operation of setting the two straight lines L1 and L2, an operation of drawing the two straight lines L1 and L2 on the cross-sectional image.

[0110] In step S113, the control unit 41 of the image processing apparatus 11 calculates the centroid positions of a plurality of cross-sections in the short-axis direction of the lumen 63 of the biological tissue 60 using the latest 3D data 52 stored in the storage unit 42. The latest 3D data 52 refers to the 3D data 52 generated in step S102 if the process of step S108 has not been performed, and refers to the 3D data 52 updated in step S108 if the process of step S108 has been performed. Here, when the already generated 3D data 52 exists, it is preferable not to regenerate all the 3D data 52 from scratch, but to update only the data at the locations corresponding to the updated tomographic data 51. In that case, the data processing amount when generating the 3D data 52 can be reduced, and the real-time performance of the 3D image 53 in the subsequent step S117 can be improved.

[0111] Specifically, as shown in FIG. 9, if the control unit 41 of the image processing apparatus 11 has generated a new corresponding cross-sectional image in step S107 for each of the plurality of cross-sectional images generated in step S101, it replaces the cross-sectional image with the new one and then binarizes it. As shown in FIG. 10, the control unit 41 extracts a point cloud of the inner surface of the biological tissue 60 from the binarized cross-sectional image. For example, the control unit 41 extracts a point cloud of the inner surface of the blood vessel by extracting one point corresponding to the inner surface of the main blood vessel along the vertical direction of the cross-sectional image with the r-axis as the horizontal axis and the θ-axis as the vertical axis. The control unit 41 may simply obtain the centroid of the extracted point cloud of the inner surface, but in that case, since the point cloud is not uniformly sampled over the inner surface, a deviation occurs in the centroid position. Therefore, in the present embodiment, the control unit 41 calculates the convex hull of the extracted point cloud of the inner surface and calculates the centroid position C using the formula for obtaining the centroid of a polygon as follows n =(C x ,C y ). However, in the following formula, it is assumed that n vertices (x0, y0), (x1, y1), ···, (x n-1 , y n-1 ) as the point cloud of the inner surface as shown in FIG. 10 exist on the convex hull in the counterclockwise direction, and (x n , y n ) is regarded as (x0, y0). [Number]

[0112] The center-of-gravity position obtained as a result is shown in FIG. 11. In FIG. 11, point Cn is the center of the cross-sectional image. Point Bp is the center of gravity of the point group on the inner surface. Point Bv is the center of gravity of the vertices of the polygon. Point Bx is the center of gravity of the polygon as the convex hull.

[0113] As a method for calculating the center-of-gravity position of the blood vessel, a method different from the method for calculating the center-of-gravity position of the polygon as the convex hull may be used. For example, in the original cross-sectional image that has not been binarized, a method of calculating the center position of the largest circle that fits within the main blood vessel as the center-of-gravity position may be used. Alternatively, in a binarized cross-sectional image with the r-axis as the horizontal axis and the θ-axis as the vertical axis, a method of calculating the average position of the pixels in the main blood vessel region as the center-of-gravity position may be used. The same methods can also be used when the biological tissue 60 is not a blood vessel.

[0114] In step S114, the control unit 41 of the image processing apparatus 11 performs smoothing on the calculation result of the center-of-gravity position in step S113.

[0115] As shown in FIG. 12, it can be seen that when the calculation result of the center-of-gravity position is regarded as a function of time, the influence of pulsation is significant. Therefore, in the present embodiment, the control unit 41 of the image processing apparatus 11 performs smoothing on the calculation result of the center-of-gravity position by using a moving average as shown by the dashed line in FIG. 13.

[0116] As a smoothing method, a method other than the moving average may be used. For example, the exponential smoothing method, the kernel method, local regression, the Ramer-Douglas-Peucker algorithm, the Savitzky-Golay method, smoothing splines, or SGM may be used. Alternatively, a method of performing a fast Fourier transform and then removing high-frequency components may be used. Alternatively, a Kalman filter, or a low-pass filter such as a Butterworth filter, a Chebyshev filter, a digital filter, an elliptic filter, or a KZ filter may be used. "SGM" is an abbreviation for the stretched grid method. "KZ" is an abbreviation for Kolmogorov-Zurbenko.

[0117] If only smoothing is performed, the center of gravity position may enter the tissue. In that case, the control unit 41 may divide the calculation result of the center of gravity position according to the positions of a plurality of cross-sections in the short-hand direction of the lumen 63 of the living tissue 60 in the longitudinal direction of the lumen 63 of the living tissue 60, and perform smoothing for each divided calculation result. That is, when the curve of the center of gravity position as shown by the broken line in FIG. 13 overlaps the tissue region, the control unit 41 may divide the curve of the center of gravity position into a plurality of sections and perform individual smoothing for each section. Alternatively, the control unit 41 may adjust the degree of smoothing performed on the calculation result of the center of gravity position according to the positions of a plurality of cross-sections in the short-hand direction of the lumen 63 of the living tissue 60 in the longitudinal direction of the lumen 63 of the living tissue 60. That is, when the curve of the center of gravity position as shown by the broken line in FIG. 13 overlaps the tissue region, the control unit 41 may reduce the degree of smoothing performed on a part of the section including the overlapping point.

[0118] In step S115, as shown in FIG. 4, the control unit 41 of the image processing apparatus 11 sets two planes that intersect with a single line Lb passing through the center of gravity position calculated in step S113 as the cutting planes P1 and P2. In the present embodiment, the control unit 41 sets the cutting planes P1 and P2 after performing smoothing on the calculation result of the center of gravity position in step S114, but the process of step S114 may be omitted.

[0119] Specifically, the control unit 41 of the image processing apparatus 11 sets the curve of the center-of-gravity position obtained as a result of the smoothing in step S114 as line Lb. The control unit 41 sets two planes that intersect with the set line Lb and each contain the two straight lines L1 and L2 set in step S112 as cutting planes P1 and P2. The control unit 41 specifies, in the latest three-dimensional data 52 stored in the storage unit 42, the three-dimensional coordinates that intersect with the cutting planes P1 and P2 of the biological tissue 60 as the three-dimensional coordinates of the edge of the opening that exposes the lumen 63 of the biological tissue 60 in the three-dimensional image 53. The control unit 41 stores the specified three-dimensional coordinates in the storage unit 42.

[0120] In step S116, the control unit 41 of the image processing apparatus 11 forms, in the three-dimensional data 52, a region that is sandwiched between the cutting planes P1 and P2 and exposes the lumen 63 of the biological tissue 60 as a cutting region 62.

[0121] Specifically, the control unit 41 of the image processing apparatus 11 sets, in the latest three-dimensional data 52 stored in the storage unit 42, the portion specified by the three-dimensional coordinates stored in the storage unit 42 to be non-displayed or transparent when the three-dimensional image 53 is displayed on the display 16. That is, the control unit 41 forms the cutting region 62 in accordance with the region 65 set in step S112.

[0122] In step S117, the control unit 41 of the image processing apparatus 11 displays the three-dimensional data 52 in which the cutting region 62 is formed in step S116 on the display 16 as the three-dimensional image 53. The control unit 41 displays, on the display 16 together with the three-dimensional image 53, a two-dimensional image 56 that represents the cross-section 64 indicated by the newly acquired tomographic data 51 by the sensor, which is represented by the cross-sectional image displayed on the display 16 in step S103, and the region 65 corresponding to the cutting region 62 in the cross-section 64.

[0123] Specifically, the control unit 41 of the image processing apparatus 11 processes the latest cross-sectional image among the cross-sectional images of the biological tissue 60 included in the tomographic data 51 stored in the storage unit 42, and generates a two-dimensional image 56 as shown in FIG. 2. The control unit 41 generates a three-dimensional image 53 as shown in FIG. 2, in which the portion specified by the three-dimensional coordinates stored in the storage unit 42 is non-displayed or transparent. The control unit 41 causes the generated two-dimensional image 56 and three-dimensional image 53 to be displayed on the display 16 via the output unit 45.

[0124] In the present embodiment, as shown in FIG. 2, the control unit 41 of the image processing apparatus 11 generates, as the two-dimensional image 56, an image in which the color of the region 65 corresponding to the cutting region 62 is represented by a color different from that of the remaining regions. For example, it is conceivable to change the white portion in a general IVUS image to red in the region 65.

[0125] In step S118, if there is an operation of setting the cutting region 62 as a user change operation, the process of step S119 is performed. If there is no user change operation, the process of step S120 is performed.

[0126] In step S119, the control unit 41 of the image processing apparatus 11 receives, via the input unit 44, an operation of setting the cutting region 62 in the same manner as the process of step S112. Then, the processes after step S115 are performed.

[0127] In step S120, if there is an update of the tomographic data 51, the processes of step S121 and step S122 are performed. If there is no update of the tomographic data 51, in step S118, the presence or absence of a user change operation is confirmed again.

[0128] In step S121, the control unit 41 of the image processing apparatus 11 processes the signal input from the probe 20 in the same manner as the process of step S101 or step S107 to newly generate a cross-sectional image of the biological tissue 60, thereby obtaining tomographic data 51 including at least one new cross-sectional image.

[0129] In step S122, the control unit 41 of the image processing apparatus 11 updates the three-dimensional data 52 of the biological tissue 60 based on the tomographic data 51 acquired in step S121. Thereafter, the processing after step S113 is performed. In step S122, it is preferable to update only the data at the location corresponding to the updated tomographic data 51. In that case, the amount of data processing when generating the three-dimensional data 52 can be reduced, and the real-time performance of the data processing after step S113 can be improved.

[0130] As described above, in this embodiment, the control unit 41 of the image processing apparatus 11 causes the display 16 to display the three-dimensional data 52 representing the biological tissue 60 as a three-dimensional image 53. The control unit 41 forms a cutting region 62 that exposes the lumen 63 of the biological tissue 60 in the three-dimensional data 52 in the three-dimensional image 53. The control unit 41 causes the display 16 to display a two-dimensional image 56 representing the cross-section 64 of the biological tissue 60 and the region 65 corresponding to the cutting region 62 in the cross-section 64 together with the three-dimensional image 53.

[0131] According to this embodiment, it is possible to show how a part of the structure of the biological tissue 60 has been cut off. Therefore, the user can grasp from the two-dimensional image 56 what the structure of the uncut and displayed part of the biological tissue 60 is in the three-dimensional image 53. For example, if the user is a surgeon, it becomes easier to perform an operation on the inside of the biological tissue 60.

[0132] In this embodiment, the control unit 41 of the image processing apparatus 11 generates and updates three-dimensional data 52 representing the biological tissue 60 based on the tomographic data 51 acquired by a sensor that acquires the tomographic data 51 of the biological tissue 60 while moving through the lumen 63 of the biological tissue 60. The control unit 41 causes the display 16 to display the three-dimensional data 52 as a three-dimensional image 53. In the three-dimensional image 53, the control unit 41 colors at least the voxels representing the inner surface 61 of the biological tissue 60, or the voxels representing the lumen 63 adjacent to the voxels representing the inner surface 61, in the first voxel group 54 corresponding to the cross section 64 indicated by the newly acquired tomographic data 51 by the sensor, distinguishing them from the second voxel group 55 corresponding to other cross sections of the biological tissue 60.

[0133] According to this embodiment, it is possible to indicate which part of the three-dimensional image 53 the cross section 64 of the biological tissue 60 indicated by the newly acquired tomographic data 51 by the sensor hits. Therefore, for the user observing the lumen 63 of the biological tissue 60 using the three-dimensional image 53, it becomes easier to understand which part of the three-dimensional image 53 the information currently obtained by the sensor, that is, the latest information, hits.

[0134] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks described in the block diagram may be integrated, or one block may be divided. Instead of executing two or more steps described in the flowchart in time series according to the description, they may be executed in parallel or in a different order according to the processing ability of the device that executes each step, or as necessary. In addition, changes can be made without departing from the spirit of the present disclosure.

Description of Reference Numerals

[0135] 10 Image processing system 11 Image processing apparatus 12 Cable 13 Driving unit 14 Keyboard 15 Mouse 16 Display 17 Connection terminal 18 Cart Unit 20 Probe 21 Drive Shaft 22 Hub 23 Sheath 24 Outer Tube 25 Ultrasonic Vibrator 26 Relay Connector 31 Scanner Unit 32 Slide Unit 33 Bottom Cover 34 Probe Connection Part 35 Scanner Motor 36 Insertion Port 37 Probe Clamp Part 38 Slide Motor 39 Switch Group 41 Control Unit 42 Memory Unit 43 Communication Unit 44 Input Unit 45 Output Unit 51 Tomographic Data 52 3D Data 53 3D Image 54 First Voxel Group 55 Second Voxel Group 56, 56a, 56b 2D Image 60 Biological Tissue 61 Inner Surface 62 Cutting Region 63 Inner Cavity 64 Cross-Section 65, 65a, 65b Region 71 Camera 80 Screen 81 Operation Panel 82 Check Box 83 Slider 84 Slider 85 Check Box 86 First Graphic Element 87 Second Graphic Element

Claims

1. An image processing apparatus that generates and updates three-dimensional data representing the biological tissue based on tomographic data acquired by a sensor that acquires tomographic data of the biological tissue while moving through the lumen of the biological tissue, and displays the three-dimensional data as a three-dimensional image on a display, In the three-dimensional image, among the first voxel group corresponding to the cross-section indicated by the tomographic data newly acquired by the sensor, at least the voxel representing the inner surface of the biological tissue, or the voxel representing the lumen adjacent to the voxel representing the inner surface, is colored differently from the second voxel group corresponding to other cross-sections of the biological tissue. An image processing apparatus comprising a control unit.

2. The image processing apparatus according to claim 1, wherein the control unit colors all the voxels representing the biological tissue in the first voxel group differently from the second voxel group.

3. The image processing apparatus according to claim 1, wherein the control unit colors not only the first voxel group but also at least the voxel representing the inner surface, or the voxel representing the lumen adjacent to the voxel representing the inner surface, among the voxel group corresponding to the cross-section adjacent to the cross-section corresponding to the first voxel group, differently from the voxel group corresponding to other cross-sections of the biological tissue.

4. The image processing apparatus according to any one of claims 1 to 3, wherein the control unit colors at least the voxel representing the inner surface, or the voxel representing the lumen adjacent to the voxel representing the inner surface, in the first voxel group with a color different from any color of the second voxel group, thereby coloring at least the voxel representing the inner surface, or the voxel representing the lumen adjacent to the voxel representing the inner surface, in the first voxel group differently from the second voxel group.

5. The image processing apparatus according to any one of claims 1 to 4, wherein the control unit displays a two-dimensional image representing the cross-section indicated by the tomographic data newly acquired by the sensor on the display together with the three-dimensional image.

6. The control unit causes the display to display, together with the three-dimensional image, a first graphic element representing a movement range of the sensor and a second graphic element representing a position of the sensor, in combination, according to any one of claims 1 to 5. The image processing apparatus according to any one of claims 1 to 5.

7. The control unit causes the display to display the first graphic element in a direction in which a longitudinal direction of the lumen in the three-dimensional image is parallel to a major axis direction of the first graphic element, according to claim 6. The image processing apparatus according to claim 6.

8. An image processing apparatus according to any one of claims 1 to 7, A probe having the sensor, and An image processing system comprising.

9. The image processing system according to claim 8, further comprising the display.

10. An image display method for generating and updating three-dimensional data representing a living tissue based on tomographic data acquired by a sensor that acquires tomographic data of the living tissue while moving through a lumen of the living tissue, and displaying the three-dimensional data as a three-dimensional image on a display, An image display method in which a computer colors, in the three-dimensional image, at least a voxel representing an inner surface of the living tissue, or a voxel representing the lumen adjacent to the voxel representing the inner surface, in a first voxel group corresponding to a cross section indicated by tomographic data newly acquired by the sensor, separately from a second voxel group corresponding to another cross section of the living tissue.

11. A computer for generating and updating three-dimensional data representing a living tissue based on tomographic data acquired by a sensor that acquires tomographic data of the living tissue while moving through a lumen of the living tissue, and displaying the three-dimensional data as a three-dimensional image on a display, An image processing program that executes a process of coloring, in the three-dimensional image, at least a voxel representing the inner surface of the living tissue or a voxel representing the lumen adjacent to the voxel representing the inner surface, in a first voxel group corresponding to a cross section indicated by tomographic data newly acquired by the sensor, separately from a second voxel group corresponding to another cross section of the living tissue.

Images