IMAGE PROCESSING APPARATUS, IMAGE PROCESSING SYSTEM, IMAGE DISPLAY METHOD, AND IMAGE PROCESSING PROGRAM

Abstract

This image processing device is capable of displaying three-dimensional data representing a body tissue as a three-dimensional image on a display, and is provided with a control unit which performs control so as to form, in the three-dimensional data, a cutting area through which an inner cavity of the body tissue is to be exposed in the three-dimensional image and to cause a two-dimensional image representing a portion corresponding to the cutting area in one cross-sectional plane of the body tissue to be displayed at a position corresponding to said one cross-sectional plane in the three-dimensional image.

Description

[Technical field]

[0001] The present disclosure relates to an image processing device, an image processing system, an image display method, and an image processing program. [Background technology]

[0002] Patent Documents 1 to 3 describe techniques for generating three-dimensional images of cardiac chambers or blood vessels using an US imaging system. "US" is an abbreviation for ultrasound. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2010 / 0215238 [Patent Document 2] U.S. Patent No. 6,385,332 [Patent Document 3] U.S. Patent No. 6,251,072 Summary of the Invention [Problem to be solved by the invention]

[0004] Treatment using IVUS is widely used in the cardiac cavity, cardiovascular system, and lower limb arterial area. "IVUS" is an abbreviation of intravascular ultrasound. IVUS is a device or method that provides two-dimensional images in a plane perpendicular to the catheter long axis.

[0005] Currently, surgeons must perform surgery while reconstructing the three-dimensional structure by stacking two-dimensional IVUS images in their minds, which is a barrier, especially for younger or inexperienced doctors. In order to remove such barriers, it is conceivable to automatically generate three-dimensional images that express the structure of biological tissues, such as cardiac chambers or blood vessels, from two-dimensional IVUS images and display the generated three-dimensional images to the surgeon.

[0006] However, in 2D IVUS images, tissue exists surrounding the cardiac or vascular lumen. Therefore, if the generated 3D image is simply displayed as is, the surgeon can only see the outer wall of the tissue and cannot perform treatment on the inside of the biological tissue. Therefore, it is conceivable to cut out a part of the biological tissue structure in the 3D image to allow a view of the lumen. In this case, if information on the cut-out area is missing, it becomes difficult to understand where you are looking within the entire biological tissue. For this reason, it is necessary to be able to supplement the information on the cut-out area.

[0007] The objective of this disclosure is to provide complementary information about a cut-out portion of a biological tissue structure. [Means for solving the problem]

[0008] An image processing device according to one embodiment of the present disclosure is an image processing device that displays three-dimensional data representing biological tissue on a display as a three-dimensional image, and includes a control unit that forms a cut area in the three-dimensional data that exposes the inner cavity of the biological tissue in the three-dimensional image, and controls the display of a two-dimensional image representing a portion of a cross-section of the biological tissue that corresponds to the cut area at a position in the three-dimensional image that corresponds to the cross-section.

[0009] In one embodiment, the control unit generates and updates the three-dimensional data based on cross-sectional data acquired by a sensor that acquires cross-sectional data of the biological tissue while moving through the inner cavity of the biological tissue, and controls the display, each time new cross-sectional data is acquired by the sensor, of an image representing a portion of the cross-section indicated by the newly acquired cross-sectional data by the sensor that corresponds to the cutting area as the two-dimensional image, at a position in the three-dimensional image corresponding to the cross-section indicated by the newly acquired cross-sectional data.

[0010] In one embodiment, the control unit sets the viewpoint when the three-dimensional image is displayed by placing a virtual camera in three-dimensional space, and if the camera is not on a plane corresponding to one of the cross sections in the three-dimensional image, controls the two-dimensional image to be displayed on the plane.

[0011] In one embodiment, when the camera is located on the plane, the control unit performs control to display the two-dimensional image at an angle with respect to the plane.

[0012] In one embodiment, the control unit performs control to display the two-dimensional image in a window separate from the three-dimensional image when the camera is on the plane.

[0013] In one embodiment, the control unit sets the viewpoint when the three-dimensional image is displayed by placing a virtual camera in three-dimensional space, and adjusts the angle at which the two-dimensional image is displayed relative to the plane, regardless of whether the camera is on a plane corresponding to one of the cross sections in the three-dimensional image, to maintain a fixed angle between the line of sight from the camera to the two-dimensional image and the two-dimensional image.

[0014] In one embodiment, the control unit sets a viewpoint when the three-dimensional image is displayed by placing a virtual camera in three-dimensional space, and adjusts the angle at which the two-dimensional image is displayed relative to the plane, regardless of whether the camera is on a plane corresponding to the one cross-section in the three-dimensional image, to maintain the absolute value of the angle at a fixed value.

[0015] In one embodiment, the control unit controls the display of the two-dimensional image to be tilted diagonally downward with respect to the plane when the camera is above the plane, and to be tilted diagonally upward with respect to the plane when the camera is below the plane.

[0016] An image processing system according to one aspect of the present disclosure includes the image processing device and a probe having a sensor that moves through an inner cavity of the biological tissue to acquire tomographic data of the biological tissue.

[0017] In one embodiment, the image processing system further comprises the display.

[0018] An image display method according to one embodiment of the present disclosure is an image display method for displaying three-dimensional data representing biological tissue on a display as a three-dimensional image, in which a computer forms a cut area in the three-dimensional data that exposes the inner cavity of the biological tissue in the three-dimensional image, and the computer controls the display of a two-dimensional image representing a portion of a cross-section of the biological tissue that corresponds to the cut area at a position in the three-dimensional image that corresponds to the cross-section.

[0019] An image processing program as one embodiment of the present disclosure causes a computer that displays three-dimensional data representing biological tissue on a display as a three-dimensional image to perform a process of forming a cut area in the three-dimensional data that exposes the inner cavity of the biological tissue in the three-dimensional image, and a process of controlling the display of a two-dimensional image representing a portion of a cross-section of the biological tissue that corresponds to the cut area at a position in the three-dimensional image that corresponds to the cross-section. Effect of the Invention

[0020] According to the present disclosure, information can be supplemented for cut-out portions of biological tissue structures. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a perspective view of an image processing system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of an image displayed on a display by an image processing system according to an embodiment of the present disclosure. [Diagram 3]1A to 1C are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to an embodiment of the present disclosure. [Figure 4] 1A and 1B are diagrams illustrating an example of a cutting area formed by an image processing system according to an embodiment of the present disclosure. [Diagram 5] FIG. 2 is a perspective view of a probe and a drive unit according to an embodiment of the present disclosure. [Figure 6] 1 is a block diagram showing a configuration of an image processing device according to an embodiment of the present disclosure. [Figure 7] 4 is a flowchart illustrating an operation of the image processing system according to an embodiment of the present disclosure. [Figure 8] 4 is a flowchart illustrating an operation of the image processing system according to an embodiment of the present disclosure. [Figure 9] FIG. 2 is a diagram showing a result of binarizing a cross-sectional image of biological tissue in an embodiment of the present disclosure. [Figure 10] FIG. 13 is a diagram showing the results of extracting a point cloud on the inner surface of biological tissue in an embodiment of the present disclosure. [Figure 11] 11A and 11B are diagrams showing the results of calculating the center of gravity position of a cross section of biological tissue in an embodiment of the present disclosure. [Figure 12] 11A to 11C are diagrams showing results of calculating the center of gravity positions of multiple cross sections of biological tissue in an embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram showing the result of performing smoothing on the result of FIG. 12. [Figure 14] 11A to 11C are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to a first modified example of the embodiment of the present disclosure. [Figure 15] 13A and 13B are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to a second modified example of the embodiment of the present disclosure. [Figure 16A] 13A and 13B are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to a third modified example of the embodiment of the present disclosure. [Figure 16B] 13A and 13B are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to a third modified example of the embodiment of the present disclosure. [Figure 16C] 13A and 13B are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to a third modified example of the embodiment of the present disclosure. [Figure 16D] 13A and 13B are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to a third modified example of the embodiment of the present disclosure. [Figure 17A] 13A to 13C are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to a fourth modified example of the embodiment of the present disclosure. [Figure 17B] 13A to 13C are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to a fourth modified example of the embodiment of the present disclosure. [Figure 17C] 13A to 13C are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to a fourth modified example of the embodiment of the present disclosure. [Figure 17D] 13A to 13C are diagrams illustrating an example of the positional relationship between an image and a camera set by an image processing system according to a fourth modified example of the embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0023] In each drawing, the same or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0024] The present embodiment will be outlined with reference to FIGS. 1 to 4 and 6. FIG.

[0025] The image processing device 11 according to this embodiment is a computer that displays three-dimensional data 52 representing a biological tissue 60 as a three-dimensional image 53 on the display 16. As shown in Fig. 4, the image processing device 11 forms a cutting region 62 that exposes an inner cavity 63 of the biological tissue 60 in the three-dimensional image 53 in the three-dimensional data 52. As shown in Fig. 2, the image processing device 11 performs control to display a two-dimensional image 54 representing a portion of one cross section Ci of the biological tissue 60 that corresponds to the cutting region 62 in the three-dimensional image 53 at a position 65 corresponding to the one cross section Ci.

[0026] According to this embodiment, by cutting out a part of the structure of the biological tissue 60 in the three-dimensional image 53, it is possible to view the lumen 63 of the biological tissue 60. In this case, if information on the cut-out area is missing, it becomes difficult to know where you are looking within the entire biological tissue 60, but in this embodiment, by displaying a two-dimensional image 54 showing a part corresponding to that area, the information on that area can be supplemented.

[0027] In this embodiment, the image processing device 11 generates and updates three-dimensional data 52 based on the tomographic data 51 acquired by a sensor that acquires the tomographic data 51 while moving through a lumen 63 of a biological tissue 60. Every time new tomographic data 51 is acquired by the sensor, the image processing device 11 performs control to display, as a two-dimensional image 54, an image representing a portion of the cross section indicated by the tomographic data 51 newly acquired by the sensor, which corresponds to the cutting region 62, at a position 65 in the three-dimensional image 53 corresponding to the cross section indicated by the newly acquired tomographic data 51. That is, in this embodiment, the cross section Ci is the cross section indicated by the tomographic data 51 last acquired by the sensor.

[0028] According to this embodiment, a cross section of the biological tissue 60 indicated by the tomographic data 51 newly acquired by the sensor can be displayed at the position where the sensor is currently located. Therefore, the missing information can be replaced by the information currently obtained by the sensor, i.e., the latest information.

[0029] In this embodiment, an image corresponding to the current position of the sensor, i.e., the most recent image, is always displayed as the two-dimensional image 54, but as a variation of this embodiment, an image corresponding to a position other than the current position of the sensor may be displayed as the two-dimensional image 54.

[0030] In this embodiment, the image processing device 11 places a virtual camera 71 in a three-dimensional space to set a viewpoint when the three-dimensional image 53 is displayed. When the camera 71 is not on a plane 66 corresponding to the cross section Ci in the three-dimensional image 53, the image processing device 11 performs control to display the two-dimensional image 54 on the plane 66, as shown in FIG.

[0031] The biological tissue 60 includes, for example, blood vessels or organs such as the heart. The biological tissue 60 is not limited to a single anatomical organ or a part thereof, but also includes tissue having a lumen across multiple organs. A specific example of such tissue is a part of the vascular tissue extending from the upper part of the inferior vena cava through the right atrium to the lower part of the superior vena cava. In the examples of Figs. 2 and 4, the biological tissue 60 is a blood vessel.

[0032] 3 and 4, the X direction and the Y direction perpendicular to the X direction correspond to the short direction of the inner cavity 63 of the biological tissue 60. The Z direction perpendicular to the X direction and the Y direction corresponds to the long direction of the inner cavity 63 of the biological tissue 60.

[0033] 4, the image processing device 11 calculates the positions of the centers of gravity B1, B2, B3, and B4 of the cross sections C1, C2, C3, and C4 of the biological tissue 60, respectively, using the three-dimensional data 52. The image processing device 11 sets two planes intersecting at a line Lb passing through the positions of the centers of gravity B1, B2, B3, and B4 as the cutting planes P1 and P2. The image processing device 11 forms in the three-dimensional data 52 a region that is sandwiched between the cutting planes P1 and P2 in the three-dimensional image 53 and exposes the inner cavity 63 of the biological tissue 60 as the cutting region 62.

[0034] In the case of a 3D model of a curved blood vessel as shown in Fig. 4, if the 3D model is cut using one plane to display the lumen 63, the inside of the blood vessel may not be displayed correctly. In this embodiment, by continuously capturing the center of gravity of the blood vessel as shown in Fig. 4, it is possible to cut the 3D model so as to reliably display the inside of the blood vessel.

[0035] In FIG. 4, for convenience, four cross sections C1, C2, C3, and C4 are shown as multiple cross sections in the short direction of the inner cavity 63 of the biological tissue 60, but the number of cross sections for calculating the center of gravity position is not limited to four, and is preferably the same as the number of cross-sectional images acquired by IVUS.

[0036] As an example different from that of Fig. 4, the center of gravity may not be used. In such an example, the image processing device 11 sets two planes intersecting at one arbitrary line, such as a straight line extending in the Z direction, as the cutting planes P1 and P2.

[0037] The configuration of an image processing system 10 according to this embodiment will be described with reference to FIG.

[0038] The image processing system 10 comprises an image processing device 11 , a cable 12 , a drive unit 13 , a keyboard 14 , a mouse 15 , and a display 16 .

[0039] In this embodiment, the image processing device 11 is a dedicated computer specialized for image diagnosis, but may be a general-purpose computer such as a PC. "PC" is an abbreviation for personal computer.

[0040] The cable 12 is used to connect the image processing device 11 and the drive unit 13 .

[0041] The drive unit 13 is a device that is connected to the probe 20 shown in FIG. 5 and drives the probe 20. The drive unit 13 is also called an MDU. "MDU" is an abbreviation for motor drive unit. The probe 20 is applied to IVUS. The probe 20 is also called an IVUS catheter or an imaging diagnostic catheter.

[0042] The keyboard 14, the mouse 15, and the display 16 are connected to the image processing device 11 via any cable or wirelessly. The display 16 is, for example, an LCD, an organic EL display, or an HMD. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electro luminescence. "HMD" is an abbreviation for head-mounted display.

[0043] The image processing system 10 further includes a connection terminal 17 and a cart unit 18 as options.

[0044] The connection terminal 17 is used to connect the image processing device 11 to an external device. The connection terminal 17 is, for example, a USB terminal. "USB" is an abbreviation for Universal Serial Bus. The external device is, for example, a recording medium such as a magnetic disk drive, a magneto-optical disk drive, or an optical disk drive.

[0045] The cart unit 18 is a cart with casters for mobility. The image processing device 11, the cable 12, and the drive unit 13 are installed in the cart body of the cart unit 18. The keyboard 14, the mouse 15, and the display 16 are installed on the top table of the cart unit 18.

[0046] The configurations of the probe 20 and the drive unit 13 according to this embodiment will be described with reference to FIG.

[0047] The probe 20 includes a drive shaft 21, a hub 22, a sheath 23, an outer tube 24, an ultrasonic transducer 25, and a relay connector 26.

[0048] The drive shaft 21 passes through a sheath 23 inserted into a body cavity of a living body and an outer tube 24 connected to the base end of the sheath 23, and extends to the inside of a hub 22 provided at the base end of the probe 20. The drive shaft 21 has an ultrasonic transducer 25 for transmitting and receiving signals at its tip, and is rotatably provided within the sheath 23 and outer tube 24. A relay connector 26 connects the sheath 23 and outer tube 24.

[0049] The hub 22, the drive shaft 21, and the ultrasonic transducer 25 are connected to each other so that they can move forward and backward in the axial direction as a unit. Therefore, for example, when the hub 22 is pushed toward the distal end, the drive shaft 21 and the ultrasonic transducer 25 move toward the distal end inside the sheath 23. For example, when the hub 22 is pulled toward the proximal end, the drive shaft 21 and the ultrasonic transducer 25 move toward the proximal end inside the sheath 23 as shown by the arrows.

[0050] The drive unit 13 includes a scanner unit 31 , a slide unit 32 , and a bottom cover 33 .

[0051] The scanner unit 31 is connected to the image processing device 11 via the cable 12. The scanner unit 31 includes a probe connection portion 34 that connects to the probe 20, and a scanner motor 35 that is a drive source that rotates the drive shaft 21.

[0052] The probe connection part 34 is detachably connected to the probe 20 via a socket 36 of a hub 22 provided at the base end of the probe 20. Inside the hub 22, the base end of the drive shaft 21 is rotatably supported, and the rotational force of a scanner motor 35 is transmitted to the drive shaft 21. In addition, signals are transmitted between the drive shaft 21 and the image processing device 11 via a cable 12. In the image processing device 11, tomographic images of the biological lumen are generated and image processing is performed based on the signals transmitted from the drive shaft 21.

[0053] The slide unit 32 carries the scanner unit 31 so as to be movable forward and backward, and is mechanically and electrically connected to the scanner unit 31. The slide unit 32 includes a probe clamp portion 37, a slide motor 38, and a group of switches 39.

[0054] The probe clamp portion 37 is disposed coaxially with the probe connection portion 34 on the distal side thereof, and supports the probe 20 connected to the probe connection portion 34 .

[0055] The slide motor 38 is a drive source that generates a driving force in the axial direction. The scanner unit 31 moves forward and backward when driven by the slide motor 38, and accordingly the drive shaft 21 moves forward and backward in the axial direction. The slide motor 38 is, for example, a servo motor.

[0056] The group of switches 39 includes, for example, a forward switch and a pullback switch that are pressed when moving the scanner unit 31 forward or backward, and a scan switch that is pressed when starting and ending image drawing. The group of switches 39 is not limited to the examples given here, and various switches may be included in the group of switches 39 as necessary.

[0057] When the forward switch is pressed, the slide motor 38 rotates forward, moving the scanner unit 31 forward. On the other hand, when the pullback switch is pressed, the slide motor 38 rotates backward, moving the scanner unit 31 backward.

[0058] When the scan switch is pressed, image drawing begins, and the scanner motor 35 and slide motor 38 are driven to move the scanner unit 31 backward. A user such as an operator connects the probe 20 to the scanner unit 31 in advance, so that when image drawing begins, the drive shaft 21 rotates and moves toward the base end in the axial direction. When the scan switch is pressed again, the scanner motor 35 and slide motor 38 stop, and image drawing ends.

[0059] The bottom cover 33 covers the bottom surface of the slide unit 32 and the entire periphery of the side surface on the bottom surface side, and is freely movable toward and away from the bottom surface of the slide unit 32 .

[0060] The configuration of the image processing device 11 will be described with reference to FIG.

[0061] The image processing device 11 includes a control unit 41, a storage unit 42, a communication unit 43, an input unit 44, and an output unit 45.

[0062] The control unit 41 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. The programmable circuit is, for example, an FPGA. "FPGA" is an abbreviation for field-programmable gate array. The dedicated circuit is, for example, an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 41 executes processes related to the operation of the image processing device 11 while controlling each part of the image processing system 10 including the image processing device 11.

[0063] The storage unit 42 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM or a ROM. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. The RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. The ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. The storage unit 42 functions, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 42 stores data used in the operation of the image processing device 11, such as the tomographic data 51, and data obtained by the operation of the image processing device 11, such as the three-dimensional data 52 and the three-dimensional image 53.

[0064] The communication unit 43 includes at least one communication interface. The communication interface is, for example, a wired LAN interface, a wireless LAN interface, or an image diagnosis interface that receives and A / D converts IVUS signals. "LAN" is an abbreviation for local area network. "A / D" is an abbreviation for analog to digital. The communication unit 43 receives data used in the operation of the image processing device 11, and transmits data obtained by the operation of the image processing device 11. In this embodiment, the drive unit 13 is connected to the image diagnosis interface included in the communication unit 43.

[0065] The input unit 44 includes at least one input interface. The input interface is, for example, a USB interface, an HDMI (registered trademark) interface, or an interface compatible with a short-range wireless communication standard such as Bluetooth (registered trademark). "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface. The input unit 44 accepts user operations such as an operation to input data used in the operation of the image processing device 11. In this embodiment, the keyboard 14 and the mouse 15 are connected to a USB interface included in the input unit 44 or an interface compatible with short-range wireless communication. When a touch screen is provided integrally with the display 16, the display 16 may be connected to a USB interface or an HDMI (registered trademark) interface included in the input unit 44.

[0066] The output unit 45 includes at least one output interface. The output interface is, for example, a USB interface, an HDMI (registered trademark) interface, or an interface compatible with a short-range wireless communication standard such as Bluetooth (registered trademark). The output unit 45 outputs data obtained by the operation of the image processing device 11. In this embodiment, the display 16 is connected to the USB interface or the HDMI (registered trademark) interface included in the output unit 45.

[0067] The functions of the image processing device 11 are realized by executing an image processing program according to this embodiment in a processor serving as the control unit 41. That is, the functions of the image processing device 11 are realized by software. The image processing program causes a computer to execute the operations of the image processing device 11, thereby causing the computer to function as the image processing device 11. That is, the computer functions as the image processing device 11 by executing the operations of the image processing device 11 in accordance with the image processing program.

[0068] The program may be stored in a non-transitory computer-readable medium. Examples of the non-transitory computer-readable medium include a flash memory, a magnetic recording device, an optical disk, a magneto-optical recording medium, or a ROM. The program may be distributed, for example, by selling, transferring, or lending a portable medium such as an SD card, DVD, or CD-ROM on which the program is stored. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program may be distributed by storing the program in the storage of a server and transferring the program from the server to another computer. The program may be provided as a program product.

[0069] A computer temporarily stores a program stored in a portable medium or transferred from a server in a main storage device. Then, the computer reads the program stored in the main storage device with a processor and executes processing according to the read program with the processor. The computer may read the program directly from a portable medium and execute processing according to the program. The computer may execute processing according to the received program each time a program is transferred from the server to the computer. Processing may be executed by a so-called ASP-type service that realizes functions only by execution instructions and result acquisition without transferring a program from the server to the computer. "ASP" is an abbreviation for application service provider. Programs include information used for processing by a computer and equivalent to a program. For example, data that is not a direct command to a computer but has the property of defining computer processing falls under " equivalent to a program.

[0070] A part or all of the functions of the image processing device 11 may be realized by a programmable circuit or a dedicated circuit as the control unit 41. In other words, a part or all of the functions of the image processing device 11 may be realized by hardware.

[0071] The operation of the image processing system 10 according to this embodiment will be described with reference to Figures 7 and 8. The operation of the image processing system 10 corresponds to an image display method according to this embodiment.

[0072] 7, the user primes the probe 20. After that, the probe 20 is fitted into the probe connection portion 34 and the probe clamp portion 37 of the drive unit 13, and is connected and fixed to the drive unit 13. Then, the probe 20 is inserted to a target site in biological tissue 60 such as a blood vessel or the heart.

[0073] In step S101, a scan switch included in the switch group 39 is pressed, and then a pullback switch included in the switch group 39 is pressed, thereby performing a so-called pullback operation. The probe 20 transmits ultrasonic waves from the ultrasonic transducer 25, which is moved back in the axial direction by the pullback operation, inside the biological tissue 60. The ultrasonic transducer 25 transmits ultrasonic waves radially while moving inside the biological tissue 60. The ultrasonic transducer 25 receives reflected waves of the transmitted ultrasonic waves. The probe 20 inputs signals of the reflected waves received by the ultrasonic transducer 25 to the image processing device 11. The control unit 41 of the image processing device 11 processes the input signals to sequentially generate cross-sectional images of the biological tissue 60, thereby acquiring tomographic data 51 including a plurality of cross-sectional images.

[0074] Specifically, the probe 20 transmits ultrasonic waves from the ultrasonic transducer 25 in a plurality of directions from the center of rotation outward while rotating the ultrasonic transducer 25 in the circumferential direction and moving it in the axial direction inside the biological tissue 60. The probe 20 receives reflected waves from reflectors present in each of a plurality of directions inside the biological tissue 60 using the ultrasonic transducer 25. The probe 20 transmits signals of the received reflected waves to the image processing device 11 via the drive unit 13 and the cable 12. The communication unit 43 of the image processing device 11 receives the signal transmitted from the probe 20. The communication unit 43 A / D converts the received signal. The communication unit 43 inputs the A / D converted signal to the control unit 41. The control unit 41 processes the input signal to calculate the intensity value distribution of the reflected waves from the reflectors present in the transmission direction of the ultrasonic waves from the ultrasonic transducer 25. The control unit 41 sequentially generates two-dimensional images having a brightness value distribution corresponding to the calculated intensity value distribution as cross-sectional images of the biological tissue 60, thereby acquiring tomographic data 51, which is a data set of the cross-sectional images. The control unit 41 stores the acquired tomographic data 51 in the storage unit 42 .

[0075] In this embodiment, the reflected wave signal received by the ultrasonic transducer 25 corresponds to raw data of the tomographic data 51, and the cross-sectional image generated by the image processing device 11 by processing the reflected wave signal corresponds to processed data of the tomographic data 51.

[0076] As a modified example of this embodiment, the control unit 41 of the image processing device 11 may store the signal input from the probe 20 as it is in the storage unit 42 as the tomographic data 51. Alternatively, the control unit 41 may store data indicating the intensity value distribution of the reflected wave calculated by processing the signal input from the probe 20 in the storage unit 42 as the tomographic data 51. In other words, the tomographic data 51 is not limited to a data set of a cross-sectional image of the biological tissue 60, but may be data that represents a cross-section of the biological tissue 60 at each moving position of the ultrasonic transducer 25 in some form.

[0077] As a modification of this embodiment, instead of the ultrasonic transducer 25 that transmits ultrasonic waves in multiple directions while rotating in the circumferential direction, an ultrasonic transducer that transmits ultrasonic waves in multiple directions without rotating may be used.

[0078] As a modified example of this embodiment, the tomographic data 51 may be acquired using OFDI or OCT instead of using IVUS. "OFDI" is an abbreviation for optical frequency domain imaging. "OCT" is an abbreviation for optical coherence tomography. When OFDI or OCT is used, a sensor that emits light in the cavity 63 of the biological tissue 60 to acquire the tomographic data 51 is used as a sensor that acquires the tomographic data 51 while moving through the cavity 63 of the biological tissue 60, instead of the ultrasonic transducer 25 that transmits ultrasonic waves in the cavity 63 of the biological tissue 60 to acquire the tomographic data 51.

[0079] As a modified example of this embodiment, instead of the image processing device 11 generating a data set of the cross-sectional image of the biological tissue 60, another device may generate a similar data set, and the image processing device 11 may acquire the data set from the other device. That is, instead of the control unit 41 of the image processing device 11 processing an IVUS signal to generate a cross-sectional image of the biological tissue 60, another device may process an IVUS signal to generate a cross-sectional image of the biological tissue 60, and input the generated cross-sectional image to the image processing device 11.

[0080] In step S102, the control unit 41 of the image processing device 11 generates three-dimensional data 52 of the biological tissue 60 based on the tomographic data 51 acquired in step S101. That is, the control unit 41 generates the three-dimensional data 52 based on the tomographic data 51 acquired by the sensor. Here, when the three-dimensional data 52 that has already been generated exists, it is preferable to update only the data corresponding to the updated tomographic data 51, rather than regenerating all the three-dimensional data 52 from scratch. In that case, the amount of data processing when generating the three-dimensional data 52 can be reduced, and the real-time performance of the three-dimensional image 53 in the subsequent step S103 can be improved.

[0081] Specifically, the control unit 41 of the image processing device 11 generates three-dimensional data 52 of the biological tissue 60 by stacking cross-sectional images of the biological tissue 60 included in the tomographic data 51 stored in the storage unit 42 to create a three-dimensional image. As a method for creating a three-dimensional image, any method from among rendering methods such as surface rendering or volume rendering, and various processes associated therewith, such as texture mapping including environment mapping, and bump mapping, is used. The control unit 41 stores the generated three-dimensional data 52 in the storage unit 42.

[0082] In step S103, the control unit 41 of the image processing device 11 causes the three-dimensional data 52 generated in step S102 to be displayed on the display 16 as a three-dimensional image 53. At this point, the control unit 41 may set the angle at which the three-dimensional image 53 is displayed to any angle.

[0083] Specifically, the control unit 41 of the image processing device 11 generates a three-dimensional image 53 from the three-dimensional data 52 stored in the storage unit 42. The control unit 41 causes the generated three-dimensional image 53 to be displayed on the display 16 via the output unit 45.

[0084] In step S104, if the user performs a change operation to set the angle at which the three-dimensional image 53 is displayed, the process proceeds to step S105. If the user does not perform a change operation, the process proceeds to step S106.

[0085] In step S105, the control unit 41 of the image processing device 11 accepts an operation to set the angle at which the three-dimensional image 53 is displayed via the input unit 44. The control unit 41 adjusts the angle at which the three-dimensional image 53 is displayed to the set angle. Then, in step S103, the control unit 41 causes the display 16 to display the three-dimensional image 53 at the angle set in step S105.

[0086] Specifically, the control unit 41 of the image processing device 11 receives, via the input unit 44, an operation by the user to rotate the three-dimensional image 53 displayed on the display 16 using the keyboard 14, the mouse 15, or a touch screen integrally provided with the display 16. The control unit 41 interactively adjusts the angle at which the three-dimensional image 53 is displayed on the display 16 in response to the user's operation. Alternatively, the control unit 41 receives, via the input unit 44, an operation by the user to input a numerical value of the angle at which the three-dimensional image 53 is displayed using the keyboard 14, the mouse 15, or a touch screen integrally provided with the display 16. The control unit 41 adjusts the angle at which the three-dimensional image 53 is displayed on the display 16 in accordance with the input numerical value.

[0087] In step S106, if the tomographic data 51 is updated, the processes of steps S107 and S108 are performed. If the tomographic data 51 is not updated, in step S104, it is confirmed again whether or not the user has performed a change operation.

[0088] In step S107, the control unit 41 of the image processing device 11 processes the signal input from the probe 20 to generate a new cross-sectional image of the biological tissue 60, similar to the processing in step S101, thereby acquiring cross-sectional data 51 including at least one new cross-sectional image.

[0089] In step S108, the control unit 41 of the image processing device 11 updates the three-dimensional data 52 of the biological tissue 60 based on the tomographic data 51 acquired in step S107. That is, the control unit 41 updates the three-dimensional data 52 based on the tomographic data 51 acquired by the sensor. Then, in step S103, the control unit 41 displays the three-dimensional data 52 updated in step S108 on the display 16 as a three-dimensional image 53. In step S108, it is preferable to update only the data at the location corresponding to the updated tomographic data 51. In that case, the amount of data processing when generating the three-dimensional data 52 can be reduced, and the real-time performance of the three-dimensional image 53 can be improved in step S108.

[0090] In step S111, if the user performs a setting operation to set the angle between the cutting planes P1 and P2 as shown in FIG. 4, the process proceeds to step S112.

[0091] In step S112, the control unit 41 of the image processing device 11 accepts, via the input unit 44, an operation for setting the angle between the cutting planes P1 and P2.

[0092] Specifically, the control unit 41 of the image processing device 11 accepts, via the input unit 44, an operation in which the user inputs the numerical value of the angle between the cut surfaces P1 and P2 using the keyboard 14, the mouse 15, or a touch screen integral with the display 16.

[0093] In step S113, the control unit 41 of the image processing device 11 calculates the center of gravity of multiple cross sections in the short direction of the lumen 63 of the biological tissue 60 using the latest three-dimensional data 52 stored in the storage unit 42. The latest three-dimensional data 52 refers to the three-dimensional data 52 generated in step S102 if the processing of step S108 has not been performed, and refers to the three-dimensional data 52 updated in step S108 if the processing of step S108 has been performed. Here, when three-dimensional data 52 that has already been generated exists, it is preferable to update only the data corresponding to the updated tomographic data 51, rather than regenerating all the three-dimensional data 52 from scratch. In that case, the amount of data processing when generating the three-dimensional data 52 can be reduced, and the real-time performance of the three-dimensional image 53 in the subsequent step S117 can be improved.

[0094] Specifically, as shown in FIG. 9, if a new cross-sectional image corresponding to each of the multiple cross-sectional images generated in step S101 has been generated in step S107, the control unit 41 of the image processing device 11 replaces the new cross-sectional image and binarizes the new cross-sectional image. As shown in FIG. 10, the control unit 41 extracts a point group of the inner surface of the biological tissue 60 from the binarized cross-sectional image. For example, the control unit 41 extracts points corresponding to the inner surface of the main blood vessel one by one along the vertical direction of the cross-sectional image with the r axis as the horizontal axis and the θ axis as the vertical axis, thereby extracting the point group of the inner surface of the blood vessel. The control unit 41 may simply obtain the center of gravity of the extracted point group of the inner surface, but in that case, the point group is not sampled uniformly over the inner surface, so that a deviation occurs in the center of gravity position. Therefore, in this embodiment, the control unit 41 calculates the convex hull of the extracted point group of the inner surface, and obtains the center of gravity position C using the following equation for obtaining the center of gravity of a polygon: n =(C x ,C y In the following formula, n vertices (x0, y0), (x1, y1), . . . , (x n-1 ,y n-1 ) lies on the convex hull in counterclockwise order, and (x n ,y n ) is considered to be (x0,y0).

number

[0095] The resulting centroid positions are shown in Figure 11. In Figure 11, point Cn is the center of the cross-sectional image. Point Bp is the centroid of the points of the inner surface. Point Bv is the centroid of the vertices of the polygon. Point Bx is the centroid of the polygon as a convex hull.

[0096] The method for calculating the center of gravity of the blood vessel may be a method other than the method for calculating the center of gravity of a polygon as a convex hull. For example, a method may be used in which the center of gravity of the largest circle that fits within the main blood vessel is calculated as the center of gravity in the original cross-sectional image that is not binarized. Alternatively, a method may be used in which the average position of pixels in the main blood vessel region is calculated as the center of gravity in a binarized cross-sectional image with the r axis as the horizontal axis and the θ axis as the vertical axis. Similar methods may be used when the biological tissue 60 is not a blood vessel.

[0097] In step S114, the control unit 41 of the image processing device 11 performs smoothing on the calculation result of the center of gravity position in step S113.

[0098] As shown in Fig. 12, when the calculation result of the center of gravity position is viewed as a function of time, it is found that the influence of pulsation occurs significantly. Therefore, in this embodiment, the control unit 41 of the image processing device 11 performs smoothing on the calculation result of the center of gravity position by using a moving average, as shown by the dashed line in Fig. 13.

[0099] As a method of smoothing, a method other than the moving average may be used. For example, an exponential smoothing method, a kernel method, a local regression, a Ramer-Douglas-Peucker algorithm, a Savitzky-Golay method, a smoothing spline, or an SGM may be used. Alternatively, a method of performing a fast Fourier transform and then removing high frequency components may be used. Alternatively, a low pass filter such as a Kalman filter, a Butterworth filter, a Chebyshev filter, a digital filter, an elliptic filter, or a KZ filter may be used. "SGM" is an abbreviation for the stretched grid method. "KZ" is an abbreviation for Kolmogorov-Zurbenko.

[0100] If smoothing is simply performed, the center of gravity may be located inside the tissue. In that case, the control unit 41 may divide the calculation result of the center of gravity according to the positions of multiple cross sections in the short direction of the lumen 63 of the biological tissue 60 in the longitudinal direction of the lumen 63 of the biological tissue 60, and perform smoothing for each of the divided calculation results. That is, when the curve of the center of gravity as shown by the dashed line in FIG. 13 overlaps with the tissue region, the control unit 41 may divide the curve of the center of gravity into multiple sections and perform individual smoothing for each section. Alternatively, the control unit 41 may adjust the degree of smoothing to be performed on the calculation result of the center of gravity according to the positions of multiple cross sections in the short direction of the lumen 63 of the biological tissue 60 in the longitudinal direction of the lumen 63 of the biological tissue 60. That is, when the curve of the center of gravity as shown by the dashed line in FIG. 13 overlaps with the tissue region, the control unit 41 may reduce the degree of smoothing to be performed on some sections including the overlapping points.

[0101] In step S115, the control unit 41 of the image processing device 11 sets two planes intersecting at one line Lb passing through the center of gravity position calculated in step S113 as cutting planes P1 and P2, as shown in Fig. 4. In this embodiment, the control unit 41 sets the cutting planes P1 and P2 after smoothing the calculation result of the center of gravity position in step S114, but the process of step S114 may be omitted.

[0102] Specifically, the control unit 41 of the image processing device 11 sets the curve of the center of gravity position obtained as a result of the smoothing in step S114 as a line Lb. The control unit 41 sets two planes that intersect at the set line Lb and form the angle set in step S112 as cutting planes P1 and P2. The control unit 41 specifies the three-dimensional coordinates that intersect with the cutting planes P1 and P2 of the biological tissue 60 in the latest three-dimensional data 52 stored in the storage unit 42 as the three-dimensional coordinates of the edge of the opening that exposes the inner cavity 63 of the biological tissue 60 in the three-dimensional image 53. The control unit 41 stores the specified three-dimensional coordinates in the storage unit 42. The positions of the cutting planes P1 and P2 may be set at any positions, but in this embodiment, they are set so that the opening is located in front on the screen of the display 16.

[0103] In step S116, the control unit 41 of the image processing device 11 forms in the three-dimensional data 52 a region sandwiched between the cutting planes P1 and P2 in the three-dimensional image 53 and exposing the inner cavity 63 of the biological tissue 60 as the cutting region 62.

[0104] Specifically, the control unit 41 of the image processing device 11 sets the part of the latest three-dimensional data 52 stored in the memory unit 42 that is identified by the three-dimensional coordinates stored in the memory unit 42 to be hidden or transparent when the three-dimensional image 53 is displayed on the display 16.

[0105] In step S117, the control unit 41 of the image processing device 11 causes the three-dimensional data 52 in which the cutting region 62 has been formed in step S116 to be displayed on the display 16 as a three-dimensional image 53. The control unit 41 performs control to display a two-dimensional image 54 representing a portion of one cross section Ci of the biological tissue 60 that corresponds to the cutting region 62, at a position 65 in the three-dimensional image 53 that corresponds to the one cross section Ci.

[0106] Specifically, the control unit 41 of the image processing device 11 cuts out the latest cross-sectional image from among the cross-sectional images of the biological tissue 60 included in the tomographic data 51 stored in the storage unit 42, and generates a two-dimensional image 54 as shown in Fig. 2. The control unit 41 generates a three-dimensional image 53 as shown in Fig. 2, in which a portion specified by the three-dimensional coordinates stored in the storage unit 42 is hidden or transparent. The control unit 41 causes the generated two-dimensional image 54 and three-dimensional image 53 to be displayed on the display 16 via the output unit 45.

[0107] In this embodiment, as shown in FIG. 3, when the camera 71 is not on the plane 66 corresponding to the cross section Ci in the three-dimensional image 53, the control unit 41 of the image processing device 11 performs control to display the two-dimensional image 54 on the plane 66. When the camera 71 is on the plane 66 in the three-dimensional image 53, the line of sight 72 from the camera 71 to the two-dimensional image 54 is located on the plane 66, so that even if an attempt is made to display the two-dimensional image 54 on the plane 66, the two-dimensional image 54 cannot be seen. In other words, when the camera 71 is on the plane 66 in the three-dimensional image 53, the two-dimensional image 54 cannot be displayed on the plane 66. Therefore, the following measures may be considered.

[0108] As a first modified example of this embodiment, the two-dimensional image 54 may be displayed at an angle. That is, the two-dimensional image 54 may be displayed at a certain angle with respect to the camera 71. Specifically, the control unit 41 of the image processing device 11 may perform control to display the two-dimensional image 54 at an angle with respect to the plane 66 when the camera 71 is on the plane 66 as shown in FIG. 14. In the first modified example, the control unit 41 performs control to display the two-dimensional image 54 on the plane 66 when the camera 71 is not on the plane 66 as shown in FIG. 3.

[0109] As a second modified example of this embodiment, a separate window may be temporarily displayed to display the two-dimensional image 54. That is, the two-dimensional image 54 may be displayed in a mini view. Specifically, the control unit 41 of the image processing device 11 may perform control to display the two-dimensional image 54 in a window separate from the three-dimensional image 53 when the camera 71 is on the plane 66 as shown in FIG. 15. In the second modified example, the control unit 41 also performs control to display the two-dimensional image 54 on the plane 66 when the camera 71 is not on the plane 66 as shown in FIG. 3.

[0110] As a third modified example of this embodiment, the two-dimensional image 54 may be always displayed at the same angle with respect to the camera 71. That is, the two-dimensional image 54 may be easily viewed by always maintaining the same angle with respect to the camera 71. Specifically, as shown in FIG. 16A to FIG. 16D, the control unit 41 of the image processing device 11 may adjust the angle at which the two-dimensional image 54 is displayed with respect to the plane 66 regardless of whether the camera 71 is on the plane 66 corresponding to the cross section Ci in the three-dimensional image 53, thereby maintaining the angle between the line of sight 72 from the camera 71 to the two-dimensional image 54 and the two-dimensional image 54 at a fixed angle. The "fixed angle" may be any angle as long as the absolute value is greater than 0 degrees and less than 90 degrees. For example, if the sensor moves along the Z direction, the position 65 at which the two-dimensional image 54 is displayed also changes along the Z direction, but in the third modified example, the angle between the line of sight 72 and the two-dimensional image 54 is always constant. That is, in FIGS. 16A, 16B, 16C, and 16D, angles A1, A2, A3, and A4 formed between line of sight 72 and two-dimensional image 54 are all the same angle.

[0111] As a fourth modification of the present embodiment, the two-dimensional image 54 may be displayed at an inclination at all times. Specifically, as shown in FIG. 17A to FIG. 17D, the control unit 41 of the image processing device 11 may adjust the angle at which the two-dimensional image 54 is displayed with respect to the plane 66 regardless of whether the camera 71 is on the plane 66 corresponding to the cross section Ci in the three-dimensional image 53, and may maintain the absolute value of the angle at a fixed value. The "fixed value" may be any angle that is greater than 0 degrees and smaller than 90 degrees. In the fourth modification, the control unit 41 performs control to display the two-dimensional image 54 at an inclination obliquely downward with respect to the plane 66 when the camera 71 is above the plane 66 as shown in FIG. 17A. In this modification, the control unit 41 performs control to display the two-dimensional image 54 at an inclination obliquely downward with respect to the plane 66 even when the camera 71 is on the plane 66 as shown in FIG. 17B, but may perform control to display the two-dimensional image 54 at an inclination obliquely upward with respect to the plane 66 when the camera 71 is on the plane 66. As shown in Figs. 17C and 17D, when the camera 71 is below the plane 66, the control unit 41 performs control to display the two-dimensional image 54 at an angle upward with respect to the plane 66. This makes it easier to avoid a situation in which the two-dimensional image 54 overlaps most of the three-dimensional image 53, making the three-dimensional image 53 almost invisible. For example, if the sensor moves along the Z direction, the position 65 at which the two-dimensional image 54 is displayed also changes along the Z direction, but in the fourth modified example, the absolute value of the angle between the plane 66 and the two-dimensional image 54 is always constant. That is, in Figs. 17A, 17B, 17C, and 17D, the absolute values ​​of the angles A5, A6, A7, and A8 between the plane 66 and the two-dimensional image 54 are all the same.

[0112] In step S118, if the user performs a change operation to set the angle between the cutting planes P1 and P2 or to set the angle at which the three-dimensional image 53 is displayed, the process of step S119 is performed. If the user does not perform a change operation, the process of step S120 is performed.

[0113] In step S119, the control unit 41 of the image processing device 11 accepts an operation to set the angle between the cutting planes P1 and P2 through the input unit 44, similar to the processing of step S112. In that case, the processing of step S115 and subsequent steps is performed. Alternatively, the control unit 41 accepts an operation to set the angle at which the three-dimensional image 53 is displayed through the input unit 44, similar to the processing of step S105. The control unit 41 adjusts the angle at which the three-dimensional image 53 is displayed to the set angle. In that case, the processing of step S115 and subsequent steps is also performed. In step S115, the control unit 41 adjusts the positions of the cutting planes P1 and P2 according to the set angle at which the three-dimensional image 53 is displayed. That is, the positions of the cutting planes P1 and P2 are adjusted again so that the opening is located in front on the screen of the display 16.

[0114] In step S120, if the tomographic data 51 is updated, the processes of steps S121 and S122 are performed. If the tomographic data 51 is not updated, in step S118, it is confirmed again whether or not the user has performed a change operation.

[0115] In step S121, the control unit 41 of the image processing device 11 processes the signal input from the probe 20 to generate a new cross-sectional image of the biological tissue 60, similar to the processing in step S101 or step S107, thereby acquiring cross-sectional data 51 including at least one new cross-sectional image.

[0116] In step S122, the control unit 41 of the image processing device 11 updates the three-dimensional data 52 of the biological tissue 60 based on the tomographic data 51 acquired in step S121. Then, the processing in step S113 and subsequent steps is performed. In step S122, it is preferable to update only the data of the location corresponding to the updated tomographic data 51. In that case, the amount of data processing required to generate the three-dimensional data 52 can be reduced, and the real-time nature of the data processing in step S113 and subsequent steps can be improved.

[0117] As described above, in this embodiment, the control unit 41 of the image processing device 11 causes the display 16 to display three-dimensional data 52 representing the biological tissue 60 as a three-dimensional image 53. The control unit 41 forms, in the three-dimensional data 52, a cutting region 62 that exposes an inner cavity 63 of the biological tissue 60 in the three-dimensional image 53. The control unit 41 performs control to display a two-dimensional image 54 representing a portion of one cross section Ci of the biological tissue 60 that corresponds to the cutting region 62 at a position 65 in the three-dimensional image 53 that corresponds to the one cross section Ci.

[0118] According to this embodiment, information about a cut portion of the structure of the biological tissue 60 can be supplemented.

[0119] The present disclosure is not limited to the above-mentioned embodiment. For example, two or more blocks shown in the block diagram may be integrated, or one block may be divided. Instead of executing two or more steps shown in the flowchart in chronological order as described, they may be executed in parallel or in a different order depending on the processing capacity of the device executing each step, or as necessary. Other modifications are possible without departing from the spirit of the present disclosure. [Explanation of symbols]

[0120] 10 Image Processing System 11 Image processing device 12 Cable 13 Drive unit 14 Keyboard 15 Mouse 16 Display 17 Connection terminal 18 Cart Unit 20 Probe 21 Drive shaft 22 Hub 23 Sheath 24 Outer tube 25 Ultrasonic transducer 26 Relay connector 31 Scanner unit 32 Slide unit 33 Bottom cover 34 Probe connection 35 Scanner motor 36 Outlet 37 Probe clamp part 38 Slide motor 39 Switches 41 Control section 42 Storage section 43 Communications Department 44 Input section 45 Output section 51 Fault Data 52 3D Data 53 3D Images 54 2D Images 60 Biological Tissue 61 Inner surface 62 Cutting area 63 Lumen 65 positions 66 plane 71 Camera 72 Gaze

Claims

1. An image processing device that displays three-dimensional data representing biological tissue as a three-dimensional image on a display, An image processing device comprising a control unit that forms a cutting area in the three-dimensional data that exposes the inner cavity of the biological tissue in the three-dimensional image, and controls the display of a two-dimensional image representing a portion of one cross-section of the biological tissue that corresponds to the cutting area at a position in the three-dimensional image that corresponds to the one cross-section.

2. 2. The image processing device according to claim 1, wherein the control unit generates and updates the three-dimensional data based on cross-sectional data acquired by a sensor that acquires cross-sectional data of the biological tissue while moving through the inner cavity of the biological tissue, and controls the display, each time new cross-sectional data is acquired by the sensor, of an image representing a portion of the cross-section indicated by the newly acquired cross-sectional data by the sensor that corresponds to the cutting area, as the two-dimensional image, at a position in the three-dimensional image corresponding to the cross-section indicated by the newly acquired cross-sectional data.

3. The image processing device according to claim 1 or claim 2, wherein the control unit sets a viewpoint when the three-dimensional image is displayed by placing a virtual camera in three-dimensional space, and when the camera is not on a plane corresponding to one of the cross sections in the three-dimensional image, controls the display of the two-dimensional image on the plane.

4. The image processing device according to claim 3 , wherein the control unit performs control such that, when the camera is located on the plane, the two-dimensional image is displayed tilted with respect to the plane.

5. The image processing device according to claim 3 , wherein the control unit performs control such that, when the camera is on the plane, the two-dimensional image is displayed in a window separate from the three-dimensional image.

6. 3. The image processing device according to claim 1, wherein the control unit sets a viewpoint when the three-dimensional image is displayed by placing a virtual camera in three-dimensional space, and adjusts the angle at which the two-dimensional image is displayed relative to the plane regardless of whether the camera is on a plane corresponding to one of the cross sections in the three-dimensional image, thereby maintaining a fixed angle between the line of sight from the camera to the two-dimensional image and the two-dimensional image.

7. 3. The image processing device according to claim 1, wherein the control unit sets a viewpoint when the three-dimensional image is displayed by placing a virtual camera in three-dimensional space, and adjusts the angle at which the two-dimensional image is displayed relative to the plane, regardless of whether the camera is on a plane corresponding to one of the cross sections in the three-dimensional image, to maintain the absolute value of the angle at a fixed value.

8. 8. The image processing device according to claim 7, wherein the control unit performs control to display the two-dimensional image tilted diagonally downward with respect to the plane when the camera is above the plane, and to display the two-dimensional image tilted diagonally upward with respect to the plane when the camera is below the plane.

9. An image processing device according to any one of claims 1 to 8; a probe having a sensor for acquiring tomographic data of the biological tissue while moving through the lumen of the biological tissue; An image processing system comprising:

10. The image processing system of claim 9 further comprising the display.

11. 1. An image display method for displaying three-dimensional data representing biological tissue on a display as a three-dimensional image, comprising: A computer forms a cutting area in the three-dimensional data that exposes the lumen of the biological tissue in the three-dimensional image; An image display method in which the computer controls to display a two-dimensional image representing a portion of one cross-section of the biological tissue, which corresponds to the cut region, at a position in the three-dimensional image corresponding to the one cross-section.

12. A computer that displays three-dimensional data representing biological tissue as a three-dimensional image on a display. forming a cutting area in the three-dimensional data that exposes the lumen of the biological tissue in the three-dimensional image; a process of controlling a two-dimensional image representing a portion of one cross section of the biological tissue, the portion corresponding to the cutting region, to be displayed at a position in the three-dimensional image corresponding to the one cross section; An image processing program that executes the above.

Images