Image processing apparatus, image processing system, image display method, and image processing program

Abstract

This image processing device refers to tomographic data which is a data set obtained using a sensor which moves within a lumen of a biological tissue, and displays, on a display, a cross sectional image showing a cross section of the biological tissue perpendicular to the movement direction of the sensor. The image processing device is provided with a control unit which performs control such that specification data for specifying at least one position in a space corresponding to the tomographic data is acquired, and while the cross sectional image is displayed, a mark which varies depending on the distance between the at least one position and the cross section in the movement direction is displayed at a position corresponding to the at least one position in the cross sectional image.

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus, an image processing system, an image display method, and an image processing program.

Background Art

[0002] Patent Documents 1 to 3 describe techniques for generating three-dimensional images of the heart cavity or blood vessels using a US image system. "US" is an abbreviation for ultrasound.

[0003] Patent Document 4 describes a technique for displaying a ring or square representing the position of the tip of a catheter with electrodes on a three-dimensional image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0005] Treatment using IVUS is widely performed for the intracardiac, cardiovascular, and lower limb artery regions. "IVUS" is an abbreviation for intravascular ultrasound. IVUS is a device or method that provides a two-dimensional image of a plane perpendicular to the catheter long axis.

[0006] Currently, the operator needs to stack 2D images of IVUS in their mind to reconstruct the three-dimensional structure while performing the procedure, which is a barrier especially for young doctors or doctors with little experience. To remove such a barrier, it is conceivable to automatically generate a 3D image representing the structure of a living tissue such as a cardiac chamber or a blood vessel from the 2D image of IVUS and display the generated 3D image to the operator.

[0007] Recently, a technique of performing electrical isolation by ablating the inside of the cardiac chamber with an ablation catheter has become popular. A 3D mapping system that mounts a position sensor on the catheter and draws a 3D image using the position information when the position sensor touches the myocardial tissue is mainly used in this technique, but it is very expensive and costly. When performing circumferential isolation of the PV or SVC, an operation of marking where ablation has been performed is required, and if such an operation can be completed using IVUS, the cost may be reduced. "PV" is an abbreviation for pulmonary vein. "SVC" is an abbreviation for superior vena cava.

[0008] In a system for marking at least one location such as a burned area of a living tissue, simply displaying a mark does not mean that the usefulness of the system is sufficient for users such as operators.

[0009] An object of the present disclosure is to improve the usefulness of a system for marking at least one location related to a living tissue.

Means for Solving the Problems

[0010] An image processing apparatus according to one aspect of the present disclosure is an image processing apparatus that displays, on a display, a cross-sectional image representing a cross-section of a living tissue orthogonal to the moving direction of a sensor, with reference to tomographic data that is a data set obtained using a sensor that moves within a lumen of the living tissue, the image processing apparatus including a control unit that acquires designation data for designating at least one location within a space corresponding to the tomographic data, and performs control to display, at a position corresponding to the at least one location in the cross-sectional image when the cross-sectional image is being displayed, marks that differ depending on the distance between the at least one location in the moving direction and the cross-section.

[0011] In one embodiment, the control unit changes the color, brightness, transparency, pattern, size, shape, or orientation of the mark according to the distance.

[0012] In one embodiment, when performing control to display the mark, the control unit further performs control to display the distance.

[0013] In one embodiment, when the distance exceeds a threshold value, the control unit makes the mark non-displayed.

[0014] In one embodiment, the control unit changes the mark depending on whether the at least one location exists in front of or behind the cross-section in the moving direction.

[0015] In one embodiment, the at least one location is a cauterized location of the living tissue, and when the cross-sectional image is being displayed, the control unit further performs control to display the distance between a catheter for cauterizing the living tissue and the at least one location.

[0016] In one embodiment, when the at least one location is only one location, the control unit displays, on the cross-sectional image, the distance between the catheter inserted into the living tissue and the one location, and when the at least one location is a plurality of locations, the control unit further performs control to display, on the cross-sectional image, the distance between the catheter and the location closest to the catheter among the plurality of locations.

[0017] In one embodiment, when only one of the at least one location is present, the control unit performs control to display, on the cross-sectional image, a line connecting the catheter inserted into the biological tissue and the one location; when a plurality of the at least one location are present, the control unit further performs control to display, on the cross-sectional image, a line connecting the catheter and the location closest to the catheter among the plurality of locations.

[0018] In one embodiment, the control unit refers to the tomographic data, generates three-dimensional data representing the biological tissue, causes the generated three-dimensional data to be displayed as a three-dimensional image on the display, when only one of the at least one location is present, performs control to display, on the three-dimensional image, the distance between the catheter inserted into the biological tissue and the one location; when a plurality of the at least one location are present, the control unit further performs control to display, on the three-dimensional image, the distance between the catheter and the location closest to the catheter among the plurality of locations.

[0019] In one embodiment, the control unit refers to the tomographic data, generates three-dimensional data representing the biological tissue, causes the generated three-dimensional data to be displayed as a three-dimensional image on the display, when only one of the at least one location is present, performs control to display, on the three-dimensional image, a line connecting the catheter inserted into the biological tissue and the one location; when a plurality of the at least one location are present, the control unit further performs control to display, on the three-dimensional image, a line connecting the catheter and the location closest to the catheter among the plurality of locations.

[0020] In one embodiment, the control unit receives a user operation for designating the at least one location on the cross-sectional image.

[0021] In one embodiment, each time a new data set is obtained using the sensor, the control unit causes a new image representing a cross-section corresponding to the position of the sensor to be displayed as the cross-sectional image on the display.

[0022] An image processing system according to one aspect of the present disclosure includes the image processing apparatus and the sensor.

[0023] As one embodiment, the image processing system further includes the display.

[0024] An image display method according to one aspect of the present disclosure is an image display method for displaying, on a display, a cross-sectional image representing a cross-section of a biological tissue orthogonal to a moving direction of a sensor, with reference to tomographic data that is a data set obtained using a sensor that moves inside a lumen of the biological tissue, the method including: a computer acquiring designation data for designating at least one location in a space corresponding to the tomographic data; and the computer performing control to display, at a position corresponding to the at least one location in the cross-sectional image when the cross-sectional image is being displayed, marks that differ depending on a distance between the at least one location in the moving direction and the cross-section.

[0025] An image processing program according to one aspect of the present disclosure causes a computer that displays, on a display, a cross-sectional image representing a cross-section of a biological tissue orthogonal to a moving direction of a sensor, with reference to tomographic data that is a data set obtained using a sensor that moves inside a lumen of the biological tissue, to execute a process of acquiring designation data for designating at least one location in a space corresponding to the tomographic data, and a process of performing control to display, at a position corresponding to the at least one location in the cross-sectional image when the cross-sectional image is being displayed, marks that differ depending on a distance between the at least one location in the moving direction and the cross-section.

Advantages of the Invention

[0026] According to the present disclosure, the usefulness of a system for marking at least one location related to a biological tissue is improved.

Brief Description of the Drawings

[0027]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Mode for Carrying Out the Invention

[0028] Hereinafter, some embodiments of the present disclosure will be described with reference to the drawings.

[0029] In each figure, the same or corresponding parts are denoted by the same reference numerals. In the description of each embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0030] An embodiment of the present disclosure will be described.

[0031] Referring to FIGS. 1 to 4, the outline of this embodiment will be described.

[0032] The image processing apparatus 11 according to this embodiment is a computer that causes a display 16 to display a cross-sectional image 54 representing a cross-section 64 of a biological tissue 60 orthogonal to the moving direction of a sensor, with reference to tomographic data 51 that is a data set obtained using a sensor that moves within a lumen 61 of the biological tissue 60.

[0033] The image processing apparatus 11 acquires designation data for designating at least one location within the space corresponding to the tomographic data 51 as a point Pd. In the example of FIG. 2, six locations on the inner wall surface 65 of the biological tissue 60 that have been cauterized with the catheter 63 are designated as points P1, P2, P3, P4, P5, and P6.

[0034] When the cross-sectional image 54 is being displayed, the image processing apparatus 11 performs control to display marks 55 that vary depending on the distance between the point Pd and the cross-section 64 in the moving direction of the sensor, at positions corresponding to the point Pd in the cross-sectional image 54. The position corresponding to the point Pd in the cross-sectional image 54 is the position where the point Pd is shifted to the same position as the cross-section 64 in the moving direction of the sensor. In the example of FIG. 2, marks M1, M2, M3, M4, M5, and M6 are respectively displayed at positions corresponding to the points P1, P2, P3, P4, P5, and P6 in the cross-sectional image 54. Marks M5 and M6 are displayed in the darkest color because the points P5 and P6 exist in the cross-section 64. Mark M4 is displayed in a lighter color than marks M5 and M6 because the point P4 is separated from the cross-section 64 by a distance Db in the moving direction of the sensor. Marks M2 and M3 are displayed in the lightest color because the points P2 and P3 are separated from the cross-section 64 by a distance Dc in the moving direction of the sensor, and the distance Dc is longer than the distance Db. Mark M1 is displayed in the same color as mark M4 because the point P1 is separated from the cross-section 64 by a distance Da in the moving direction of the sensor, and the distance Da is equal to the distance Db.

[0035] According to the present embodiment, in a system for marking at least one location in the space corresponding to the tomographic data 51, the relative position of the location in the moving direction of the sensor can be intuitively understood by the user. Therefore, the usability of the system is improved.

[0036] In the present embodiment, the image processing apparatus 11 generates and updates three-dimensional data 52 representing the biological tissue 60 with reference to the tomographic data 51, which is a data set obtained using a sensor. The image processing apparatus 11 causes the three-dimensional data 52 to be displayed on the display 16 as a three-dimensional image 53 together with the cross-sectional image 54. That is, the image processing apparatus 11 causes the three-dimensional image 53 and the cross-sectional image 54 to be displayed on the display 16 with reference to the tomographic data 51.

[0037] The image processing apparatus 11 forms an opening 62 in the three-dimensional data 52 to expose the lumen 61 of the biological tissue 60 in the three-dimensional image 53. In the example of FIG. 2, the opening 62 is formed so that all of the points P1, P2, P3, P4, P5, and P6 can be seen. Then, according to the position of the opening 62, the viewpoint when the three-dimensional image 53 is displayed on the screen is adjusted. The viewpoint is the position of a virtual camera arranged in the three-dimensional space.

[0038] According to the present embodiment, in the three-dimensional image 53, by cutting out a part of the structure of the biological tissue 60, it is possible to look into the lumen 61 of the biological tissue 60.

[0039] The biological tissue 60 includes, for example, blood vessels or organs such as the heart. The biological tissue 60 is not limited to anatomically single organs or parts thereof, but also includes tissues having lumens across a plurality of organs. As an example of such a tissue, specifically, a part of the vascular tissue that passes from the upper part of the inferior vena cava through the right atrium and reaches the lower part of the superior vena cava can be mentioned. In the examples of FIGS. 2 and 3, the biological tissue 60 is a blood vessel.

[0040] In FIG. 2, the Z direction corresponds to the moving direction of the sensor. However, as shown in FIG. 3, for convenience, the Z direction may be regarded as corresponding to the longitudinal direction of the lumen 61 of the biological tissue 60. The X direction orthogonal to the Z direction, and the Y direction orthogonal to the Z direction and the X direction may each be regarded as corresponding to the short-side direction of the lumen 61 of the biological tissue 60.

[0041] In the example of FIG. 3, the image processing apparatus 11 calculates the positions of the centroids B1, B2, B3, B4 of the respective cross-sections C1, C2, C3, C4 of the biological tissue 60 using the three-dimensional data 52. The image processing apparatus 11 sets a pair of planes intersecting with one line Lb passing through the positions of the centroids B1, B2, B3, B4 as the cutting planes D1, D2. The image processing apparatus 11 forms in the three-dimensional data 52 a cutting region 66 that is sandwiched between the cutting planes D1, D2 in the three-dimensional image 53 and exposes the lumen 61 of the biological tissue 60. In the three-dimensional image 53, an opening 62 as shown in FIG. 2 is formed by setting the cutting region 66 as non-display or transparent.

[0042] In the case of a three-dimensional model of a blood vessel bent as shown in FIG. 3, if the three-dimensional model is cut with one plane to display the lumen 61, there are cases where the inside of the blood vessel cannot be correctly displayed. In the present embodiment, as shown in FIG. 3, by continuously capturing the centroid of the blood vessel, it is possible to cut the three-dimensional model so that the inside of the blood vessel can be surely displayed.

[0043] In FIG. 3, for convenience, four cross-sections C1, C2, C3, C4 are shown as a plurality of cross-sections of the biological tissue 60 orthogonal to the Z direction. However, the number of cross-sections for which the centroid position is calculated is not limited to four, and preferably is the same as the number of cross-sectional images acquired by IVUS.

[0044] Referring to FIG. 1, the configuration of the image processing system 10 according to the present embodiment will be described.

[0045] The image processing system 10 includes an image processing apparatus 11, a cable 12, a drive unit 13, a keyboard 14, a mouse 15, and a display 16.

[0046] The image processing apparatus 11 is a dedicated computer specialized for image diagnosis in this embodiment, but it may also be a general-purpose computer such as a PC. "PC" is an abbreviation for personal computer.

[0047] The cable 12 is used to connect the image processing apparatus 11 and the drive unit 13.

[0048] The drive unit 13 is connected to and used with the probe 20 shown in FIG. 5, and is a device for driving the probe 20. The drive unit 13 is also called an MDU. "MDU" is an abbreviation for motor drive unit. The probe 20 is applied to IVUS. The probe 20 is also called an IVUS catheter or an image diagnosis catheter.

[0049] The keyboard 14, the mouse 15, and the display 16 are connected to the image processing apparatus 11 via an arbitrary cable or wirelessly. The display 16 is, for example, an LCD, an organic EL display, or an HMD. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electro luminescence. "HMD" is an abbreviation for head-mounted display.

[0050] The image processing system 10 further includes a connection terminal 17 and a cart unit 18 as options.

[0051] The connection terminal 17 is used to connect the image processing apparatus 11 and an external device. The connection terminal 17 is, for example, a USB terminal. "USB" is an abbreviation for Universal Serial Bus. The external device is, for example, a recording medium such as a magnetic disk drive, a magneto-optical disk drive, or an optical disk drive.

[0052] The cart unit 18 is a cart with casters for movement. An image processing device 11, a cable 12, and a drive unit 13 are installed on the cart body of the cart unit 18. A keyboard 14, a mouse 15, and a display 16 are installed on the top table of the cart unit 18.

[0053] Referring to FIG. 5, the configuration of the probe 20 and the drive unit 13 according to this embodiment will be described.

[0054] The probe 20 includes a drive shaft 21, a hub 22, a sheath 23, an outer tube 24, an ultrasonic vibrator 25, and a relay connector 26.

[0055] The drive shaft 21 passes through a sheath 23 inserted into the body cavity of a living body and an outer tube 24 connected to the proximal end of the sheath 23, and extends to the inside of a hub 22 provided at the proximal end of the probe 20. The drive shaft 21 has an ultrasonic vibrator 25 for transmitting and receiving signals at its tip and is rotatably provided in the sheath 23 and the outer tube 24. The relay connector 26 connects the sheath 23 and the outer tube 24.

[0056] The hub 22, the drive shaft 21, and the ultrasonic vibrator 25 are connected to each other so that they can move forward and backward axially integrally. Therefore, for example, when an operation is performed to push the hub 22 toward the tip side, the drive shaft 21 and the ultrasonic vibrator 25 move toward the tip side inside the sheath 23. For example, when an operation is performed to pull the hub 22 toward the proximal end side, the drive shaft 21 and the ultrasonic vibrator 25 move toward the proximal end side inside the sheath 23 as indicated by the arrow.

[0057] The drive unit 13 includes a scanner unit 31, a slide unit 32, and a bottom cover 33.

[0058] The scanner unit 31 is connected to the image processing device 11 via the cable 12. The scanner unit 31 includes a probe connection portion 34 that connects to the probe 20 and a scanner motor 35 that is a drive source for rotating the drive shaft 21.

[0059] The probe connection part 34 is detachably connected to the probe 20 through the insertion port 36 of the hub 22 provided at the proximal end of the probe 20. Inside the hub 22, the proximal end of the drive shaft 21 is rotatably supported, and the rotational force of the scanner motor 35 is transmitted to the drive shaft 21. Also, signals are transmitted and received between the drive shaft 21 and the image processing device 11 via the cable 12. In the image processing device 11, based on the signals transmitted from the drive shaft 21, tomography images of the living body lumen are generated and image processing is performed.

[0060] The slide unit 32 has the scanner unit 31 mounted thereon so as to be movable forward and backward, and is mechanically and electrically connected to the scanner unit 31. The slide unit 32 includes a probe clamp part 37, a slide motor 38, and a switch group 39.

[0061] The probe clamp part 37 is arranged coaxially with and on the tip side of the probe connection part 34, and supports the probe 20 connected to the probe connection part 34.

[0062] The slide motor 38 is a drive source that generates a driving force in the axial direction. When the slide motor 38 is driven, the scanner unit 31 moves forward and backward, and accordingly, the drive shaft 21 moves forward and backward in the axial direction. The slide motor 38 is, for example, a servo motor.

[0063] The switch group 39 includes, for example, a forward switch and a pull-back switch that are pressed when the scanner unit 31 is moved forward and backward, and a scan switch that is pressed when image drawing starts and ends. Without being limited to this example, various switches are included in the switch group 39 as necessary.

[0064] When the forward switch is pressed, the slide motor 38 rotates forward and the scanner unit 31 moves forward. On the other hand, when the pull-back switch is pressed, the slide motor 38 rotates backward and the scanner unit 31 moves backward.

[0065] When the scan switch is pressed, image drawing starts, the scanner motor 35 is driven, and the slide motor 38 is driven to retract the scanner unit 31. A user such as an operator connects the probe 20 to the scanner unit 31 in advance so that the drive shaft 21 rotates and moves axially toward the proximal end side at the start of image drawing. The scanner motor 35 and the slide motor 38 stop when the scan switch is pressed again, and image drawing ends.

[0066] The bottom cover 33 covers the bottom surface and the entire circumferential side surface on the bottom surface side of the slide unit 32 and is movable closer to and away from the bottom surface of the slide unit 32.

[0067] Referring to FIG. 4, the configuration of the image processing apparatus 11 will be described.

[0068] The image processing apparatus 11 includes a control unit 41, a storage unit 42, a communication unit 43, an input unit 44, and an output unit 45.

[0069] The control unit 41 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or a GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. The programmable circuit is, for example, an FPGA. "FPGA" is an abbreviation for field-programmable gate array. The dedicated circuit is, for example, an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 41 executes processes related to the operation of the image processing apparatus 11 while controlling each part of the image processing system 10 including the image processing apparatus 11.

[0070] The storage unit 42 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM or a ROM. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. The RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. The ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. The storage unit 42 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 42 stores data used in the operation of the image processing apparatus 11, such as tomographic data 51, and data obtained by the operation of the image processing apparatus 11, such as three-dimensional data 52 and three-dimensional image 53.

[0071] The communication unit 43 includes at least one communication interface. The communication interface is, for example, a wired LAN interface, a wireless LAN interface, or an image diagnosis interface that receives and A / D-converts an IVUS signal. "LAN" is an abbreviation for local area network. "A / D" is an abbreviation for analog to digital. The communication unit 43 receives data used in the operation of the image processing apparatus 11 and transmits data obtained by the operation of the image processing apparatus 11. In the present embodiment, the drive unit 13 is connected to the image diagnosis interface included in the communication unit 43.

[0072] The input unit 44 includes at least one input interface. The input interface is an interface corresponding to a short-range wireless communication standard such as, for example, a USB interface, an HDMI (registered trademark) interface, or Bluetooth (registered trademark). "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface. The input unit 44 receives user operations such as an operation of inputting data used for the operation of the image processing apparatus 11. In the present embodiment, the keyboard 14 and the mouse 15 are connected to the USB interface included in the input unit 44 or an interface corresponding to short-range wireless communication. When the touch screen is provided integrally with the display 16, the display 16 may be connected to the USB interface or the HDMI (registered trademark) interface included in the input unit 44.

[0073] The output unit 45 includes at least one output interface. The output interface is an interface corresponding to a short-range wireless communication standard such as, for example, a USB interface, an HDMI (registered trademark) interface, or Bluetooth (registered trademark). The output unit 45 outputs data obtained by the operation of the image processing apparatus 11. In the present embodiment, the display 16 is connected to the USB interface or the HDMI (registered trademark) interface included in the output unit 45.

[0074] The functions of the image processing apparatus 11 are realized by executing the image processing program according to the present embodiment by a processor as the control unit 41. That is, the functions of the image processing apparatus 11 are realized by software. The image processing program causes a computer to execute the operation of the image processing apparatus 11, thereby causing the computer to function as the image processing apparatus 11. That is, the computer functions as the image processing apparatus 11 by executing the operation of the image processing apparatus 11 according to the image processing program.

[0075] The program can be stored in a non-transitory computer-readable medium. The non-transitory computer-readable medium is, for example, a flash memory, a magnetic recording device, an optical disc, a magneto-optical recording medium, or a ROM. The distribution of the program is carried out, for example, by selling, transferring, or lending a portable medium such as an SD card storing the program, a DVD, or a CD-ROM. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program may be stored in the server's storage and distributed by transferring the program from the server to other computers. The program may be provided as a program product.

[0076] The computer stores, for example, a program stored in a portable medium or a program transferred from a server, once, in the main memory device. Then, the computer reads the program stored in the main memory device with the processor and executes the processing according to the read program with the processor. The computer may directly read the program from the portable medium and execute the processing according to the program. The computer may sequentially execute the processing according to the received program each time the program is transferred from the server to the computer. The processing may be executed by a so-called ASP type service that realizes the function only by execution instructions and result acquisition without transferring the program from the server to the computer. "ASP" is an abbreviation for application service provider. The program includes information for use in processing by an electronic computer that conforms to the program. For example, data that is not a direct instruction to the computer but has the property of defining the computer's processing corresponds to "what conforms to the program".

[0077] Some or all of the functions of the image processing apparatus 11 may be realized by a programmable circuit or a dedicated circuit as the control unit 41. That is, some or all of the functions of the image processing apparatus 11 may be realized by hardware.

[0078] Referring to FIG. 6, the operation of the image processing system 10 according to the present embodiment will be described. The operation of the image processing system 10 corresponds to the image display method according to the present embodiment.

[0079] Before the start of the flow of FIG. 6, the probe 20 is primed by the user. Thereafter, the probe 20 is fitted into the probe connection portion 34 and the probe clamp portion 37 of the drive unit 13 and connected and fixed to the drive unit 13. Then, the probe 20 is inserted to a target site in a living tissue 60 such as a blood vessel or the heart.

[0080] In step S101, by pressing a scan switch included in the switch group 39 and further pressing a pull-back switch included in the switch group 39, a so-called pull-back operation is performed. The probe 20 transmits ultrasonic waves by an ultrasonic vibrator 25 that moves backward in the axial direction by the pull-back operation inside the living tissue 60. The ultrasonic vibrator 25 transmits ultrasonic waves radially while moving inside the living tissue 60. The ultrasonic vibrator 25 receives the reflected waves of the transmitted ultrasonic waves. The probe 20 inputs the signal of the reflected waves received by the ultrasonic vibrator 25 to the image processing apparatus 11. The control unit 41 of the image processing apparatus 11 processes the input signal and sequentially generates cross-sectional images of the living tissue 60, thereby acquiring tomographic data 51 including a plurality of cross-sectional images.

[0081] Specifically, while rotating the ultrasonic vibrator 25 circumferentially and moving it axially inside the biological tissue 60, the probe 20 transmits ultrasonic waves in a plurality of directions from the center of rotation toward the outside by the ultrasonic vibrator 25. The probe 20 receives reflected waves from reflectors existing in each of the plurality of directions inside the biological tissue 60 by the ultrasonic vibrator 25. The probe 20 transmits the received reflected wave signals to the image processing apparatus 11 via the drive unit 13 and the cable 12. The communication unit 43 of the image processing apparatus 11 receives the signals transmitted from the probe 20. The communication unit 43 performs A / D conversion on the received signals. The communication unit 43 inputs the A / D converted signals to the control unit 41. The control unit 41 processes the input signals and calculates the intensity value distribution of the reflected waves from the reflectors existing in the transmission direction of the ultrasonic waves of the ultrasonic vibrator 25. The control unit 41 sequentially generates a two-dimensional image having a luminance value distribution corresponding to the calculated intensity value distribution as a cross-sectional image of the biological tissue 60, thereby obtaining tomographic data 51 which is a data set of the cross-sectional image. The control unit 41 stores the obtained tomographic data 51 in the storage unit 42.

[0082] In the present embodiment, the signal of the reflected wave received by the ultrasonic vibrator 25 corresponds to the raw data of the tomographic data 51, and the cross-sectional image generated by the image processing apparatus 11 processing the reflected wave signal corresponds to the processed data of the tomographic data 51.

[0083] As a modification of the present embodiment, the control unit 41 of the image processing apparatus 11 may store the signals input from the probe 20 in the storage unit 42 as the tomographic data 51 as they are. Alternatively, the control unit 41 may store data indicating the intensity value distribution of the reflected waves calculated by processing the signals input from the probe 20 in the storage unit 42 as the tomographic data 51. That is, the tomographic data 51 is not limited to the data set of the cross-sectional image of the biological tissue 60, and may be data representing the cross-section of the biological tissue 60 at each moving position of the ultrasonic vibrator 25 in any form.

[0084] As a modification example of the present embodiment, instead of the ultrasonic vibrator 25 that transmits ultrasonic waves in a plurality of directions while rotating in the circumferential direction, an ultrasonic vibrator that transmits ultrasonic waves in a plurality of directions without rotating may be used.

[0085] As a modification example of the present embodiment, instead of the tomographic data 51 being acquired using IVUS, it may be acquired using OFDI or OCT. "OFDI" is an abbreviation for optical frequency domain imaging. "OCT" is an abbreviation for optical coherence tomography. When OFDI or OCT is used, as a sensor for acquiring tomographic data 51 while moving through the lumen 61 of the biological tissue 60, instead of the ultrasonic vibrator 25 that transmits ultrasonic waves in the lumen 61 of the biological tissue 60 to acquire tomographic data 51, a sensor that emits light in the lumen 61 of the biological tissue 60 to acquire tomographic data 51 is used.

[0086] As a modification example of the present embodiment, instead of the image processing device 11 generating a data set of cross-sectional images of the biological tissue 60, another device may generate a similar data set, and the image processing device 11 may acquire the data set from the other device. That is, instead of the control unit 41 of the image processing device 11 processing the IVUS signal to generate a cross-sectional image of the biological tissue 60, another device may process the IVUS signal to generate a cross-sectional image of the biological tissue 60 and input the generated cross-sectional image to the image processing device 11.

[0087] In step S102, the control unit 41 of the image processing apparatus 11 generates three-dimensional data 52 of the biological tissue 60 based on the tomographic data 51 acquired in step S101. That is, the control unit 41 generates the three-dimensional data 52 based on the tomographic data 51 acquired by the sensor. Here, when there is already generated three-dimensional data 52, rather than generating all the three-dimensional data 52 from scratch, it is preferable to update only the data at the location corresponding to the updated tomographic data 51. In that case, the data processing amount when generating the three-dimensional data 52 can be reduced, and the real-time performance of the three-dimensional image 53 in the subsequent step S103 can be improved.

[0088] Specifically, the control unit 41 of the image processing apparatus 11 generates the three-dimensional data 52 of the biological tissue 60 by stacking and three-dimensionally processing the cross-sectional images of the biological tissue 60 included in the tomographic data 51 stored in the storage unit 42. As the three-dimensional processing method, any method among rendering methods such as surface rendering or volume rendering, and various processes such as texture mapping including environment mapping and bump mapping associated therewith can be used. The control unit 41 stores the generated three-dimensional data 52 in the storage unit 42.

[0089] When a catheter 63 different from the IVUS catheter, such as an ablation catheter, is inserted into the biological tissue 60, the tomographic data 51 includes the data of the catheter 63 in the same way as the data of the biological tissue 60. Therefore, in step S102, the three-dimensional data 52 generated by the control unit 41 also includes the data of the catheter 63 in the same way as the data of the biological tissue 60.

[0090] The control unit 41 of the image processing apparatus 11 classifies the pixel groups of the cross-sectional images included in the tomographic data 51 acquired in step S101 into two or more classes. These two or more classes include at least the "tissue" class to which the biological tissue 60 belongs and the "catheter" class to which the catheter 63 belongs, and may further include the "blood cell" class, medical instruments other than "catheters" such as guide wires, the "implant" class such as an indwelling stent, or the "lesion" class such as lime or plaque. As a classification method, any method may be used, but in this embodiment, a method of classifying the pixel groups of the cross-sectional images by a learned model is used. The learned model is trained in advance by performing machine learning so as to be able to detect regions corresponding to each class from the cross-sectional images of the IVUS serving as samples.

[0091] In step S103, the control unit 41 of the image processing apparatus 11 causes the display 16 to display the 3D data 52 generated in step S102 as a 3D image 53. At this point, the control unit 41 may set the angle at which the 3D image 53 is displayed to an arbitrary angle. The control unit 41 causes the display 16 to display the latest cross-sectional image 54 included in the tomographic data 51 acquired in step S101 together with the 3D image 53.

[0092] Specifically, the control unit 41 of the image processing apparatus 11 generates a 3D image 53 from the 3D data 52 stored in the storage unit 42. The control unit 41 causes the display 16 to display the latest cross-sectional image 54 and the generated 3D image 53 among the cross-sectional images of the biological tissue 60 included in the tomographic data 51 stored in the storage unit 42 via the output unit 45.

[0093] With reference to FIG. 7, the procedure of the process further executed in step S103 will be described.

[0094] When at least one location within the space corresponding to the tomographic data 51 has been designated as the point Pd, the processes of step S301 and step S302 are executed. When there is no designated location, the processes of step S301 and step S302 are skipped.

[0095] In step S301, the control unit 41 of the image processing apparatus 11 calculates the distance from the point Pd to the cross-section 64 of the biological tissue 60 represented by the cross-sectional image 54 in the moving direction of the sensor.

[0096] In step S302, the control unit 41 of the image processing apparatus 11 performs control to display different marks 55 at the position corresponding to the point Pd in the cross-sectional image 54 according to the distance calculated in step S301. In this embodiment, the control unit 41 changes the color of the mark 55 according to the calculated distance, but may also change the brightness, transparency, pattern, size, shape, orientation, or any combination thereof, either together with the color or instead of the color. For example, if the point Pd is close to the cross-section 64, the mark 55 may be enlarged, and if it is far away, the mark 55 may be reduced. Alternatively, if the point Pd is present on the cross-section 64, the mark 55 may be made rectangular, and if it is present on another cross-section, the mark 55 may be made circular or some other shape other than rectangular. Alternatively, if the point Pd is present on the cross-section 64, the mark 55 may be surrounded by a white frame or blinked. According to these examples, it is possible to clarify how far the past ablation position is and in which angular direction ablation has already been performed on one screen.

[0097] In the example of FIG. 2, assuming that points P1, P2, P3, P4, and P5 are specified, the control unit 41 of the image processing apparatus 11 calculates a distance Da for point P1, distances Dc for points P2 and P3, a distance Db for point P4, and a distance 0 for point P5. If the distance Da is equal to the distance Db and the distance Dc is longer than the distance Db, the control unit 41 places a darkest-colored mark M5 at the position corresponding to point P5 in the cross-sectional image 54, lighter-colored marks M1 and M4 than mark M5 at the positions corresponding to points P1 and P4 in the cross-sectional image 54, and the lightest-colored marks M2 and M3 at the positions corresponding to points P2 and P3 in the cross-sectional image 54 for display control.

[0098] As a modification of this embodiment, when performing the control to display the mark 55, the control unit 41 of the image processing apparatus 11 may further perform control to display the distance between the point Pd and the cross-section 64 in the moving direction of the sensor. The unit of the displayed distance may be, for example, millimeters. In the example of FIG. 2, the control unit 41 may perform control to display the distance Da next to the mark M1, the distance Dc next to the marks M2 and M3, and the distance Db next to the mark M4.

[0099] As a modification of this embodiment, when the distance between the point Pd and the cross-section 64 in the moving direction of the sensor exceeds a threshold value, the control unit 41 of the image processing apparatus 11 may turn off the display of the mark 55. In the example of FIG. 2, if the threshold value is smaller than the distance Dc, the control unit 41 may turn off the display of the marks M2 and M3.

[0100] As a modification example of the present embodiment, the control unit 41 of the image processing apparatus 11 may change the mark 55 depending on whether the point Pd exists in front of or behind the cross section 64 in the moving direction of the sensor. In the example of FIG. 2, the point P1 exists in front of the cross section 64 in the moving direction of the sensor, that is, above the cross section 64. The points P2, P3, and P4 exist behind the cross section 64 in the moving direction of the sensor, that is, below the cross section 64. Therefore, the control unit 41 may set the color, brightness, transparency, pattern, size, shape, orientation, or any combination thereof of the mark M1 to be different from those of the marks M2, M3, and M4. For example, the mark M1 may be an upwardly convex triangle, and the marks M2, M3, and M4 may be downwardly convex triangles. When setting the color of the mark M1 to be different from those of the marks M2, M3, and M4, the color may be set so as to ensure a difference according to the distance. For example, if the color of the mark M1 is set to red and the colors of the marks M2, M3, and M4 are set to blue, the darkness of the red color of the mark M1 may be set to be about the same as the darkness of the blue color of the mark M4, and the darkness of the blue colors of the marks M2 and M3 may be set to be lighter than the blue color of the mark M4. According to these examples, it is possible to clarify in one screen how far the past ablation positions are separated in the up and down directions.

[0101] As a modification example of the present embodiment, the control unit 41 of the image processing apparatus 11 may further perform control to display the distance between the catheter 63 and the point Pd. The unit of the displayed distance is, for example, millimeter. The displayed distance may be a distance on a plane, but is a distance in a three-dimensional space, that is, an actual distance. In the example of FIG. 2, the control unit 41 may perform control to display the distance from the catheter 63 to the point P5 next to the point P5 having the shortest distance from the catheter 63. The control unit 41 may further perform control to display a mark different from the mark 55 at a position corresponding to the tip of the catheter 63 in the cross-sectional image 54.

[0102] In step S303, the control unit 41 of the image processing apparatus 11 determines whether the catheter 63 is in contact with the inner wall surface 65 of the biological tissue 60. Specifically, the control unit 41 analyzes the cross-sectional image 54 to detect the biological tissue 60 and the catheter 63 within the cross-sectional image 54. Then, the control unit 41 measures the distance between the biological tissue 60 and the tip of the catheter 63 to determine whether the biological tissue 60 and the tip of the catheter 63 are in contact. Alternatively, the control unit 41 analyzes the three-dimensional data 52 to detect the tip of the catheter 63 included in the three-dimensional data 52. Then, the control unit 41 measures the distance between the biological tissue 60 and the tip of the catheter 63 to determine whether the biological tissue 60 and the tip of the catheter 63 are in contact. The control unit 41 may receive an input of position data indicating the position where the tip of the catheter 63 is in contact from an external system that determines whether the tip of the catheter 63 is in contact with the inner wall surface 65 of the biological tissue 60 using an electrode provided at the tip of the catheter 63, via the communication unit 43 or the input unit 44. Then, the control unit 41 may correct the analysis result of the cross-sectional image 54 or the three-dimensional data 52 with reference to the input position data.

[0103] The process of step S303 may be executed using AI. "AI" is an abbreviation for artificial intelligence. As a variant of this embodiment, instead of executing the process of step S303, a human may determine whether the catheter 63 is in contact with the inner wall surface 65 of the biological tissue 60.

[0104] If it is determined that the catheter 63 is in contact with the inner wall surface 65 of the biological tissue 60, the processes of step S304 and step S305 are executed. If it is determined that the catheter 63 is not in contact with the inner wall surface 65 of the biological tissue 60, the processes of step S304 and step S305 are skipped.

[0105] In step S304, the control unit 41 of the image processing apparatus 11 acquires designation data for designating, as a point Pd, the location on the inner wall surface 65 of the biological tissue 60 where the catheter 63 is in contact. If at least one location within the space corresponding to the tomographic data 51 has already been designated as the point Pd before this time, then the location designated as the point Pd will have one additional location added. In this embodiment, the control unit 41 acquires, as the designation data, the data for designating the point Pd by receiving a user operation for designating at least one location as the point Pd on the cross-sectional image 54. However, the data for designating the point Pd may be acquired as the designation data by automatically detecting, as the point Pd, the position where the tip of the catheter 63 is in contact in step S303.

[0106] In step S305, the control unit 41 of the image processing apparatus 11 performs control to display a new mark at the position corresponding to the location designated by the acquired designation data in step S304 in the cross-sectional image 54.

[0107] In the example of FIG. 2, assuming that the point P6 has been cauterized, the control unit 41 of the image processing apparatus 11 acquires, as the designation data, the data for designating the point P6. The control unit 41 performs control to display a mark M6 of the same color as the mark M5 at the position corresponding to the point P6 in the cross-sectional image 54.

[0108] In this embodiment, each time a new data set is obtained using the sensor, the control unit 41 of the image processing apparatus 11 causes a new image representing the cross-section 64 corresponding to the position of the sensor to be displayed on the display 16 as the cross-sectional image 54. Therefore, when the sensor is moved by the pull-back operation, the distance from the point Pd in the moving direction of the sensor to the cross-section 64 changes, and the mark 55 also changes accordingly with this change in distance. The user can obtain a sense that the sensor is approaching the point Pd or moving away from the point Pd by observing the change in this mark 55 during the pull-back operation.

[0109] As a modification example of the present embodiment, portions of the inner wall surface 65 of the biological tissue 60 other than the portions cauterized by the catheter 63 may be marked. That is, the point Pd is not limited to the ablation point. For example, assuming that the biological tissue 60 is a blood vessel, as shown in FIG. 8, the base of the branch 72 of the blood vessel may be designated as the point Pd. Alternatively, as shown in FIG. 9, the base of the aneurysm 74 formed in the blood vessel may be designated as the point Pd. Alternatively, as shown in FIG. 10, the point where the nerve 75 intersects the blood vessel may be designated as the point Pd. Alternatively, as shown in FIG. 11, one location of the tumor 76 formed around the blood vessel may be designated as the point Pd.

[0110] In FIG. 8, the upper side shows images of each cross section of the blood vessel actually displayed on the display 16 as the cross-sectional image 54, and the lower side is a schematic diagram of the longitudinal section of the blood vessel. In this schematic diagram, each dotted line indicates the position of each cross section as the cross section 64. The same applies to FIG. 9 as to FIG. 8. FIG. 10 is a schematic diagram of the longitudinal section of the blood vessel. The same applies to FIG. 11 as to FIG. 10.

[0111] In the examples of FIGS. 8 to 11, it is assumed that the size of the mark 55 changes depending on the distance from the point Pd to the cross section 64 in the moving direction of the sensor. In the example of FIG. 8, if the user wants to place the stent 71 at a certain distance so as not to cover the branch 72, the user can easily identify the position where the stent 71 should be placed by checking the change in the size of the mark 55 while performing the pull-back operation. In the example of FIG. 9, if the user wants to place the stent graft 73 across a certain distance so as to cover the aneurysm 74, the user can easily identify the position where the stent graft 73 should be placed by checking the change in the size of the mark 55 while performing the pull-back operation. As a modification, if the user wants to place the stent graft 73 with the hole of the stent graft 73 aligned with the blood vessel branch, the user can easily identify the position where the stent graft 73 should be placed by checking the change in the size of the mark 55 while performing the pull-back operation. According to this example, the distance and direction from the branch of the position where the stent graft 73 is placed can also be easily confirmed. In the example of FIG. 10, if the user wants to perform ablation while avoiding the nerve 75 intersecting the blood vessel or wants to perform ablation around the nerve 75, the user can easily identify the position where ablation should be performed by checking the change in the size of the mark 55 while performing the pull-back operation. The nerve 75 may be another blood vessel intersecting the blood vessel. In the example of FIG. 11, if the user wants to inject a drug at a certain distance from the tumor 76 around the blood vessel, the user can easily identify the position where the drug should be injected by checking the change in the size of the mark 55 while performing the pull-back operation. In any of the examples, the distance from the point Pd to the cross section 64 in the moving direction of the sensor may be displayed numerically. Alternatively, when the distance from the point Pd to the cross section 64 in the moving direction of the sensor reaches the target distance, the display method such as the color of the mark 55 may be changed, or the mark 55 may be made non-displayed.

[0112] In step S104, if there is an operation to set the angle at which the three-dimensional image 53 is to be displayed as a user change operation, the process of step S105 is executed. If there is no user change operation, the process of step S106 is executed.

[0113] In step S105, the control unit 41 of the image processing apparatus 11 receives, via the input unit 44, an operation to set the angle at which the three-dimensional image 53 is to be displayed. The control unit 41 adjusts the angle at which the three-dimensional image 53 is to be displayed to the set angle. Then, in step S103, the control unit 41 causes the three-dimensional image 53 to be displayed on the display 16 at the angle set in step S105.

[0114] Specifically, the control unit 41 of the image processing apparatus 11 receives, via the input unit 44, an operation in which the user rotates the three-dimensional image 53 displayed on the display 16 using the keyboard 14, the mouse 15, or the touch screen provided integrally with the display 16. The control unit 41 interactively adjusts the angle at which the three-dimensional image 53 is to be displayed on the display 16 according to the user's operation. Alternatively, the control unit 41 receives, via the input unit 44, an operation in which the user inputs a numerical value of the angle at which the three-dimensional image 53 is to be displayed using the keyboard 14, the mouse 15, or the touch screen provided integrally with the display 16. The control unit 41 adjusts the angle at which the three-dimensional image 53 is to be displayed on the display 16 according to the input numerical value.

[0115] In step S106, if there is an update of the tomographic data 51, the processes of step S107 and step S108 are executed. If there is no update of the tomographic data 51, in step S104, it is confirmed again whether there is a user change operation.

[0116] In step S107, the control unit 41 of the image processing apparatus 11 processes the signal input from the probe 20 in the same manner as the process of step S101 to newly generate a cross-sectional image 54 of the living tissue 60, thereby obtaining tomographic data 51 including at least one new cross-sectional image 54.

[0117] In step S108, the control unit 41 of the image processing apparatus 11 updates the three-dimensional data 52 of the biological tissue 60 based on the tomographic data 51 acquired in step S107. That is, the control unit 41 updates the three-dimensional data 52 based on the tomographic data 51 acquired by the sensor. Then, in step S103, the control unit 41 causes the display 16 to display the three-dimensional data 52 updated in step S108 as a three-dimensional image 53. The control unit 41 causes the display 16 to display the latest cross-sectional image 54 included in the tomographic data 51 acquired in step S107 together with the three-dimensional image 53. In step S108, it is preferable to update only the data at the location corresponding to the updated tomographic data 51. In that case, the amount of data processing for generating the three-dimensional data 52 can be reduced, and the real-time performance of the three-dimensional image 53 can be improved in step S108.

[0118] As described above, in the present embodiment, the control unit 41 of the image processing apparatus 11 refers to the tomographic data 51 which is a data set obtained using a sensor that moves within the lumen 61 of the biological tissue 60, and causes the display 16 to display a cross-sectional image 54 representing a cross-section 64 of the biological tissue 60 orthogonal to the moving direction of the sensor. The control unit 41 acquires designation data for designating at least one location within the space corresponding to the tomographic data 51 as a point Pd. When the cross-sectional image 54 is being displayed, the control unit 41 performs control to display marks 55 that differ according to the distance between the point Pd and the cross-section 64 in the moving direction of the sensor at the position corresponding to the point Pd in the cross-sectional image 54. Therefore, according to the present embodiment, the usefulness of the system for marking the point Pd related to the biological tissue 60 is improved.

[0119] According to this embodiment, ablation procedures can be guided and ablation points can be marked. By checking the ablation points in the cross-sectional image 54, more detailed and accurate information can be obtained compared to the case of checking in the three-dimensional image 53. Although circumferential isolation may be performed obliquely with respect to the axis of the IVUS catheter instead of on the same plane, in this embodiment, all ablation points can be checked even in such a case. Moreover, it is possible to check whether each ablation point is in the cross-section 64 represented by the cross-sectional image 54, or if not, how far away it is from the cross-section 64.

[0120] In this embodiment, the tomographic data 51 includes, for each ultrasonic image, volume data in which pixel groups are stacked in the moving direction of the sensor by classifying each pixel on the ultrasonic image into classes such as "tissue", "blood cell" or "lumen", and "catheter" other than the IVUS catheter, as a data set. This volume data corresponds to voxel information. As designated data, data indicating the position of the point Pd is also incorporated into the data set as volume data of a class called "marked location" separately from classes such as "tissue", "blood cell" or "lumen", and "catheter", and the mark 55 is displayed based on the volume data. As will be described later, when adjusting the marked location by obtaining a vector from the centroid, after vector calculation, instead of the data itself indicating the position of the point Pd as the "marked location", a vector, that is, data indicating the direction may be incorporated into the data set.

[0121] In the present embodiment, as a method for designating the point Pd, a method in which a user such as an operator designates the position of the point Pd on a two-dimensional image is used. For example, a method in which the user clicks the point Pd on the two-dimensional image with the mouse 15 is used. As a modification of the present embodiment, a method in which the user designates the position of the point Pd on the three-dimensional image 53 may be used. For example, a method in which the user clicks the point Pd on the three-dimensional image 53 with the mouse 15 may be used. Alternatively, a method may be used in which the area where the ablation catheter is in contact is automatically designated as the point Pd based on the information that ablation has been performed. The information that ablation has been performed may be manually input to the image processing apparatus 11, or may be input to the image processing apparatus 11 from a device that controls the ablation catheter. In any of the modifications, when one point is designated in two-dimensional coordinates or three-dimensional coordinates, a certain range centered on the designated point is marked as one location. When one range centered on a certain point is designated, the designated range is marked as one location. For example, a range of a certain size may be designated as the ablation point with a circular or spherical pointer. As the distance between the point Pd and the cross section 64 in the moving direction of the sensor, the distance from the center of the designated range to the cross section 64 may be calculated.

[0122] Referring to FIG. 12, the operation of the image processing system 10 according to the present embodiment will be further described.

[0123] In step S111, if there is an operation for setting the cutting area 66 as a user setting operation, the process of step S112 is executed.

[0124] In step S112, the control unit 41 of the image processing apparatus 11 receives the operation for setting the cutting area 66 via the input unit 44.

[0125] In step S113, the control unit 41 of the image processing apparatus 11 calculates the centroid positions of a plurality of cross-sections in the short-axis direction of the lumen 61 of the biological tissue 60 using the latest three-dimensional data 52 stored in the storage unit 42. The latest three-dimensional data 52 is the three-dimensional data 52 generated in step S102 if the process of step S108 has not been executed, and is the three-dimensional data 52 updated in step S108 if the process of step S108 has been executed. Here, when the already generated three-dimensional data 52 exists, it is preferable not to regenerate all of the three-dimensional data 52 from scratch, but to update only the data at the locations corresponding to the updated tomographic data 51. In that case, the amount of data processing when generating the three-dimensional data 52 can be reduced, and the real-time performance of the three-dimensional image 53 in subsequent step S117 can be improved.

[0126] Specifically, as shown in FIG. 13, if the control unit 41 of the image processing apparatus 11 has generated a corresponding new cross-sectional image 54 in step S107 for each of the plurality of cross-sectional images generated in step S101, it replaces the cross-sectional image with the new cross-sectional image 54 and then binarizes it. As shown in FIG. 14, the control unit 41 extracts a point cloud of the inner wall surface 65 of the biological tissue 60 from the binarized cross-sectional image. For example, the control unit 41 extracts a point cloud of the inner wall surface of the blood vessel by extracting one point corresponding to the inner wall surface of the main blood vessel along the vertical direction of the cross-sectional image with the r-axis as the horizontal axis and the θ-axis as the vertical axis. The control unit 41 may simply obtain the centroid of the extracted point cloud of the inner wall surface, but in that case, since the point cloud is not uniformly sampled over the inner wall surface, a deviation occurs in the centroid position. Therefore, in the present embodiment, the control unit 41 calculates the convex hull of the extracted point cloud of the inner wall surface and obtains the centroid position C using the formula for obtaining the centroid of a polygon as follows n =(C x ,C y ). However, in the following formula, as shown in FIG. 14, n vertices (x0, y0), (x1, y1), ···, (x n-1 , y n-1 ) of the point cloud of the inner wall surface exist counterclockwise on the convex hull, and (x n , y n ) is regarded as (x0, y0).

Number

[0127] The center-of-gravity position obtained as a result is shown in FIG. 15. In FIG. 15, point Cn is the center of the cross-sectional image. Point Bp is the center of gravity of the point group on the inner wall surface. Point Bv is the center of gravity of the vertices of the polygon. Point Bx is the center of gravity of the polygon as the convex hull.

[0128] As a method for calculating the center-of-gravity position of the blood vessel, a method different from the method for calculating the center-of-gravity position of the polygon as the convex hull may be used. For example, in the original cross-sectional image that has not been binarized, a method of calculating the center position of the largest circle that fits within the main blood vessel as the center-of-gravity position may be used. Alternatively, in the binarized cross-sectional image with the r-axis as the horizontal axis and the θ-axis as the vertical axis, a method of calculating the average position of the pixels in the main blood vessel region as the center-of-gravity position may be used. The same methods can also be used when the biological tissue 60 is not a blood vessel.

[0129] In step S114, the control unit 41 of the image processing apparatus 11 executes smoothing on the calculation result of the center-of-gravity position in step S113.

[0130] As shown in FIG. 16, it can be seen that when the calculation result of the center-of-gravity position is regarded as a function of time, the influence of pulsation is large. Therefore, in the present embodiment, the control unit 41 of the image processing apparatus 11 executes smoothing on the calculation result of the center-of-gravity position by using a moving average as shown by the dashed line in FIG. 17.

[0131] As a smoothing method, a method other than the moving average may be used. For example, the exponential smoothing method, the kernel method, local regression, the Ramer-Douglas-Peucker algorithm, the Savitzky-Golay method, smoothing splines, or SGM may be used. Alternatively, a method of performing a fast Fourier transform and then removing high-frequency components may be used. Alternatively, a Kalman filter, or a low-pass filter such as a Butterworth filter, a Chebyshev filter, a digital filter, an elliptic filter, or a KZ filter may be used. "SGM" is an abbreviation for the stretched grid method. "KZ" is an abbreviation for Kolmogorov-Zurbenko.

[0132] If only smoothing is performed, the center of gravity position may enter the tissue. In that case, the control unit 41 may divide the calculation result of the center of gravity position according to the positions of a plurality of cross-sections of the biological tissue 60 orthogonal to the Z direction in the Z direction, and perform smoothing for each divided calculation result. That is, when the curve of the center of gravity position as shown by the broken line in FIG. 17 overlaps the tissue region, the control unit 41 may divide the curve of the center of gravity position into a plurality of sections and perform individual smoothing for each section. Alternatively, the control unit 41 may adjust the degree of smoothing performed on the calculation result of the center of gravity position according to the positions of a plurality of cross-sections of the biological tissue 60 orthogonal to the Z direction in the Z direction. That is, when the curve of the center of gravity position as shown by the broken line in FIG. 17 overlaps the tissue region, the control unit 41 may reduce the degree of smoothing performed on a part of the section including the overlapping point.

[0133] In step S115, as shown in FIG. 3, the control unit 41 of the image processing apparatus 11 sets two planes that intersect with a single line Lb passing through the center of gravity position calculated in step S113 as the cutting planes D1 and D2. In the present embodiment, the control unit 41 sets the cutting planes D1 and D2 after performing smoothing on the calculation result of the center of gravity position in step S114, but the process of step S114 may be omitted.

[0134] Specifically, the control unit 41 of the image processing apparatus 11 sets the curve of the center of gravity position obtained as a result of the smoothing in step S114 as line Lb. The control unit 41 sets a pair of planes intersecting with the set line Lb as cutting planes D1 and D2. In the latest three-dimensional data 52 stored in the storage unit 42, the control unit 41 identifies the three-dimensional coordinates intersecting with the cutting planes D1 and D2 of the biological tissue 60 as the three-dimensional coordinates of the edge of the opening 62 that exposes the lumen 61 of the biological tissue 60 in the three-dimensional image 53. The control unit 41 stores the identified three-dimensional coordinates in the storage unit 42.

[0135] In step S116, the control unit 41 of the image processing apparatus 11 forms, in the three-dimensional data 52, a region sandwiched between the cutting planes D1 and D2 in the three-dimensional image 53 and exposing the lumen 61 of the biological tissue 60 as a cutting region 66.

[0136] Specifically, in the latest three-dimensional data 52 stored in the storage unit 42, the control unit 41 of the image processing apparatus 11 sets the portion specified by the three-dimensional coordinates stored in the storage unit 42 to be non-displayed or transparent when the three-dimensional image 53 is displayed on the display 16. That is, the control unit 41 forms the cutting region 66 set in step S112.

[0137] In step S117, the control unit 41 of the image processing apparatus 11 displays the three-dimensional data 52 in which the cutting region 66 is formed in step S116 on the display 16 as the three-dimensional image 53. The control unit 41 displays the cross-sectional image 54, that is, the two-dimensional image, displayed on the display 16 in step S103 together with the three-dimensional image 53 on the display 16.

[0138] Specifically, the control unit 41 of the image processing apparatus 11 generates a three-dimensional image 53 as shown in FIG. 2 in which the portion specified by the three-dimensional coordinates stored in the storage unit 42 is non-displayed or transparent. Among the cross-sectional images of the biological tissue 60 included in the tomographic data 51 stored in the storage unit 42, the control unit 41 displays the latest cross-sectional image 54 and the generated three-dimensional image 53 on the display 16 via the output unit 45.

[0139] In step S117, similar to step S103, the process shown in FIG. 7 is further executed.

[0140] In step S118, if there is an operation to set the cutting area 66 as the user's change operation, the process of step S119 is executed. If there is no user's change operation, the process of step S120 is executed.

[0141] In step S119, the control unit 41 of the image processing apparatus 11 receives an operation to set the cutting area 66 via the input unit 44, similar to the process of step S112. Then, the processes after step S115 are executed.

[0142] In step S120, if there is an update of the tomographic data 51, the processes of step S121 and step S122 are executed. If there is no update of the tomographic data 51, in step S118, the presence or absence of the user's change operation is confirmed again.

[0143] In step S121, the control unit 41 of the image processing apparatus 11 processes the signal input from the probe 20 to newly generate a cross-sectional image 54 of the biological tissue 60, similar to the process of step S101 or step S107, thereby obtaining tomographic data 51 including at least one new cross-sectional image 54.

[0144] In step S122, the control unit 41 of the image processing apparatus 11 updates the three-dimensional data 52 of the biological tissue 60 based on the tomographic data 51 acquired in step S121. Then, the processes after step S113 are executed. In step S122, it is preferable to update only the data at the corresponding location of the updated tomographic data 51. In that case, the data processing amount when generating the three-dimensional data 52 can be reduced, and the real-time performance of the data processing after step S113 can be improved.

[0145] In this embodiment, marking is performed using a two-dimensional image. As a modification of this embodiment, marking may be performed using a three-dimensional image 53. When marking is performed using a two-dimensional image, even if a point Pd is marked as shown in FIG. 18, if the axis of the probe 20 is displaced as shown in FIG. 19, the axis in the three-dimensional space is displaced and the meaning of the marking is lost. For example, in FIG. 18, the position of the center Pc of the cross-sectional image 54 coincides with the position of the center of gravity Pb of the cross-sectional image 54, but in FIG. 19, the position of the center Pc of the cross-sectional image 54 is greatly displaced from the position of the center of gravity Pb of the cross-sectional image 54. Therefore, in FIG. 18, the point Pd is present on the inner wall surface 65, but in FIG. 19, the point Pd is greatly displaced from the inner wall surface 65. To address such a problem, in this embodiment, as shown in FIG. 20, a mark 55 is displayed at the intersection of the straight line connecting the point Pd and the center of gravity Pb of the lumen 61 and the inner wall surface 65 of the biological tissue 60. Even when the axis of the probe 20 is displaced as shown in FIG. 20, the position of the center of gravity Pb does not change, and the direction from the center of gravity Pb to the point Pd does not change. Therefore, the displacement of the marking as shown in FIG. 19 can be eliminated. Even if the axis is not displaced, the inner wall surface 65 may move due to the influence of pulsation, but the displacement of the marking can be eliminated even in such a case. Similarly, when marking is performed using the three-dimensional image 53, even if the point Pd is marked as shown in FIG. 22, if the position of the center Pc is displaced as shown in FIG. 23, the position of the point Pd also shifts. Therefore, similar to this embodiment, as shown in FIG. 24, for example, by displaying the mark 55 at the relative position from the center of gravity B2 of the cross-section C2, the displacement of the marking as shown in FIG. 23 can be eliminated.

[0146] Referring to FIG. 21, the procedure of the marking process will be described.

[0147] The process of step S311 is the same as the process of step S303 in FIG. 7, so the description thereof is omitted.

[0148] When it is determined that the catheter 63 is in contact with the inner wall surface 65 of the living tissue 60, the processes after step S312 are executed. When it is determined that the catheter 63 is not in contact with the inner wall surface 65 of the living tissue 60, the flow in FIG. 21 ends.

[0149] In step S312, the control unit 41 of the image processing apparatus 11 acquires designation data for designating, as a point Pd, the location on the inner wall surface 65 of the living tissue 60 where the catheter 63 is in contact, in the same manner as in step S304 of FIG. 7. In the present embodiment, the control unit 41 acquires, as the designation data, data for designating the point Pd by receiving a user operation for designating at least one location as the point Pd on the cross-sectional image 54. However, the data for designating the point Pd may be acquired as the designation data by automatically detecting, as the point Pd, the position where the tip of the catheter 63 is in contact in step S311.

[0150] In step S313, the control unit 41 of the image processing apparatus 11 refers to the tomographic data 51 and specifies, as the designated direction, the direction of the point Pd designated by the designation data acquired in step S304 from the centroid Pb in the cross-section 64.

[0151] In step S314, the control unit 41 of the image processing apparatus 11 specifies, as the corresponding position, the position corresponding to the point Pd in the cross-section 64 according to the designated direction specified in step S313 and the position of the centroid Pb. Specifically, the control unit 41 of the image processing apparatus 11 refers to the tomographic data 51 and detects the inner wall surface 65 of the living tissue 60 existing in the cross-section 64. The control unit 41 specifies, as the corresponding position, the position where the straight line extending in the designated direction specified in step S313 from the position of the centroid Pb in the cross-sectional image 54 intersects the detected inner wall surface 65.

[0152] In step S315, the control unit 41 of the image processing apparatus 11 performs control to display the mark 55 at the corresponding position specified in step S314.

[0153] As a modification example of the present embodiment, in step S314, the control unit 41 of the image processing apparatus 11 may specify, as a corresponding position, a position shifted toward the lumen 61 side from a position where a straight line extending in the specified direction specified in step S313 intersects the detected inner wall surface 65 from the position of the center of gravity Pb in the cross-sectional image 54. That is, the mark 55 may be displayed slightly outside the wall from the intersection of the straight line connecting the point Pd and the center of gravity Pb and the inner wall surface 65 of the biological tissue 60. According to this modification example, it is possible to avoid the edge of the inner wall surface 65 being hidden by the mark 55 and the information on the edge portion disappearing. In this modification example, the distance between the inner wall surface 65 and the display position of the mark 55 is stored in the storage unit 42 and read out and applied each time the mark 55 is displayed.

[0154] As a modification example of the present embodiment, in step S314, the control unit 41 of the image processing apparatus 11 may specify, as a corresponding position, a position shifted to the opposite side of the lumen 61 from a position where a straight line extending in the specified direction specified in step S313 intersects the detected inner wall surface 65 from the position of the center of gravity Pb in the cross-sectional image 54. That is, the mark 55 may be displayed slightly inside the wall from the intersection of the straight line connecting the point Pd and the center of gravity Pb and the inner wall surface 65 of the biological tissue 60. According to this modification example, it is possible to avoid the edge of the inner wall surface 65 being hidden by the mark 55 and the information on the edge portion disappearing. In this modification example, the distance between the inner wall surface 65 and the display position of the mark 55, or the distance between the outer wall surface of the biological tissue 60 and the display position of the mark 55 is stored in the storage unit 42 and read out and applied each time the mark 55 is displayed. Alternatively, the relative display position of the mark 55 between the inner wall surface 65 and the outer wall surface is stored in the storage unit 42 and read out and applied each time the mark 55 is displayed. For example, when injecting iPS cells into the wall of the left ventricle, even if the wall thickness fluctuates due to pulsation, the mark 55 is always displayed inside the wall, so that the position where the cells should be injected can be easily specified.

[0155] In this modification example, as shown in FIG. 25, the mark 55 may indicate the ablation position. In the example of FIG. 25, the lumen 61 of the biological tissue 60 is displayed as a three-dimensional object, and the biological tissue 60 is made non-displayed so that the shape of the lumen 61 can be understood. The outer surface of the three-dimensional object representing the lumen 61 corresponds to the inner wall surface 65 of the biological tissue 60. By arranging spheres as the marks 55 slightly outside this outer surface, the ablation position becomes easier to see compared to the case where the spheres are placed on the outer surface or inside the outer surface.

[0156] As a modification example of the present embodiment, in step S314, the control unit 41 of the image processing apparatus 11 may calculate the distance from the center of gravity Pb to the point Pd in the cross section 64 with reference to the tomographic data 51. The control unit 41 may specify, as a corresponding position, a position that is separated from the position of the center of gravity Pb in the specified direction by the calculated distance on a straight line extending in the specified direction from the position of the center of gravity Pb in the cross-sectional image 54.

[0157] As described above, in the present embodiment, the control unit 41 of the image processing apparatus 11 refers to the tomographic data 51, which is a data set obtained using a sensor that moves the lumen 61 of the biological tissue 60, and causes an image representing the biological tissue 60 to be displayed on the display 16. The control unit 41 acquires designation data for designating at least one location in the space corresponding to the tomographic data 51 as the point Pd. The control unit 41 refers to the tomographic data 51 and specifies, as a specified direction, the direction of the point Pd from the center of gravity Pb in the cross section 64 of the biological tissue 60 that is orthogonal to the moving direction of the sensor and includes the point Pd. The control unit 41 specifies, as a corresponding position, the position corresponding to the point Pd in the image representing the biological tissue 60 according to the specified specified direction and the position of the center of gravity Pb. The control unit 41 performs control to display the mark 55 at the specified corresponding position when the image representing the biological tissue 60 is being displayed. Therefore, according to the present embodiment, it is possible to eliminate the deviation in marking in the system for marking the point Pd of the biological tissue 60.

[0158] In this embodiment, the cross-sectional image 54 is used as the "image representing the biological tissue 60", but instead of the cross-sectional image 54, a three-dimensional image 53 may be used. In that case, the control unit 41 of the image processing apparatus 11 may perform control to display the mark 55 by setting the first region including the corresponding position and the second region around the first region in different colors on the entire inner wall surface of the biological tissue 60 in the image representing the biological tissue 60. When the point Pd is specified, if one point is specified in two-dimensional coordinates or three-dimensional coordinates, a certain range centered on the specified point is marked as the first region. When one range centered on a certain point is specified, the specified range is marked as the first region. For example, a range of a certain size may be specified as the first region with a circular or spherical pointer.

[0159] In this embodiment, when a plurality of locations of the biological tissue 60 have been specified as the point Pd, the mark 55 is displayed at each position corresponding to the plurality of locations. However, as a modification of this embodiment, the mark 55 may be displayed only at the locations existing in the cross-section 64 corresponding to the position of the sensor among the plurality of locations. In that case, the information of the image with the mark 55 set is stored, and when the same image is displayed as the cross-sectional image 54, the set mark 55 may be displayed.

[0160] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks described in the block diagram may be integrated, or one block may be divided. Instead of executing two or more steps described in the flowchart in time series according to the description, each step may be executed in parallel or in a different order according to the processing ability of the apparatus that executes each step or as necessary. In addition, modifications can be made without departing from the spirit of the present disclosure.

[0161] For example, as shown in FIG. 26, when a plurality of locations are specified as the point Pd, the control unit 41 of the image processing apparatus 11 may further perform control to display on the cross-sectional image 54 the distance between the catheter 63 inserted into the biological tissue 60 and the location among the plurality of locations that is closest to the catheter 63. Alternatively, when only one location is specified as the point Pd, the control unit 41 may further perform control to display on the cross-sectional image 54 the distance between the catheter 63 and the one location. The unit of the displayed distance is, for example, millimeters. The displayed distance may be a distance on a plane, but is a distance in a three-dimensional space, that is, an actual distance. In the example of FIG. 26, six locations on the inner wall surface 65 of the biological tissue 60 that have been cauterized by the catheter 63 are specified as the points P1, P2, P3, P4, P5, and P6. Assuming that the location specified as the point P6 among these six locations is the closest to the catheter 63, the distance between the catheter 63 and the position corresponding to the point P6 in the cross-sectional image 54 is displayed on the cross-sectional image 54. In the example of FIG. 26, the text "15.7 mm" is displayed as the distance. As shown in FIG. 26, the control unit 41 may further perform control to display a mark different from the mark 55 at the position corresponding to the catheter 63 in the cross-sectional image 54.

[0162] As shown in FIG. 26, when a plurality of locations are specified as the points Pd, the control unit 41 of the image processing apparatus 11 may further perform control to display a line 56 connecting the catheter 63 and the location closest to the catheter 63 among the plurality of locations on the cross-sectional image 54. Alternatively, when only one location is specified as the point Pd, the control unit 41 may further perform control to display a line connecting the catheter 63 and the one location on the cross-sectional image 54. In the example of FIG. 26, six locations on the inner wall surface 65 of the biological tissue 60 that have been cauterized by the catheter 63 are specified as the points P1, P2, P3, P4, P5, and P6. Assuming that the location specified as the point P6 is the closest to the catheter 63 among these six locations, as the line 56, a straight line connecting the catheter 63 and the position corresponding to the point P6 in the cross-sectional image 54 is displayed on the cross-sectional image 54. In the example of FIG. 26, a mark different from the mark 55 is displayed at the position corresponding to the catheter 63 in the cross-sectional image 54, and a straight line connecting this mark and the mark M6 is displayed.

[0163] In catheter ablation for arrhythmia, the aim is to disconnect the source of abnormal electrical signals from the heart area by cauterizing the source linearly or circumferentially using an ablation catheter. However, in the point-by-point ablation method, the ablation points may be too far apart, resulting in failure to disconnect the electrical connection and failure to achieve the treatment goal. Therefore, it is necessary to reduce the interval between ablation points. According to the above example, in a two-dimensional image, the next ablation point can be determined while observing the distance between the ablation catheter and the ablation point closest to the ablation catheter. Thus, it becomes easier to reduce the interval between ablation points. As a result, the possibility of achieving the treatment goal can be increased.

[0164] For example, as shown in FIG. 26, when a plurality of locations are specified as point Pd, the control unit 41 of the image processing apparatus 11 may further perform control to display, on the three-dimensional image 53, the distance between the catheter 63 and the location among the plurality of locations that is closest to the catheter 63. Alternatively, when only one location is specified as point Pd, the control unit 41 may further perform control to display, on the three-dimensional image 53, the distance between the catheter 63 and the one location. The unit of the displayed distance is, for example, millimeters. The displayed distance is the distance in three-dimensional space, that is, the actual distance. In the example of FIG. 26, six locations on the inner wall surface 65 of the biological tissue 60 that have been cauterized by the catheter 63 are specified as points P1, P2, P3, P4, P5, and P6. Assuming that the location specified as point P6 among these six locations is the closest to the catheter 63, the distance between the catheter 63 and point P6 is displayed on the three-dimensional image 53. In the example of FIG. 26, the text "15.7 mm" is displayed as the distance.

[0165] As shown in FIG. 26, when a plurality of locations are specified as point Pd, the control unit 41 of the image processing apparatus 11 may further perform control to display, on the three-dimensional image 53, a line 57 connecting the catheter 63 and the location among the plurality of locations that is closest to the catheter 63. Alternatively, when only one location is specified as point Pd, the control unit 41 may further perform control to display, on the three-dimensional image 53, a line connecting the catheter 63 and the one location. In the example of FIG. 26, six locations on the inner wall surface 65 of the biological tissue 60 that have been cauterized by the catheter 63 are specified as points P1, P2, P3, P4, P5, and P6. Assuming that the location specified as point P6 among these six locations is the closest to the catheter 63, as the line 57, a straight line connecting the catheter 63 and the position corresponding to point P6 in the cross-sectional image 54 is displayed on the three-dimensional image 53. In the example of FIG. 26, a straight line connecting the tip of the catheter 63 and point P6 in the three-dimensional image 53 is displayed.

[0166] According to the above example, in the three-dimensional image 53, the next ablation point can be determined while looking at the distance between the ablation catheter and the ablation point closest to the ablation catheter. Therefore, it becomes easier to narrow the interval between the ablation points. As a result, the possibility of achieving the procedure goal can be increased.

Explanation of Signs

[0167] 10 Image processing system 11 Image processing device 12 Cable 13 Drive unit 14 Keyboard 15 Mouse 16 Display 17 Connection terminal 18 Cart unit 20 Probe 21 Drive shaft 22 Hub 23 Sheath 24 Outer tube 25 Ultrasonic vibrator 26 Relay connector 31 Scanner unit 32 Slide unit 33 Bottom cover 34 Probe connection part 35 Scanner motor 36 Insertion port 37 Probe clamp part 38 Slide motor 39 Switch group 41 Control unit 42 Memory unit 43 Communication unit 44 Input unit 45 Output unit 51 Tomographic data 52 Three-dimensional data 53 Three-dimensional image 54 Cross-sectional image 55 Mark 56 Line 57 Line 60 Biological tissue 61 Inner lumen 62 Opening 63 Catheter 64 Cross-section 65 Inner wall surface 66 Cutting region 71 Stent 72 Branch 73 Stent graft 74 Aneurysm 75 Nerve 76 Tumor

Claims

1. An image processing apparatus that displays, on a display, a cross-sectional image representing a cross-section of the living tissue orthogonal to the moving direction of a sensor, with reference to tomographic data that is a dataset obtained using a sensor that moves within the lumen of the living tissue, the image processing apparatus comprising a control unit that acquires designation data for designating at least one location within the space corresponding to the tomographic data, and performs control to display, at a position corresponding to the at least one location in the cross-sectional image when the cross-sectional image is being displayed, marks that differ according to the distance between the at least one location in the moving direction and the cross-section.

2. The image processing apparatus according to claim 1, wherein the control unit changes the color, brightness, transparency, pattern, size, shape, or orientation of the mark according to the distance.

3. The image processing apparatus according to claim 1 or claim 2, wherein the control unit further performs control to display the distance when performing control to display the mark.

4. The image processing apparatus according to any one of claims 1 to 3, wherein the control unit makes the mark non-displayed when the distance exceeds a threshold value.

5. The image processing apparatus according to any one of claims 1 to 4, wherein the control unit changes the mark according to whether the at least one location exists in front of or behind the cross-section in the moving direction.

6. The at least one location is a cauterized location of the living tissue, and the image processing apparatus according to any one of claims 1 to 5, wherein the control unit further performs control to display the distance between a catheter for cauterizing the living tissue and the at least one location when the cross-sectional image is being displayed.

7. The image processing apparatus according to any one of claims 1 to 5, wherein when there is only one of the at least one location, the control unit performs control to display, on the cross-sectional image, the distance between the catheter inserted into the living tissue and the one location, and when there are a plurality of the at least one location, the control unit further performs control to display, on the cross-sectional image, the distance between the catheter and the location closest to the catheter among the plurality of locations.

8. When there is only one of the at least one location, the control unit performs control to display, on the cross-sectional image, a line connecting the catheter inserted into the biological tissue and the one location; when there are a plurality of the at least one location, the control unit further performs control to display, on the cross-sectional image, a line connecting the catheter and the location closest to the catheter among the plurality of locations. The image processing apparatus according to any one of claims 1 to 5.

9. The control unit refers to the tomographic data to generate three-dimensional data representing the biological tissue, and causes the display to display the generated three-dimensional data as a three-dimensional image. When there is only one of the at least one location, the control unit performs control to display, on the three-dimensional image, the distance between the catheter inserted into the biological tissue and the one location; when there are a plurality of the at least one location, the control unit further performs control to display, on the three-dimensional image, the distance between the catheter and the location closest to the catheter among the plurality of locations. The image processing apparatus according to any one of claims 1 to 5.

10. The control unit refers to the tomographic data to generate three-dimensional data representing the biological tissue, and causes the display to display the generated three-dimensional data as a three-dimensional image. When there is only one of the at least one location, the control unit performs control to display, on the three-dimensional image, a line connecting the catheter inserted into the biological tissue and the one location; when there are a plurality of the at least one location, the control unit further performs control to display, on the three-dimensional image, a line connecting the catheter and the location closest to the catheter among the plurality of locations. The image processing apparatus according to any one of claims 1 to 5.

11. The control unit of the image processing apparatus according to any one of claims 1 to 10 acquires the designated data by receiving a user operation for designating the at least one location on the cross-sectional image.

12. The control unit of the image processing apparatus according to any one of claims 1 to 11 causes the display to display, as the cross-sectional image, a new image representing a cross-section corresponding to the position of the sensor each time a new data set is obtained using the sensor.

13. An image processing system comprising the image processing apparatus according to any one of claims 1 to 12, the sensor, and.

14. The image processing system according to claim 13, further comprising the display.

15. An image display method for displaying on a display a cross-sectional image representing a cross-section of the living tissue orthogonal to the moving direction of the sensor, with reference to tomographic data which is a dataset obtained using a sensor that moves within the lumen of the living tissue, comprising: a computer acquiring designation data for designating at least one location within the space corresponding to the tomographic data; the computer performing control to display, at a position corresponding to the at least one location in the cross-sectional image, marks that differ depending on the distance between the at least one location in the moving direction and the cross-section, when the cross-sectional image is being displayed. **Claim 16** A computer for causing a display to display a cross-sectional image representing a cross-section of the living tissue orthogonal to the moving direction of the sensor, with reference to tomographic data which is a dataset obtained using a sensor that moves within the lumen of the living tissue, a process of acquiring designation data for designating at least one location within the space corresponding to the tomographic data; and a process of performing control to display, at a position corresponding to the at least one location in the cross-sectional image, marks that differ depending on the distance between the at least one location in the moving direction and the cross-section, when the cross-sectional image is being displayed An image processing program for causing the computer to execute.

Images