Medical image processing equipment

Abstract

The goal is to estimate the angle of the lumen structure of an object without performing extensive calculations. [Solution] The medical image processing apparatus according to the embodiment comprises an acquisition unit, a identification unit, a setting unit, a measurement unit, and an estimation unit. The acquisition unit acquires three-dimensional medical image data including a tubular structure. The identification unit identifies the lumen region of the tubular structure in the acquired medical image data. The setting unit sets a virtual rod that penetrates the identified lumen region. The measurement unit measures the overlap between the body of the tubular structure and the rod for each angle between the lumen region and the rod. The estimation unit estimates the angle of the tubular structure based on the measured overlap.

Description

Technical Field

[0005] , , , ,

[0001] The embodiments disclosed in this specification and the drawings relate to a medical image processing apparatus.

Background Art

[0002] Regarding a medical image of an object having a lumen structure, accurately estimating the angle of the lumen structure is useful for diagnosis and treatment. A lumen is the inner space in a tubular organ. Examples of the object include blood vessels, small intestine, large intestine, etc. For example, by using machine learning, by recognizing the lumen structure of the object, the angle of the lumen structure can be estimated based on the relationship between the lumen structure and its surroundings. When the object is the heart and the lumen structure is a valve, auxiliary data such as an electrocardiogram is required.

[0003] However, with means such as machine learning, it is necessary to pre-train a model dedicated to each object. In the pre-training, there is a problem that a large amount of calculation is required to make the object recognize the model dedicated to each object.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

Means for Solving the Problems

[0006] The medical image processing apparatus according to the embodiment comprises an acquisition unit, a identification unit, a setting unit, a measurement unit, and an estimation unit. The acquisition unit acquires three-dimensional medical image data including a tubular structure. The identification unit identifies the lumen region of the tubular structure in the acquired medical image data. The setting unit sets a virtual rod that penetrates the identified lumen region. The measurement unit measures the overlap between the body of the tubular structure and the rod for each angle between the lumen region and the rod. The estimation unit estimates the angle of the tubular structure based on the measured overlap. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a perspective view showing an example of the external appearance of an ultrasound diagnostic device according to the first embodiment. [Figure 2] Figure 2 is a block diagram showing the configuration of an ultrasound diagnostic apparatus equipped with an ultrasound image processing device according to the first embodiment. [Figure 3] Figure 3 is a block diagram showing the functions of the ultrasound diagnostic device according to the first embodiment. [Figure 4] Figure 4 is a flowchart showing the processing of the ultrasound diagnostic apparatus 1 according to the first embodiment. [Figure 5] Figure 5 shows an example of projection and noise reduction according to the first embodiment. [Figure 6] Figure 6 shows an example of setting up and rotating a virtual cylinder according to the first embodiment. [Figure 7] Figure 7 shows an example of angle estimation according to the first embodiment. [Figure 8] Figure 8 shows a specific example of a rectangular parallelepiped range including the luminal region according to the first embodiment. [Figure 9] Figure 9 shows an example of a tubular structure and a hypothetical cylinder according to the first embodiment. [Modes for carrying out the invention]

[0008] The embodiments of the medical image processing device will be described in detail below with reference to the drawings.

[0009] [First Embodiment] Figure 1 is a perspective view showing an example of the external appearance of an ultrasound diagnostic apparatus 1 according to the first embodiment. As shown in Figure 1, the ultrasound diagnostic apparatus 1 comprises an ultrasound image processing device 10 and an ultrasound probe 20. The ultrasound image processing device 10 includes various circuits housed in a main body case with casters, as well as an input interface 30 and a display 40. The ultrasound image processing device 10 is an example of a medical image processing device.

[0010] The input interface 30 is a device that allows the user to input various data and information to the ultrasound image processing device 10, or to set various operating modes to the ultrasound image processing device 10, through user operation. The input interface 30 is configured to include, for example, two devices: an operation panel and a touch panel.

[0011] The control panel is equipped with various operating devices, such as a trackball, various switches, and dials. By operating these devices, the user can input various types of data and information into the ultrasound image processing device 10.

[0012] On the other hand, a touch panel is a display and input device composed of a touchscreen superimposed on a display panel such as an LCD panel. By touching or pressing the touchscreen according to the display panel's display, the user can input various data and information into the ultrasound image processing device 10.

[0013] The display 40 displays ultrasound images and various data generated by the various circuits of the ultrasound image processing device 10. The display 40 is configured to include, for example, a liquid crystal display panel or an organic EL (Electro Luminescence) panel.

[0014] Figure 2 is a block diagram showing the configuration of an ultrasound diagnostic apparatus 1 equipped with an ultrasound image processing apparatus 10 according to the first embodiment. The ultrasound image processing apparatus 10 includes an ultrasound transmitting circuit 11, an ultrasound receiving circuit 12, an image memory 13, a network interface 14, a processing circuit 15, and a main memory 16. The ultrasound transmitting circuit 11 and the ultrasound receiving circuit 12 are configured using application-specific integrated circuits (ASICs), etc. However, it is not limited to this case, and all or part of the functions of the ultrasound transmitting circuit 11 and the ultrasound receiving circuit 12 may be realized by the processing circuit 15 executing a computer program.

[0015] The ultrasonic transmitting circuit 11 and the ultrasonic receiving circuit 12 control the transmission directivity and reception directivity of ultrasound under the control of the processing circuit 15. The case in which both the ultrasonic transmitting circuit 11 and the ultrasonic receiving circuit 12 are provided in the ultrasonic image processing device 10 will be described. At least one of the ultrasonic transmitting circuit 11 and the ultrasonic receiving circuit 12 may be provided in the ultrasonic probe 20, or both the ultrasonic image processing device 10 and the ultrasonic probe 20 may be provided.

[0016] The ultrasonic transmitting circuit 11 has the ability to instantaneously change the transmission frequency, transmission drive voltage, etc., in order to execute a predetermined scan sequence based on instructions from the processing circuit 15. In particular, the function of changing the transmission drive voltage is realized, for example, by a linear amplifier type oscillator circuit that can switch its value instantaneously, or by a mechanism that electrically switches multiple power supply units.

[0017] Here, when 3D scanning, i.e., volume scanning, is performed, as the ultrasonic probe 20, a 2D array probe equipped with a scanning method such as a linear type, a convex type, a sector type, etc. is used. Or, when volume scanning is performed, as the ultrasonic probe 20, a 1D probe equipped with a scanning method such as a linear type, a convex type, etc. and having a mechanism that mechanically swings in the elevation direction is used. The latter probe is also called a mechanical 4D probe.

[0018] The image memory 13 has, for example, a magnetic or optical recording medium, or a recording medium readable by a processor such as a semiconductor memory, etc. The image memory 13 stores a plurality of ultrasonic images under the control of the processing circuit 15.

[0019] The network interface 14 implements various information communication protocols according to the form of the network. Also, the network interface 14 may implement various protocols for non-contact wireless communication.

[0020] The processing circuit 15 means a dedicated or general-purpose CPU (Central Processing Unit), MPU (Micro Processor unit), or GPU (Graphics Processing Unit), as well as an ASIC, and a programmable logic device, etc.

[0021] The main memory 16 is composed of semiconductor memory elements such as RAM (Random Access Memory), flash memory (Flash Memory), a hard disk, an optical disk, etc. The main memory 16 may be composed of portable media such as a USB (Universal Serial Bus) memory and a DVD (Digital Video Disk). The main memory 16 stores various processing programs (including other application programs and an OS (Operating System), etc.) used in the processing circuit 15 and data necessary for the execution of the programs.

[0022] Figure 3 is a block diagram showing the functions of the ultrasound diagnostic device 1. The processing circuit 15 is a processor that, by calling and executing a program in the main memory 16, realizes the following functions, as shown in Figure 2: system control function 151, imaging control function 152, image processing function 153, memory control function 154, display control function 155, lumen identification function 156, cylinder setting function 157, overlap measurement function 158, and angle estimation function 159. Alternatively, the processing circuit 15 may be configured by combining multiple independent processors, with each processor executing a program to realize each function.

[0023] The system control function 151 includes, for example, a function that stores command signals from the operator input through the input interface 30, and information such as various initial setting conditions, and then transmits this information to each processing function of the processing circuit 15.

[0024] The imaging control function 152 includes, for example, a function that reads information from the system control function 151 and controls the transmission and reception of ultrasonic waves.

[0025] The image processing function 153 includes a function to read image data stored in the image memory 13, perform image processing on the image data, and then store the processed image data back in the image memory 13. The image processing function 153 may also read three-dimensional medical image data including tubular structures (hereinafter referred to as "voxel data") from the image memory 13.

[0026] The memory control function 154 includes the function of storing various data in the main memory 16 or reading data from the main memory 16. The memory control function 154 may also read voxel data, including the lumen structure, from the main memory 16.

[0027] The display control function 155 includes, for example, a function to read signals from the system control function 151, acquire desired ultrasound image data from the image memory 13, and display it on the display 40.

[0028] The lumen identification function 156 includes a function to identify the lumen region of a tubular structure in the voxel data acquired by the image processing function 153 or the memory control function 154.

[0029] The cylinder setting function 157 includes the function of setting a virtual cylinder that penetrates the lumen region identified by the lumen identification function 156. The cylinder is an example of a rod.

[0030] The overlap measurement function 158 includes a function to measure the overlap between the main body of the lumen structure and the cylinder set by the cylinder setting function 157 for each angle between the lumen region identified by the lumen identification function 156 and the cylinder set by the cylinder setting function 157.

[0031] The angle estimation function 159 includes a function to estimate the angle of the lumen structure based on the overlap between the lumen region and the cylinder at each angle, as measured by the overlap measurement function 158.

[0032] Figure 4 is a flowchart showing the processing of the ultrasound diagnostic device 1 according to the first embodiment. Figure 5 is a diagram showing an example of projection and noise reduction according to the first embodiment. Figure 6 is a diagram showing an example of setting up and rotating a virtual cylinder according to the first embodiment. Figure 7 is a diagram showing an example of angle estimation according to the first embodiment. Figure 8 is a diagram showing an example of specifying a rectangular parallelepiped range including a lumen region according to the first embodiment. Figure 9 is a diagram showing an example of a lumen structure and a virtual cylinder according to the first embodiment.

[0033] The angle estimation process according to the first embodiment will be described below, following Figure 4 and with reference to Figures 5 to 9.

[0034] In step ST1, the image processing function 153 or the memory control function 154 acquires voxel data (i.e., 3D image data) including the lumen structure from the image memory 13 or the main memory 16. If the image processing function 153 acquires voxel data from a source other than the image memory 13, it stores the voxel data in the image memory 13 for storage.

[0035] In step ST2, the lumen identification function 156 includes a function to identify the lumen region of the tubular structure in the voxel data acquired in step ST1. For example, the lumen identification function 156 extracts the three-dimensional coordinates of each pixel (for example, each pixel on the inner wall surface) that constitutes the lumen region included in the voxel data.

[0036] In step ST3, the image processing function 153 determines the coordinates of the center point of the lumen region identified in step ST2. The details are explained below.

[0037] First, the image processing function 153 projects the voxel data onto the plane formed by the L and S axes in a direction parallel to the A axis, as shown in Figure 5(a). Next, the image processing function 153 removes noise from the resulting planar image. Noise here refers to small regions located around the periphery of the planar image with 100 or fewer pixels. In the planar image of Figure 5(b), four circles other than the largest circle are noise, so the image processing function 153 removes these four circles from the planar image, leaving only the largest circle. In the planar image of Figure 5(c), there are two circles, but neither is noise, so the image processing function 153 leaves these two circles.

[0038] The image processing function 153 then determines the coordinates of the center point of the luminal region TS1 from the denoised image data. If, as shown in Figure 5(b), one closed region remains after noise removal, the image processing function 153 determines the coordinates of the center point of that closed region. The center point may be, for example, the centroid of the figure of that closed region. If, as shown in Figure 5(c), two or more closed regions remain after noise removal, the image processing function 153 may determine the coordinates of the center points of the polygons formed by the center points of each closed region, or it may adjust the coordinates of the center points by weighting them according to the area of ​​each closed region. For example, in Figure 5(c), the image processing function 153 may use the point that divides the line segment connecting the centers of the two circles internally according to the inverse ratio of the areas of the two circles as the center point. In this case, the center point is not the midpoint of the line segment, but is closer to the center of the larger circle than the center of the smaller circle.

[0039] In step ST4, the cylinder setting function 157 sets a virtual rod-shaped structure that penetrates the lumen region TS1 and has a central axis that passes through the center point of the lumen region TS1 determined in step ST3 and makes a predetermined angle with respect to the A axis. In this embodiment, an example is described in which the virtual rod-shaped structure is a cylinder VC. The cylinder setting function 157 appropriately determines the diameter and orientation of the virtual cylinder VC. For example, if one closed region remains after step ST3, the cylinder setting function 157 may set a virtual cylinder VC that has a circular cross-section overlapping the main body TS2 outside the closed region and has a central axis that makes a predetermined angle with respect to the A axis. For example, the cylinder setting function 157 sets the virtual cylinder VC shown in Figure 6(b) for the lumen region TS1 shown in Figure 6(a). Figure 6(c) is a side view showing the lumen region TS1 and the set virtual cylinder VC.

[0040] In step ST5, the image processing function 153 rotates the virtual cylinder VC around two central axes that pass through the center point of the lumen region TS1 and are parallel to the S-axis and L-axis, respectively. The image processing function 153 rotates the virtual cylinder VC within an angular range of -22 to +23 degrees. By minimizing the angular range, the computational load can be reduced, thereby lowering the cost.

[0041] In this embodiment, the A-axis coordinate of the center point (hereinafter referred to as the "rotation center point") when the virtual cylinder VC is rotated is the A-axis coordinate of the center point of the image showing the lumen structure TS. For example, if the coordinates of the center point of the image are (128, 128, 128), then the coordinates of the rotation center point will be (S, 128, L). S and L are the S-axis coordinate and L-axis coordinate of the center point of the lumen region TS1 obtained in step ST3, respectively. Note that the method of determining the A-axis coordinate of the rotation center point is not limited and may be within the range of the image. For example, as shown in Figures 6(c) and (d), the rotation center point may be at one end of the image.

[0042] The overlap measurement function 158 then measures the overlap between the virtual cylinder VC and the main body TS2 of the lumen structure TS for each rotation angle with the two axes as the central axes. Overlap refers to the overlap between the pixels constituting the virtual cylinder VC and the pixels constituting the main body TS2, for example, the number of pixels included in the overlapping portion. The shaded area shown in Figure 6(c) indicates this overlap. The overlap measurement function 158 may also measure the surface area, volume, or weighted volume considering density of the overlapping portion between the main body TS2 of the lumen structure TS and the virtual cylinder VC. Although it is stated that the image processing function 153 rotates the virtual cylinder VC, it may also rotate the main body TS2 of the lumen structure TS.

[0043] In step ST6, the angle estimation function 159 estimates the angle between the lumen region TS1 and the virtual cylinder VC when the overlap is minimized as the angle of the lumen structure TS. As shown in Figure 7, the rotation angle alpha around the axis parallel to the S axis is approximately 18°, and the rotation angle beta around the axis parallel to the L axis is approximately 23°. Figure 6(d) is a side view showing the state in which the virtual cylinder VC and the main body TS2 of the lumen structure TS do not overlap.

[0044] Note that, referring to Figures 6(c) and (d), the closer the rotation angle is to 0°, the smaller the overlap becomes, because the lumen region is cylindrical. In reality, the lumen region is not cylindrical, but rather winding, uneven, and folds, so the rotation angle at which the overlap is small is not necessarily 0°.

[0045] In step ST7, the image processing function 153 rotates the original voxel data to a standard angle to determine the rectangular area containing the lumen. The details are described below.

[0046] First, the image processing function 153 rotates the original voxel data around each axis by the two angles estimated in step ST6. This aligns the orientation of the original voxel data with the standard angle, i.e., the A-axis direction. Figure 8(a) shows a projected image obtained by projecting the rotated voxel data onto a plane containing the L-axis and S-axis.

[0047] Next, the image processing function 153 determines the range of the rectangular parallelepiped including the lumen region based on the projected image. As shown in Figure 8(a), a rectangle is set to enclose the shape of the projected lumen region (in this case, a circle). A margin M of, for example, about 20 pixels is provided between the lumen region and the rectangle.

[0048] The image processing function 153 then uses the set rectangle as a cross-section to determine the range of a rectangular prism with depth in the rotated voxel data. This allows obtaining the minimum range of a rectangular prism in which the orientation of the lumen region included in the original voxel data is adjusted in the A-axis direction, and for example, this range becomes the coordinate range of the valve.

[0049] Figure 8(b) shows the smallest rectangular prism containing the lumen region from the rotated voxel data. As shown in Figure 8(b), the width of the rectangular prism is L1-L0, which is smaller than the width L1 of the rotated voxel data. The height of the rectangular prism is S1-S0, which is smaller than the height S1 of the rotated voxel data. The depth of the rectangular prism is A1-0, which is the same as the depth A1 of the rotated voxel data.

[0050] As described above, instead of recognizing the object itself, such as a valve, from the ultrasound image and estimating the angle of that object, the angle of the lumen structure TS identified from the ultrasound image is estimated, allowing for real-time processing. Next, compared to conventional machine learning-based methods, the computational load is significantly reduced because prior model training and object recognition are not required. Furthermore, it can be used as a general-purpose method rather than a specialized method for specific objects. Moreover, even if the lumen region TS1 of the lumen structure TS is meandering, or if there are irregularities or folds on the inner surface of the lumen region TS1, the angle of the lumen structure TS can be estimated robustly.

[0051] According to at least one embodiment described above, the angle of the lumen structure of the object can be estimated without performing a large amount of calculations.

[0052] Note that the image processing function 153 is an example of an acquisition unit and a specific unit. The memory control function 154 is an example of an acquisition unit. The lumen specificity function 156 is an example of a specific unit. The cylinder setting function 157 is an example of a setting unit. The overlap measurement function 158 is an example of a measurement unit. The angle estimation function 159 is an example of an estimation unit.

[0053] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0054] 1… Ultrasound diagnostic equipment 10… Ultrasonic imaging device 13…Image memory 16…Main memory 30…Input Interface 40…Display 153…Image processing function 154...Memory control function 156…lumen specific function 157...Cylinder setting function 158... Overlap measurement function 159...Angle estimation function P...Subject TS…Lumen structure TS1…lumen area TS2...Main unit VC... virtual cylinder

Claims

1. An acquisition unit that acquires three-dimensional medical image data including tubular structures, A specific unit that identifies the luminal region of the tubular structure in the acquired medical image data, A setting unit for setting a virtual rod that penetrates the identified lumen region, For each angle between the lumen region and the rod, a measuring unit measures the overlap between the main body of the lumen structure and the rod. An estimation unit that estimates the angle of the lumen structure based on the measured overlap, A medical image processing device equipped with [a specific feature].

2. The setting unit sets a cylinder whose central axis is the axis passing through the center of the lumen region as the rod. The medical image processing apparatus according to claim 1.

3. The measuring unit measures the overlap while rotating the main body of the cylinder or the tubular structure around an axis passing through the center of the lumen region and a plurality of axes. The medical image processing apparatus according to claim 2.

4. The measurement unit measures the surface area, volume, or weighted volume (considering density) of the overlapping portion where the main body of the tubular structure and the cylinder overlap. The medical image processing apparatus according to claim 2.

5. The estimation unit estimates the angle between the lumen region and the rod when the overlap is minimized as the angle of the lumen structure. The medical image processing apparatus according to claim 1.

Images