Medical image processing equipment

Abstract

To estimate the angle of an object in medical image data without performing extensive calculations. [Solution] The medical image processing apparatus according to the embodiment comprises an acquisition unit, an extraction unit, and an estimation unit. The acquisition unit acquires three-dimensional medical image data at multiple time phases. The extraction unit extracts temporal feature values ​​of each pixel contained in the medical image data from the medical image data at multiple time phases. The estimation unit estimates the angle of the fluctuating region in the medical image data based on the temporal feature values.

Description

Technical Field

[0006] , , , , ,

[0001] The embodiments disclosed in this specification and the drawings relate to a medical image processing apparatus.

Background Art

[0002] In medical ultrasonic images, the relative angle of an object in the captured image may change depending on the angle of the probe that a doctor manually operates during imaging. For example, transesophageal echocardiography (TEE) is a technique in which an ultrasonic probe is inserted into a patient's esophagus and imaging is performed from the back side of the heart, but the angle of the ultrasonic probe changes each time imaging is performed. In such a case, when separately captured images are displayed simultaneously, it is difficult to directly compare them because the relative angles of the objects in the images are different. In that case, by adjusting the angle of the object, it becomes easier to compare the objects in the separately captured images.

[0003] In conventional automatic processing, means such as machine learning are used to recognize the shape and contour of an object, determine the angle of the object, and further perform angle adjustment for easier viewing.

[0004] However, with means such as machine learning, it is necessary to pre-train a model dedicated to each object. In the pre-training, there is a problem that a large amount of calculation is required to make the object recognized by the model dedicated to each object.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

[0007] The medical image processing apparatus according to this embodiment comprises an acquisition unit, an extraction unit, and an estimation unit. The acquisition unit acquires three-dimensional medical image data at multiple time phases. The extraction unit extracts temporal feature values ​​for each pixel contained in the medical image data from the medical image data at multiple time phases. The estimation unit estimates the angle of the fluctuating region in the medical image data based on the temporal feature values. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a perspective view showing an example of the external appearance of an ultrasound diagnostic device according to the first embodiment. [Figure 2] Figure 2 is a block diagram showing the configuration of an ultrasound diagnostic apparatus equipped with an ultrasound image processing device according to the first embodiment. [Figure 3] Figure 3 is a block diagram showing the functions of the ultrasound diagnostic device according to the first embodiment. [Figure 4] Figure 4 is a flowchart showing the processing of the ultrasound diagnostic apparatus according to the first embodiment. [Figure 5] Figure 5 is a schematic diagram showing voxel data, feature images, and binarized images according to the first embodiment. [Figure 6] Figure 6 shows an example of the original image according to the first embodiment and an image obtained by rotating the original image to a reference angle. [Modes for carrying out the invention]

[0009] The embodiments of the medical image processing device will be described in detail below with reference to the drawings.

[0010] [First Embodiment] Figure 1 is a perspective view showing an example of the external appearance of an ultrasound diagnostic apparatus 1 according to the first embodiment. As shown in Figure 1, the ultrasound diagnostic apparatus 1 comprises an ultrasound image processing device 10 and an ultrasound probe 20. The ultrasound image processing device 10 includes various circuits housed in a main body case with casters, as well as an input interface 30 and a display 40. The ultrasound image processing device 10 is an example of a medical image processing device.

[0011] The input interface 30 is a device that allows the user to input various data and information to the ultrasound image processing device 10, or to set various operating modes to the ultrasound image processing device 10, through user operation. The input interface 30 is configured to include, for example, two devices: an operation panel and a touch panel.

[0012] The control panel is equipped with various operating devices, such as a trackball, various switches, and dials. By operating these devices, the user can input various types of data and information into the ultrasound image processing device 10.

[0013] On the other hand, a touch panel is a display and input device composed of a touchscreen superimposed on a display panel such as an LCD panel. By touching or pressing the touchscreen according to the display panel's display, the user can input various data and information into the ultrasound image processing device 10.

[0014] The display 40 displays ultrasound images and various data generated by the various circuits of the ultrasound image processing device 10. The display 40 is configured to include, for example, a liquid crystal display panel or an organic EL (Electro Luminescence) panel.

[0015] Figure 2 is a block diagram showing the configuration of an ultrasound diagnostic apparatus 1 equipped with an ultrasound image processing apparatus 10 according to the first embodiment. The ultrasound image processing apparatus 10 includes an ultrasound transmitting circuit 11, an ultrasound receiving circuit 12, an image memory 13, a network interface 14, a processing circuit 15, and a main memory 16. The ultrasound transmitting circuit 11 and the ultrasound receiving circuit 12 are configured using application-specific integrated circuits (ASICs), etc. However, it is not limited to this case, and all or part of the functions of the ultrasound transmitting circuit 11 and the ultrasound receiving circuit 12 may be realized by the processing circuit 15 executing a computer program.

[0016] The ultrasonic transmitting circuit 11 and the ultrasonic receiving circuit 12 control the transmission directivity and reception directivity of ultrasound under the control of the processing circuit 15. The case in which both the ultrasonic transmitting circuit 11 and the ultrasonic receiving circuit 12 are provided in the ultrasonic image processing device 10 will be described. At least one of the ultrasonic transmitting circuit 11 and the ultrasonic receiving circuit 12 may be provided in the ultrasonic probe 20, or both the ultrasonic image processing device 10 and the ultrasonic probe 20 may be provided.

[0017] The ultrasonic transmitting circuit 11 has the ability to instantaneously change the transmission frequency, transmission drive voltage, etc., in order to execute a predetermined scan sequence based on instructions from the processing circuit 15. In particular, the function of changing the transmission drive voltage is realized, for example, by a linear amplifier type oscillator circuit that can switch its value instantaneously, or by a mechanism that electrically switches multiple power supply units.

[0018] Here, when 3D scanning, that is, volume scanning, is performed, as the ultrasonic probe 20, a 2D array probe equipped with scanning methods such as linear type, convex type, and sector type is used. Or, when volume scanning is performed, as the ultrasonic probe 20, a 1D probe equipped with scanning methods such as linear type and convex type and having a mechanism that mechanically swings in the elevation direction is used. The latter probe is also called a mechanical 4D probe.

[0019] The image memory 13 has, for example, a magnetic or optical recording medium, or a recording medium readable by a processor such as a semiconductor memory. The image memory 13 stores a plurality of ultrasonic images under the control of the processing circuit 15.

[0020] The network interface 14 implements various information communication protocols according to the form of the network. Also, the network interface 14 may implement various protocols for non-contact wireless communication.

[0021] The processing circuit 15 means a dedicated or general-purpose CPU (Central Processing Unit), MPU (Micro Processor unit), or GPU (Graphics Processing Unit), as well as an ASIC and a programmable logic device, etc.

[0022] The main memory 16 is composed of semiconductor memory elements such as RAM (Random Access Memory) and flash memory, a hard disk, an optical disk, etc. The main memory 16 may be composed of portable media such as a USB (Universal Serial Bus) memory and a DVD (Digital Video Disk). The main memory 16 stores various processing programs (including application programs and an OS (Operating System), etc.) used in the processing circuit 15 and data necessary for the execution of the programs.

[0023] Figure 3 is a block diagram showing the functions of the ultrasound diagnostic device 1. The processing circuit 15 is a processor that, by calling and executing a program in the main memory 16, realizes the system control function 151, the imaging control function 152, the image processing function 153, the memory control function 154, the display control function 155, the feature extraction function 156, the angle estimation function 157, and the reference angle setting function 158, as shown in Figure 3. Alternatively, the processing circuit 15 may be configured by combining multiple independent processors, with each processor executing a program to realize each function.

[0024] The system control function 151 includes, for example, a function that stores command signals input by the operator from the input interface 30, as well as information such as various initial setting conditions, and then transmits this information to each processing function of the processing circuit 15.

[0025] The imaging control function 152 includes, for example, a function that reads information from the system control function 151 and controls the transmission and reception of ultrasonic waves.

[0026] The image processing function 153 includes a function to read image data stored in the image memory 13, perform image processing on the image data, and then store the processed image data back into the image memory 13. The image processing function 153 may also read three-dimensional medical image data (hereinafter referred to as "voxel data") from multiple time phases from the image memory 13.

[0027] The memory control function 154 includes the function of storing various data in the main memory 16 or reading data from the main memory 16. The memory control function 154 may also read voxel data from multiple time phases from the main memory 16.

[0028] The display control function 155 includes, for example, a function to read signals from the system control function 151, acquire desired ultrasound image data from the image memory 13, and display it on the display 40. The display control function 155 may also display on the display 40 voxel data in which the angle of the hollow region included in the voxel data has been rotated to a reference angle.

[0029] The feature extraction function 156 includes a function to extract temporal feature values ​​for each pixel contained in voxel data from voxel data at multiple time phases read from the image memory 13 or main memory 16 by the image processing function 153 or the memory control function 154.

[0030] The angle estimation function 157 includes a function to estimate the angle of the fluctuating region in voxel data across multiple time phases, based on the temporal feature values ​​extracted by the feature extraction function 156.

[0031] The reference angle setting function 158 includes a function for setting the reference angle of voxel data in multiple time phases.

[0032] Figure 4 is a flowchart showing the processing of the ultrasound diagnostic apparatus 1 according to the first embodiment. Figure 5 is a schematic diagram showing voxel data, feature images, and binarized images according to the first embodiment. Figure 6(a) is a diagram showing an example of the original image according to the first embodiment. Figure 6(b) is a diagram showing an example of the original image according to the first embodiment rotated to a reference angle.

[0033] The fluctuating region represents, for example, the movement of the mitral valve, tricuspid valve, or the heart. In the voxel data, the movement of the fluctuating region differs from the movement of other regions. Therefore, it is expected that in each voxel data captured at different times, the change in pixel values ​​in the fluctuating region will be large, while the change in pixel values ​​outside the fluctuating region will be small or no change at all. The angle estimation process according to the first embodiment will be described below with reference to Figure 4, and then to Figures 5 and 6.

[0034] In step S1, the image processing function 153 or the memory control function 154 acquires voxel data (i.e., 4-dimensional image data) at multiple time phases from the image memory 13 or the main memory 16. If the image processing function 153 acquires voxel data from a source other than the image memory 13, it stores the voxel data in the image memory 13 for preservation. The left side of Figure 5 schematically shows an example of voxel data. Each cubic voxel data contains, for example, 3D data of a heart.

[0035] In step S2, the feature extraction function 156 extracts the temporal feature values ​​of each pixel contained in each voxel data from the voxel data at multiple time phases obtained in step S1. The feature extraction function 156 may also calculate statistical values ​​related to the brightness value of each pixel at the same position (coordinate) in each voxel data at multiple time phases as temporal feature values. Statistical values ​​include the mean, variance, and standard deviation.

[0036] In step S3, the image processing function 153 generates a feature image using the temporal feature values ​​of each pixel extracted in step S2. The feature image is an image in which the temporal features of each pixel are reflected in that pixel. For example, the feature image is an image in which the pixel data of the color indicated by the brightness value, which is the temporal feature value of each pixel, is placed at the position (coordinates) of each pixel in the voxel data. Therefore, when the temporal feature values ​​of each pixel in the voxel data are reflected in that pixel, the features of each pixel become apparent. An example of a 3D feature image is schematically shown in the center of Figure 5.

[0037] In step S4, the image processing function 153 binarizes the feature image generated in step S3 using a predetermined threshold, selecting pixels corresponding to temporal feature values ​​(for example, statistical values ​​related to the brightness value of each pixel). An example of a binarized image is schematically shown on the right side of Figure 5. The image processing function 153, for example, makes pixels corresponding to statistical values ​​above the threshold white and pixels corresponding to statistical values ​​below the threshold black.

[0038] Since the time-averaged brightness values ​​of pixels differ between the variable region and the non-variable region, they can be binarized using an appropriate threshold.

[0039] Furthermore, it is assumed that the variation in pixel brightness values ​​is large in the variable region and small in the non-variable region. Therefore, when using variance or standard deviation, which indicates the degree of data variation, as a statistical value, the region corresponding to statistical values ​​above the threshold is the variable region, and the region corresponding to statistical values ​​below the threshold is the non-variable region. In this case, in a binarized image, the variable region will appear white, and the non-variable region will appear black.

[0040] In step S5, the angle estimation function 157 estimates the angle of the hollow region in the binarized feature image as the angle of the variable region. More specifically, the angle estimation function 157 estimates the angle of the hollow region based on the image data binarized in step S4. By binarizing each pixel corresponding to the temporal feature value using a predetermined threshold, the shape of the hollow region becomes apparent, making it possible to estimate the angle of the hollow region.

[0041] The angle estimation function 157 estimates the angle of a hollow region based on the geometric features of the hollow region, which change regularly over time in the voxel data. A regularly changing hollow region is, for example, a region where valve movement or intestinal peristalsis occurs. Geometric features are an example of a specific part. Figure 6(a) shows the original image of the mitral valve. The upper part of Figure 6(a) shows the axial plane images of cases 1 and 2, and the lower part of Figure 6(a) shows the sagittal plane images of cases 1 and 2. In order to clearly show the changes before and after rotation processing, line segments may be drawn connecting points located in the middle of ranges that fluctuate over multiple time phases, or line segments may be drawn connecting points that do not fluctuate over multiple time phases.

[0042] In step S6, the reference angle setting function 158 sets the reference angle of the hollow region in the voxel data. The reference angle setting function 158 sets the reference angle to a predetermined direction in the reference coordinate system of the voxel data (e.g., the X-axis direction, the Z-axis direction, etc.). Figure 6(b) shows the original image of the mitral valve rotated to the reference angle. The upper part of Figure 6(b) shows the axial plane images of cases 1 and 2, and the lower part of Figure 6(b) shows the sagittal plane images of cases 1 and 2. The reference angle setting function 158 may also set the reference angle to an angle specified by the user via the input interface 30.

[0043] In step S7, the image processing function 153 rotates the angle of the hollow region (e.g., mitral valve) of the voxel data to the reference angle set in step S6, and stores the voxel data in the image memory 13.

[0044] In step S8, the display control function 155 reads the original voxel data acquired in step S1 and the voxel data obtained in step S7 by rotating the angle of the hollow region to the reference angle from the image memory 13 and displays them on the display 40.

[0045] As described above, instead of recognizing the object itself, such as a valve, from the ultrasound image and estimating the angle of that object, the angle of the feature portion extracted from the ultrasound image is estimated, allowing for real-time processing. Furthermore, compared to conventional machine learning-based methods, the computational load is significantly reduced because prior model training and object recognition are not required. And it can be used as a general-purpose method rather than a specialized method for specific objects.

[0046] According to at least one embodiment described above, the angle of an object in medical image data can be estimated without performing a large amount of calculations.

[0047] Note that the image processing function 153 and the memory control function 154 are examples of the acquisition unit. The feature extraction function 156 is an example of the extraction unit. The angle estimation function 157 is an example of the estimation unit. The reference angle setting function 158 is an example of the setting unit.

[0048] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0049] 1… Ultrasound diagnostic equipment 10… Ultrasonic imaging device 13…Image memory 15…Processing circuit 16…Main memory 40…Display 153…Image processing function 154...Memory control function 155…Display control function 156...Feature extraction function 157...Angle estimation function 158...Reference angle setting function P...Subject

Claims

1. An acquisition unit that acquires three-dimensional medical image data at multiple time phases, An extraction unit that extracts the temporal characteristic value of each pixel contained in the medical image data from the medical image data at the aforementioned multiple time phases, An estimation unit that estimates the angle of the fluctuating region in the medical image data based on the aforementioned temporal feature values, A medical image processing device equipped with [a specific feature].

2. The estimation unit estimates the angle of the hollow region in the medical image data as the angle of the fluctuating region based on the temporal feature value. The medical image processing apparatus according to claim 1.

3. A setting unit for setting the reference angle of the aforementioned medical image data, A display control unit that displays the medical image data obtained by rotating the angle of the hollow region to the reference angle on the display unit, The medical image processing apparatus according to claim 2, further comprising the above.

4. The estimation unit estimates the angle of the hollow region based on a specific portion of the hollow region that changes regularly over time in the medical image data. The medical image processing apparatus according to claim 2.

5. The extraction unit calculates statistical values ​​relating to the brightness values ​​of each pixel included in the medical image data at multiple time phases as the temporal feature values. The estimation unit binarizes the pixels corresponding to the statistical values ​​using a predetermined threshold, and estimates the angle of the hollow region based on the image data obtained by binarizing the pixels. The medical image processing apparatus according to claim 2.

6. The aforementioned variable region indicates the movement of the mitral valve, tricuspid valve, or heart. The medical image processing apparatus according to claim 1.

Images