Image processing device for evaluating cardiac images and ventricular state recognition method

Abstract

The present invention provides an image processing device for evaluating cardiac images and a method for identifying ventricular states. In this method, regions of interest are identified from multiple target images, grayscale value changes of multiple pixels in the regions of interest in those target images are determined, and one or more representative images are obtained based on the grayscale value changes. The target images are pixels within the endocardial contour of the left ventricle, with the boundaries of the region of interest roughly located on both sides of the bottom of the endocardial contour, and the region of interest corresponds to the mitral valve. Grayscale value changes are related to the movement of the mitral valve. The representative images are used to evaluate the state of the left ventricle. In this way, the end-systole and end-diastole of the heart can be quickly and accurately identified.

Description

Technical Field

[0001] The present invention relates to an image recognition technology, and in particular to an image processing device for evaluating cardiac images and a ventricular state recognition method. Background Art

[0002] Clinically, there are many methods for assessing cardiac health. One method involves measuring the left ventricular ejection fraction (LVEF), which measures the volume of blood ejected from the left ventricle with each heartbeat. It is important to note that measuring LVEF requires reference to the end-systolic volume (ESV) and the end-diastolic volume (EDV). Therefore, it is necessary to identify the end-systolic (ES) and end-diastolic (ED) phases during the heart rate cycle in order to calculate EDV and ESV using the Simpson formula. Current technology can identify ED based on the R wave in the electrocardiogram (ECG), when the left ventricular volume is at its maximum. Additionally, ES can be identified at the end of the T wave on the ECG, when the left ventricle volume is at its minimum. However, identifying ES on the ECG is not easy. Summary of the Invention

[0003] Clinically, there are many methods for assessing cardiac health. One method involves measuring the left ventricular ejection fraction (LVEF), which measures the volume of blood ejected from the left ventricle with each heartbeat. It is important to note that measuring LVEF requires reference to the end-systolic volume (ESV) and the end-diastolic volume (EDV). Therefore, it is necessary to identify the end-systolic (ES) and end-diastolic (ED) phases during the heart rate cycle in order to calculate EDV and ESV using the Simpson formula. Current technology can identify ED based on the R wave in the electrocardiogram (ECG), when the left ventricular volume is at its maximum. Additionally, ES can be identified at the end of the T wave on the ECG, when the left ventricle volume is at its minimum. However, identifying ES on the ECG is not easy. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The accompanying drawings are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the present invention and together with the description serve to explain the principles of the present invention.

[0005] Figure 1 is a block diagram of components of an image processing device according to an embodiment of the present invention;

[0006] Figure 2 is a flow chart of a ventricular state identification method according to an embodiment of the present invention;

[0007] Figure 3 is a schematic diagram of left ventricular cutting according to an embodiment of the present invention;

[0008] Figure 4A is a schematic diagram of the end diastole (ED) of the left ventricle according to one embodiment of the present invention;

[0009] Figure 4B is a schematic diagram of the left ventricle from end-diastole to end-systole (ES) according to one embodiment of the present invention;

[0010] Figure 4C FIG. 1 is a schematic diagram of the end-systole of the left ventricle according to an embodiment of the present invention.

[0011] Explanation of Figure Numbers

[0012] 100: image processing device;

[0013] 110: memory;

[0014] 130: processor;

[0015] S210~S250: step;

[0016] ROI: region of interest;

[0017] 401: anterior lobe;

[0018] 403: posterior lobe. DETAILED DESCRIPTION

[0019] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0020] Figure 1 is a block diagram of components of an image processing apparatus 100 according to an embodiment of the present invention. Figure 1The image processing device 100 includes (but is not limited to) a memory 110 and a processor 130. The image processing device 100 can be a desktop computer, a laptop computer, a smart phone, a tablet computer, a server, a medical detection instrument, or other computing devices.

[0021] The memory 110 may be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, a hard disk drive (HDD), a solid-state drive (SSD), or the like. In one embodiment, the memory 110 is used to store program code, software modules, configurations, data (e.g., images, grayscale values, statistics, status, or volume), or files, and an embodiment thereof will be described in detail below.

[0022] The processor 130 is coupled to the memory 110 and may be a central processing unit (CPU), a graphics processing unit (GPU), or other programmable general-purpose or special-purpose microprocessor, a digital signal processor (DSP), a programmable logic controller (PLC), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a neural network accelerator, or other similar components or a combination thereof. In one embodiment, the processor 130 is configured to perform all or part of the operations of the image processing device 100 and may load and execute various program codes, software modules, files, and data stored in the memory 110.

[0023] Hereinafter, the method according to the embodiment of the present invention will be described with reference to various devices, components, and modules in the image processing device 100. The various processes of the method can be adjusted according to the implementation situation and are not limited thereto.

[0024] Figure 2 This is a flow chart of a ventricular state recognition method according to an embodiment of the present invention. Figure 2, the processor 130 can determine the region of interest (ROI) from multiple target images (step S210). Specifically, those target images are pixels within the endocardial contour of the left ventricle. In one embodiment, the processor 130 can obtain one or several cardiac cycles and continuous frames of A4C (Apical Four Chamber View) or A2C (Apical Two Chamber View) cardiac ultrasound images (or cardiac ultrasound images (Echocardiography). The source of these continuous frames of cardiac ultrasound images can be a film shot of the heart that is divided into continuous frames of cardiac images or a plurality of frames of cardiac images detected by an ultrasound probe and transmitted wirelessly or wired. In addition, continuous images refer to multiple cardiac images that are captured by an ultrasound probe at a specific frame rate and are sorted according to time.

[0025] In one embodiment, the processor 130 may input the A4C or A2C cardiac ultrasound image into a machine learning model (e.g., based on deep learning, a multi-layer perceptron (MLP), or a support vector machine (SVM)) to segment the image within the endocardial contour of the left ventricle and output the image as the endocardial image of the left ventricle. In other embodiments, the processor 130 may identify the endocardial contour based on scale-invariant feature transform (SIFT), Haar features, Adaboost, or other recognition techniques, and segment the target image accordingly.

[0026] Figure 3 This is a schematic diagram of left ventricular cutting according to an embodiment of the present invention. Figure 3 The left figure shows the cardiac ultrasound image of A2C, which can be identified to obtain the endocardial contour of the left ventricle as shown in the right figure. The processor 130 can segment the target image from the cardiac ultrasound image based on the endocardial contour. It should be noted that Figure 3 The image segmentation of a single frame is shown, and the same or similar methods can be used for the segmentation of other frames.

[0027] In some embodiments, the processor 130 may also directly obtain the target image that has been segmented from the cardiac image, that is, other devices or medical instruments segment the target image first.

[0028] In one embodiment, the boundaries of the region of interest are approximately located on both sides of the bottom of the endocardial contour, and the region of interest corresponds to the mitral valve. The bottom refers to the lower half of the endocardium, or a portion of other proportions or ranges.

[0029] In one embodiment, the processor 130 may determine the region of interest (ROI) based on a binary image of the endocardial contour. This binary image contains only two grayscale values ​​(e.g., completely black and completely white). Therefore, the processor 130 may calculate the coordinate positions of the first left and right pixels, starting from the base of the left ventricular endocardium (i.e., corresponding to the base). The positions of these two pixels are the left and right boundaries of the ROI. These boundaries are approximately located at the leftmost and rightmost edges of the anterior and posterior leaflets of the mitral valve in A4C or A2C views.

[0030] In one embodiment, the thickness of the region of interest (i.e., the vertical width in the A4C or A2C view, or the apical and basal borders) can be the thickness of the anterior or posterior leaflet of the mitral valve. It should be noted that the thickness may vary among individuals due to factors such as age and gender. For example, the thickness of the anterior leaflet is 1.3 millimeters (mm) for individuals under 20 years old, 1.6 mm for those between 20 and 56 years old, and 3.2 mm for those over 60 years old.

[0031] Figure 4A is a schematic diagram of the left ventricular end-diastole (ED) according to one embodiment of the present invention. Figure 4A The region of interest (ROI) shown in the figure roughly corresponds to the anterior lobe 401 and the posterior lobe 403. The left boundary of the region of interest (ROI) corresponds to the leftmost side of the anterior lobe 401, and the left boundary of the region of interest (ROI) corresponds to the rightmost side of the posterior lobe 403. The thickness of the region of interest (ROI) is slightly greater than or approximately equal to the thickness of the anterior lobe 401 and the posterior lobe 403.

[0032] It should be noted that the shape of the region of interest ROI is not limited to Figure 4A The region of interest is shown as a rectangle in FIG. , but in other embodiments, the region of interest may be a geometric shape such as an ellipse, a diamond, or an irregular shape corresponding to the contour of the mitral valve.

[0033] In another embodiment, the processor 130 may determine the boundaries of the region of interest based on image recognition technology (e.g., neural network or feature matching). For example, the processor 130 may identify the region of interest based on a classifier derived from training samples with labeled mitral valve positions or based on image features of the mitral valve, where the region of interest substantially corresponds to the outline of the mitral valve when fully open.

[0034] The processor 130 can determine the grayscale value changes of multiple pixels in the region of interest in those target images (step S230). Specifically, the grayscale value changes are related to the movement of the mitral valve. When the heart is in contraction and relaxation, its mitral valve will open and close. In the heart image of consecutive frames, the process of the mitral valve being fully opened to fully closed (or fully closed to fully open) will be presented. It is worth noting that the cardiac ultrasound image showing A4C or A2C usually uses B-mode. Therefore, when the ultrasound probe emits the sound wave, the intensity of the echo reflected back can be represented by the brightness of the point. The image composed of these points (as pixels) is a grayscale image. In a grayscale image, the intensity of each pixel (i.e., grayscale value) is a value between 0 and 255, where the value 0 represents the blackest and the value 255 represents the whitest. As Figure 4A As shown, the mitral valve (composed of the anterior leaflet 401 and the posterior leaflet 403 ) appears generally white.

[0035] From a ROI perspective (i.e., analyzing or viewing images within the ROI only), the movement of the mitral valve will cause different areas of the mitral valve to appear within the ROI. For example, as the mitral valve moves from fully open to fully closed, portions of the anterior or posterior leaflet will gradually move away from the ROI.

[0036] In one embodiment, the processor 130 may determine the sum of the grayscale values ​​of the pixels in the region of interest in each target image. It is worth noting that the area of ​​the mitral valve included in the region of interest in different target images may be different. The mitral valve in the grayscale image is roughly white (i.e., the grayscale value is higher), and its surroundings are roughly black (i.e., the grayscale value is lower). The sum of the grayscale values ​​can reflect the area of ​​the mitral valve included in the region of interest. The processor 130 can determine the difference in the sum between those target images (i.e., the change in the area of ​​the mitral valve included in the region of interest). And this difference corresponds to the aforementioned change in grayscale value. That is, the difference in the sum of the grayscale values ​​reflects the movement of the mitral valve.

[0037] For example, Figure 4B is a schematic diagram of the left ventricle from end-diastole to end-systole according to one embodiment of the present invention, and Figure 4C is a schematic diagram of the left ventricular end-systole according to an embodiment of the present invention. Figures 4A to 4C During the process of cardiac contraction and relaxation, the area of ​​the anterior leaflet 401 and the area of ​​the posterior leaflet 403 included in the region of interest ROI may be different.

[0038] In some embodiments, the processor 130 may enhance the contrast of the target image through histogram equalization or other image processing, and then determine the total grayscale value of the pixels.

[0039] In one embodiment, the processor 130 may determine the maximum and minimum sums in those target images. The difference in the aforementioned sums is related to the maximum and minimum. It is worth noting that, clinically, when the mitral valve is completely closed, it is the end diastole (ED) of the heart, and when the mitral valve is fully open, it is the end systole (ES) of the heart. The minimum sum of grayscale values ​​in the region of interest corresponds to the state where the mitral valve is completely closed, that is, corresponds to the end diastole of the left ventricle. The maximum sum corresponds to the state where the mitral valve is fully open, that is, corresponds to the end systole of the left ventricle. The processor 130 may use the maximum and minimum sums of grayscale values ​​as representatives of grayscale value changes. That is, the processor 130 uses the end diastole and the end systole as representatives.

[0040] In some embodiments, the processor 130 may calculate the average of the sum of grayscale values ​​or the number of maximum grayscale values ​​and use this to determine the maximum and minimum values ​​in the target images. For example, the average of the sum corresponding to the end-diastole is the smallest, while the average of the sum corresponding to the end-systole is the largest. Alternatively, the number of maximum grayscale values ​​corresponding to the end-diastole is the smallest, while the number corresponding to the end-systole is the largest.

[0041] The processor 130 can obtain one or more representative images according to the grayscale value change (step S250). Specifically, these representative images can be used to evaluate the state of the left ventricle. Clinically, the volume of the left ventricle is the largest at the end of diastole (e.g., Figure 4A ), while the volume of the left ventricle at the end of systole is the smallest (as shown in Figure 4C In one embodiment, the processor 130 may obtain the target image with the maximum and the target image with the minimum sum of the grayscale values ​​as representative images. That is, these two representative images correspond to the end-systolic and end-diastolic states, respectively.

[0042] As can be seen, embodiments of the present invention can determine whether a target image represents end-systole or end-diastole based on the motion changes of the mitral valve within the region of interest. Compared to analyzing the septal annulus, embodiments of the present invention can quickly identify end-systole or end-diastole. For example, within a single heart rate cycle, embodiments of the present invention can identify end-systole or end-diastole states.

[0043] In one embodiment, the processor 130 may calculate the volume of the left ventricle in each representative image and determine the volume change of the left ventricle between the largest and smallest images (i.e., the difference in volume between the largest and smallest images). For example, the processor 130 may calculate the area occupied by pixels within the endocardial contour of the left ventricle in the representative image. The volume change is used to assess blood ejection volume. For example, the processor 130 may determine the left ventricular ejection fraction (LVEF) based on the end-systolic volume (ESV) and the end-diastolic volume (EDV).

[0044] In other embodiments, according to different requirements, the processor 130 may also obtain target images of the ventricle in other states as representative images based on different grayscale value sums.

[0045] It should be noted that embodiments of the present invention can identify the end-systole or end-diastole of the heart from cardiac images in real-time or non-real-time. In one embodiment, in real-time, the ultrasound probe transmits several frames of cardiac images, so the processor 130 may accumulate images spanning several cardiac cycles before identifying the end-systole or end-diastole. In another embodiment, in non-real-time, the processor 130 may segment the A4C or A2C cardiac ultrasound film into several cardiac images, extract the cardiac images spanning several cardiac cycles, and then further identify the end-systole or end-diastole.

[0046] In summary, in the image processing device and ventricular state identification method for evaluating cardiac images according to embodiments of the present invention, grayscale value changes of pixels within a region of interest (ROI) in an image are analyzed to determine the motion of the mitral valve within that region of interest. Furthermore, embodiments of the present invention can identify the cardiac state at end-systole or end-diastole based on the sum of the pixel grayscale values, and use this to assess blood ejection volume. This allows for rapid and accurate identification of the different cardiac states during systole and diastole.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A ventricular state recognition method, characterized in that: include: determining a region of interest from a plurality of target images, wherein the target image is pixels within an endocardial contour of the left ventricle, boundaries of the region of interest are approximately located on both sides of a bottom of the endocardial contour, and the region of interest corresponds to a contour of a mitral valve in a fully open state; Determining a change in grayscale values ​​of a plurality of pixels in the region of interest in the target image, wherein the change in grayscale values ​​is related to movement of the mitral valve, the step of determining the change in grayscale values ​​of the pixels in the region of interest in the target image comprising: determining a sum of the grayscale values ​​of the pixels in the region of interest, wherein the sum of the grayscale values ​​of the pixels reflects an area of ​​the region of interest including the mitral valve; and determining a difference in the sums between the target images, wherein the difference is related to a maximum and a minimum of the sums in the target images, the minimum of the sums of the grayscale values ​​of the pixels corresponds to a state in which the mitral valve is fully closed and corresponds to end-diastole of the left ventricle, and the maximum of the sums of the grayscale values ​​of the pixels corresponds to a state in which the mitral valve is fully open and corresponds to end-systole of the left ventricle; and At least one representative image is obtained according to the grayscale value change, wherein the representative image is used to evaluate the state of the left ventricle.

2. The ventricular state recognition method according to claim 1, characterized in that: The step of determining the difference in the sum between the target images comprises: The maximum and the minimum are used as representatives of the grayscale value change.

3. The ventricular state recognition method according to claim 2, characterized in that: The step of obtaining the representative image according to the grayscale value change includes: The largest target image and the smallest target image are obtained as the representative images.

4. The ventricular state recognition method according to claim 3, characterized in that: Also includes: A volume change of the left ventricle between the maximum and the minimum is determined, wherein the volume change is used to estimate blood ejection volume.

5. An image processing apparatus for evaluating a cardiac image, comprising: a memory for storing a plurality of program codes; as well as a processor coupled to the memory, wherein the processor is configured to load and execute the program code to: determining a region of interest from a plurality of target images, wherein the target image is pixels within an endocardial contour of the left ventricle, boundaries of the region of interest are approximately located on both sides of a bottom of the endocardial contour, and the region of interest corresponds to a contour of a mitral valve in a fully open state; determining grayscale value changes of a plurality of pixels in the region of interest in the target image, wherein the grayscale value changes are related to movement of the mitral valve, the processor being further configured to: determining a sum of the grayscale values ​​of the pixels in the region of interest, wherein the sum of the grayscale values ​​of the pixels reflects an area of ​​the region of interest including the mitral valve; and determining a difference in the sums between the target images, wherein the difference is related to a maximum and a minimum of the sums in the target images, the minimum of the sums of the grayscale values ​​of the pixels corresponds to a fully closed state of the mitral valve and to end-diastole of the left ventricle, and the maximum of the sums of the grayscale values ​​of the pixels corresponds to a fully open state of the mitral valve and to end-systole of the left ventricle; as well as At least one representative image is obtained according to the grayscale value change, wherein the representative image is used to evaluate the state of the left ventricle.

6. The image processing device for evaluating cardiac images according to claim 5, characterized in that The processor is further configured to: The maximum and the minimum are used as representatives of the grayscale value change.

7. The image processing device for evaluating cardiac images according to claim 6, characterized in that The processor is further configured to: The largest target image and the smallest target image are obtained as the representative images.

8. The image processing device for evaluating cardiac images according to claim 7, characterized in that The processor is further configured to: A volume change of the left ventricle between the maximum and the minimum is determined, wherein the volume change is used to estimate blood ejection volume.

Images