Methods, electronic devices, and storage media for obtaining heart valve parameters

Abstract

This invention provides a method, electronic device, and storage medium for acquiring heart valve parameters. The method includes: acquiring location information of a region of interest (ROI) of a target heart valve based on a current frame of an echocardiogram; segmenting the ROI of the target heart valve corresponding to the current frame of the echocardiogram using a first segmentation model based on the location information of the ROI to obtain an inner contour mask image of the target heart valve in the current frame; and acquiring the opening area corresponding to the current frame of the echocardiogram based on the inner contour mask image of the target heart valve in the current frame. This invention enables an end-to-end algorithm flow, automatically calculates heart valve parameters, and effectively reduces variability issues that may be caused by human factors.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for acquiring heart valve parameters, an electronic device, and a storage medium. Background Technology

[0002] Aortic valve disease is a common and dangerous cardiovascular disease that seriously endangers human health. Echocardiography is the preferred method for diagnosing and assessing the severity of target valvular heart disease. Measurement of the target valvular opening area, the number and morphology of the aortic valve leaflets are important tools for analyzing cardiac function.

[0003] Currently, the measurement of the target valvular opening area mainly utilizes acquired echocardiographic images. Specially trained medical personnel or specialists analyze and process the cardiac images to determine the image frames for delineating the opening area, which is then manually delineated. Manual delineation requires a high level of expertise and experience and is time-consuming. Therefore, in-depth analysis of aortic valve structural abnormalities using medical imaging is of great significance for the prevention and diagnosis of valvular heart disease.

[0004] It should be noted that the information disclosed in the background section of this invention is intended only to enhance the understanding of the general background of this invention, and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method, electronic device and storage medium for acquiring heart valve parameters, which can automatically identify, segment and calculate the opening area of ​​the target heart valve. The algorithm has high overall accuracy and can effectively reduce the cumbersome operation of human-computer interaction.

[0006] To achieve the above objectives, the present invention provides a method for obtaining heart valve parameters, comprising:

[0007] Based on the current frame of the cardiac image, obtain the location information of the region of interest of the target heart valve;

[0008] Based on the location information of the target heart valve region of interest, the first segmentation model is used to segment the target heart valve region of interest corresponding to the current frame cardiac image to obtain the inner contour mask image of the target heart valve in the current frame.

[0009] Based on the target heart valve inner contour mask image of the current frame, obtain the opening area corresponding to the current frame cardiac image.

[0010] Optionally, the step of segmenting the target heart valve region of interest corresponding to the current frame echocardiogram using a first segmentation model based on the location information of the target heart valve region of interest includes:

[0011] Based on the location information of the target heart valve region of interest, the corresponding region is cropped from the current frame of the cardiac image to obtain the current frame image of the target heart valve region of interest.

[0012] The first segmentation model is used to segment the region of interest image of the target heart valve in the current frame.

[0013] Optionally, before segmenting the current frame target heart valve region of interest image using the first segmentation model, the method further includes:

[0014] Adjust the size of the current frame target heart valve region of interest image to a preset size.

[0015] Optionally, adjusting the size of the current frame target heart valve region of interest image to a preset size includes:

[0016] The target side length is defined by the length dimension of the current frame's target heart valve region of interest image.

[0017] The current frame target heart valve region of interest image is filled along the width direction to adjust the width dimension of the current frame target heart valve region of interest image to the target side length dimension;

[0018] The current frame target heart valve region of interest image, whose width dimension is adjusted to the target side length dimension, is enlarged or reduced to adjust the size of the current frame target heart valve region of interest image to a preset size.

[0019] Optionally, obtaining the opening area corresponding to the current frame echocardiogram based on the target heart valve inner contour mask image of the current frame includes:

[0020] Connected component analysis is performed on the inner contour mask image of the target heart valve in the current frame to obtain the opening area of ​​the target heart valve corresponding to the cardiac image in the current frame.

[0021] Optionally, performing connected component analysis on the target heart valve inner contour mask image of the current frame to obtain the target heart valve opening area corresponding to the current frame echocardiogram includes:

[0022] Perform connected component analysis on the current frame target heart valve inner contour mask image to obtain the maximum connected component;

[0023] The pixel area of ​​the largest connected component is obtained based on the number of pixels in the largest connected component.

[0024] Based on the pixel area of ​​the largest connected region and the correspondence between the pre-obtained pixel area and the actual area, the target heart valve opening area corresponding to the current frame cardiac image is obtained.

[0025] Optionally, the method further includes:

[0026] Based on the temporal sequence corresponding to each frame of the cardiac image and the target heart valve opening area, an opening area diagram is drawn to represent the correspondence between the frame number and the opening area.

[0027] Based on the opening area diagram, the final target heart valve opening area is obtained.

[0028] Optionally, obtaining the final target heart valve opening area based on the opening area map includes:

[0029] The opening area map is filtered to obtain an opening area curve.

[0030] Based on the opening area curve, determine the peak opening area for each cardiac cycle;

[0031] The average of all peak opening areas is taken as the final target heart valve opening area.

[0032] Optionally, the step of filtering the opening area map to obtain an opening area curve includes:

[0033] The opening area map is transformed from the time domain to the frequency domain to obtain the frequency amplitude map;

[0034] The frequency amplitude graph is filtered to remove the portion of the frequency amplitude graph whose frequency is higher than a preset threshold.

[0035] The filtered frequency amplitude graph is converted from the frequency domain to the time domain to obtain the aperture area curve.

[0036] Optionally, the method further includes:

[0037] Based on the location information of the target heart valve region of interest, the first segmentation model is used to segment the target heart valve region of interest corresponding to the current frame cardiac image to obtain the outer contour mask image of the target heart valve in the current frame.

[0038] Based on the inner contour mask image of the target heart valve in the current frame and the outer contour mask image of the target heart valve in the current frame, the position information of the target heart valve corresponding to the current frame cardiac image is obtained.

[0039] Optionally, after obtaining the location information of the region of interest of the target heart valve based on the current frame of the echocardiogram, the method further includes:

[0040] Based on the location information of the target heart valve region of interest, a second segmentation model is used to segment the target heart valve region of interest corresponding to the current frame cardiac image to obtain the target heart valve leaflet mask image of the current frame.

[0041] Optionally, obtaining the location information of the target heart valve region of interest based on the current frame of the echocardiogram includes:

[0042] A target detection model is used to detect the current frame of the cardiac image to obtain the location information of the target heart valve region of interest.

[0043] Optionally, the step of using a target detection model to detect the current frame of the cardiac image to obtain the location information of the target heart valve region of interest includes:

[0044] A target detection model is used to detect the current frame of the cardiac image to obtain candidate location information of the target heart valve region of interest;

[0045] Based on the candidate location information of the target heart valve region of interest, calculate the location information of the target heart valve region of interest after magnification by a preset factor.

[0046] The location information of the target heart valve region of interest after being magnified by a preset factor is used as the location information of the target heart valve region of interest.

[0047] Optionally, after using a target detection model to detect the acquired current frame of the cardiac image to obtain the location information of the target heart valve region of interest, the method further includes:

[0048] Based on the temporal sequence corresponding to the current frame of the cardiac image, the location information of the region of interest of the target heart valve is corrected.

[0049] To achieve the above objectives, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the heart valve parameter acquisition method described above.

[0050] To achieve the above objectives, the present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the heart valve parameter acquisition method described above.

[0051] Compared with existing technologies, the heart valve parameter acquisition method, electronic device, and storage medium provided by this invention have the following advantages: This invention first obtains the location information of the region of interest (ROI) of the target heart valve corresponding to the current frame of the echocardiogram; then, based on the location information of the ROI, a first segmentation model is used to segment the corresponding region in the current frame of the echocardiogram to obtain the inner contour mask image of the target heart valve in the current frame; finally, based on the inner contour mask image of the target heart valve in the current frame, the opening area corresponding to the current frame of the echocardiogram can be obtained. Therefore, this invention can achieve an end-to-end algorithm flow, not only automatically calculating heart valve parameters and effectively reducing the variability problems that may be caused by human factors, but also better assisting doctors in improving diagnostic accuracy and providing a basis for the analysis of heart valve abnormalities, while also reducing the cumbersome operation of human-computer interaction. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a method for obtaining heart valve parameters according to one embodiment of the present invention.

[0053] Figure 2a This is a heartbeat image from a specific example of the present invention;

[0054] Figure 2b From Figure 2a A region of interest image of the target heart valve (aortic valve) cropped from a cardiac image;

[0055] Figure 2c To Figure 2b Image of the region of interest of the target heart valve (aortic valve) after filling;

[0056] Figure 3a This is a region of interest image of the target heart valve (aortic valve) during ventricular diastole in a specific example of the present invention;

[0057] Figure 3b To Figure 3a The target heart valve (aortic valve) outer contour mask image obtained by segmentation;

[0058] Figure 3c To Figure 3a The inner contour mask image of the target heart valve (aortic valve) obtained by segmentation;

[0059] Figure 3d This is a region of interest image of the target heart valve (aortic valve) during ventricular systole in a specific example of the present invention;

[0060] Figure 3e To Figure 3dThe target heart valve (aortic valve) outer contour mask image obtained by segmentation;

[0061] Figure 3f To Figure 3d The inner contour mask image of the target heart valve (aortic valve) obtained by segmentation;

[0062] Figure 4a This is a diagram showing the opening area in a specific example of the present invention;

[0063] Figure 4b To Figure 4a The frequency amplitude diagram obtained by transforming the aperture area diagram shown;

[0064] Figure 4c To Figure 4b The frequency amplitude diagram shown is obtained by filtering the frequency amplitude diagram.

[0065] Figure 4d To Figure 4c The aperture area curve is obtained by inverse transformation of the frequency amplitude graph after filtering.

[0066] Figure 5 This is a heartbeat image with an outer contour, an inner contour, and an opening area drawn in a specific example of the present invention.

[0067] Figure 6a To Figure 3a The target heart valve (aortic valve) leaflet mask image obtained by segmentation;

[0068] Figure 6b To Figure 3d The target heart valve (aortic valve) leaflet mask image obtained by segmentation;

[0069] Figure 7 This is a schematic diagram of the target detection model in a specific example of the present invention;

[0070] Figure 8a This is a schematic diagram of the structure of the first residual module in a specific example of the present invention;

[0071] Figure 8b This is a schematic diagram of the structure of the second residual module in a specific example of the present invention;

[0072] Figure 8c This is a schematic diagram of the structure of the third residual module in a specific example of the present invention;

[0073] Figure 8d This is a schematic diagram of the structure of the fourth residual module in a specific example of the present invention;

[0074] Figure 9This is a schematic diagram of the structure of a densely connected block in a specific example;

[0075] Figure 10 This is a schematic diagram of the structure of the first segmentation model in a specific example of the present invention;

[0076] Figure 11 This is a schematic diagram of the bottleneck layer in a specific example of the present invention;

[0077] Figure 12 This is a schematic diagram of the transition block in a specific example of the present invention;

[0078] Figure 13 This is a schematic diagram of the upward transition block in a specific example of the present invention;

[0079] Figure 14 This is a block diagram of an electronic device provided according to an embodiment of the present invention.

[0080] The reference numerals in the attached figures are as follows:

[0081] Processor-101; Communication interface-102; Memory-103; Communication bus-104. Detailed Implementation

[0082] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, further illustrates the heart valve parameter acquisition method, electronic device, and storage medium proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clearly illustrate the embodiments of this invention. Please refer to the drawings to make the objectives, features, and advantages of this invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for illustrative purposes and to enable those skilled in the art to understand and read them, and are not intended to limit the implementation conditions of this invention. Any modifications to the structure, changes in proportions, or adjustments to the size, provided that the effects and objectives achieved by this invention are the same or similar, should still fall within the scope of the technical content disclosed in this invention.

[0083] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0084] Furthermore, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0085] The core idea of ​​this invention is to provide a method, electronic device and storage medium for obtaining heart valve parameters, which can automatically realize the identification, segmentation and opening area calculation of the target heart valve. Not only is the overall algorithm highly accurate, but it can also effectively reduce the cumbersome operation of human-computer interaction.

[0086] It should be noted that the method for acquiring heart valve parameters according to the embodiments of the present invention can be applied to the electronic devices described in the embodiments of the present invention. These electronic devices can be personal computers, mobile terminals, etc., and the mobile terminals can be hardware devices with various operating systems, such as mobile phones and tablets. Furthermore, it should be noted that although this article uses echocardiography as an example, as those skilled in the art will understand, the echocardiogram can also be acquired by other devices besides ultrasound (e.g., cardiac endoscopes), and the present invention does not limit this. Additionally, it should be noted that although this article uses the aortic valve as the target heart valve in the description, as those skilled in the art will understand, the target heart valve can also be the mitral valve, tricuspid valve, pulmonary valve, etc., and the present invention does not limit this. It should also be noted that in this article, the long side direction of the image is defined as the length direction, and the short side direction of the image is defined as the width direction.

[0087] To achieve the above-mentioned goals, this invention provides a method for obtaining heart valve parameters. Please refer to [the relevant documentation]. Figure 1 A flowchart illustrating a method for obtaining heart valve parameters according to an embodiment of the present invention is given. Figure 1 As shown, the method for obtaining heart valve parameters includes the following steps:

[0088] Step S100: Obtain the location information of the region of interest of the target heart valve based on the current frame of the cardiac image.

[0089] Step S200: Based on the location information of the target heart valve region of interest, the first segmentation model is used to segment the target heart valve region of interest corresponding to the current frame cardiac image to obtain the inner contour mask image of the target heart valve in the current frame.

[0090] Step S300: Obtain the opening area corresponding to the current frame cardiac image based on the mask image of the inner contour of the target heart valve in the current frame.

[0091] Specifically, the area of ​​the inner contour of the target heart valve in the current frame target heart valve inner contour mask image is the target heart valve opening area corresponding to the current frame echocardiogram image. Therefore, this invention enables an end-to-end algorithm flow, not only automatically calculating heart valve parameters (e.g., heart valve opening area), effectively reducing potential human-induced variability, but also better assisting doctors in improving diagnostic accuracy, providing a basis for heart valve anomaly analysis, effectively reducing the risk of misdiagnosis in existing technologies using echocardiography for heart valve anomaly analysis, and also reducing cumbersome human-computer interaction operations.

[0092] Specifically, the current frame echocardiogram is extracted from the acquired echocardiogram video (each video contains multiple cardiac cycles). The video resolution can be set according to specific circumstances, such as 600×800. The echocardiogram video specifically refers to a PSAX-AV cross-sectional image acquired by an ultrasound device. It should be noted that, as those skilled in the art will understand, the current frame is dynamically changing, meaning the current frame echocardiogram changes over time. After acquiring the heart valve parameters of the current frame echocardiogram, the next frame echocardiogram is extracted as the new current frame echocardiogram to continue acquiring heart valve parameters, until the heart valve parameters of all frames echocardiograms are acquired.

[0093] In one exemplary embodiment, the method further includes:

[0094] Based on the location information of the target heart valve region of interest, the first segmentation model is used to segment the target heart valve region of interest corresponding to the current frame cardiac image to obtain the outer contour mask image of the target heart valve in the current frame.

[0095] Based on the inner contour mask image of the target heart valve in the current frame and the outer contour mask image of the target heart valve in the current frame, the position information of the target heart valve corresponding to the current frame cardiac image is obtained.

[0096] Since the outer contour is the envelope region of the outer edge of the target heart valve, that is, the region between the outer contour and the inner contour is the region where the target heart valve is located, the position information of the target heart valve corresponding to the current frame echocardiogram can be obtained by performing a difference operation between the mask image of the inner contour of the target heart valve in the current frame and the mask image of the outer contour of the target heart valve in the current frame. It should be noted that, as those skilled in the art will understand, the first segmentation model in this invention has two output layers, one output layer is used to output the inner contour image of the target heart valve, and the other output layer is used to output the outer contour image of the target heart valve. That is, by using the first segmentation model in this invention to segment the region of interest of the target heart valve corresponding to the current frame echocardiogram, both the inner contour image and the outer contour image of the target heart valve can be obtained simultaneously.

[0097] In one exemplary implementation, obtaining the location information of the target heart valve region of interest based on the current frame of the echocardiogram includes:

[0098] A target detection model is used to detect the current frame of the cardiac image to obtain the location information of the target heart valve region of interest.

[0099] Further, the step of using a target detection model to detect the current frame of the cardiac image to obtain the location information of the target heart valve region of interest includes:

[0100] A target detection model is used to detect the current frame of the cardiac image to obtain candidate location information of the target heart valve region of interest;

[0101] Based on the candidate location information of the target heart valve region of interest, calculate the location information of the target heart valve region of interest after magnification by a preset factor.

[0102] The location information of the target heart valve region of interest after being magnified by a preset factor is used as the location information of the target heart valve region of interest.

[0103] While object detection models can detect the target heart valve region of interest in the current frame of the echocardiogram, providing preliminary localization for subsequent segmentation models, they also result in the loss of detailed information such as the surrounding tissue of the target heart valve. Therefore, this invention calculates the location information of the target heart valve region of interest after magnification by a preset factor based on the candidate location information of the target heart valve region of interest. Specifically, the original bounding box of the target heart valve region of interest detected by the object detection model is magnified by a preset factor, for example, 1.3 times, to obtain the magnified bounding box. The area defined by this magnified bounding box is the final target heart valve region of interest. Since the area defined by this magnified bounding box includes detailed information such as the surrounding tissue of the target heart valve, the segmentation accuracy of the subsequent segmentation model can be further improved. It should be noted that, as those skilled in the art will understand, the center position of the magnified bounding box is the same as the center position of the original bounding box.

[0104] In one exemplary implementation, after using a target detection model to detect the acquired current frame of the echocardiogram to obtain location information of the target heart valve region of interest, the method further includes:

[0105] Based on the temporal sequence corresponding to the current frame of the cardiac image, the location information of the region of interest of the target heart valve is corrected.

[0106] Therefore, by correcting the location information of the target heart valve region of interest obtained by the target detection model, the accuracy of the target heart valve region of interest extraction can be further improved, thus ensuring the accuracy of subsequent segmentation. Specifically, optical flow can be used to correct the location information of the target heart valve region of interest to obtain the corrected location information. Optical flow is a method that uses the temporal changes of pixels in an image sequence and the correlation between adjacent frames to find the correspondence between the previous frame and the current frame, thereby calculating the motion information of objects between adjacent frames.

[0107] In one exemplary embodiment, the step of segmenting the target heart valve region of interest corresponding to the current frame echocardiogram using a first segmentation model based on the location information of the target heart valve region of interest includes:

[0108] Based on the location information of the target heart valve region of interest, the corresponding region is cropped from the current frame of the cardiac image to obtain the current frame image of the target heart valve region of interest.

[0109] The first segmentation model is used to segment the region of interest image of the target heart valve in the current frame.

[0110] Therefore, by first cropping the target heart valve region of interest from the current frame of the cardiac image to obtain the target heart valve region of interest, and then using the first segmentation model to segment the current frame image of the target heart valve region of interest, the computational load of the segmentation model can be further reduced, thereby further improving computational efficiency. For details, please refer to... Figure 2a and Figure 2b ,in Figure 2a A heartbeat image is shown as an example. Figure 2b The illustration shows from Figure 2a The region of interest image of the target heart valve (aortic valve) cropped from the cardiac image.

[0111] In one exemplary embodiment, before segmenting the current frame target heart valve region of interest image using a first segmentation model, the method further includes:

[0112] Adjust the size of the current frame target heart valve region of interest image to a preset size.

[0113] When the first segmentation model is a neural network model, since neural network models require images of uniform size as input, adjusting the size of the current frame's target heart valve region of interest image to a preset size can meet the input requirements of the first segmentation model. Specifically, the preset size can be set according to specific circumstances. As a preferred embodiment, in the preset size, the length and width dimensions of the image are consistent, that is, the image after adjustment to the preset size is a square image, for example, the preset size is 320*320. Therefore, by setting the length and width dimensions in the preset size to be consistent, it is easier to adjust the size of the current frame's target heart valve region of interest image to the preset size.

[0114] Correspondingly, the first segmentation model is used to segment the current frame target heart valve region of interest image, including:

[0115] The first segmentation model is used to segment the current frame target heart valve region of interest image, which has been adjusted to a preset size.

[0116] In this embodiment, the inner contour mask image and the outer contour mask image of the target heart valve in the current frame are obtained by segmenting the region of interest image of the target heart valve in the current frame, which has been adjusted to a preset size. Therefore, based on the inner contour mask image and the outer contour mask image of the target heart valve in the current frame, the heart valve parameters (including opening area, target heart valve position information, etc.) corresponding to the region of interest image of the target heart valve in the current frame, which has been adjusted to a preset size, are directly obtained. Therefore, it is necessary to adjust the heart valve parameters corresponding to the region of interest image of the target heart valve in the current frame, which has been adjusted to a preset size, in order to obtain the heart valve parameters corresponding to the region of interest image of the target heart valve in the current frame before adjustment. It should be noted that, as those skilled in the art will understand, in some other embodiments, the sizes of the inner contour mask image and the outer contour mask image of the current frame target heart valve, obtained by segmenting the current frame target heart valve region of interest image adjusted to a preset size, can be adjusted to the original size of the current frame target heart valve region of interest image. Then, based on the adjusted original size inner contour mask image and outer contour mask image of the current frame target heart valve, the heart valve parameters corresponding to the current frame cardiac image can be directly obtained.

[0117] In one exemplary implementation, adjusting the size of the current frame target heart valve region of interest image to a preset size includes:

[0118] The target side length is defined by the length dimension of the current frame's target heart valve region of interest image.

[0119] The current frame target heart valve region of interest image is filled along the width direction to adjust the width dimension of the current frame target heart valve region of interest image to the target side length dimension;

[0120] The current frame target heart valve region of interest image, whose width dimension is adjusted to the target side length dimension, is enlarged or reduced to adjust the size of the current frame target heart valve region of interest image to a preset size.

[0121] For details, please refer to Figure 2b and Figure 2c ,in Figure 2c The illustration shows the... Figure 2b Image of the region of interest of the target heart valve (aortic valve) after filling. Figure 2b and Figure 2c As shown, the region of interest (ROI) image of the target heart valve in the current frame can be filled with black pixels (pixel value of 0) along the width direction to adjust the width dimension of the ROI image to be consistent with the length dimension, that is, to adjust the ROI image of the target heart valve in the current frame to a square image. Then, by enlarging or reducing the square ROI image of the target heart valve in the current frame by a certain factor, the size of the current frame echocardiogram can be adjusted to a preset size. It should be noted that, as those skilled in the art will understand, if the square ROI image of the target heart valve in the current frame is enlarged by R times to reach the preset size, then the heart valve parameters calculated based on the inner contour mask image and / or the outer contour mask image of the target heart valve in the current frame can be divided by R (i.e., reduced by R times) to obtain the actual heart valve parameters corresponding to the current frame echocardiogram.

[0122] Please continue to refer to this. Figures 3a to 3f ,in Figure 3a An image of the region of interest of the target heart valve (aortic valve) during ventricular diastole in a specific example of the present invention is shown schematically. Figure 3b The illustration shows the... Figure 3a The target heart valve (aortic valve) outer contour mask image obtained by segmentation, where the white area represents the region defined by the outer contour; Figure 3c The illustration shows the... Figure 3a The target heart valve (aortic valve) inner contour mask image obtained by segmentation, where the white area represents the region defined by the inner contour; Figure 3dAn image of the region of interest of a target heart valve (aortic valve) during ventricular systole is schematically shown in a specific example of the present invention. Figure 3e The illustration shows the... Figure 3d The target heart valve (aortic valve) outer contour mask image obtained by segmentation, where the white area represents the region defined by the outer contour; Figure 3f The illustration shows the... Figure 3d The mask image of the inner contour of the target heart valve (aortic valve) obtained through segmentation. (Example) Figures 3a to 3f As shown, when the ventricle is in diastole, the opening area of ​​the target heart valve (aortic valve) is not 0. When the ventricle is in systole, the opening area of ​​the target heart valve (aortic valve) is 0, that is, the inner contour does not exist, and the obtained inner contour mask image of the target heart valve (aortic valve) is pure black.

[0123] In one exemplary embodiment, obtaining the opening area corresponding to the current frame echocardiogram based on the target heart valve inner contour mask image of the current frame includes:

[0124] Connected component analysis is performed on the inner contour mask image of the target heart valve in the current frame to obtain the opening area of ​​the target heart valve corresponding to the cardiac image in the current frame.

[0125] In one exemplary embodiment, performing connected component analysis on the target heart valve inner contour mask image of the current frame to obtain the target heart valve opening area corresponding to the current frame echocardiogram includes:

[0126] Perform connected component analysis on the current frame target heart valve inner contour mask image to obtain the maximum connected component;

[0127] The pixel area of ​​the largest connected component is obtained based on the number of pixels in the largest connected component.

[0128] Based on the pixel area of ​​the largest connected region and the correspondence between the pre-obtained pixel area and the actual area, the target heart valve opening area corresponding to the current frame cardiac image is obtained.

[0129] Due to the limitations of the segmentation accuracy of the first segmentation model, interference regions may exist on the segmented mask image of the inner contour of the target heart valve in the current frame. Since the area of ​​these interference regions is smaller than the area of ​​the inner contour, performing connected component analysis on the mask image of the inner contour of the target heart valve in the current frame to obtain the maximum connected component effectively avoids the influence of interference regions. The extracted maximum connected component is the region defined by the inner contour of the target heart valve. Therefore, based on the maximum connected component, the opening area of ​​the target heart valve corresponding to the current frame echocardiogram can be obtained. Specifically, the correspondence between pixel area and actual area can be obtained based on the area scale (i.e., the number of pixels per square centimeter) at the time of echocardiogram acquisition. Therefore, based on the pixel area of ​​the maximum connected component and the pre-obtained correspondence between pixel area and actual area, the actual opening area of ​​the target heart valve corresponding to the current frame echocardiogram (in square centimeters) can be obtained.

[0130] In one exemplary embodiment, the method further includes:

[0131] Based on the temporal sequence corresponding to each frame of the cardiac image and the target heart valve opening area, an opening area diagram is drawn to represent the correspondence between the frame number and the opening area.

[0132] Based on the opening area diagram, the final target heart valve opening area is obtained.

[0133] Please refer to Figure 4a The diagram schematically illustrates the opening area in a specific example of the present invention, where the vertical axis represents the opening area, and the horizontal and vertical axes represent the frame number. For example... Figure 4a As shown, by drawing an opening area map representing the correspondence between frame number and opening area based on the temporal sequence of each frame of the cardiac image and the opening area of ​​the target heart valve, doctors can more intuitively observe the opening area value corresponding to each frame of the cardiac image, thus facilitating diagnosis. Furthermore, by obtaining the final target heart valve opening area based on the opening area map, doctors can further facilitate the diagnosis of target heart valve abnormalities based on the obtained final target heart valve opening area. For example, doctors can determine whether a patient has aortic valve stenosis based on the final aortic valve opening area.

[0134] In one exemplary embodiment, obtaining the final target heart valve opening area based on the opening area map includes:

[0135] The opening area map is filtered to obtain an opening area curve.

[0136] Based on the opening area curve, determine the peak opening area for each cardiac cycle;

[0137] The average of all peak opening areas is taken as the final target heart valve opening area.

[0138] This invention first filters the opening area map to effectively remove noise information, and then averages the opening area values ​​(i.e., area peaks) of all peak points under each cardiac cycle on the opening area curve obtained by the filtering process to obtain the final target heart valve opening area. It can be seen that this invention fully considers the numerical fluctuations that may be caused by image acquisition equipment and model accuracy when calculating the opening area, which can effectively improve the accuracy of the final target heart valve opening area, and further improve the accuracy of doctors' diagnosis.

[0139] In one exemplary embodiment, filtering the opening area map to obtain an opening area curve includes:

[0140] The opening area map is transformed from the time domain to the frequency domain to obtain the frequency amplitude map;

[0141] The frequency amplitude graph is filtered to remove the portion of the frequency amplitude graph whose frequency is higher than a preset threshold.

[0142] The filtered frequency amplitude graph is converted from the frequency domain to the time domain to obtain the aperture area curve.

[0143] Please continue to refer to this. Figures 4b to 4d ,in Figure 4b The illustration shows the... Figure 4a The frequency amplitude diagram obtained by transforming the aperture area diagram shown; Figure 4c The illustration shows the... Figure 4b The frequency amplitude diagram shown is obtained by filtering the frequency amplitude diagram. Figure 4d The illustration shows the... Figure 4c The image shows the aperture area curve obtained by inverse transformation of the filtered frequency amplitude graph. Figures 4b to 4dAs shown, by performing a time-domain to frequency-domain transformation on the opening area map, such as a Fast Fourier Transform, the opening area signal can be converted from the original domain (time domain) to the frequency domain. By filtering the acquired frequency amplitude map, the portion of the frequency amplitude map with smaller amplitudes and frequencies higher than a preset threshold (e.g., frequencies higher than 0.1) (i.e., the high-frequency portion) can be filtered out. For example, the amplitude of the high-frequency portion with smaller amplitudes (e.g., frequencies higher than 0.1) can be set to 0. By performing an inverse transformation on the filtered frequency amplitude map, such as an inverse Fast Fourier Transform, the area signal can be converted from the frequency domain to the time domain, thereby obtaining a curve of the opening area changing with image frames after removing the high-frequency portion, i.e., the opening area curve. Then, based on the peak value of the opening area under each cardiac cycle on the acquired opening area curve and taking the average value, the final value of the target heart valve opening area can be obtained. Figure 4d The shown opening area curve contains three peak points (one peak point corresponds to one cardiac cycle), meaning there are three area peaks. It should be noted that, as those skilled in the art will understand, when determining the peak points, only the peak points during cardiac cycles that include both an opening area increase phase and an opening area decrease phase are considered.

[0144] In one exemplary embodiment, after obtaining the opening area corresponding to the current frame of the heart rate image, the method further includes:

[0145] The text content depicting the outer contour, inner contour, and opening area of ​​the target heart valve is drawn on the current frame of the cardiac image and then output.

[0146] Please continue to refer to this. Figure 5 The diagram schematically illustrates a cardiac motion image with an outer contour, an inner contour, and an opening area in a specific example of the present invention, wherein the curve located on the outer side is the outer contour, and the curve located on the inner side is the inner contour. For example... Figure 5 As shown, the opening area corresponding to the cardiac image in this example is 3.91 square centimeters. Therefore, by drawing the outer contour, inner contour, and opening area of ​​the target heart valve on each frame of the cardiac image, doctors can more intuitively view the test results, which is more conducive to improving the accuracy of doctors' diagnoses.

[0147] Furthermore, after drawing the text content describing the outer contour, inner contour, and opening area of ​​the target heart valve on the current frame echocardiogram, the method further includes:

[0148] The current frame of the heartbeat image is denoised to remove noise from the heartbeat image.

[0149] Specifically, median filtering can be used to set the grayscale value of each pixel in the current frame of the heart rate image to the median of the grayscale values ​​of all pixels within its neighborhood window. The size parameter of the filtering kernel can be set according to specific circumstances, for example, to 5×5. Thus, median filtering can effectively remove salt-and-pepper noise from the current frame of the heart rate image. It should be noted that, as those skilled in the art will understand, in other embodiments, other filtering methods besides median filtering can be used to filter the current frame of the heart rate image, and this invention does not limit this to such methods.

[0150] In one exemplary embodiment, after detecting the acquired current frame of cardiac image using a target detection model frequency amplitude map to obtain location information of the target heart valve region of interest, the method further includes:

[0151] Based on the location information of the target heart valve region of interest, a second segmentation model is used to segment the target heart valve region of interest corresponding to the current frame cardiac image to obtain the target heart valve leaflet mask image of the current frame.

[0152] Specifically, as described above, the current frame's target heart valve region of interest image can be obtained based on the location information of the target heart valve's region of interest. Then, the current frame's target heart valve region of interest image, adjusted to a preset size, is input into a pre-trained second segmentation model for segmentation to obtain the current frame's target heart valve leaflet mask image. Based on these target heart valve leaflet mask images, the shape and outline of the leaflets can be visually and intuitively displayed, thus better assisting doctors in clinical diagnosis (e.g., diagnosing whether the aortic valve has bicuspid or quadruple valve deformities), reducing the experience required for clinical diagnosis and the possibility of misdiagnosis. Please continue to refer to... Figure 6a and Figure 6b ,in Figure 6a The illustration shows the... Figure 3a The target heart valve (aortic valve) leaflet mask image obtained by segmentation; where Figure 6b The illustration shows the... Figure 3d The target heart valve (aortic valve) leaflet mask image obtained through segmentation. (Example) Figure 6a and 6b As shown, in this example, the aortic valve is a normal tricuspid valve, without problems such as bicuspid or quadruple cuspid aberrations.

[0153] Furthermore, for visual visualization, the outline of the target heart valve (e.g., aortic valve) leaflet can be drawn on the current frame of the echocardiogram. Median filtering is then used to set the grayscale value of each pixel in the current frame of the echocardiogram to the median of the grayscale values ​​of all pixels within its neighborhood window. The size parameter of the filtering kernel can be set according to specific circumstances, for example, to 5×5. Thus, median filtering effectively removes salt-and-pepper noise from the current frame of the echocardiogram. It should be noted that, as those skilled in the art will understand, in other embodiments, other filtering methods besides median filtering can be used to filter the current frame of the echocardiogram; this invention does not limit this to such methods.

[0154] In one exemplary implementation, the object detection model is a ResNet50 neural network model. Because ResNet uses skip connections (or shortcuts), it directly transmits the activation values ​​of one network layer to deeper layers. Furthermore, skip connections only transmit data; through skip connections, the signal can be transmitted without attenuation during backpropagation, without worrying about gradient changes, thus enabling the transmission of effective gradients to the next layer. Therefore, skip connections effectively alleviate the gradient vanishing problem caused by deepening network layers. By stacking residual blocks, very deep network models can be constructed, allowing for effective training even at deep network layers.

[0155] Furthermore, the target detection model includes a first convolutional layer, a first pooling layer, multiple cascaded residual subnetworks, a second pooling layer, and a fully connected layer. The first convolutional layer is used to extract target heart valve (e.g., aortic valve) features from the input current frame cardiac image. The first pooling layer is used to perform pooling operations on the output of the first convolutional layer. The residual subnetwork is used to extract target heart valve (e.g., aortic valve) features from the output of the first pooling layer or the output of the previous level residual subnetwork. The second pooling layer is used to perform pooling operations on the output of the last level residual subnetwork. The fully connected layer is used to perform nonlinear mapping regression on the output of the second pooling layer to obtain the location information of the target heart valve's region of interest.

[0156] Furthermore, each of the residual subnetworks includes multiple cascaded residual modules, and each residual module includes multiple cascaded second convolutional layers, wherein the input of the first-level second convolutional layer and the output of the last-level second convolutional layer are added together to obtain the output of the residual module.

[0157] Furthermore, the size of the convolution kernel in the first level of the second convolutional layer and the size of the convolution kernel in the last level of the second convolutional layer are both 1×1.

[0158] For details, please refer to Figure 7 The diagram illustrates the structural schematic of a target detection model in a specific example of the present invention. Figure 7 As shown in this example, the target detection model includes a first convolutional layer, a first pooling layer, a first residual subnetwork, a second residual subnetwork, a third residual subnetwork, a fourth residual subnetwork, a second pooling layer, and a fully connected layer. In this configuration, the first convolutional layer extracts target heart valve (e.g., aortic valve) features from the input current frame echocardiogram image; the first pooling layer performs pooling on the output of the first convolutional layer; the first residual subnetwork extracts target heart valve (e.g., aortic valve) features from the output of the first pooling layer; the second residual subnetwork extracts target heart valve (e.g., aortic valve) features from the output of the first residual subnetwork; the third residual subnetwork extracts target heart valve (e.g., aortic valve) features from the output of the second residual subnetwork; the fourth residual subnetwork extracts target heart valve (e.g., aortic valve) features from the output of the third residual subnetwork; the second pooling layer performs pooling on the output of the fourth residual subnetwork; and the fully connected layer performs nonlinear mapping regression on the output of the second pooling layer to obtain the location information of the target heart valve's region of interest. Further, the first pooling layer is a max pooling layer, and the second pooling layer is an average pooling layer.

[0159] Furthermore, the first residual subnetwork includes three cascaded first residual modules, the second residual subnetwork includes four cascaded second residual modules, the third residual subnetwork includes six cascaded third residual modules, and the fourth residual subnetwork includes three cascaded fourth residual modules. Please continue to refer to... Figure 8a The diagram illustrates the structure of the first residual module in a specific example of the present invention. Figure 8a As shown, the first residual module includes three cascaded second convolutional layers: second convolutional layer A1, second convolutional layer A2, and second convolutional layer A3. Second convolutional layer A1 has a 1×1 kernel size, 64 output channels, and a stride of 1; second convolutional layer A2 has a 3×3 kernel size, 64 output channels, and a stride of 1; and second convolutional layer A3 has a 1×1 kernel size, 256 output channels, and a stride of 1. The identity mapping of the input to second convolutional layer A1 is added to the output of second convolutional layer A3 to obtain the output of the first residual module. Please refer to [reference needed]. Figure 8b The diagram illustrates the structure of the second residual module in a specific example of the present invention, as shown below. Figure 8bAs shown, the second residual module includes three cascaded second convolutional layers: second convolutional layer B1, second convolutional layer B2, and second convolutional layer B3. The second convolutional layer B1 has a 1×1 kernel, 128 output channels, and a stride of 1. The second convolutional layer B2 has a 3×3 kernel, 128 output channels, and a stride of 2. The second convolutional layer B3 has a 1×1 kernel, 512 output channels, and a stride of 1. The identity mapping of the input to the second convolutional layer B1 is added to the output of the second convolutional layer B3 to obtain the output of the second residual module. Please refer to [reference needed]. Figure 8c The diagram illustrates the structure of the third residual module in a specific example of the present invention, as shown below. Figure 8c As shown, the third residual module includes three cascaded second convolutional layers: second convolutional layer C1, second convolutional layer C2, and second convolutional layer C3. Second convolutional layer C1 has a 1×1 kernel size, 256 output channels, and a stride of 1; second convolutional layer C2 has a 3×3 kernel size, 256 output channels, and a stride of 2; and second convolutional layer C3 has a 1×1 kernel size, 1024 output channels, and a stride of 1. The identity mapping of the input to second convolutional layer C1 is added to the output of second convolutional layer C3 to obtain the output of the third residual module. Please refer to [reference needed]. Figure 8d The diagram illustrates the structure of the fourth residual module in a specific example of the present invention, as shown below. Figure 8d As shown, the fourth residual module includes three cascaded second convolutional layers, namely second convolutional layer D1, second convolutional layer D2, and second convolutional layer D3. The second convolutional layer D1 has a kernel size of 1×1, 512 output channels, and a stride of 1. The second convolutional layer D2 has a kernel size of 3×3, 512 output channels, and a stride of 2. The second convolutional layer D3 has a kernel size of 1×1, 2048 output channels, and a stride of 1. The identity mapping of the input of the second convolutional layer D1 is added to the output of the second convolutional layer D3 to obtain the output of the fourth residual module.

[0160] Since the skip connections between the second convolutional layer A1 and the second convolutional layer A3, the skip connections between the second convolutional layer B1 and the second convolutional layer B3, the skip connections between the second convolutional layer C1 and the second convolutional layer C3, and the skip connections between the second convolutional layer D1 and the second convolutional layer D3 all adopt identity mapping connections, the training speed of the object detection model can be accelerated and the training effect of the object detection model can be improved without increasing additional parameters and computational load.

[0161] In one exemplary embodiment, the first segmentation model is a DenseNet neural network model. Since the DenseNet neural network model is a densely connected convolutional neural network where the input of each layer comes from the outputs of all preceding layers, this neural network structure enhances feature transfer and utilizes features more effectively. Furthermore, the DenseNet neural network model has good anti-overfitting performance, making it particularly suitable for applications with relatively scarce training data. Therefore, using the DenseNet neural network model as the first segmentation model in this invention can effectively improve the segmentation efficiency and accuracy of the inner and outer contours of the target heart valve (e.g., the aortic valve). Specifically, the DenseNet neural network model consists of multiple densely connected blocks connected by transition blocks; that is, any two adjacent densely connected blocks are connected by a transition block, and the number of convolutional output channels within each densely connected block is consistent, facilitating the superposition of feature information from each layer.

[0162] One layer in a densely connected block is called a bottleneck layer. Dense connections in DenseNet connect each layer in a densely connected block to all subsequent layers, enabling feature reuse. Please refer to [link / reference]. Figure 9 It schematically illustrates the structure of a densely connected block in a specific example. For example... Figure 9 As shown, suppose a densely connected block has L bottleneck layers, X0 is the input of the densely connected block (defined as the output of layer 0), and layer l takes the outputs of all previous layers X0, ..., X... L-1 As input, the input of layer l satisfies the following relationship with the outputs of all previous layers:

[0163]

[0164] in, This indicates that the outputs from layers 0 to (L-1) are combined and concatenated to serve as the input to the bottleneck layer of layer L. H L This represents all operations of the Lth bottleneck layer. Specifically, each bottleneck layer includes multiple operations: batch normalization (BN), ReLU activation function, and 3×3 convolution.

[0165] Please continue to refer to this. Figure 10 The diagram illustrates the structure of the first segmentation model in a specific example of the present invention. Figure 10As shown, in this example, the segmentation model includes a third convolutional layer, a third pooling layer (preferably a max pooling layer), a first dense connection block, a first transition block, a second dense connection block, a second transition block, a third dense connection block, a third transition block, a fourth dense connection block, a first upward transition block, a second upward transition block, and a convolutional block, wherein the convolutional block includes two fourth convolutional layers with a kernel size of 1×1. The third convolutional layer is used to extract target heart valve (e.g., aortic valve) features from the input image. The third pooling layer is used to perform pooling operations on the output of the third convolutional layer to remove unnecessary redundant information from the image. The first dense connection block is used to extract target heart valve (e.g., aortic valve) features from the output of the third pooling layer. The first transition block is used to compress the output of the first dense connection block to reduce the size of the feature map output by the first dense connection block. The second dense connection block is used to extract target heart valve (e.g., aortic valve) features from the output of the first transition block. The second transition block is used to compress the output of the second dense connection block to reduce the size of the feature map output by the second dense connection block. The third dense connection block is used to extract target heart valve (e.g., aortic valve) features from the output of the first transition block. The second transition block is used to compress the output of the second dense connection block to reduce the size of the feature map output by the second dense connection block. The output of the transition block is used to extract features of the target heart valve (e.g., aortic valve). The third transition block is used to compress the output of the third dense connection block to reduce the size of the feature map output by the third dense connection block. The fourth dense connection block is used to extract features of the target heart valve (e.g., aortic valve) from the output of the third transition block. The first upward transition block is used to deconvolve the output of the fourth dense connection block to increase the size of the feature map output by the fourth dense connection block. The second upward transition block is used to deconvolve the output of the first upward transition block to increase the size of the feature map output by the first upward transition block. The convolution block is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the segmentation results of the outer and inner contours of the target heart valve (e.g., aortic valve). Specifically, the fourth convolutional layer A in the convolutional block is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the segmentation result of the outer contour of the target heart valve (e.g., the aortic valve), and the fourth convolutional layer B in the convolutional block is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the segmentation result of the inner contour of the target heart valve (e.g., the aortic valve).

[0166] Sex mapping regression is used to obtain segmentation results of the inner contour of the target heart valve (e.g., the aortic valve).

[0167] Specifically, the fourth convolutional layer A and the fourth convolutional layer B can perform nonlinear mapping regression on the output of the second upward transition block using the sigmoid function. The formula for the sigmoid function is as follows:

[0168]

[0169] As shown in the above equation, the Sigmoid function can map any input real number to the real number mapping interval (0,1). When the input value x is large, the output value g tends to 1, and when the input value x is small, the output value g tends to 0.

[0170] It should be noted that, as those skilled in the art will understand, the first dense connection block, the second dense connection block, the third dense connection block, and the fourth dense connection block all include multiple bottleneck layers, and the number of bottleneck layers in the first dense connection block, the second dense connection block, the third dense connection block, and the fourth dense connection block can be the same or different. The specific number can be set according to actual needs, and the present invention does not limit this. For example, the first dense connection block may have 6 bottleneck layers, the second dense connection block may have 12 bottleneck layers, the third dense connection block may have 24 bottleneck layers, and the fourth dense connection block may have 16 bottleneck layers.

[0171] Please continue to refer to this. Figure 11 The diagram illustrates the structure of the bottleneck layer in a specific example of the present invention. Figure 11 As shown, the bottleneck layer comprises a first batch normalization layer A, a first activation layer A, a fifth convolutional layer A, a first batch normalization layer B, a first activation layer B, and a fifth convolutional layer B connected in sequence. The kernel size of the fifth convolutional layer A is 1×1, and the kernel size of the fifth convolutional layer B is 3×3. Therefore, by adding a 1×1 convolution before the 3×3 convolution in the bottleneck layer, this invention can reduce the number of feature maps and lower the dimensionality of each feature map, thereby reducing computational cost and fusing features from various channels. Furthermore, since the bottleneck layer performs batch normalization (BN) and ReLU activation operations before performing the 1×1 and 3×3 convolution operations, training speed and convergence efficiency can be further improved.

[0172] Please continue to refer to this. Figure 12 The diagram illustrates the structure of a transition block in a specific example of the present invention. Figure 12As shown, the first transition block, the second transition block, and the third transition block each include a second batch normalization layer, a second activation layer, a sixth convolutional layer, and a fourth pooling layer (preferably an average pooling layer) connected in sequence. The kernel size of the sixth convolutional layer is 1×1. Thus, the convolution operation of the sixth convolutional layer can reduce the dimensionality of the feature map, and the average pooling operation of the fourth pooling layer can solve the problem of excessive channel count in the feature map, preventing model complexity caused by too many densely connected blocks. Furthermore, since each transition block performs batch normalization (BN) and ReLU activation operations before the 1×1 convolution operation, the number of parameters can be further compressed.

[0173] Please continue to refer to this. Figure 13 The diagram illustrates the structure of the upward transition block in a specific example of the present invention. Figure 13 As shown, both the first upward transition block and the second upward transition block include a third batch normalization layer A, a third activation layer A, a seventh convolutional layer A, a third batch normalization layer B, a third activation layer B, a seventh convolutional layer B, a third batch normalization layer C, a third activation layer C, and a deconvolutional layer connected in sequence. The convolutional kernels of the seventh convolutional layer A and the seventh convolutional layer B are both 3×3.

[0174] In one exemplary implementation, the first segmentation model uses the binary cross-entropy loss function during training, the formula of which is shown below:

[0175] ;

[0176] ;

[0177] In the formula, For real labels, This is the predicted result.

[0178] Furthermore, after completing the training of the first segmentation model, this invention also uses the Dice coefficient formula to evaluate the algorithm accuracy of the first segmentation model. The Dice coefficient formula is as follows:

[0179]

[0180] In the formula, X represents the prediction result, and Y represents the true label.

[0181] The value of Dice ranges from 0 to 1. The closer the Dice value is to 1, the higher the segmentation accuracy of the first segmentation model.

[0182] As an example, during the training of the first segmentation model, the learning rate is set to 1e-3 (i.e., 0.001), and Adam (adaptive moment estimation) is used as the optimizer. The learning rate of each parameter is dynamically adjusted using the first moment estimation and second moment estimation of the gradient, and clipnorm=0.001 is added to the parameters of the optimizer to clip the gradient.

[0183] In one exemplary implementation, the second segmentation model is also a DenseNet neural network model. The structure of the second segmentation model is largely the same as that of the first segmentation model. The difference is that the second segmentation model includes only a 1×1 fourth convolutional layer, which is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the segmentation result of the target heart valve (e.g., aortic valve) leaflet.

[0184] Based on the same inventive concept, the present invention also provides an electronic device, please refer to [reference needed]. Figure 14 A block diagram illustrating an embodiment of the electronic device provided by the present invention is shown. Figure 14 As shown, the electronic device includes a processor 101 and a memory 103. The memory 103 stores a computer program. When the computer program is executed by the processor 101, it implements the heart valve parameter acquisition method described above. Since the electronic device provided by this invention and the heart valve parameter acquisition method described above belong to the same inventive concept, the electronic device provided by this invention has all the advantages of the heart valve parameter acquisition method described above, and therefore will not be described in detail here.

[0185] like Figure 14 As shown, the electronic device also includes a communication interface 102 and a communication bus 104, wherein the processor 101, the communication interface 102, and the memory 103 communicate with each other via the communication bus 104. The communication bus 104 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 104 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 102 is used for communication between the aforementioned electronic device and other devices.

[0186] The processor 101 referred to in this invention can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 101 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and lines.

[0187] The memory 103 can be used to store the computer program. The processor 101 implements various functions of the electronic device by running or executing the computer program stored in the memory 103 and calling the data stored in the memory 103.

[0188] The memory 103 may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0189] This invention also provides a readable storage medium storing a computer program that, when executed by a processor, can implement the heart valve parameter acquisition method described above. Since the readable storage medium provided by this invention and the heart valve parameter acquisition method described above belong to the same inventive concept, the readable storage medium provided by this invention possesses all the advantages of the heart valve parameter acquisition method described above, and therefore will not be elaborated further.

[0190] The readable storage medium of embodiments of the present invention can be any combination of one or more computer-readable media. The readable medium can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable computer hard disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device.

[0191] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0192] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0193] In summary, compared with the prior art, the heart valve parameter acquisition method, electronic device, and storage medium provided by the present invention have the following advantages: The present invention first obtains the location information of the region of interest (ROI) of the target heart valve corresponding to the current frame of the echocardiogram based on the acquired current frame of the echocardiogram; then, based on the location information of the ROI of the target heart valve, a first segmentation model is used to segment the corresponding region in the current frame of the echocardiogram to obtain the inner contour mask image of the target heart valve in the current frame; and finally, based on the inner contour mask image of the target heart valve in the current frame, the opening area corresponding to the current frame of the echocardiogram can be obtained. Therefore, the present invention can realize an end-to-end algorithm flow, which can not only automatically calculate heart valve parameters, effectively reduce the variability problems that may be caused by human factors, but also better assist doctors in improving the accuracy of diagnosis, provide a basis for the analysis of heart valve abnormalities, and reduce the cumbersome operation of human-computer interaction.

[0194] It should be noted that the apparatus and methods disclosed in the embodiments herein can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, program, or part of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system to perform the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0195] In addition, the functional modules in the various embodiments of this article can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0196] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for obtaining heart valve parameters, characterized in that, include: Based on the current frame of the cardiac image, obtain the location information of the region of interest of the target heart valve; Based on the location information of the target heart valve region of interest, the first segmentation model is used to segment the target heart valve region of interest corresponding to the current frame cardiac image to obtain the inner contour mask image of the target heart valve in the current frame. Based on the target heart valve inner contour mask image of the current frame, obtain the opening area corresponding to the current frame cardiac image; The method further includes: Based on the temporal sequence corresponding to each frame of the cardiac image and the target heart valve opening area, an opening area diagram is drawn to represent the correspondence between the frame number and the opening area. The opening area map is transformed from the time domain to the frequency domain to obtain the frequency amplitude map; The frequency amplitude graph is filtered to remove the portion of the frequency amplitude graph whose frequency is higher than a preset threshold. The filtered frequency amplitude graph is converted from the frequency domain to the time domain to obtain the aperture area curve. Based on the opening area curve, determine the peak opening area for each cardiac cycle that has both an opening area increase period and an opening area decrease period. The average of all peak opening areas is taken as the final target heart valve opening area.

2. The method for obtaining heart valve parameters according to claim 1, characterized in that, The step of segmenting the target heart valve region of interest in the current frame echocardiogram using a first segmentation model based on the location information of the target heart valve region of interest includes: Based on the location information of the target heart valve region of interest, the corresponding region is cropped from the current frame of the cardiac image to obtain the current frame image of the target heart valve region of interest. The first segmentation model is used to segment the region of interest image of the target heart valve in the current frame.

3. The method for obtaining heart valve parameters according to claim 2, characterized in that, Before segmenting the current frame target heart valve region of interest image using the first segmentation model, the method further includes: Adjust the size of the current frame target heart valve region of interest image to a preset size.

4. The method for obtaining heart valve parameters according to claim 3, characterized in that, Adjusting the size of the current frame target heart valve region of interest image to a preset size includes: The target side length is defined by the length dimension of the current frame's target heart valve region of interest image. The current frame target heart valve region of interest image is filled along the width direction to adjust the width dimension of the current frame target heart valve region of interest image to the target side length dimension; The current frame target heart valve region of interest image, whose width dimension is adjusted to the target side length dimension, is enlarged or reduced to adjust the size of the current frame target heart valve region of interest image to a preset size.

5. The method for obtaining heart valve parameters according to claim 1, characterized in that, The step of obtaining the opening area corresponding to the current frame echocardiogram based on the target heart valve inner contour mask image of the current frame includes: Connected component analysis is performed on the inner contour mask image of the target heart valve in the current frame to obtain the opening area of ​​the target heart valve corresponding to the cardiac image in the current frame.

6. The method for obtaining heart valve parameters according to claim 5, characterized in that, The step of performing connected component analysis on the target heart valve inner contour mask image of the current frame to obtain the target heart valve opening area corresponding to the current frame echocardiogram includes: Perform connected component analysis on the current frame target heart valve inner contour mask image to obtain the maximum connected component; The pixel area of ​​the largest connected component is obtained based on the number of pixels in the largest connected component. Based on the pixel area of ​​the largest connected region and the correspondence between the pre-obtained pixel area and the actual area, the target heart valve opening area corresponding to the current frame cardiac image is obtained.

7. The method for obtaining heart valve parameters according to claim 1, characterized in that, The method further includes: Based on the location information of the target heart valve region of interest, the first segmentation model is used to segment the target heart valve region of interest corresponding to the current frame cardiac image to obtain the outer contour mask image of the target heart valve in the current frame. Based on the inner contour mask image of the target heart valve in the current frame and the outer contour mask image of the target heart valve in the current frame, the position information of the target heart valve corresponding to the current frame cardiac image is obtained.

8. The method for obtaining heart valve parameters according to claim 1, characterized in that, After using a target detection model to detect the current frame of the cardiac image to obtain the location information of the target heart valve region of interest, the method further includes: Based on the location information of the target heart valve region of interest, a second segmentation model is used to segment the target heart valve region of interest corresponding to the current frame cardiac image to obtain the target heart valve leaflet mask image of the current frame.

9. The method for obtaining heart valve parameters according to claim 1, characterized in that, The step of obtaining the location information of the region of interest of the target heart valve based on the current frame of the cardiac image includes: A target detection model is used to detect the current frame of the cardiac image to obtain the location information of the target heart valve region of interest.

10. The method for obtaining heart valve parameters according to claim 9, characterized in that, The step of using a target detection model to detect the current frame of the cardiac image to obtain the location information of the target heart valve region of interest includes: A target detection model is used to detect the current frame of the cardiac image to obtain candidate location information of the target heart valve region of interest; Based on the candidate location information of the target heart valve region of interest, calculate the location information of the target heart valve region of interest after magnification by a preset factor. The location information of the target heart valve region of interest after being magnified by a preset factor is used as the location information of the target heart valve region of interest.

11. The method for obtaining heart valve parameters according to claim 9, characterized in that, After using a target detection model to detect the current frame of the cardiac image to obtain the location information of the target heart valve region of interest, the method further includes: Based on the temporal sequence corresponding to the current frame of the cardiac image, the location information of the region of interest of the target heart valve is corrected.

12. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the method of any one of claims 1 to 11.

13. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, implements the method of any one of claims 1 to 11.

Images