Heart valve opening area detection method, electronic device, and storage medium

Abstract

The application provides a heart valve opening area detection method, an electronic device and a storage medium, and the detection method comprises the following steps: extracting a region of interest of each frame of heart image respectively to obtain a target heart valve region of interest image; adopting a heart valve segmentation model to segment each frame of target heart valve region of interest image respectively to obtain a target heart valve inner contour mask image and a target heart valve outer contour mask image; determining all target heart images according to each frame of target heart valve outer contour mask image; determining a target heart valve opening area of each frame of target heart image according to each frame of target heart valve inner contour mask image; and determining a target heart valve opening area of a to-be-confirmed heart video according to the target heart valve opening area of each frame of target heart image. The application can improve the accuracy of the finally obtained target heart valve opening area.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for detecting the opening area of ​​heart valves, an electronic device, and a storage medium. Background Technology

[0002] Aortic valve disease is a common and dangerous cardiovascular disease that seriously endangers human health. Echocardiography is the preferred method for diagnosing and assessing the severity of target valvular heart disease. Measurement of the target valvular opening area, the number and morphology of the aortic valve leaflets are important tools for analyzing cardiac function.

[0003] Currently, the measurement of the target aortic valve orifice area primarily utilizes acquired echocardiographic images. Specially trained medical personnel or specialists analyze and process the cardiac images to determine the image frame from which the orifice area is delineated manually. Manually delineating the aortic valve orifice area yields highly variable results and requires significant time and effort. Furthermore, accurately locating the image frame for measurement within one or more cardiac cycles is challenging. Moreover, measuring the aortic valve orifice area using a single image frame is significantly influenced by random factors. Therefore, in-depth analysis of aortic valve structural abnormalities using medical imaging is crucial for the prevention and diagnosis of valvular heart disease.

[0004] It should be noted that the information disclosed in the background section of this invention is intended only to enhance the understanding of the general background of this invention, and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method, electronic device and storage medium for detecting the opening area of ​​a heart valve, which can automatically detect the opening area of ​​a target heart valve. The overall algorithm has high accuracy and can effectively reduce the cumbersome operation of human-computer interaction.

[0006] To achieve the above objectives, the present invention provides a method for detecting the opening area of ​​a heart valve, comprising:

[0007] The region of interest of the target heart valve is extracted from each frame of the acquired cardiac video to be confirmed, so as to obtain the region of interest image of the target heart valve corresponding to each frame of cardiac video.

[0008] The heart valve segmentation model is used to segment the region of interest image of the target heart valve corresponding to each frame of the acquired cardiac images, so as to obtain the inner contour mask image and the outer contour mask image of the target heart valve corresponding to each frame of cardiac images.

[0009] Based on the target heart valve outer contour mask image corresponding to each frame of cardiac images, all target cardiac images are determined;

[0010] Based on the target heart valve inner contour mask image corresponding to each frame of target cardiac images, determine the target heart valve opening area corresponding to each frame of target cardiac images;

[0011] The target heart valve opening area corresponding to the heart video to be confirmed is determined based on the target heart valve opening area corresponding to each frame of the target heart image.

[0012] Optionally, determining all target cardiac images based on the target heart valve outer contour mask image corresponding to each frame of cardiac images includes:

[0013] For each frame of cardiac images, the pixel area of ​​the target heart valve outer contour corresponding to that frame of cardiac images is determined based on the target heart valve outer contour mask image corresponding to that frame of cardiac images.

[0014] Based on the frame number corresponding to each frame of the cardiac image and the pixel area of ​​the outer contour of the target heart valve, obtain the correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number.

[0015] Based on the correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number, the cardiac image corresponding to the peak value of the pixel area of ​​the outer contour of the target heart valve in each cardiac cycle is taken as a candidate cardiac image.

[0016] Based on each candidate heartbeat image, all target heartbeat images are determined.

[0017] Optionally, the step of using the cardiac images corresponding to the peak value of the target heart valve outer contour pixel area in each cardiac cycle as a candidate cardiac image based on the correspondence between the target heart valve outer contour pixel area and the frame number includes:

[0018] The correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number is filtered to obtain the correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number after filtering.

[0019] Based on the correspondence between the pixel area of ​​the target heart valve outer contour after filtering and the frame number, the cardiac image corresponding to the peak pixel area of ​​the target heart valve outer contour in each cardiac cycle is used as a candidate cardiac image.

[0020] Optionally, the step of using the correlation between the pixel area of ​​the target heart valve's outer contour after filtering and the frame number to select the cardiac image corresponding to the peak pixel area of ​​the target heart valve's outer contour during each cardiac cycle as a candidate cardiac image includes:

[0021] Step A: For each frame of cardiac images, based on the correspondence between the pixel area of ​​the outer contour of the target heart valve after filtering and the frame number, form a data pair with the pixel area of ​​the outer contour of the target heart valve corresponding to that frame of cardiac images after filtering and the frame number.

[0022] Step B: Obtain the data pair with the largest pixel area of ​​the outer contour of the target heart valve after filtering as the target data pair, and take the heart image corresponding to the frame number in the target data pair as the candidate heart image.

[0023] Step C: Traverse all data pairs except the target data pairs until a data pair is found in which the absolute value of the difference between the frame number and the frame number corresponding to each target data pair is greater than the cardiac cycle value. Then, take the cardiac image corresponding to the frame number in the target data pair as a candidate cardiac image.

[0024] Step D: Determine if all data pairs have been traversed. If not, return to step C.

[0025] Optionally, the step of filtering the correspondence between the pixel area of ​​the target heart valve's outer contour and the frame number to obtain the filtered correspondence between the pixel area of ​​the target heart valve's outer contour and the frame number includes:

[0026] Based on the correspondence between the outer contour pixel area of ​​the target heart valve and the frame number, obtain the outer contour pixel area curve.

[0027] The outer contour pixel area curve is transformed from the time domain to the frequency domain to obtain the frequency amplitude map;

[0028] The frequency amplitude graph is filtered to remove the portion of the frequency amplitude graph whose frequency is higher than a preset threshold.

[0029] The frequency amplitude map after filtering is transformed from the frequency domain to the time domain to obtain the outer contour pixel area curve after filtering.

[0030] Based on the filtered outer contour pixel area curve, the correspondence between the filtered target heart valve outer contour pixel area and the frame number is obtained.

[0031] Optionally, the cardiac cycle value is obtained through the following steps:

[0032] According to the frequency amplitude diagram, the frequency corresponding to the second amplitude peak is taken as the target frequency;

[0033] The reciprocal of the target frequency is used as the cardiac cycle value.

[0034] Optionally, determining all target motion images based on each candidate motion image frame includes:

[0035] For each candidate heartbeat image, both the candidate heartbeat image and the neighboring heartbeat images are used as target heartbeat images. The difference between the frame number of the candidate heartbeat image and the frame number of the neighboring heartbeat images is within a preset range.

[0036] Optionally, determining the target heart valve opening area corresponding to each frame of target echocardiogram based on the target heart valve inner contour mask image corresponding to each frame of target echocardiogram includes:

[0037] For each frame of cardiac images, the pixel area of ​​the inner contour of the target heart valve corresponding to the frame of cardiac images is determined based on the inner contour mask image of the target heart valve corresponding to the frame of cardiac images. Based on the pixel area of ​​the inner contour of the target heart valve corresponding to the frame of cardiac images and the correspondence between the pre-obtained pixel area and the actual area, the opening area of ​​the target heart valve corresponding to the frame of cardiac images is obtained.

[0038] Based on the frame number corresponding to each frame of the cardiac image and the target heart valve opening area, obtain the correspondence between the target heart valve opening area and the frame number;

[0039] The correspondence between the target heart valve opening area and the frame number is determined to obtain the filtered correspondence between the target heart valve opening area and the frame number.

[0040] Based on the correspondence between the target heart valve opening area after filtering and the frame number, and the frame number corresponding to each frame of target cardiac images, the target heart valve opening area corresponding to each frame of target cardiac images is determined.

[0041] Optionally, determining the target heart valve opening area corresponding to the heart video to be confirmed based on the target heart valve opening area corresponding to each frame of the target heart image includes:

[0042] The average value of the target heart valve opening area corresponding to each frame of the target cardiac image is taken as the target heart valve opening area corresponding to the cardiac video to be confirmed.

[0043] To achieve the above objectives, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the heart valve opening area detection method described above.

[0044] To achieve the above objectives, the present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the heart valve opening area detection method described above.

[0045] Compared with the prior art, the method, electronic device, and storage medium for detecting the heart valve opening area provided by the present invention have the following advantages:

[0046] The method for detecting the opening area of ​​a heart valve provided by this invention first extracts the region of interest (ROI) of the target heart valve in each frame of the acquired cardiac video to be confirmed, thereby obtaining the ROI image of the target heart valve corresponding to each frame of the cardiac video; then, a heart valve segmentation model is used to segment the ROI image of the target heart valve corresponding to each frame of the cardiac video, thereby obtaining the inner contour mask image and the outer contour mask image of the target heart valve corresponding to each frame of the cardiac video; then, all target cardiac images are determined based on the outer contour mask image of the target heart valve corresponding to each frame of the cardiac video; then, the opening area of ​​the target heart valve corresponding to each frame of the target cardiac video is determined based on the inner contour mask image of the target heart valve corresponding to each frame of the target cardiac video; finally, the opening area of ​​the target heart valve corresponding to the cardiac video to be confirmed is determined based on the opening area of ​​the target heart valve corresponding to each frame of the target cardiac video. Since the outer contour of the target heart valve (e.g., the aortic valve) is more regular than its inner contour, and the segmentation accuracy of the target heart valve outer contour mask image (e.g., the aortic valve outer contour mask image) is higher than that of the target heart valve inner contour mask image (e.g., the aortic valve inner contour mask image) based on the performance of the heart valve segmentation model (e.g., the aortic valve segmentation model), all target heart valve images can be determined by using the target heart valve outer contour mask image corresponding to each frame of the heartbeat image. Then, based on the target heart valve inner contour mask image corresponding to each frame of the target heartbeat image, the target heart valve opening area corresponding to the heartbeat video to be confirmed can be determined, effectively improving the accuracy of the final obtained target heart valve opening area. Furthermore, the heart valve opening area detection method provided by this invention can achieve an end-to-end algorithm flow, which not only effectively reduces the variability problems that may be caused by human factors, but also better assists doctors in improving diagnostic accuracy and providing a basis for heart valve abnormality analysis, but also reduces the cumbersome operation of human-computer interaction.

[0047] Since the electronic device and storage medium provided by this invention belong to the same inventive concept as the method for detecting the opening area of ​​the heart valves provided by this invention, the electronic device and storage medium provided by this invention have all the advantages of the method for detecting the opening area of ​​the heart valves provided by this invention. Therefore, the beneficial effects of the electronic device and storage medium provided by this invention will not be described in detail here. Attached Figure Description

[0048] Figure 1 This is a schematic flowchart of a method for detecting the opening area of ​​a heart valve according to an embodiment of the present invention.

[0049] Figure 2a A heartbeat image provided as a specific example of the present invention;

[0050] Figure 2b From Figure 2a The target heart valve image is cropped from the cardiac images;

[0051] Figure 2c To Figure 2b The target heart valve region of interest image obtained after resizing the target heart valve region of interest image shown;

[0052] Figure 3 This is a schematic diagram of the structure of a heart valve segmentation model provided in a specific example of the present invention;

[0053] Figure 4a This is a specific example of the present invention, showing a region of interest image of a target heart valve in an open ventricular state;

[0054] Figure 4b To Figure 4a The target heart valve outer contour mask image obtained by segmentation;

[0055] Figure 4c To Figure 4a The target heart valve inner contour mask image obtained by segmentation;

[0056] Figure 4d This is a specific example of the present invention, showing a region of interest image of a target heart valve in a ventricular closed state;

[0057] Figure 4e To Figure 4d The target heart valve outer contour mask image obtained by segmentation;

[0058] Figure 4f To Figure 4d The target heart valve inner contour mask image obtained by segmentation;

[0059] Figure 5A schematic diagram of the bottleneck layer provided as a specific example of the present invention;

[0060] Figure 6 This is a schematic diagram of the structure of a transition block provided in a specific example of the present invention;

[0061] Figure 7 This is a schematic diagram of the structure of an upward transition block provided in a specific example of the present invention;

[0062] Figure 8a The outer contour pixel area curve is provided as a specific example of the present invention;

[0063] Figure 8b To Figure 8a The frequency amplitude diagram obtained by transforming the outer contour pixel area curve shown;

[0064] Figure 8c To Figure 8b The frequency amplitude diagram shown is the frequency amplitude diagram obtained after filtering.

[0065] Figure 8d To Figure 8c The filtered outer contour pixel area curve is obtained by inverse transformation of the frequency amplitude map shown.

[0066] Figure 9 This is a schematic flowchart of a process for obtaining candidate heart rate images according to an embodiment of the present invention;

[0067] Figure 10a An opening area curve is provided as a specific example of the present invention;

[0068] Figure 10b To Figure 10a The frequency amplitude diagram obtained by transforming the shown opening area curve;

[0069] Figure 10c To Figure 10b The frequency amplitude diagram shown is the frequency amplitude diagram obtained after filtering.

[0070] Figure 10d To Figure 10c The filtered aperture area curve is obtained by inverse transformation of the frequency amplitude graph shown.

[0071] Figure 11 A heartbeat image with an outer contour, an inner contour, and an opening area is provided as a specific example of the present invention.

[0072] Figure 12 This is a block diagram of an electronic device provided according to an embodiment of the present invention.

[0073] The accompanying figure is labeled as follows:

[0074] Processor-101; Communication interface-102; Memory-103; Communication bus-104. Detailed Implementation

[0075] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, further illustrates the heart valve opening area detection method, electronic device, and storage medium proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clarify the illustration of the embodiments of this invention. Please refer to the drawings to make the objectives, features, and advantages of this invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for illustrative purposes and to enable those skilled in the art to understand and read them, and are not intended to limit the implementation conditions of this invention. Any modifications to the structure, changes in proportions, or adjustments to the size, provided they produce the same or similar effects and achieve the same objectives as this invention, should still fall within the scope of the technical content disclosed in this invention. Furthermore, if the methods described herein include a series of steps, the order of these steps presented herein is not necessarily the only possible order in which these steps can be performed, and some steps may be omitted and / or other steps not described herein may be added to the method.

[0076] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The singular forms “a,” “an,” and “the” include plural objects. The term “or” is generally used to mean “and / or,” the term “several” is generally used to mean “at least one,” and the term “at least two” is generally used to mean “two or more.” Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0077] Furthermore, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0078] The core idea of ​​this invention is to provide a method, electronic device and storage medium for detecting the opening area of ​​a heart valve, which can automatically detect the opening area of ​​a target heart valve. The overall algorithm has high accuracy and can effectively reduce the cumbersome operation of human-computer interaction.

[0079] It should be noted that the method for detecting the opening area of ​​heart valves provided by this invention can be applied to the electronic device provided by this invention. This electronic device can be a personal computer, a mobile terminal, etc., and the mobile terminal can be a mobile phone, tablet computer, or other hardware device with various operating systems. Furthermore, it should be noted that although this article uses echocardiography as an example of the cardiac video to be confirmed, as those skilled in the art will understand, the cardiac video to be confirmed can also be cardiac video acquired by other devices besides ultrasound (e.g., a cardiac endoscope), and this invention does not limit this. Similarly, although this article uses the aortic valve as the target heart valve as an example, as those skilled in the art will understand, the target heart valve can also be the mitral valve, tricuspid valve, pulmonary valve, etc., and this invention does not limit this. Additionally, it should be noted that, as those skilled in the art will understand, the outer contour referred to herein refers to the envelope region of the outer edge of the target heart valve (e.g., the aortic valve), and the area between the outer contour and the inner contour is the area where the target heart valve is located. The area of ​​the inner contour is the opening area of ​​the target heart valve (e.g., the aortic valve).

[0080] To achieve the above-mentioned goals, this invention provides a method for detecting the opening area of ​​heart valves. Please refer to [the relevant documentation]. Figure 1 The diagram illustrates a flowchart of a method for detecting the opening area of ​​a heart valve according to an embodiment of the present invention. Figure 1 As shown, the method for detecting the opening area of ​​the heart valve includes the following steps:

[0081] Step S100: Extract the region of interest of the target heart valve for each frame of the acquired heart video to be confirmed, so as to obtain the region of interest image of the target heart valve corresponding to each frame of heart video.

[0082] Step S200: The heart valve segmentation model is used to segment the region of interest image of the target heart valve corresponding to each frame of the acquired cardiac images, so as to obtain the inner contour mask image and the outer contour mask image of the target heart valve corresponding to each frame of the cardiac images.

[0083] Step S300: Determine all target cardiac images based on the target heart valve outer contour mask image corresponding to each frame of cardiac images.

[0084] Step S400: Determine the target heart valve opening area corresponding to each frame of target cardiac images based on the target heart valve inner contour mask image corresponding to each frame of target cardiac images.

[0085] Step S500: Determine the target heart valve opening area corresponding to the heart video to be confirmed based on the target heart valve opening area corresponding to each frame of target heart image.

[0086] Since the outer contour of the target heart valve (e.g., the aortic valve) is more regular than its inner contour, and the segmentation accuracy of the target heart valve outer contour mask image (e.g., the aortic valve outer contour mask image) is higher than that of the target heart valve inner contour mask image (e.g., the aortic valve inner contour mask image) based on the performance of the heart valve segmentation model (e.g., the aortic valve segmentation model), all target heart valve images can be determined by using the target heart valve outer contour mask image corresponding to each frame of the heartbeat image. Then, based on the target heart valve inner contour mask image corresponding to each frame of the target heartbeat image, the target heart valve opening area corresponding to the heartbeat video to be confirmed can be determined, effectively improving the accuracy of the final obtained target heart valve opening area. Furthermore, the heart valve opening area detection method provided by this invention can achieve an end-to-end algorithm flow, which not only effectively reduces the variability problems that may be caused by human factors, but also better assists doctors in improving diagnostic accuracy and providing a basis for heart valve abnormality analysis, but also reduces the cumbersome operation of human-computer interaction.

[0087] As a specific example, the acquired cardiac video to be confirmed can be a short-axis section image video of the aorta acquired by an ultrasound device. The acquired cardiac video to be confirmed contains multiple cardiac cycles, and the resolution of the cardiac video to be confirmed can be set according to specific circumstances, such as 600×800 pixels.

[0088] In one exemplary embodiment, the step of extracting the region of interest (ROI) of the target heart valve from each frame of the acquired cardiac video to be confirmed, to obtain the ROI image of the target heart valve corresponding to each frame of the cardiac video, includes:

[0089] For each frame of the acquired video of the heartbeat to be confirmed:

[0090] The target detection model was used to detect the region of interest of the target heart valve in each frame of the cardiac image to obtain the location information of the candidate target heart valve region of interest corresponding to the frame of the cardiac image.

[0091] Based on the location information of the candidate target heart valve region of interest corresponding to the frame of the heart image and the preset magnification, the location information of the target heart valve region of interest corresponding to the frame of the heart image is obtained.

[0092] Based on the location information of the target heart valve region of interest corresponding to the cardiac image frame, the corresponding region is cropped from the cardiac image frame to obtain the target heart valve region of interest image corresponding to the cardiac image frame.

[0093] Therefore, by employing a target detection model to detect the target heart valve region of interest (e.g., aortic valve region of interest) in each frame of the cardiac video to be confirmed, the location information of the candidate target heart valve region of interest (e.g., aortic valve region of interest) corresponding to each frame of the cardiac video can be obtained efficiently and accurately. While object detection models can detect candidate regions of interest (ROIs) for heart valves (e.g., aortic valve ROIs) in echocardiograms, providing preliminary localization for subsequent heart valve segmentation models (e.g., aortic valve segmentation models), they also result in the loss of detailed information about the surrounding tissues of the target heart valve. Therefore, this invention calculates the location information of the target heart valve ROI after magnification (e.g., 1.3 times) based on the location information of the candidate target heart valve ROIs. This involves magnifying the bounding box of the candidate target heart valve ROIs detected by the object detection model by a predetermined factor, such as 1.3 times, to obtain the magnified bounding box. The area defined by this magnified bounding box is the final target heart valve ROI. Since the area defined by this magnified bounding box includes detailed information about the surrounding tissues of the target heart valve, the segmentation accuracy of subsequent segmentation models can be further improved. It should be noted that, as those skilled in the art will understand, the center position of the magnified bounding box is the same as the center position of the unmagnified bounding box. Specifically, please refer to... Figure 2a and Figure 2b ,in Figure 2aThe cardiac image provided by a specific example of the present invention is illustrated. Figure 2b From Figure 2a A region of interest image of the target heart valve (aortic valve) cropped from a cardiac image. For example... Figure 2a and Figure 2b As shown, by cropping the region of interest image of the target heart valve (aortic valve) from the cardiac image, the interference of the background can be effectively removed, laying a good foundation for obtaining efficient and accurate inner contour mask images and outer contour mask images of the target heart valve.

[0094] In one exemplary implementation, the object detection model is a ResNet50 neural network model. Because ResNet uses skip connections (or shortcuts), it directly passes the activation values ​​of one network layer to deeper layers. Furthermore, skip connections only transmit data; through skip connections, the signal can be transmitted without attenuation during backpropagation, without worrying about gradient changes, thus enabling the transmission of effective gradients to the next layer. Therefore, skip connections effectively alleviate the gradient vanishing problem caused by deepening network layers. By stacking residual blocks, very deep network models can be constructed, allowing for effective training even at deep network layers.

[0095] It should be noted that, as those skilled in the art will understand, the samples used in the training process of the target detection model are cardiac images of target heart valve regions of interest (e.g., aortic valve regions of interest) that have been labeled.

[0096] In one exemplary embodiment, before segmenting the target heart valve region of interest image corresponding to each frame of echocardiogram using a standard heart valve segmentation model, the heart valve opening area detection method further includes:

[0097] For each frame of cardiac images, the size of the target heart valve region of interest image corresponding to that frame of cardiac images is adjusted to a preset size.

[0098] Correspondingly, the step of using a heart valve segmentation model to segment the region of interest image of the target heart valve corresponding to each frame of cardiac images to obtain the inner contour mask image and the outer contour mask image of the target heart valve corresponding to each frame of cardiac images includes:

[0099] For each frame of the heartbeat image:

[0100] The heart valve segmentation model is used to segment the region of interest image of the target heart valve, which is adjusted to a preset size, corresponding to the heart image frame, to obtain the inner contour mask image and the outer contour mask image of the target heart valve, which are adjusted to a preset size, corresponding to the heart image frame.

[0101] The sizes of the target heart valve inner contour mask image and the target heart valve outer contour mask image, which are adjusted to a preset size according to the heart rate image of the frame, are adjusted to the size of the target heart valve region of interest image corresponding to the heart rate image of the frame, so as to obtain the target heart valve inner contour mask image and the target heart valve outer contour mask image corresponding to the heart rate image of the frame.

[0102] Since the heart valve segmentation model requires images of a uniform size as input, adjusting the size of the target heart valve region of interest image corresponding to each frame of echocardiogram to a preset size can meet the input requirements of the heart valve segmentation model. Specifically, the preset size can be set according to specific circumstances. As a preferred embodiment, in the preset size, the length and width dimensions of the image are consistent, that is, the target heart valve region of interest image after adjustment to the preset size is a square image, for example, the preset size is 320*320. Therefore, by setting the length and width dimensions in the preset size to be consistent, it is easier to adjust the size of the target heart valve region of interest image to the preset size.

[0103] Further, adjusting the size of the region of interest image of the target heart valve corresponding to the frame of the echocardiogram to a preset size includes:

[0104] The larger of the height and width dimensions of the target heart valve region of interest image corresponding to the frame of the cardiac image is taken as the target side length dimension;

[0105] The target heart valve region of interest image corresponding to the frame of cardiac image is filled to adjust the smaller of the width and height of the target heart valve region of interest image to the target side length.

[0106] The target heart valve region of interest image, adjusted to the target side length size, is magnified or reduced to adjust the size of the target heart valve region of interest image to the preset size.

[0107] For details, please refer to Figure 2b and Figure 2c ,in, Figure 2c To Figure 2b The target heart valve region of interest image (aortic valve region of interest image) shown is obtained after resizing the target heart valve region of interest image (aortic valve region of interest image). For example... Figure 2b and Figure 2c As shown, due to Figure 2b The width of the target heart valve region of interest image (aortic valve region of interest image) is smaller than its height. Therefore, along the height direction, the target heart valve region of interest image (aortic valve region of interest image) is filled with black pixels (pixel value of 0) to adjust the height of the target heart valve region of interest image (aortic valve region of interest image) to match the width, that is, to adjust the target heart valve region of interest image (aortic valve region of interest image) into a square image. Then, the square target heart valve region of interest image is enlarged or reduced by a certain factor, so that the size of the target heart valve region of interest image (aortic valve region of interest image) can be adjusted to a preset size.

[0108] In one exemplary embodiment, the heart valve segmentation model is a DenseNet neural network model. Since the DenseNet neural network model is a densely connected convolutional neural network, where the input of each layer comes from the outputs of all preceding layers, this neural network structure enhances feature transfer and utilizes features more effectively. Furthermore, the DenseNet neural network model has good anti-overfitting performance, making it particularly suitable for applications with relatively scarce training data. Therefore, using the DenseNet neural network model as the heart valve segmentation model (e.g., the aortic valve segmentation model) in this invention can effectively improve the segmentation efficiency and accuracy of the target heart valve outer contour mask image (e.g., the aortic valve outer contour mask image) and the target heart valve inner contour mask image (e.g., the aortic valve inner contour mask image). Specifically, the DenseNet neural network model consists of multiple densely connected blocks connected by transition blocks; that is, any two adjacent densely connected blocks are connected by a transition block, and the number of convolutional output channels within each densely connected block is consistent, facilitating the superposition of feature information from each layer. One layer in a densely connected block is called a bottleneck layer. Dense connections in DenseNet connect each layer in a densely connected block to all subsequent layers, enabling feature reuse.

[0109] Please continue to refer to this. Figure 3 The diagram illustrates the structure of a heart valve segmentation model provided in a specific example of the present invention. Figure 3As shown in this specific example, the heart valve segmentation model includes a first convolutional layer, a first pooling layer (preferably a max pooling layer), a first dense connection block, a first transition block, a second dense connection block, a second transition block, a third dense connection block, a third transition block, a fourth dense connection block, a first upward transition block, a second upward transition block, and a convolutional block, wherein the convolutional block includes two second convolutional layers with a kernel size of 1×1. In this configuration, the first convolutional layer extracts target heart valve features (e.g., aortic valve features) from the input image; the first pooling layer performs pooling operations on the output of the first convolutional layer to remove unnecessary redundant information from the image; the first dense connection block extracts target heart valve features (e.g., aortic valve features) from the output of the first pooling layer; the first transition block compresses the output of the first dense connection block to reduce the size of the feature map output by the first dense connection block; the second dense connection block extracts target heart valve features (e.g., aortic valve features) from the output of the first transition block; the second transition block compresses the output of the second dense connection block to reduce the size of the feature map output by the second dense connection block; and the third dense connection block extracts target heart valve features (e.g., aortic valve features) from the output of the second transition block. The third transition block is used to compress the output of the third dense connection block to reduce the size of the feature map output by the third dense connection block. The fourth dense connection block is used to extract the target heart valve features (e.g., aortic valve features) from the output of the third transition block. The first upward transition block is used to perform a deconvolution operation on the output of the fourth dense connection block to increase the size of the feature map output by the fourth dense connection block. The second upward transition block is used to perform a deconvolution operation on the output of the first upward transition block to increase the size of the feature map output by the first upward transition block. The convolution block is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the segmentation results of the outer and inner contours of the target heart valve (e.g., aortic valve), that is, to obtain the outer contour mask image (e.g., aortic valve outer contour mask image) and the inner contour mask image (e.g., aortic valve inner contour mask image) of the target heart valve.Specifically, the second convolutional layer A in the convolutional block is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the segmentation result of the outer contour of the target heart valve (e.g., the aortic valve) (i.e., output two), that is, to obtain the outer contour mask image of the target heart valve (e.g., the outer contour mask image of the aortic valve). The second convolutional layer B in the convolutional block is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the segmentation result of the inner contour of the target heart valve (e.g., the aortic valve) (i.e., output one), that is, to obtain the inner contour mask image of the target heart valve (e.g., the inner contour mask image of the aortic valve).

[0110] Please refer to Figures 4a to 4f ,in, Figure 4a This is a region of interest image of a target heart valve (aortic valve) in an open state, provided as a specific example of the present invention. Figure 4b To Figure 4a The target heart valve (aortic valve) outer contour mask image obtained by segmentation, where the white area represents the region defined by the outer contour; Figure 4c To Figure 4a The target heart valve (aortic valve) inner contour mask image obtained by segmentation, where the white area represents the region defined by the inner contour; Figure 4d This is a region of interest image of a target heart valve (aortic valve) under ventricular closure state, provided as a specific example of the present invention. Figure 4e To Figure 4d The target heart valve (aortic valve) outer contour mask image obtained by segmentation, where the white area represents the region defined by the outer contour; Figure 4f To Figure 4d The inner contour mask image of the target heart valve (aortic valve) obtained through segmentation. (Example) Figures 4a to 4f As shown, when the ventricle is open, the opening area of ​​the target heart valve (aortic valve) is not 0. When the ventricle is closed, the opening area of ​​the target heart valve (aortic valve) is 0, that is, the inner contour does not exist, and the obtained inner contour mask image of the target heart valve (aortic valve) is pure black.

[0111] Specifically, the second convolutional layer A and the second convolutional layer B can perform nonlinear mapping regression on the output of the second upward transition block using the sigmoid function. The formula for the sigmoid function is as follows:

[0112]

[0113] As shown in the above equation, the Sigmoid function can map any input real number to the real number mapping interval (0,1). When the input value x is large, the output value g tends to 1, and when the input value x is small, the output value g tends to 0.

[0114] It should be noted that, as those skilled in the art will understand, the first dense connection block, the second dense connection block, the third dense connection block, and the fourth dense connection block all include multiple bottleneck layers, and the number of bottleneck layers in the first dense connection block, the second dense connection block, the third dense connection block, and the fourth dense connection block can be the same or different. The specific number can be set according to actual needs, and the present invention does not limit this. For example, the first dense connection block may have 6 bottleneck layers, the second dense connection block may have 12 bottleneck layers, the third dense connection block may have 24 bottleneck layers, and the fourth dense connection block may have 16 bottleneck layers.

[0115] Please continue to refer to this. Figure 5 The diagram illustrates the structure of the bottleneck layer provided in a specific example of the present invention. Figure 5 As shown, the bottleneck layer comprises a first batch normalization layer A, a first activation layer A, a third convolutional layer A, a first batch normalization layer B, a first activation layer B, and a third convolutional layer B connected in sequence. The kernel size of the third convolutional layer A is 1×1, and the kernel size of the third convolutional layer B is 3×3. Therefore, by adding a 1×1 convolution before the 3×3 convolution in the bottleneck layer, this invention reduces the number of feature maps and the dimensionality of each feature map, thereby reducing computational cost and fusing features from various channels. Furthermore, since the bottleneck layer performs batch normalization (BN) and ReLU activation operations before both the 1×1 and 3×3 convolution operations, training speed and convergence efficiency can be further improved.

[0116] Please continue to refer to this. Figure 6 The diagram illustrates the structure of a transition block provided in a specific example of the present invention. Figure 6 As shown, the first transition block, the second transition block, and the third transition block each include a second batch normalization layer, a second activation layer, a fourth convolutional layer, and a second pooling layer (preferably an average pooling layer) connected in sequence. The kernel size of the fourth convolutional layer is 1×1. Thus, the convolutional operation of the fourth convolutional layer can reduce the dimensionality of the feature map, and the average pooling operation of the second pooling layer can solve the problem of excessive channels in the feature map, preventing model complexity caused by too many densely connected blocks. Furthermore, since each transition block performs batch normalization (BN) and ReLU activation operations before the 1×1 convolutional operation, the number of parameters can be further compressed.

[0117] Please continue to refer to this. Figure 7 The diagram illustrates the structure of an upward transition block provided in a specific example of the present invention. Figure 7 As shown, both the first upward transition block and the second upward transition block include a third batch normalization layer A, a third activation layer A, a fifth convolutional layer A, a third batch normalization layer B, a third activation layer B, a fifth convolutional layer B, a third batch normalization layer C, a third activation layer C, and a deconvolutional layer connected in sequence. The convolutional kernels of the fifth convolutional layer A and the fifth convolutional layer B are both 3×3.

[0118] In one exemplary implementation, the heart valve segmentation model uses the binary cross-entropy loss function during training, the formula of which is shown below:

[0119]

[0120]

[0121] In the formula, y i For real labels, This is the predicted result.

[0122] Furthermore, after completing the training of the heart valve segmentation model, this invention also uses the Dice coefficient formula to evaluate the algorithm accuracy of the heart valve segmentation model. The Dice coefficient formula is as follows:

[0123]

[0124] In the formula, X represents the prediction result, and Y represents the true label.

[0125] The value of Dice ranges from 0 to 1. The closer the Dice value is to 1, the higher the segmentation accuracy of the heart valve segmentation model.

[0126] As an example, during the training of the heart valve segmentation model, the learning rate is set to 1e-3 (i.e., 0.001), and Adam (adaptive moment estimation) is used as the optimizer. The learning rate of each parameter is dynamically adjusted using the first moment estimation and second moment estimation of the gradient, and clipnorm = 0.001 is added to the parameters of the optimizer to clip the gradient.

[0127] Furthermore, the samples used during the training of the heart valve segmentation model (e.g., the aortic valve segmentation model) are training images of the target heart valve region of interest (e.g., aortic valve region of interest training images) and corresponding mask labels for the target heart valve's outer contour (e.g., aortic valve outer contour mask label image) and inner contour (e.g., aortic valve inner contour mask label image). Specifically, an OpenCV contour extraction algorithm can be used to find the outer and inner contours of the target heart valve (e.g., the aortic valve) in the target heart valve region of interest training images (e.g., aortic valve region of interest training images) to obtain the target heart valve outer contour mask label image (e.g., aortic valve outer contour mask label image) and the target heart valve inner contour mask label image (e.g., aortic valve inner contour mask label image).

[0128] In one exemplary embodiment, determining all target cardiac images based on the target heart valve outer contour mask image corresponding to each frame of cardiac images includes:

[0129] For each frame of cardiac images, the pixel area of ​​the target heart valve outer contour corresponding to that frame of cardiac images is determined based on the target heart valve outer contour mask image corresponding to that frame of cardiac images.

[0130] Based on the frame number corresponding to each frame of the cardiac image and the pixel area of ​​the outer contour of the target heart valve, obtain the correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number.

[0131] Based on the correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number, the cardiac image corresponding to the peak value of the pixel area of ​​the outer contour of the target heart valve in each cardiac cycle is taken as a candidate cardiac image.

[0132] Based on each candidate heartbeat image, all target heartbeat images are determined.

[0133] Specifically, the pixel area of ​​the target heart valve's outer contour (e.g., the area defined by the outer contour) can be obtained by counting the number of pixels in the largest connected region (i.e., the region defined by the outer contour) in the target heart valve's outer contour mask image (e.g., the aortic valve's outer contour mask image) corresponding to the cardiac image. Since the inner contour area (i.e., the opening area) is also the largest when the outer contour area is the largest, it is reasonable and can reduce errors by using the cardiac images corresponding to the peak pixel area of ​​the target heart valve's outer contour in each cardiac cycle (i.e., the cardiac images with the largest outer contour pixel area in each cardiac cycle) as candidate cardiac images for calculating the opening area of ​​the final cardiac video to be confirmed. This approach avoids the influence of random factors such as large fluctuations in single-frame cardiac images.

[0134] Further, the step of using the cardiac image corresponding to the peak value of the target heart valve outer contour pixel area in each cardiac cycle as a candidate cardiac image, based on the correspondence between the target heart valve outer contour pixel area and the frame number, includes:

[0135] The correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number is filtered to obtain the correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number after filtering.

[0136] Based on the correspondence between the pixel area of ​​the target heart valve outer contour after filtering and the frame number, the cardiac image corresponding to the peak pixel area of ​​the target heart valve outer contour in each cardiac cycle is used as a candidate cardiac image.

[0137] Therefore, the heart valve opening area detection method provided by the present invention can effectively filter out noise information by first filtering the correspondence between the target heart valve outer contour pixel area and the frame number, and then obtaining candidate cardiac images based on the correspondence between the filtered target heart valve outer contour pixel area and the frame number. This can effectively improve the accuracy of the final obtained target heart valve opening area, and further improve the accuracy of doctors' diagnosis.

[0138] Further, the step of filtering the correspondence between the pixel area of ​​the target heart valve's outer contour and the frame number to obtain the filtered correspondence between the pixel area of ​​the target heart valve's outer contour and the frame number includes:

[0139] Based on the correspondence between the outer contour pixel area of ​​the target heart valve and the frame number, obtain the outer contour pixel area curve.

[0140] The outer contour pixel area curve is transformed from the time domain to the frequency domain to obtain the frequency amplitude map;

[0141] The frequency amplitude graph is filtered to remove the portion of the frequency amplitude graph whose frequency is higher than a preset threshold.

[0142] The frequency amplitude map after filtering is transformed from the frequency domain to the time domain to obtain the outer contour pixel area curve after filtering.

[0143] Based on the filtered outer contour pixel area curve, the correspondence between the filtered target heart valve outer contour pixel area and the frame number is obtained.

[0144] Please refer to Figures 8a to 8d ,in, Figure 8a The outer contour pixel area curve is provided as a specific example of the present invention; Figure 8b To Figure 8a The frequency amplitude diagram obtained by transforming the outer contour pixel area curve shown; Figure 8c To Figure 8b The frequency amplitude diagram shown is the frequency amplitude diagram obtained after filtering. Figure 8d To Figure 8c The image shown is a filtered outer contour pixel area curve obtained by inverse transformation of the filtered frequency amplitude map. Figures 8a to 8d As shown, by performing a time-domain to frequency-domain transformation on the outer contour pixel area curve, such as a Fast Fourier Transform, the outer contour pixel area signal can be converted from the original domain (time domain) to the frequency domain. By filtering the acquired frequency amplitude map, the portion of the frequency amplitude map with smaller amplitudes and frequencies higher than a preset threshold (e.g., frequencies higher than 0.1) (i.e., the high-frequency portion) can be filtered out. For example, the amplitude of the high-frequency portion with smaller amplitudes (e.g., the portion with frequencies higher than 0.1) can be set to 0. By performing an inverse transformation on the filtered frequency amplitude map, such as an inverse Fast Fourier Transform, the outer contour pixel area signal can be converted from the frequency domain to the time domain, thereby obtaining a curve showing the change of the outer contour pixel area with image frames after removing the high-frequency portion, i.e., the filtered outer contour pixel area curve. Then, based on the frame number corresponding to the peak point of the outer contour pixel area in each cardiac cycle in the obtained filtered outer contour pixel area curve (the correspondence between the filtered target heart valve outer contour pixel area and the frame number), all candidate cardiac images can be determined. Figure 8dIn the filtered outer contour pixel area curve shown, there are three peak points of outer contour pixel area (one peak point corresponds to one cardiac cycle). It should be noted that, as those skilled in the art will understand, when determining the peak point of outer contour pixel area, only the peak point of outer contour pixel area under cardiac cycles that have both an increasing and decreasing phase of outer contour pixel area is considered.

[0145] Please continue to refer to this. Figure 9 The diagram illustrates a flowchart of obtaining candidate heart rate images according to an embodiment of the present invention. Figure 9 As shown, in one exemplary embodiment, the step of using the cardiac image corresponding to the peak value of the target heart valve outer contour pixel area in each cardiac cycle as a candidate cardiac image, based on the correspondence between the filtered target heart valve outer contour pixel area and the frame number, includes:

[0146] Step A: For each frame of cardiac images, based on the correspondence between the pixel area of ​​the outer contour of the target heart valve after filtering and the frame number, form a data pair with the pixel area of ​​the outer contour of the target heart valve corresponding to that frame of cardiac images after filtering and the frame number.

[0147] Step B: Obtain the data pair with the largest pixel area of ​​the outer contour of the target heart valve after filtering as the target data pair, and take the heart image corresponding to the frame number in the target data pair as the candidate heart image.

[0148] Step C: Traverse all data pairs except the target data pairs until a data pair is found in which the absolute value of the difference between the frame number and the frame number corresponding to each target data pair is greater than the cardiac cycle value. Then, take the cardiac image corresponding to the frame number in the target data pair as a candidate cardiac image.

[0149] Step D: Determine if all data pairs have been traversed. If not, return to step C.

[0150] Therefore, through steps A to D above, all target data pairs can be found, and the heartbeat image corresponding to the frame number in the target data pair is the candidate heartbeat image.

[0151] Furthermore, to improve search efficiency, the data pairs corresponding to all frames of cardiac images can be arranged in descending order of the pixel area of ​​the target heart valve outer contour after filtering to obtain a data pair set. The data pair ranked first is taken as the target data pair. Then, the data pairs in the data pair set, excluding the target data pair, are traversed sequentially until a data pair is found in which the absolute value of the difference between the frame number and the frame number corresponding to each target data pair is greater than the cardiac cycle value. This data pair is then taken as the new target data pair. The process of traversing the data pairs in the data pair set, excluding the target data pair, continues until all data pairs in the data pair set have been traversed.

[0152] Furthermore, the cardiac cycle value is obtained through the following steps:

[0153] According to the frequency amplitude diagram, the frequency corresponding to the second amplitude peak is taken as the target frequency;

[0154] The reciprocal of the target frequency is used as the cardiac cycle value.

[0155] Specifically, such as Figure 8b As shown, Figure 8b Counting from left to right, the frequency corresponding to the second peak value is the target frequency, and the reciprocal of the target frequency is the corresponding cardiac cycle value.

[0156] In one exemplary implementation, determining all target motion images based on each candidate motion image frame includes:

[0157] For each candidate heartbeat image, both the candidate heartbeat image and the neighboring heartbeat images are used as target heartbeat images. The difference between the frame number of the candidate heartbeat image and the frame number of the neighboring heartbeat images is within a preset range.

[0158] It should be noted that, as those skilled in the art will understand, the preset range can be set according to specific circumstances. For example, the preset range can be set to [-2, 2], and the present invention does not limit this. Therefore, by using the candidate cardiac image and the preceding few frames (e.g., the first two frames) and the following few frames (e.g., the last two frames) of cardiac images located near the candidate cardiac image as target cardiac images, the numerical fluctuations that may be caused by sampling equipment and model accuracy (including the accuracy of the target detection model and the accuracy of the heart valve segmentation model) can be fully considered. This further ensures the accuracy of the target heart valve opening area (e.g., the aortic valve opening area) corresponding to the cardiac video to be confirmed, calculated based on the target heart valve opening area (e.g., the aortic valve opening area) corresponding to each target cardiac image, effectively reducing errors.

[0159] In one exemplary embodiment, determining the target heart valve opening area corresponding to each frame of target echocardiogram based on the target heart valve inner contour mask image corresponding to each frame of target echocardiogram includes:

[0160] For each frame of the target heartbeat image:

[0161] Based on the target heart valve inner contour mask image corresponding to the target heart image in that frame, obtain the pixel area of ​​the target heart valve inner contour corresponding to the target heart image in that frame.

[0162] Based on the pixel area of ​​the inner contour of the target heart valve corresponding to the target cardiac image frame and the correspondence between the pre-obtained pixel area and the actual area, the opening area of ​​the target heart valve corresponding to the target cardiac image frame is obtained.

[0163] It should be noted that, as those skilled in the art can understand, the correspondence between pixel area and actual area can be obtained based on the area scale (i.e., the number of pixels per square centimeter) during the acquisition of the cardiac video to be confirmed. Thus, based on the pixel area of ​​the inner contour of the target heart valve corresponding to each frame of the target cardiac image and the correspondence between the pre-obtained pixel area and the actual area, the target heart valve opening area (in square centimeters) corresponding to each frame of the target cardiac image can be obtained.

[0164] In one exemplary embodiment, determining the target heart valve opening area corresponding to each frame of target echocardiogram based on the target heart valve inner contour mask image corresponding to each frame of target echocardiogram includes:

[0165] For each frame of cardiac images, the pixel area of ​​the inner contour of the target heart valve corresponding to the frame of cardiac images is determined based on the inner contour mask image of the target heart valve corresponding to the frame of cardiac images. Based on the pixel area of ​​the inner contour of the target heart valve corresponding to the frame of cardiac images and the correspondence between the pre-obtained pixel area and the actual area, the opening area of ​​the target heart valve corresponding to the frame of cardiac images is obtained.

[0166] Based on the frame number corresponding to each frame of the cardiac image and the target heart valve opening area, obtain the correspondence between the target heart valve opening area and the frame number;

[0167] The correspondence between the target heart valve opening area and the frame number is filtered to obtain the filtered correspondence between the target heart valve opening area and the frame number.

[0168] Based on the correspondence between the target heart valve opening area after filtering and the frame number, and the frame number corresponding to each frame of target cardiac images, the target heart valve opening area corresponding to each frame of target cardiac images is determined.

[0169] Therefore, by obtaining the correspondence between the target heart valve opening area and the frame number, and filtering the correspondence between the target heart valve opening area and the frame number to obtain the filtered correspondence between the target heart valve opening area and the frame number, and then determining the target heart valve opening area corresponding to each frame of target cardiac images based on the filtered correspondence between the target heart valve opening area and the frame number, the influence of noise can be effectively eliminated, further improving the accuracy of the heart valve opening area detection method provided by the present invention.

[0170] Further, the step of filtering the correspondence between the target heart valve opening area and the frame number to obtain the filtered correspondence between the target heart valve opening area and the frame number includes:

[0171] Based on the correspondence between the target heart valve opening area and the frame number, obtain the opening area curve.

[0172] The opening area curve is transformed from the time domain to the frequency domain to obtain the frequency amplitude curve.

[0173] The frequency amplitude graph is filtered to remove the portion of the frequency amplitude graph whose frequency is higher than a preset threshold.

[0174] The frequency amplitude graph after filtering is converted from the frequency domain to the time domain to obtain the aperture area curve after filtering.

[0175] Based on the filtered opening area curve, the correspondence between the filtered target heart valve opening area and the frame number is obtained.

[0176] For details, please refer to Figures 10a to 10d ,in, Figure 10a An opening area curve is provided as a specific example of the present invention; Figure 10b To Figure 10a The frequency amplitude diagram obtained by transforming the shown opening area curve; Figure 10c To Figure 10b The frequency amplitude diagram shown is the frequency amplitude diagram obtained after filtering. Figure 10d To Figure 10c The image shows the filtered frequency amplitude graph, which is then inversely transformed to obtain the filtered aperture area curve. (See image for details.) Figures 10a to 10dAs shown, by performing a time-domain to frequency-domain transformation on the aperture area curve, such as a Fast Fourier Transform, the aperture area signal can be converted from the original domain (time domain) to the frequency domain. By filtering the obtained frequency amplitude map, the portion of the frequency amplitude map with smaller amplitudes and frequencies higher than a preset threshold (e.g., frequencies higher than 0.1) (i.e., the high-frequency portion) can be filtered out. For example, the amplitude of the high-frequency portion with smaller amplitudes (e.g., frequencies higher than 0.1) can be set to 0. By performing an inverse transformation on the filtered frequency amplitude map, such as an inverse Fast Fourier Transform, the aperture area signal can be converted from the frequency domain to the time domain, thereby obtaining a curve showing the aperture area changing with the image frame after removing the high-frequency portion, i.e., the filtered aperture area curve. Therefore, based on the aperture area curve, the correspondence between the filtered target heart valve aperture area and the frame number can be obtained, that is, the filtered aperture area corresponding to each frame of cardiac images can be obtained. Thus, for each frame of target cardiac images, the aperture area (filtered aperture area) corresponding to each frame of target cardiac images can be obtained.

[0177] In one exemplary embodiment, determining the target heart valve opening area corresponding to the cardiac video to be confirmed based on the target heart valve opening area corresponding to each frame of the target cardiac image includes:

[0178] The average value of the target heart valve opening area corresponding to each frame of the target cardiac image is taken as the target heart valve opening area corresponding to the cardiac video to be confirmed.

[0179] Therefore, by using the average value of the target heart valve opening area corresponding to each frame of the target cardiac image as the target heart valve opening area corresponding to the cardiac video to be confirmed, the accuracy of the target heart valve opening area corresponding to the cardiac video to be confirmed can be further guaranteed, and the error can be further reduced.

[0180] In one exemplary embodiment, after obtaining the target heart valve opening area corresponding to the cardiac video to be confirmed, the heart valve opening area detection method further includes:

[0181] The text content of drawing the outer contour and inner contour of the target heart valve, as well as the opening area of ​​the target heart valve corresponding to the heart video to be confirmed, on each frame of the heart video to be confirmed is output.

[0182] Please continue to refer to this. Figure 11This is a specific example of the present invention, providing a cardiac image with an outer contour, an inner contour, and an opening area, wherein the curve on the outer side represents the outer contour of the target heart valve (aortic valve outer contour), and the curve on the inner side represents the inner contour of the target heart valve (aortic valve inner contour). For example... Figure 11 As shown, in this specific example, the orifice area of ​​the target heart valve (e.g., aortic valve) corresponding to the cardiac video to be confirmed, where the cardiac image is located, is 3.91 square centimeters. Therefore, by drawing the outer and inner contours of the target heart valve (e.g., aortic valve) and the text content of the orifice area of ​​the target heart valve (e.g., aortic valve) corresponding to the cardiac video to be confirmed on each frame of the cardiac image, doctors can more intuitively view the test results, which is more conducive to improving the accuracy of doctors' diagnoses.

[0183] Furthermore, after drawing the outer contour and inner contour of the target heart valve, as well as the text content of the target heart valve opening area corresponding to the heart video to be confirmed, on each frame of the cardiac images, the method further includes:

[0184] The cardiac images in each frame are denoised to remove noise.

[0185] Specifically, median filtering can be used to set the grayscale value of each pixel in the heart rate image to the median of the grayscale values ​​of all pixels within its neighborhood window. The size parameter of the filtering kernel can be set according to specific circumstances, for example, to 5×5. Thus, median filtering can effectively remove salt-and-pepper noise from the heart rate image. It should be noted that, as those skilled in the art will understand, in other embodiments, other filtering methods besides median filtering can be used to filter the heart rate image, and this invention does not limit this to such methods.

[0186] Based on the same inventive concept, the present invention also provides an electronic device, please refer to... Figure 12 The diagram illustrates a block structure of an electronic device according to an embodiment of the present invention. Figure 12 As shown, the electronic device includes a processor 101 and a memory 103. The memory 103 stores a computer program. When the computer program is executed by the processor 101, it implements the heart valve opening area detection method described above. Since the electronic device provided by this invention and the heart valve opening area detection method described above belong to the same inventive concept, the electronic device provided by this invention has all the advantages of the heart valve opening area detection method described above. Therefore, the beneficial effects of the electronic device provided by this invention will not be described in detail here.

[0187] like Figure 12As shown, the electronic device also includes a communication interface 102 and a communication bus 104, wherein the processor 101, the communication interface 102, and the memory 103 communicate with each other via the communication bus 104. The communication bus 104 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 104 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 102 is used for communication between the aforementioned electronic device and other devices.

[0188] The processor 101 referred to in this invention can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 101 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and lines.

[0189] The memory 103 can be used to store the computer program. The processor 101 implements various functions of the electronic device by running or executing the computer program stored in the memory 103 and calling the data stored in the memory 103.

[0190] The memory 103 may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0191] This invention also provides a readable storage medium storing a computer program that, when executed by a processor, can implement the aforementioned method for detecting the opening area of ​​heart valves. Since the readable storage medium provided by this invention and the aforementioned method for detecting the opening area of ​​heart valves belong to the same inventive concept, the readable storage medium provided by this invention possesses all the advantages of the aforementioned method for detecting the opening area of ​​heart valves. Therefore, the beneficial effects of the readable storage medium provided by this invention will not be elaborated upon here.

[0192] The readable storage medium of embodiments of the present invention can be any combination of one or more computer-readable media. The readable medium can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer hard disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device.

[0193] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0194] In summary, compared with the prior art, the method, electronic device, and storage medium for detecting the heart valve opening area provided by the present invention have the following advantages:

[0195] The method for detecting the opening area of ​​a heart valve provided by this invention first extracts the region of interest (ROI) of the target heart valve in each frame of the acquired cardiac video to be confirmed, thereby obtaining the ROI image of the target heart valve corresponding to each frame of the cardiac video; then, a heart valve segmentation model is used to segment the ROI image of the target heart valve corresponding to each frame of the cardiac video, thereby obtaining the inner contour mask image and the outer contour mask image of the target heart valve corresponding to each frame of the cardiac video; then, all target cardiac images are determined based on the outer contour mask image of the target heart valve corresponding to each frame of the cardiac video; then, the opening area of ​​the target heart valve corresponding to each frame of the target cardiac video is determined based on the inner contour mask image of the target heart valve corresponding to each frame of the target cardiac video; finally, the opening area of ​​the target heart valve corresponding to the cardiac video to be confirmed is determined based on the opening area of ​​the target heart valve corresponding to each frame of the target cardiac video. Since the outer contour of the target heart valve (e.g., the aortic valve) is more regular than its inner contour, and the segmentation accuracy of the target heart valve outer contour mask image (e.g., the aortic valve outer contour mask image) is higher than that of the target heart valve inner contour mask image (e.g., the aortic valve inner contour mask image) based on the performance of the heart valve segmentation model (e.g., the aortic valve segmentation model), all target heart valve images can be determined by using the target heart valve outer contour mask image corresponding to each frame of the heartbeat image. Then, based on the target heart valve inner contour mask image corresponding to each frame of the target heartbeat image, the target heart valve opening area corresponding to the heartbeat video to be confirmed can be determined, effectively improving the accuracy of the final obtained target heart valve opening area. Furthermore, the heart valve opening area detection method provided by this invention can achieve an end-to-end algorithm flow, which not only effectively reduces the variability problems that may be caused by human factors, but also better assists doctors in improving diagnostic accuracy and providing a basis for heart valve abnormality analysis, but also reduces the cumbersome operation of human-computer interaction.

[0196] Since the electronic device and storage medium provided by this invention belong to the same inventive concept as the method for detecting the opening area of ​​the heart valves provided by this invention, the electronic device and storage medium provided by this invention have all the advantages of the method for detecting the opening area of ​​the heart valves provided by this invention. Therefore, the beneficial effects of the electronic device and storage medium provided by this invention will not be described in detail here.

[0197] It should be noted that, as those skilled in the art will understand, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0198] It should be noted that the apparatus and methods disclosed in the embodiments herein can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, program, or part of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system to perform the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of this article can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0199] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for detecting the opening area of ​​a heart valve, characterized in that, include: The region of interest of the target heart valve is extracted from each frame of the acquired cardiac video to be confirmed, so as to obtain the region of interest image of the target heart valve corresponding to each frame of cardiac video. The heart valve segmentation model is used to segment the region of interest image of the target heart valve corresponding to each frame of the acquired cardiac images, so as to obtain the inner contour mask image and the outer contour mask image of the target heart valve corresponding to each frame of cardiac images. Based on the target heart valve outer contour mask image corresponding to each frame of cardiac images, all target cardiac images are determined; Based on the target heart valve inner contour mask image corresponding to each frame of target cardiac images, determine the target heart valve opening area corresponding to each frame of target cardiac images; The target heart valve opening area corresponding to the heart video to be confirmed is determined based on the target heart valve opening area corresponding to each frame of the target heart image. The step of determining the target heart valve opening area corresponding to each frame of target echocardiogram based on the target heart valve inner contour mask image corresponding to each frame of target echocardiogram includes: For each frame of cardiac images, the pixel area of ​​the inner contour of the target heart valve corresponding to the frame of cardiac images is determined based on the inner contour mask image of the target heart valve corresponding to the frame of cardiac images. Based on the pixel area of ​​the inner contour of the target heart valve corresponding to the frame of cardiac images and the correspondence between the pre-obtained pixel area and the actual area, the opening area of ​​the target heart valve corresponding to the frame of cardiac images is obtained. Based on the frame number corresponding to each frame of the cardiac image and the target heart valve opening area, obtain the correspondence between the target heart valve opening area and the frame number; The correspondence between the target heart valve opening area and the frame number is filtered to obtain the filtered correspondence between the target heart valve opening area and the frame number. Based on the correspondence between the target heart valve opening area after filtering and the frame number, and the frame number corresponding to each frame of target cardiac images, the target heart valve opening area corresponding to each frame of target cardiac images is determined.

2. The method for detecting the opening area of ​​heart valves according to claim 1, characterized in that, The process of determining all target cardiac images based on the target heart valve outer contour mask image corresponding to each frame of cardiac images includes: For each frame of cardiac images, the pixel area of ​​the target heart valve outer contour corresponding to that frame of cardiac images is determined based on the target heart valve outer contour mask image corresponding to that frame of cardiac images. Based on the frame number corresponding to each frame of the cardiac image and the pixel area of ​​the outer contour of the target heart valve, obtain the correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number. Based on the correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number, the cardiac image corresponding to the peak value of the pixel area of ​​the outer contour of the target heart valve in each cardiac cycle is taken as a candidate cardiac image. Based on each candidate heartbeat image, all target heartbeat images are determined.

3. The method for detecting the opening area of ​​heart valves according to claim 2, characterized in that, The step of using the correspondence between the pixel area of ​​the target heart valve's outer contour and the frame number to select the cardiac image corresponding to the peak pixel area of ​​the target heart valve's outer contour during each cardiac cycle as a candidate cardiac image includes: The correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number is filtered to obtain the correspondence between the pixel area of ​​the outer contour of the target heart valve and the frame number after filtering. Based on the correspondence between the pixel area of ​​the target heart valve outer contour after filtering and the frame number, the cardiac image corresponding to the peak pixel area of ​​the target heart valve outer contour in each cardiac cycle is used as a candidate cardiac image.

4. The method for detecting the opening area of ​​heart valves according to claim 3, characterized in that, The step of using the correspondence between the pixel area of ​​the target heart valve's outer contour after filtering and the frame number to select the cardiac image corresponding to the peak pixel area of ​​the target heart valve's outer contour during each cardiac cycle as a candidate cardiac image includes: Step A: For each frame of cardiac images, based on the correspondence between the pixel area of ​​the outer contour of the target heart valve after filtering and the frame number, form a data pair with the pixel area of ​​the outer contour of the target heart valve corresponding to that frame of cardiac images after filtering and the frame number. Step B: Obtain the data pair with the largest pixel area of ​​the outer contour of the target heart valve after filtering as the target data pair, and take the heart image corresponding to the frame number in the target data pair as the candidate heart image. Step C: Traverse all data pairs except the target data pairs until a data pair is found in which the absolute value of the difference between the frame number and the frame number corresponding to each target data pair is greater than the cardiac cycle value. Then, take the cardiac image corresponding to the frame number in the target data pair as a candidate cardiac image. Step D: Determine whether all data pairs have been traversed. If not, return to step C.

5. The method for detecting the opening area of ​​a heart valve according to claim 3, characterized in that, The step of filtering the correspondence between the pixel area of ​​the target heart valve's outer contour and the frame number to obtain the filtered correspondence between the pixel area of ​​the target heart valve's outer contour and the frame number includes: Based on the correspondence between the outer contour pixel area of ​​the target heart valve and the frame number, obtain the outer contour pixel area curve. The outer contour pixel area curve is transformed from the time domain to the frequency domain to obtain the frequency amplitude map; The frequency amplitude graph is filtered to remove the portion of the frequency amplitude graph whose frequency is higher than a preset threshold. The frequency amplitude map after filtering is transformed from the frequency domain to the time domain to obtain the outer contour pixel area curve after filtering. Based on the filtered outer contour pixel area curve, the correspondence between the filtered target heart valve outer contour pixel area and the frame number is obtained.

6. The method for detecting the opening area of ​​a heart valve according to claim 5, characterized in that, The cardiac cycle value is obtained through the following steps: According to the frequency amplitude diagram, the frequency corresponding to the second amplitude peak is taken as the target frequency; The reciprocal of the target frequency is used as the cardiac cycle value.

7. The method for detecting the opening area of ​​a heart valve according to claim 2, characterized in that, The process of determining all target heartbeat images based on each candidate heartbeat image frame includes: For each candidate heartbeat image, both the candidate heartbeat image and the neighboring heartbeat images are used as target heartbeat images. The difference between the frame number of the candidate heartbeat image and the frame number of the neighboring heartbeat images is within a preset range.

8. The method for detecting the opening area of ​​a heart valve according to claim 1, characterized in that, The step of determining the target heart valve opening area corresponding to the heart video to be confirmed based on the target heart valve opening area corresponding to each frame of the target heart image includes: The average value of the target heart valve opening area corresponding to each frame of the target cardiac image is taken as the target heart valve opening area corresponding to the cardiac video to be confirmed.

9. An electronic device, characterized in that, The method includes a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the method for detecting the heart valve opening area as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, implements the method for detecting the heart valve opening area as described in any one of claims 1 to 8.

Images