Heart valve image segmentation method, electronic device and storage medium

Abstract

The application provides a heart valve image segmentation method, an electronic device and a storage medium. The method comprises the following steps: adopting a target detection model to extract a target heart valve region of interest from each frame of heart dynamic image in an obtained heart dynamic video, so as to obtain a corresponding target heart valve region of interest image; adopting a valve ring segmentation model to segment the target heart valve region of interest image, so as to obtain a corresponding target heart valve ring image; inputting the target heart valve ring image into a valve generation model, so as to generate a corresponding target heart valve mask; and inputting the target heart valve mask and the target heart valve region of interest image corresponding to the target heart valve mask into a valve segmentation model, so as to obtain a corresponding target heart valve segmentation image. The application can not only improve the segmentation accuracy and continuity of the target heart valve segmentation image (for example, an aortic valve segmentation image) obtained through final segmentation, but also reduce the problem of differentiation possibly caused by human factors.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a heart valve image segmentation method, an electronic device and a storage medium. BACKGROUND

[0002] In modern medical imaging, ultrasound images have the advantages of low intensity, low price, no harm to the human body, etc., and have unique advantages in the detection of soft tissues and the observation of blood flow of cardiovascular organs. With the improvement of living standards and the aggravation of population aging, more and more common heart valve diseases such as aortic valve malformation occur. The main diagnosis method for such diseases in clinical practice is to observe the shape and movement of the valve by using an ultrasound device, and echocardiography is a good tool for detecting heart valve diseases. The first step in the analysis is echocardiogram image segmentation. Because the ultrasound image has many speckle noises, the target motion is complex, and the target and background have low gray scale contrast, it is difficult to segment it. In actual ultrasound image processing and analysis, the identification, positioning and quantitative analysis of the target and lesion mainly rely on manual segmentation by doctors' experience. Therefore, doctors need to have rich clinical medical knowledge and keen spatial position sense to segment the aortic valve from the heart ultrasound image mixed with a large number of speckle noises and artifacts. A group of ultrasound sequences usually consists of dozens or even hundreds of pictures, and if the doctors manually segment them, it will be a very large workload.

[0003] With the development of image processing technology, the segmentation of the aortic valve based on neural network algorithm has been developed to a certain extent. However, due to the motion characteristics of the aortic valve and the existence of blurring or even invisibility of the aortic valve during B-ultrasound acquisition, the continuity of the segmentation result of the aortic valve is poor.

[0004] It should be noted that the information disclosed in the background section of the present application is only intended to deepen the understanding of the general background of the present application, and should not be regarded as acknowledging or implying in any form that the information constitutes prior art known to those skilled in the art. SUMMARY

[0005] The purpose of the present application is to provide a heart valve image segmentation method, an electronic device and a storage medium, to solve the problem of poor continuity of the segmentation result obtained by directly using a neural network to segment the aortic valve ultrasound image in the prior art.

[0006] To solve the above technical problems, the present application provides a heart valve image segmentation method, comprising:

[0007] using a target detection model to extract the target heart valve region of interest of each frame of heart image in the acquired heart video to obtain the corresponding target heart valve region of interest image;

[0008] segment the target heart valve region of interest image in the target heart valve image to obtain a corresponding target heart valve annulus image;

[0009] input the target heart valve annulus image into a valve generation model to generate a corresponding target heart valve mask;

[0010] input the target heart valve mask and the target heart valve region of interest image corresponding thereto into a valve segmentation model to obtain a corresponding target heart valve segmentation image.

[0011] Optionally, the extracting the target heart valve region of interest from each frame of the heart video image to obtain a corresponding target heart valve region of interest image includes:

[0012] extracting the target heart valve region of interest from each frame of the heart video image to obtain a corresponding target heart valve region of interest position information;

[0013] performing curve fitting according to the target heart valve region of interest position information corresponding to each frame of the heart video image, and correcting the target heart valve region of interest position information corresponding to each frame of the heart video image according to the fitted result;

[0014] cropping the corresponding target heart valve region of interest from each frame of the heart video image according to the corrected target heart valve region of interest position information corresponding to each frame of the heart video image to obtain a corresponding target heart valve region of interest image.

[0015] Optionally, the performing curve fitting according to the target heart valve region of interest position information corresponding to each frame of the heart video image, and correcting the target heart valve region of interest position information corresponding to each frame of the heart video image according to the fitted result includes:

[0016] performing curve fitting according to the target heart valve region of interest position information corresponding to each frame of the heart video image extracted by the target detection model to obtain a corresponding relationship between the fitted image frame and the target heart valve region of interest position information;

[0017] correcting the target heart valve region of interest position information corresponding to each frame of the heart video image according to the corresponding relationship between the fitted image frame and the target heart valve region of interest position information to obtain corrected target heart valve region of interest position information corresponding to each frame of the heart video image.

[0018] Optionally, the position information of the target heart valve region of interest corresponding to each frame of the heart motion image is corrected according to the correspondence between the fitted image frame and the position information of the target heart valve region of interest, so as to obtain the corrected position information of the target heart valve region of interest corresponding to each frame of the heart motion image, comprising:

[0019] For each frame of the heart motion image:

[0020] According to the correspondence between the fitted image frame and the position information of the target heart valve region of interest, the position information of the fitted target heart valve region of interest corresponding to the frame of the heart motion image is obtained;

[0021] According to the absolute value of the difference between the position information of the target heart valve region of interest corresponding to the frame of the heart motion image extracted by the target detection model and the position information of the fitted target heart valve region of interest corresponding to the frame of the heart motion image, the first position deviation information corresponding to the frame of the heart motion image is obtained;

[0022] According to the first position deviation information corresponding to the frame of the heart motion image and the confidence probability value of the target heart valve region of interest corresponding to the frame of the heart motion image extracted by the target detection model, the second position deviation information corresponding to the frame of the heart motion image is obtained;

[0023] According to the second position deviation information corresponding to the frame of the heart motion image, it is judged whether the position information of the target heart valve region of interest corresponding to the frame of the heart motion image extracted by the target detection model is accurate;

[0024] If yes, the position information of the target heart valve region of interest corresponding to the frame of the heart motion image extracted by the target detection model is taken as the corrected position information of the target heart valve region of interest corresponding to the frame of the heart motion image;

[0025] If not, the corrected position information of the target heart valve region of interest corresponding to the frame of the heart motion image is obtained according to the position information of the target heart valve region of interest corresponding to the heart motion image with accurate position information adjacent to the previous frame and the position information of the target heart valve region of interest corresponding to the heart motion image with accurate position information adjacent to the next frame.

[0026] Optionally, the judgment of whether the position information of the target heart valve region of interest corresponding to the frame of the heart motion image extracted by the target detection model is accurate according to the second position deviation information corresponding to the frame of the heart motion image comprises:

[0027] According to the first position deviation information corresponding to each frame of the heart motion image, the first position deviation mean information corresponding to the heart motion video is obtained;

[0028] The average confidence probability value of the target heart valve region of interest corresponding to each frame of the heart rate image is extracted based on the target detection model, and the average confidence probability value corresponding to the heart rate video is obtained.

[0029] Based on the mean first positional deviation information corresponding to the heartbeat video and the mean confidence probability corresponding to the heartbeat video, the mean second positional deviation information corresponding to the heartbeat video is obtained.

[0030] The position judgment threshold is obtained based on the preset multiple threshold and the average value of the second position deviation corresponding to the heartbeat video;

[0031] For each frame of the cardiac motion video, based on the second position deviation information corresponding to that frame of cardiac motion image and the position judgment threshold, it is determined whether the position information of the target heart valve region of interest corresponding to that frame of cardiac motion image extracted by the target detection model is accurate.

[0032] Optionally, before segmenting the target heart valve region of interest image using a valve annulus segmentation model, the segmentation method further includes:

[0033] The target side length is defined by the length dimension of the region of interest image of the target heart valve.

[0034] The region of interest image of the target heart valve is filled along the width direction to adjust the width dimension of the region of interest image of the target heart valve to the target side length dimension;

[0035] The target heart valve region of interest image is magnified or reduced by adjusting the width dimension to the target side length dimension, so as to adjust the size of the target heart valve region of interest image to a preset size.

[0036] Optionally, inputting the target heart valve annulus image into the valve generation model includes:

[0037] The annular state classification model is used to determine the open and closed states of the target heart valve corresponding to the annular image of the target heart valve.

[0038] If the target heart valve corresponding to the target heart valve annulus image is in an open state, then the target heart valve annulus image is input into the open valve generation model;

[0039] If the target heart valve corresponding to the target heart valve annulus image is in a closed state, then the target heart valve annulus image is input into the closed valve generation model.

[0040] Optionally, the valve generation model is a generator in a generative adversarial network.

[0041] Optionally, the valve generation model is trained through the following steps:

[0042] Acquire training samples, which include images of heart valve annulus and corresponding heart valve label images;

[0043] The pre-acquired generative adversarial network is trained based on the training samples to obtain a trained generative adversarial network.

[0044] The generator in the trained generative adversarial network is used as the valve generation model.

[0045] Optionally, training the pre-acquired generative adversarial network based on the training samples includes:

[0046] The generator and discriminator in the generative adversarial network are trained using an alternating training method based on the training samples until the preset training termination condition is met.

[0047] Optionally, the preset training termination condition is that the generator and the discriminator reach equilibrium.

[0048] Optionally, the valve segmentation model includes a first input layer, a second input layer, a first convolutional layer, a first pooling layer (preferably a max pooling layer), a first dense connection block, a first transition block, a second dense connection block, a second transition block, a third dense connection block, a third transition block, a fourth dense connection block, a first upward transition block, a second upward transition block, and a second convolutional layer.

[0049] The first input layer is used to receive the target heart valve mask, and the second input layer is used to receive the corresponding target heart valve region of interest image.

[0050] The first convolutional layer is used to extract the annular features of the target heart valve from the region of interest image of the target heart valve. The first pooling layer is used to perform pooling operations on the output of the first convolutional layer. The first dense connection block is used to extract the annular features of the target heart valve from the output of the first pooling layer or the result of adding the output of the first pooling layer to the target heart valve mask. The first transition block is used to compress the output of the first dense connection block or the result of adding the output of the first dense connection block to the target heart valve mask. The second dense connection block is used to extract the annular features of the target heart valve from the output of the first transition block or the result of adding the output of the first transition block to the target heart valve mask. The second transition block is used to compress the output of the second dense connection block or the result of adding the output of the second dense connection block to the target heart valve mask. The third dense connection block... The connecting block is used to extract the target heart valve annulus features from the output of the second transition block or the result of adding the output of the second transition block to the target heart valve mask. The third transition block is used to compress the output of the third dense connecting block or the result of adding the output of the third dense connecting block to the target heart valve mask. The fourth dense connecting block is used to extract the target heart valve annulus features from the output of the third transition block or the result of adding the output of the third transition block to the target heart valve mask. The first upward transition block is used to deconvolve the output of the fourth dense connecting block or the result of adding the output of the fourth dense connecting block to the target heart valve mask. The second upward transition block is used to deconvolve the output of the first upward transition block. The second convolutional layer is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the target heart valve segmentation result.

[0051] To address the aforementioned technical problems, the present invention also provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it implements the heart valve image segmentation method described above.

[0052] To address the aforementioned technical problems, the present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the heart valve image segmentation method described above.

[0053] Compared with existing technologies, the heart valve image segmentation method, electronic device, and storage medium provided by this invention have the following advantages: This invention first uses a target detection model to extract the region of interest (ROI) of the target heart valve from each frame of the acquired cardiac video, obtaining the corresponding ROI image of the target heart valve; then, it uses a valve annulus segmentation model to segment the ROI image of the target heart valve, obtaining the corresponding valve annulus image of the target heart valve; next, it inputs the valve annulus image of the target heart valve into a valve generation model to generate the corresponding target heart valve mask; finally, it inputs the target heart valve mask and its corresponding ROI image of the target heart valve into the valve segmentation model to obtain the corresponding segmented image of the target heart valve. Since the target heart valve mask in this invention is directly generated using a valve generation model based on the corresponding target heart valve annulus image, this target heart valve mask can be used as shape prior information and input together with the corresponding target heart valve region of interest image into the valve segmentation model. This not only improves the segmentation accuracy and continuity of the final segmented target heart valve image (e.g., aortic valve segmentation image), but also reduces the variability that may be caused by human factors. Furthermore, this invention enables an end-to-end algorithm flow and has strong versatility, thus better assisting doctors in improving diagnostic efficiency and reducing the risks caused by errors in using echocardiography for heart valve abnormality analysis. Attached Figure Description

[0054] Figure 1 A schematic flowchart of a heart valve image segmentation method provided in one embodiment of the present invention;

[0055] Figure 2a A heartbeat image provided as a specific example of the present invention;

[0056] Figure 2b From Figure 2a A region of interest image of the target heart valve (aortic valve) cropped from a cardiac image;

[0057] Figure 2c To Figure 2b Image of the region of interest of the target heart valve (aortic valve) after filling;

[0058] Figure 3 This is a schematic diagram of the structure of a lobe ring segmentation model provided in a specific example of the present invention;

[0059] Figure 4 A schematic diagram of the bottleneck layer provided as a specific example of the present invention;

[0060] Figure 5 This is a schematic diagram of the structure of a transition block provided in a specific example of the present invention;

[0061] Figure 6 This is a schematic diagram of the structure of an upward transition block provided in a specific example of the present invention;

[0062] Figure 7a An image of the region of interest of a target heart valve (aortic valve) in an open state, provided as a specific example of the present invention;

[0063] Figure 7b To Figure 7a Image of the target heart valve (aortic valve) annulus obtained by segmentation;

[0064] Figure 7c A region of interest image of a target heart valve (aortic valve) in a closed state, provided as a specific example of the present invention;

[0065] Figure 7d To Figure 7c Image of the target heart valve (aortic valve) annulus obtained by segmentation;

[0066] Figure 7e According to Figure 7b The target heart valve (aortic valve) annulus image shown is generated from the target heart valve (aortic valve) annulus image;

[0067] Figure 7f According to Figure 7d The target heart valve (aortic valve) annulus image shown is generated from the target heart valve (aortic valve) annulus image;

[0068] Figure 8 This is a schematic diagram of the network structure of a generator provided in a specific example of the present invention;

[0069] Figure 9 This is a schematic diagram of the structure of a valve segmentation model provided in a specific example of the present invention;

[0070] Figure 10 This is a block diagram of an electronic device according to one embodiment of the present invention.

[0071] The reference numerals in the attached figures are as follows:

[0072] Processor-101; Communication interface-102; Memory-103; Communication bus-104. Detailed Implementation

[0073] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, further illustrates the heart valve image segmentation method, electronic device, and storage medium proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clearly illustrate the embodiments of this invention. Please refer to the drawings to make the objectives, features, and advantages of this invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for illustrative purposes and to enable those skilled in the art to understand and read them, and are not intended to limit the implementation conditions of this invention. Any modifications to the structure, changes in proportions, or adjustments to the size, provided they produce the same or similar effects and achieve the same objectives as this invention, should still fall within the scope of the technical content disclosed in this invention.

[0074] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] Furthermore, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0076] The core idea of ​​this invention is to provide a method for segmenting heart valve images, an electronic device, and a storage medium to solve the problem of poor continuity of segmentation results obtained by directly using neural networks to segment aortic valve ultrasound images in the prior art.

[0077] It should be noted that the heart valve image segmentation method of this invention can be applied to the electronic device described in this invention. This electronic device can be a personal computer, a mobile terminal, etc., and the mobile terminal can be a mobile phone, tablet computer, or other hardware device with various operating systems. Furthermore, although this article uses echocardiography as an example, as those skilled in the art will understand, the echocardiogram can also be acquired by other devices besides ultrasound (e.g., a cardiac endoscope), and this invention does not limit this. Additionally, although this article uses the aortic valve as the target heart valve, as those skilled in the art will understand, the target heart valve can also be the mitral valve, tricuspid valve, pulmonary valve, etc., and this invention does not limit this. It should also be noted that in this article, the long side direction of the image is defined as the length direction, and the short side direction of the image is defined as the width direction. Furthermore, it should be noted that the target heart valve annulus referred to in this invention refers to the vascular cavity corresponding to the target heart valve; for example, the aortic valve annulus refers to the aortic vascular cavity corresponding to the aortic valve.

[0078] To achieve the above-mentioned goals, this invention provides a method for segmenting heart valve images. Please refer to [the relevant documentation]. Figure 1 The diagram illustrates a flowchart of a cardiac valve image segmentation method according to an embodiment of the present invention. Figure 1 As shown, the heart valve image segmentation method includes the following steps:

[0079] Step S100: Use a target detection model to extract the region of interest of the target heart valve from each frame of the acquired cardiac video to obtain the corresponding region of interest image of the target heart valve.

[0080] Step S200: Use a valve annulus segmentation model to segment the region of interest image of the target heart valve to obtain the corresponding target heart valve annulus image.

[0081] Step S300: Input the target heart valve annulus image into the valve generation model to generate the corresponding target heart valve mask.

[0082] Step S400: Input the target heart valve mask and the corresponding target heart valve region of interest image into the valve segmentation model to obtain the corresponding target heart valve segmentation image.

[0083] Therefore, this invention first acquires an image of the target heart valve's region of interest (ROI); then, it segments the ROI image using a pre-trained valve annulus segmentation model. This not only improves the accuracy of the acquired target heart valve annulus image (e.g., aortic valve annulus image), laying a solid foundation for generating an accurate target heart valve mask, but also further reduces the computational load of the valve annulus segmentation model, improving computational efficiency. Furthermore, since the target heart valve mask in this invention is directly generated using a valve generation model based on its corresponding target heart valve annulus image, the target heart valve mask can be used as shape prior information and input into the valve segmentation model along with its corresponding ROI image. This not only improves the segmentation accuracy and continuity of the final segmented target heart valve image (e.g., aortic valve segmentation image), but also reduces potential variability caused by human factors. Moreover, this invention enables an end-to-end algorithm flow with strong versatility, thus better assisting doctors in improving diagnostic efficiency and reducing the risks caused by errors in echocardiographic analysis of heart valve abnormalities.

[0084] As an example, the cardiac video is an echocardiogram (each cardiac video contains multiple cardiac cycles), and the resolution of the echocardiogram can be set according to specific circumstances, such as 600×800. Specifically, the echocardiogram is a PSAX-AV cross-sectional image acquired by an ultrasound device, and the echocardiogram contains several cardiac cycles.

[0085] In one exemplary embodiment, the step of using a target detection model to extract the region of interest (ROI) of the target heart valve from each frame of the acquired cardiac video to obtain the corresponding ROI image of the target heart valve includes:

[0086] A target detection model was used to extract the region of interest (ROI) of the target heart valve in each frame of the acquired cardiac video to obtain the location information of the corresponding target heart valve ROI.

[0087] Curve fitting is performed based on the location information of the target heart valve region of interest corresponding to each frame of cardiac images, and the location information of the target heart valve region of interest corresponding to each frame of cardiac images is corrected based on the fitting results.

[0088] Based on the location information of the corrected target heart valve region of interest corresponding to each frame of the cardiac image, the corresponding target heart valve region of interest is cropped from each frame of the cardiac image to obtain the corresponding target heart valve region of interest image.

[0089] Therefore, this invention corrects the location information of the target heart valve region of interest in each frame of cardiac images extracted by the target detection model, and then crops the corresponding target heart valve region of interest image from the cardiac images based on the corrected location information. This allows for the acquisition of a more accurate target heart valve annulus image, laying a good foundation for generating a more accurate target heart valve mask and effectively improving the accuracy of the final obtained target heart valve segmentation image. Please refer to [reference needed]. Figure 2a and Figure 2b ,in Figure 2a The cardiac image provided by a specific example of the present invention is illustrated. Figure 2b The illustration shows from Figure 2a The region of interest image of the target heart valve (aortic valve) cropped from the cardiac image.

[0090] In one exemplary embodiment, the step of performing curve fitting based on the location information of the target heart valve region of interest corresponding to each frame of echocardiogram, and correcting the location information of the target heart valve region of interest corresponding to each frame of echocardiogram based on the fitting result, includes:

[0091] Based on the location information of the target heart valve region of interest corresponding to each frame of cardiac images extracted by the target detection model, curve fitting is performed to obtain the correspondence between the fitted image frames and the location information of the target heart valve region of interest.

[0092] Based on the correspondence between the fitted image frames and the location information of the target heart valve region of interest, the location information of the target heart valve region of interest corresponding to each frame of cardiac images is corrected to obtain the corrected location information of the target heart valve region of interest corresponding to each frame of cardiac images.

[0093] Therefore, by performing curve fitting on the position information of the target heart valve region of interest corresponding to each frame of cardiac images extracted by the target detection model, the correspondence between the fitted image frame and the position information of the target heart valve region of interest can be obtained. Thus, based on the correspondence between the fitted image frame and the position information of the target heart valve region of interest, the position information of the fitted target heart valve region of interest corresponding to each frame of cardiac images can be obtained.

[0094] In one exemplary embodiment, the step of correcting the position information of the target heart valve region of interest corresponding to each frame of the echocardiogram based on the correspondence between the fitted image frames and the position information of the target heart valve region of interest, to obtain the corrected position information of the target heart valve region of interest corresponding to each frame of the echocardiogram, includes:

[0095] For each frame of the heartbeat image:

[0096] Based on the correspondence between the fitted image frame and the location information of the target heart valve region of interest, the location information of the fitted target heart valve region of interest corresponding to the cardiac image frame is obtained.

[0097] The first positional deviation information corresponding to the cardiac image is obtained based on the absolute value of the difference between the positional information of the target heart valve region of interest corresponding to the cardiac image frame extracted by the target detection model and the positional information of the fitted target heart valve region of interest corresponding to the cardiac image frame.

[0098] Based on the first positional deviation information corresponding to the frame of cardiac images and the confidence probability value of the target heart valve region of interest extracted by the target detection model, the second positional deviation information corresponding to the frame of cardiac images is obtained.

[0099] Based on the second position deviation information corresponding to the frame of the heart image, it is determined whether the position information of the target heart valve region of interest corresponding to the frame of the heart image extracted by the target detection model is accurate;

[0100] If so, the location information of the target heart valve region of interest corresponding to the frame of cardiac image extracted by the target detection model is used as the corrected location information of the target heart valve region of interest corresponding to the frame of cardiac image.

[0101] If not, then based on the location information of the target heart valve region of interest corresponding to the previous frame of the cardiac image with accurate location information and the location information of the target heart valve region of interest corresponding to the next frame of the cardiac image with accurate location information, the corrected location information of the target heart valve region of interest corresponding to the current frame of the cardiac image is obtained.

[0102] Further, the step of performing curve fitting based on the location information of the target heart valve region of interest corresponding to each frame of the cardiac image extracted by the target detection model, to obtain the correspondence between the fitted image frame and the location information of the target heart valve region of interest, includes:

[0103] Based on the x-coordinate and y-coordinate information of the first corner point and the second corner point of the target heart valve region of interest corresponding to each frame of cardiac images extracted by the target detection model, curve fitting is performed on the x-coordinate, y-coordinate, and y-coordinate of the first corner point of the target heart valve region of interest, respectively. This is to obtain the correspondence between the fitted image frame and the x-coordinate, y-coordinate, and y-coordinate of the first corner point of the target heart valve region of interest, respectively.

[0104] Specifically, the first corner point can be the upper left corner of the target heart valve region of interest extracted by the target detection model in each frame of the cardiac image, and the second corner point can be the lower right corner of the target heart valve region of interest extracted by the target detection model in each frame of the cardiac image. Thus, the position information of the first corner point and the position information of the second corner point can represent the position information of the target heart valve region of interest. By curve fitting the abscissa of the first corner point of the target heart valve region of interest corresponding to each frame of cardiac images extracted by the target detection model, a curve representing the correspondence between the fitted image frame and the abscissa of the first corner point of the target heart valve region of interest can be obtained; by curve fitting the ordinate of the first corner point of the target heart valve region of interest corresponding to each frame of cardiac images extracted by the target detection model, a curve representing the correspondence between the fitted image frame and the ordinate of the first corner point of the target heart valve region of interest can be obtained; by curve fitting the abscissa of the second corner point of the target heart valve region of interest corresponding to each frame of cardiac images extracted by the target detection model, a curve representing the correspondence between the fitted image frame and the second corner point of the target heart valve region of interest can be obtained; by curve fitting the ordinate of the second corner point of the target heart valve region of interest corresponding to each frame of cardiac images extracted by the target detection model, a curve representing the correspondence between the fitted image frame and the ordinate of the second corner point of the target heart valve region of interest can be obtained. Therefore, based on these four fitted curves, we can obtain the x-coordinate and y-coordinate information of the first corner point and the second corner point of the fitted target heart valve region of interest corresponding to any frame of cardiac image, and thus obtain the position information of the fitted target heart valve region of interest corresponding to any frame of cardiac image.

[0105] In one exemplary embodiment, the step of correcting the position information of the target heart valve region of interest corresponding to each frame of the echocardiogram based on the correspondence between the fitted image frames and the position information of the target heart valve region of interest, to obtain the corrected position information of the target heart valve region of interest corresponding to each frame of the echocardiogram, includes:

[0106] For each frame of the heartbeat image:

[0107] Based on the correspondence between the fitted image frame and the location information of the target heart valve region of interest, the location information of the fitted target heart valve region of interest corresponding to the cardiac image frame is obtained.

[0108] The first positional deviation information corresponding to the cardiac image is obtained based on the absolute value of the difference between the positional information of the target heart valve region of interest corresponding to the cardiac image frame extracted by the target detection model and the positional information of the fitted target heart valve region of interest corresponding to the cardiac image frame.

[0109] Based on the first positional deviation information corresponding to the frame of cardiac images and the confidence probability value of the target heart valve region of interest extracted by the target detection model, the second positional deviation information corresponding to the frame of cardiac images is obtained.

[0110] Based on the second position deviation information corresponding to the frame of the heart image, it is determined whether the position information of the target heart valve region of interest corresponding to the frame of the heart image extracted by the target detection model is accurate;

[0111] If so, the location information of the target heart valve region of interest corresponding to the frame of cardiac image extracted by the target detection model is used as the corrected location information of the target heart valve region of interest corresponding to the frame of cardiac image.

[0112] If not, then based on the location information of the target heart valve region of interest corresponding to the previous frame of the cardiac image with accurate location information and the location information of the target heart valve region of interest corresponding to the next frame of the cardiac image with accurate location information, the corrected location information of the target heart valve region of interest corresponding to the current frame of the cardiac image is obtained.

[0113] Therefore, for each frame of cardiac image, based on the location information of the target heart valve region of interest extracted by the target detection model and the fitted location information of the target heart valve region of interest, the first positional deviation information corresponding to that frame of cardiac image is obtained. Then, based on the first positional deviation information and the confidence probability value of the target heart valve region of interest extracted by the target detection model, the second positional deviation information corresponding to that frame of cardiac image is obtained. Finally, based on the second positional deviation information, it is determined whether the location information of the target heart valve region of interest extracted by the target detection model for that frame of cardiac image is accurate. This allows for a more accurate determination of the accuracy of the location information of the target heart valve region of interest extracted by the target detection model for that frame of cardiac image, effectively avoiding misjudgments and improving the correction effect of the location information of the target heart valve region of interest. This further lays a good foundation for obtaining accurate cardiac video classification results.

[0114] Furthermore, the step of obtaining the second positional deviation information corresponding to the cardiac image frame based on the first positional deviation information corresponding to the frame and the confidence probability value of the target heart valve region of interest extracted by the target detection model, includes:

[0115] The second positional deviation information corresponding to this frame of the heartbeat image is obtained according to the following formula:

[0116] e i =E i *(1-p i )

[0117] In the formula, e i E represents the second positional deviation corresponding to the i-th frame of the heartbeat image. i p represents the first positional deviation corresponding to the i-th frame of the heartbeat image. i This represents the confidence probability value of the target heart valve region of interest extracted by the target detection model from the i-th frame of the cardiac image.

[0118] Specifically, taking the i-th frame of the heart rate image as an example, assuming that the location information of the target heart valve region of interest corresponding to the i-th frame of the heart rate image extracted by the target detection model is (w 1i ,h 1i ,w 2i ,h 2i ), where w 1i h 1i w 2i h 2iLet w' represent the x-coordinate of the first corner point, the y-coordinate of the first corner point, the x-coordinate of the second corner point, and the y-coordinate of the second corner point of the target heart valve region of interest corresponding to the i-th frame of the cardiac image extracted by the target detection model; the position information of the fitted target heart valve region of interest corresponding to the i-th frame of the cardiac image obtained according to the fitting result is (w' 1i ,h' 1i ,w' 2i ,h' 2i ), where w' 1i h' 1i w' 2i h' 2i Let |w| represent the x-coordinate, y-coordinate, and y-coordinate of the first corner point, the second corner point, and the third corner point, respectively, of the fitted region of interest (ROI) corresponding to the i-th frame of the cardiac image. Then, the absolute value of the difference between the x-coordinate of the first corner point of the ROI extracted by the target detection model and the x-coordinate of the first corner point of the fitted ROI of the ROI is |w|. 1i -w' 1i The absolute value of the difference between the ordinate of the first corner point of the target heart valve region of interest extracted by the target detection model and the ordinate of the first corner point of the fitted target heart valve region of interest is |h 1i -h' 1i The absolute value of the difference between the x-coordinate of the second corner point of the target heart valve region of interest extracted by the target detection model and the x-coordinate of the fitted target heart valve region of interest is |w 2i -w' 2i The absolute value of the difference between the ordinate of the second corner point of the target heart valve region of interest extracted by the target detection model and the ordinate of the second corner point of the fitted target heart valve region of interest is |h 2i -h' 2i |;That is, the first positional deviation E corresponding to the i-th frame of the heartbeat image. i for:

[0119] E i =(|w 1i -w' 1i |,|h 1i -h' 1i |,|w 2i -w' 2i |,|h 2i -h' 2i |)

[0120] Then the second position deviation e corresponding to the i-th frame of the heart image i for:

[0121] e i =(|w 1i -w' 1i |*(1-p i ),|h 1i -h' 1i |*(1-p i ),|w 2i -w' 2i |*(1-p i ),|h 2i -h' 2i |*(1-p i ))

[0122] In one exemplary embodiment, determining whether the location information of the target heart valve region of interest corresponding to the frame of the cardiac image extracted by the target detection model is accurate based on the second positional deviation information corresponding to the frame of the cardiac image includes:

[0123] Based on the first position deviation information corresponding to each frame of the heartbeat image, the mean first position deviation information corresponding to the heartbeat video is obtained;

[0124] The average confidence probability value of the target heart valve region of interest corresponding to each frame of the heart rate image is extracted based on the target detection model, and the average confidence probability value corresponding to the heart rate video is obtained.

[0125] Based on the mean first positional deviation information corresponding to the heartbeat video and the mean confidence probability corresponding to the heartbeat video, the mean second positional deviation information corresponding to the heartbeat video is obtained.

[0126] The position judgment threshold is obtained based on the preset multiple threshold and the average value of the second position deviation corresponding to the heartbeat video;

[0127] For each frame of the cardiac motion video, based on the second position deviation information corresponding to that frame of cardiac motion image and the position judgment threshold, it is determined whether the position information of the target heart valve region of interest corresponding to that frame of cardiac motion image extracted by the target detection model is accurate.

[0128] Specifically, the average first position deviation of the cardiac video is obtained by averaging the first position deviations corresponding to each frame of cardiac images; the average confidence probability of the cardiac video is obtained by averaging the confidence probability values ​​corresponding to each frame of cardiac images; the product of the average first position deviation and the average confidence probability of the cardiac video is the average second position deviation of the cardiac video; the product of the average second position deviation and a preset multiple threshold is the position judgment threshold. Since the position judgment threshold is obtained based on the average second position deviation of the cardiac video and the preset multiple threshold, the position judgment threshold is different for different cardiac videos. That is, the position judgment threshold in this invention is dynamically changing, which can further improve the correction effect of the position information of the target heart valve region of interest, laying a good foundation for obtaining accurate position information of the target heart valve region of interest.

[0129] Assuming the heartbeat video comprises n frames of heartbeat images, then the average first position deviation corresponding to the heartbeat video is... It can be represented as:

[0130]

[0131] The mean confidence probability corresponding to the heartbeat video It can be represented as:

[0132]

[0133] The mean of the second position deviation corresponding to the heartbeat video It can be represented as:

[0134]

[0135] Assuming the preset magnification factor is m, then the position determination threshold T corresponding to the heartbeat video is... th It can be represented as:

[0136]

[0137] It should be noted that, as those skilled in the art will understand, during location determination, the x-coordinate, y-coordinate, and y-coordinate of the first corner point, the second corner point, and the third corner point of the target heart valve region of interest are determined respectively, and the x-coordinate, y-coordinate, and y-coordinate of the first corner point, the second corner point, and the third corner point of the target heart valve region of interest are corrected accordingly based on the determination results. Specifically, taking the i-th frame of the echocardiogram as an example, if the x-coordinate of the first corner point of the target heart valve region of interest corresponding to the i-th frame of the echocardiogram satisfies the following condition: |w1i -w' 1i |*(1-p i (greater than) This indicates that the x-coordinate error of the first corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is large. Therefore, the average of the x-coordinates of the first corner point of the target heart valve region of interest in the previous frame (i.e., the x-coordinates of the first corner point of the region of interest extracted by the target detection model) and the x-coordinates of the first corner point of the target heart valve region of interest in the next frame (i.e., the x-coordinates of the first corner point of the region of interest extracted by the target detection model) is taken as the corrected x-coordinate of the first corner point of the target heart valve region of interest in the i-th frame of the cardiac image; if |w 1i -w' 1i |*(1-p i Less than or equal to This indicates that the x-coordinate of the first corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is accurate. Therefore, the x-coordinate of the first corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is directly used as the x-coordinate of the first corner point of the target heart valve region of interest in the i-th frame of the cardiac image after correction.

[0138] If the ordinate of the first corner point of the region of interest of the target heart valve corresponding to the i-th frame of the cardiac image satisfies the following condition: |h 1i -h' 1i |*(1-p i (greater than) This indicates that the error in the ordinate of the first corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is large. Therefore, the average of the ordinates of the first corner point of the target heart valve region of interest in the previous frame (where the ordinates are accurate, i.e., the ordinates of the first corner point of the region of interest extracted by the target detection model are accurate) and the ordinates of the first corner point of the target heart valve region of interest in the next frame (where the ordinates are accurate, i.e., the ordinates of the first corner point of the region of interest extracted by the target detection model are accurate) is taken as the corrected ordinate of the first corner point of the target heart valve region of interest in the i-th frame of the cardiac image; if |h 1i -h'1i |*(1-p i Less than or equal to This indicates that the ordinate of the first corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is accurate. Therefore, the ordinate of the first corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is directly used as the ordinate of the first corner point of the target heart valve region of interest in the i-th frame of the cardiac image after correction.

[0139] If the x-coordinate of the second corner point of the region of interest of the target heart valve corresponding to the i-th frame of the cardiac image satisfies the following condition: |w 2i -w' 2i |*(1-p i (greater than) This indicates that the x-coordinate error of the second corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is large. Therefore, the average of the x-coordinates of the second corner point of the target heart valve region of interest in the previous frame (i.e., the x-coordinates of the second corner point of the region of interest extracted by the target detection model are accurate) and the x-coordinates of the second corner point of the target heart valve region of interest in the next frame (i.e., the x-coordinates of the second corner point of the region of interest extracted by the target detection model are accurate) is taken as the corrected x-coordinate of the second corner point of the target heart valve region of interest in the i-th frame of the cardiac image; if |w 2i -w' 2i |*(1-p i Less than or equal to This indicates that the x-coordinate of the second corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is accurate. Therefore, the x-coordinate of the second corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is directly used as the x-coordinate of the second corner point of the target heart valve region of interest in the i-th frame of the cardiac image after correction.

[0140] If the ordinate of the second corner point of the region of interest of the target heart valve corresponding to the i-th frame of the cardiac image satisfies the following condition: |h 2i -h' 2i |*(1-p i (greater than) This indicates that the error in the ordinate of the second corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is large. Therefore, the average of the ordinates of the second corner point of the target heart valve region of interest in the previous frame (where the ordinates of the second corner point are accurate, i.e., the ordinates of the second corner point of the region of interest extracted by the target detection model are accurate) and the ordinates of the second corner point of the target heart valve region of interest in the next frame (where the ordinates of the second corner point are accurate, i.e., the ordinates of the second corner point of the region of interest extracted by the target detection model are accurate) is taken as the corrected ordinate of the second corner point of the target heart valve region of interest in the i-th frame of the cardiac image; if |h 2i -h' 2i |*(1-p i Less than or equal to This indicates that the ordinate of the second corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is accurate. Therefore, the ordinate of the second corner point of the target heart valve region of interest extracted by the target detection model in the i-th frame of the cardiac image is directly used as the ordinate of the second corner point of the target heart valve region of interest in the i-th frame of the cardiac image after correction.

[0141] In one exemplary implementation, the object detection model is a ResNet50 neural network model. Because ResNet uses skip connections (or shortcuts), it directly passes the activation values ​​of one network layer to deeper layers. Furthermore, skip connections only transmit data; through skip connections, the signal can be transmitted without attenuation during backpropagation, without worrying about gradient changes, thus enabling the transmission of effective gradients to the next layer. Therefore, skip connections effectively alleviate the gradient vanishing problem caused by deepening network layers. By stacking residual blocks, very deep network models can be constructed, allowing for effective training even at deep network layers.

[0142] Further, the step of cropping the corresponding target heart valve region of interest from each frame of the echocardiogram based on the location information of the corrected target heart valve region of interest corresponding to each frame of the echocardiogram to obtain the corresponding target heart valve region of interest image includes:

[0143] Based on the location information of the corrected target heart valve region of interest corresponding to each frame of cardiac images, calculate the location information of the target heart valve region of interest after being magnified by a preset factor for each frame of cardiac images.

[0144] The location information of the target heart valve region of interest after being magnified by a preset factor is used as the location information of the target heart valve region of interest corresponding to the frame of the cardiac image.

[0145] While object detection models can identify the target heart valve region of interest in echocardiograms, providing initial localization for subsequent segmentation models, they also result in the loss of detailed information such as the surrounding tissue. Therefore, this invention calculates the location information of the target heart valve region of interest after magnification based on the corrected location information of the target heart valve region of interest in each frame of the echocardiogram. Specifically, the bounding box of the corrected target heart valve region of interest is magnified by a predetermined factor, such as 1.3 times, to obtain the magnified bounding box. The area defined by this magnified bounding box is the final target heart valve region of interest. Since the area defined by this magnified bounding box includes detailed information such as the surrounding tissue, the segmentation accuracy of subsequent segmentation models can be further improved. It should be noted that, as those skilled in the art will understand, the center position of the magnified bounding box is the same as the center position of the unmagnified bounding box.

[0146] In one exemplary embodiment, the method further includes, prior to segmenting the region of interest image of the target heart valve using a segmentation model:

[0147] The target side length is defined by the length dimension of the region of interest image of the target heart valve.

[0148] The region of interest image of the target heart valve is filled along the width direction to adjust the width dimension of the region of interest image of the target heart valve to the target side length dimension;

[0149] The target heart valve region of interest image is magnified or reduced by adjusting the width dimension to the target side length dimension, so as to adjust the size of the target heart valve region of interest image to a preset size.

[0150] When the valve segmentation model is a neural network model, since neural network models require images of a uniform size as input, adjusting the size of the target heart valve region of interest image to a preset size can meet the input requirements of the valve segmentation model. Specifically, the preset size can be set according to specific circumstances. As a preferred embodiment, in the preset size, the length and width dimensions of the image are consistent, that is, the image after adjustment to the preset size is a square image, for example, the preset size is 320*320. Therefore, by setting the length and width dimensions in the preset size to be consistent, it is easier to adjust the size of the target heart valve region of interest image to the preset size.

[0151] For details, please refer to Figure 2b and Figure 2c ,in Figure 2c The illustration shows the... Figure 2b Image of the region of interest of the target heart valve (aortic valve) after filling. Figure 2b and Figure 2c As shown, the target heart valve region of interest image can be filled with black pixels (pixel value 0) along the width direction to adjust the width dimension of the target heart valve region of interest image to be consistent with the length dimension, that is, to adjust the target heart valve region of interest image to a square image, and then enlarge or reduce the target heart valve region of interest image to a certain factor.

[0152] In one exemplary embodiment, the valve annulus segmentation model is a DenseNet neural network model. Since the DenseNet neural network model is a densely connected convolutional neural network, where the input of each layer comes from the outputs of all preceding layers, this neural network structure enhances feature transfer and utilizes features more effectively. Furthermore, the DenseNet neural network model has good anti-overfitting performance, making it particularly suitable for applications with relatively scarce training data. Therefore, using the DenseNet neural network model as the valve annulus segmentation model in this invention can effectively improve the segmentation efficiency and accuracy of the inner and outer contours of the target heart valve (e.g., the aortic valve). Specifically, the DenseNet neural network model consists of multiple densely connected blocks connected by transition blocks; that is, any two adjacent densely connected blocks are connected by a transition block, and the number of convolutional output channels within each densely connected block is consistent, facilitating the superposition of feature information from each layer.

[0153] One layer in a densely connected block is called a bottleneck layer. Dense connections in DenseNet connect each layer in a densely connected block to all subsequent layers, enabling feature reuse.

[0154] Please continue to refer to this. Figure 3 The diagram illustrates the structure of a lobe ring segmentation model provided in a specific example of the present invention. Figure 3As shown, in this example, the lobe ring segmentation model includes a first convolutional layer, a first pooling layer (preferably a max pooling layer), a first dense connection block, a first transition block, a second dense connection block, a second transition block, a third dense connection block, a third transition block, a fourth dense connection block, a first upward transition block, a second upward transition block, and a second convolutional layer (with a kernel size of 1×1) connected in sequence. In this configuration, the first convolutional layer extracts features such as the valve annulus boundary and texture of the target heart valve (e.g., aortic valve) from the input image. The first pooling layer performs pooling operations on the output of the first convolutional layer to remove unnecessary redundant information, such as background noise, from the image. The first dense connection block extracts valve annulus features of the target heart valve (e.g., aortic valve) from the output of the first pooling layer. The first transition block compresses the output of the first dense connection block to reduce the size of the feature map output by the first dense connection block. The second dense connection block extracts valve annulus features of the target heart valve (e.g., aortic valve) from the output of the first transition block. The second transition block compresses the output of the second dense connection block to reduce the size of the feature map output by the second dense connection block. The third dense connection block... The first upward transition block is used to extract the annular features of the target heart valve (e.g., aortic valve) from the output of the second transition block. The third transition block is used to compress the output of the third dense connection block to reduce the size of the feature map output by the third dense connection block. The fourth dense connection block is used to extract the annular features of the target heart valve (e.g., aortic valve) from the output of the third transition block. The first upward transition block is used to perform a deconvolution operation on the output of the fourth dense connection block to increase the size of the feature map output by the fourth dense connection block. The second upward transition block is used to perform a deconvolution operation on the output of the first upward transition block to increase the size of the feature map output by the first upward transition block. The second convolutional layer is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the segmentation result of the annulus of the target heart valve (e.g., aortic valve).

[0155] Specifically, the second convolutional layer can perform a non-linear mapping regression on the output of the second upward transition block using the sigmoid function, the formula of which is shown below:

[0156]

[0157] As shown in the above equation, the Sigmoid function can map any input real number to the real number mapping interval (0,1). When the input value x is large, the output value g tends to 1, and when the input value x is small, the output value g tends to 0.

[0158] It should be noted that, as those skilled in the art will understand, the first dense connection block, the second dense connection block, the third dense connection block, and the fourth dense connection block all include multiple bottleneck layers, and the number of bottleneck layers in the first dense connection block, the second dense connection block, the third dense connection block, and the fourth dense connection block can be the same or different. The specific number can be set according to actual needs, and the present invention does not limit this. For example, the first dense connection block may have 6 bottleneck layers, the second dense connection block may have 12 bottleneck layers, the third dense connection block may have 24 bottleneck layers, and the fourth dense connection block may have 16 bottleneck layers.

[0159] Please continue to refer to this. Figure 4 The diagram illustrates the structure of the bottleneck layer provided in a specific example of the present invention. Figure 4 As shown, the bottleneck layer comprises a first batch normalization layer A, a first activation layer A, a third convolutional layer A, a first batch normalization layer B, a first activation layer B, and a third convolutional layer B connected in sequence. The kernel size of the third convolutional layer A is 1×1, and the kernel size of the third convolutional layer B is 3×3. Therefore, by adding a 1×1 convolution before the 3×3 convolution in the bottleneck layer, this invention reduces the number of feature maps and the dimensionality of each feature map, thereby reducing computational cost and fusing features from various channels. Furthermore, since the bottleneck layer performs batch normalization (BN) and ReLU activation operations before both the 1×1 and 3×3 convolution operations, training speed and convergence efficiency can be further improved.

[0160] Please continue to refer to this. Figure 5 The diagram illustrates the structure of a transition block provided in a specific example of the present invention. Figure 5 As shown, the first transition block, the second transition block, and the third transition block each include a second batch normalization layer, a second activation layer, a fourth convolutional layer, and a second pooling layer (preferably an average pooling layer) connected in sequence. The kernel size of the fourth convolutional layer is 1×1. Thus, the convolutional operation of the fourth convolutional layer can reduce the dimensionality of the feature map, and the average pooling operation of the second pooling layer can solve the problem of excessive channels in the feature map, preventing model complexity caused by too many densely connected blocks. Furthermore, since each transition block performs batch normalization (BN) and ReLU activation operations before the 1×1 convolutional operation, the number of parameters can be further compressed.

[0161] Please continue to refer to this. Figure 6 The diagram illustrates the structure of an upward transition block provided in a specific example of the present invention. Figure 6As shown, both the first upward transition block and the second upward transition block include a third batch normalization layer A, a third activation layer A, a fifth convolutional layer A, a third batch normalization layer B, a third activation layer B, a fifth convolutional layer B, a third batch normalization layer C, a third activation layer C, and a first deconvolutional layer connected in sequence. The size of the convolutional kernels of the fifth convolutional layer A and the fifth convolutional layer B is 3×3.

[0162] Furthermore, the samples used in the training process of the valve annulus segmentation model are sample cardiac images with the target heart valve region of interest (ROI) labeled, and the corresponding target heart valve annulus mask image. Specifically, the OpenCV contour extraction algorithm can be used to find the target heart valve annulus contour in the ROI of the target heart valve in the sample cardiac images, so as to segment the target heart valve annulus mask image. It should be noted that, as those skilled in the art will understand, since the neural network model requires images of a uniform size as input, it is necessary to convert the sample cardiac images with the ROI labeled and their corresponding target heart valve annulus mask images to a preset size, such as 320×320.

[0163] In one exemplary implementation, the loss function used during training of the loop segmentation model is the binary cross-entropy loss function, the formula of which is shown below:

[0164]

[0165]

[0166] In the formula, y i For real labels, This is the predicted result.

[0167] Furthermore, after training the loop segmentation model, this invention also uses the Dice coefficient formula to evaluate the algorithm accuracy of the loop segmentation model. The Dice coefficient formula is as follows:

[0168]

[0169] In the formula, X represents the prediction result, and Y represents the true label.

[0170] The value of Dice ranges from 0 to 1. The closer the Dice value is to 1, the higher the segmentation accuracy of the lobe ring segmentation model.

[0171] As an example, during the training of the lobe ring segmentation model, the learning rate is set to 1e-3 (i.e., 0.001), and Adam (adaptive moment estimation) is used as the optimizer. The learning rate of each parameter is dynamically adjusted using the first moment estimation and second moment estimation of the gradient. Clipnorm = 0.001 is added to the parameters of the optimizer to clip the gradient.

[0172] Please continue to refer to this. Figures 7a to 7d ,in Figure 7a An image of the region of interest of a target heart valve (aortic valve) in an open state, provided by a specific example of the present invention, is schematically given. Figure 7b The illustration shows the... Figure 7a Image of the target heart valve (aortic valve) annulus obtained by segmentation; Figure 7c An image of the region of interest of a target heart valve (aortic valve) in a closed state, provided by a specific example of the present invention, is schematically given. Figure 7d The illustration shows the... Figure 7c The segmented image of the target heart valve (aortic valve) annulus. Figures 7a to 7d As shown, by using the valve annulus segmentation model in this invention to segment the region of interest of the target heart valve corresponding to the cardiac image, the valve annulus image of the target heart valve corresponding to the cardiac image can be accurately obtained.

[0173] In one exemplary embodiment, for visual demonstration, the outline of the target heart valve (e.g., aortic valve) annulus can be drawn on the cardiac image, and a median filter is used to set the gray value of each pixel in the cardiac image to the median of the gray values ​​of all pixels within its neighborhood window. The size parameter of the filter kernel can be set according to specific circumstances, for example, to 5×5. Thus, the salt-and-pepper noise in the cardiac image can be effectively removed by using median filtering. It should be noted that, as those skilled in the art will understand, in other embodiments, other filtering methods besides median filtering can be used to filter the cardiac image, and this invention does not limit this to such methods.

[0174] In one exemplary embodiment, inputting the target heart valve annulus image into the valve generation model includes:

[0175] The annular state classification model is used to determine the open and closed states of the target heart valve corresponding to the annular image of the target heart valve.

[0176] If the target heart valve corresponding to the target heart valve annulus image is in an open state, then the target heart valve annulus image is input into the open valve generation model;

[0177] If the target heart valve corresponding to the target heart valve annulus image is in a closed state, then the target heart valve annulus image is input into the closed valve generation model.

[0178] Therefore, this invention first uses a valve annulus state classification model to determine the open and closed states of the target heart valve corresponding to the target heart valve annulus image, and then selects the corresponding valve generation model to generate a corresponding target heart valve mask based on the target heart valve annulus image, which can further improve the accuracy of the generated target heart valve mask. Please continue to refer to... Figure 7e and Figure 7f ,in Figure 7e According to Figure 7b The target heart valve (aortic valve) mask generated from the target heart valve (aortic valve) annulus image shown; Figure 7f According to Figure 7d The target heart valve (aortic valve) mask is generated from the target heart valve (aortic valve) annulus image shown. Figure 7e and Figure 7f As shown, by selecting the appropriate valve generation model to generate the corresponding target heart valve mask based on whether the target heart valve corresponding to the target heart valve annulus image is in an open or closed state, the accuracy of the generated target heart valve mask can be effectively improved. It should be noted that, as those skilled in the art will understand, in some other embodiments, the open or closed state of the target heart valve (aortic valve) corresponding to the target heart valve annulus image can be determined based on the opening area of ​​the target heart valve (aortic valve) mask.

[0179] In one exemplary embodiment, the valve annulus state classification model is also a DenseNet neural network model. The difference between the valve annulus state classification model and the valve annulus segmentation model described above is that in the valve annulus state classification model, the second convolutional layer is followed by a fully connected layer with 2 nodes and a softmax activation function. After passing through the fully connected layer and the softmax activation function, the valve annulus state classification model will output the probability that the target heart valve mask belongs to the open state and the closed state. Based on the preset probability threshold, it can be determined whether the input target heart valve annulus image belongs to the open state or the closed state.

[0180] In one exemplary implementation, the valve generation model is a generator in a generative adversarial network.

[0181] Generative Adversarial Networks (GANs) are a novel unsupervised architecture capable of generating highly realistic images. They consist of two independent networks: a generator and a discriminator. The generator aims to create "fake" images that closely resemble "real" images to deceive the discriminator, which distinguishes between real and fake images. During training, the discriminator receives both "real" images from the dataset and "fake" images generated by the generator. Its task is to classify the generator's images as 0 (fake) and the "real" images as 1 (real). The parameters of both the generator and discriminator can be fine-tuned based on the final output. If the discriminator correctly identifies the image, the generator's parameters are adjusted to make the generated "fake" images more realistic; if the discriminator misidentifies the image, its parameters are adjusted to prevent future errors. Training continues until both the generator and discriminator reach a balanced and harmonious state. The trained generative adversarial network includes a high-quality automatic generator and a discriminator with strong judgment capabilities. Therefore, by employing the generator from the trained generative adversarial network as the valve generation model, this invention enables the valve generation model to generate more realistic heart valve images, thereby better assisting doctors in improving diagnostic efficiency.

[0182] In one exemplary implementation, the valve generation model is trained through the following steps:

[0183] Acquire training samples, which include images of heart valve annulus and corresponding heart valve label images;

[0184] The pre-acquired generative adversarial network is trained based on the training samples to obtain a trained generative adversarial network.

[0185] The generator in the trained generative adversarial network is used as the valve generation model.

[0186] Specifically, the open valve generation model and the closed valve generation model are trained on their respective pre-built generative adversarial networks using different training samples. For the open valve generation model, the training samples include heart valve annulus images obtained by segmenting heart images in the open state and corresponding heart valve label images. For the closed valve generation model, the training samples include heart valve annulus images obtained by segmenting heart images in the closed state and corresponding heart valve label images. It should be noted that, as those skilled in the art will understand, the structure of the generative adversarial network corresponding to the open valve generation model is the same as that corresponding to the closed valve generation model, and the training processes for both models are also largely the same.

[0187] Further, training the pre-acquired generative adversarial network based on the training samples includes:

[0188] The generator and discriminator in the generative adversarial network are trained using an alternating training method based on the training samples until the preset training termination condition is met.

[0189] Specifically, the parameters of the discriminator can be fixed first, and the generator can be trained: first, the heart valve annulus image in the training sample is used as the input of the generator in the generative adversarial network to generate the corresponding heart valve generated image; then, the heart valve generated image and the corresponding heart valve label image are used together as the input of the discriminator in the generative adversarial network to obtain the probability of the heart valve generated image; then, the loss function of the generator is calculated according to the probability, and the parameters of the generator are adjusted according to the loss function of the generator. After the generator is trained once or multiple times, the discriminator can be trained. Then, with the generator parameters fixed, the discriminator is trained: first, the heart valve annulus image in the training samples is used as input to the generator in the generative adversarial network to generate the corresponding heart valve generated image; then, the heart valve generated image and the corresponding heart valve label image are used together as input to the discriminator in the generative adversarial network to obtain the probability of the heart valve generated image; then, the loss function of the discriminator is calculated based on the probability, and then the parameters of the discriminator are adjusted based on the loss function of the discriminator. After the discriminator is trained once or multiple times, the generator can be trained.

[0190] Furthermore, the preset training termination condition is that the generator and the discriminator reach equilibrium.

[0191] Specifically, when the probability output by the discriminator is close to 0.5, it means that the discriminator cannot distinguish between true and false images and can only guess randomly, thus achieving a steady-state Nash equilibrium. At this point, the heart valve image generated by the generator is very close to the real heart valve image. It should be noted that, as those skilled in the art will understand, in some other embodiments, the preset training termination condition can also be that the number of training iterations reaches a preset number.

[0192] In one exemplary implementation, the generator uses a UNet-based network architecture. Please refer to [link / reference needed]. Figure 8 The diagram illustrates the network structure of a generator provided in a specific example of the present invention. Figure 8 As shown, the generator includes a decoding module and an encoding module; wherein, the decoding module includes multiple cascaded first neural network groups and a sixth convolutional layer (with a kernel size of 3×3), the first neural network group includes a cascaded seventh convolutional layer (with a kernel size of 3×3) and a max pooling layer, the seventh convolutional layer is used to extract image feature information from the image input to the first neural network group or the output image of the previous layer of the first neural network group, the max pooling layer is used to pool the output image of the seventh convolutional layer, and the sixth convolutional layer is used to extract image feature information from the output image of the deepest first neural network group. The encoding module includes multiple cascaded second neural network groups, an eighth convolutional layer (with a kernel size of 1×1), and a first output layer. The second neural network groups correspond one-to-one with the first neural network groups. Each second neural network group includes a cascaded second deconvolutional layer, a merging layer, and a ninth convolutional layer (with a kernel size of 3×3). The second deconvolutional layer is used to perform the reverse operation of the pooling operation of the corresponding max pooling layer in the decoding module. The merging layer is used to linearly add and merge the output image of the second deconvolutional layer with the output image of the corresponding seventh convolutional layer in the decoding module. The ninth convolutional layer is used to recover the image feature information lost during the pooling process of the corresponding max pooling layer in the decoding module. The eighth convolutional layer is used to perform logistic regression on the output result of the deepest ninth convolutional layer.

[0193] It should be noted that, Figure 8In the network structure of the generator shown, the number of first neural network groups in the decoding module and the number of second neural network groups in the encoding module are merely examples and should not be construed as limiting the implementation of this application. The number of first neural network groups in the decoding network and the number of second neural network groups in the encoding module can be set according to specific needs. It should be noted that, since encoding and decoding have a one-to-one correspondence, in the network structure of the generator provided in this application, the number of first neural network groups in the decoding module is equal to the number of second neural network groups in the encoding module. Furthermore, the number of seventh convolutional layers in the first neural network group and the number of ninth convolutional layers in the second neural network group are not limited to two; they can also be three or more, and this invention does not impose any limitations on them.

[0194] Please continue to refer to this. Figure 9 The diagram illustrates the structure of a valve segmentation model provided in one embodiment of the present invention. Figure 9As shown, in an exemplary embodiment, the valve segmentation model also employs the DenseNet neural network model. The difference between this valve segmentation model and the valve annulus segmentation model described above is that the valve segmentation model includes two input layers: a first input layer and a second input layer. The first input layer is used to receive the target heart valve mask, and the second input layer is used to receive the corresponding target heart valve region of interest image. Thus, the target heart valve mask and the target heart valve region of interest image input into the valve segmentation model can be fused to achieve feature fusion, thereby improving the segmentation accuracy of the final target heart valve segmentation image based on the shape prior information of the target heart valve mask. Specifically, the first convolutional layer is used to extract the annular features of the target heart valve from the region of interest image of the target heart valve; the first pooling layer is used to perform pooling operations on the output of the first convolutional layer; the first dense connection block is used to extract the annular features of the target heart valve from the output of the first pooling layer or the result of adding the output of the first pooling layer to the target heart valve mask; the first transition block is used to compress the output of the first dense connection block or the result of adding the output of the first dense connection block to the target heart valve mask; the second dense connection block is used to extract the annular features of the target heart valve from the output of the first transition block or the result of adding the output of the first transition block to the target heart valve mask; the second transition block is used to compress the output of the second dense connection block or the result of adding the output of the second dense connection block to the target heart valve mask; and the third... Dense connection blocks are used to extract target heart valve annulus features from the output of the second transition block or the result of adding the output of the second transition block to the target heart valve mask. The third transition block is used to compress the output of the third dense connection block or the result of adding the output of the third dense connection block to the target heart valve mask. The fourth dense connection block is used to extract target heart valve annulus features from the output of the third transition block or the result of adding the output of the third transition block to the target heart valve mask. The first upward transition block is used to deconvolve the output of the fourth dense connection block or the result of adding the output of the fourth dense connection block to the target heart valve mask. The second upward transition block is used to deconvolve the output of the first upward transition block. The second convolutional layer is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the target heart valve segmentation result. It should be noted that although... Figure 9Therefore, the target heart valve mask received by the first input layer is only fused with the output of the third transition block. However, as those skilled in the art will understand, in some other embodiments, the target heart valve mask may be fused with the output of any one or more of the second input layer, the first convolutional layer, the first pooling layer, the first dense connection block, the first transition block, the second dense connection block, the second transition block, the third dense connection block, the third transition block, and the fourth dense connection block.

[0195] Based on the same inventive concept, the present invention also provides an electronic device, please refer to... Figure 10 The diagram illustrates a block structure of an electronic device according to an embodiment of the present invention. Figure 10 As shown, the electronic device includes a processor 101 and a memory 103. The memory 103 stores a computer program. When the computer program is executed by the processor 101, it implements the heart valve image segmentation method described above.

[0196] like Figure 10 As shown, the electronic device also includes a communication interface 102 and a communication bus 104, wherein the processor 101, the communication interface 102, and the memory 103 communicate with each other via the communication bus 104. The communication bus 104 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 104 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 102 is used for communication between the aforementioned electronic device and other devices.

[0197] The processor 101 referred to in this invention can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 101 is the control center of the electronic device, connecting various parts of the electronic device through various interfaces and lines.

[0198] The memory 103 can be used to store the computer program. The processor 101 implements various functions of the electronic device by running or executing the computer program stored in the memory 103 and calling the data stored in the memory 103.

[0199] The memory 103 may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0200] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, can implement the heart valve image segmentation method described above.

[0201] The readable storage medium of embodiments of the present invention can be any combination of one or more computer-readable media. The readable medium can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer hard disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device.

[0202] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0203] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0204] In summary, compared with the prior art, the heart valve image segmentation method, electronic device, and storage medium provided by the present invention have the following advantages: The present invention first uses a target detection model to extract the region of interest (ROI) of the target heart valve in each frame of the acquired cardiac video to obtain the corresponding target heart valve ROI image; then, it uses a valve annulus segmentation model to segment the target heart valve ROI image to obtain the corresponding target heart valve annulus image; then, it inputs the target heart valve annulus image into a valve generation model to generate the corresponding target heart valve mask; finally, it inputs the target heart valve mask and its corresponding target heart valve ROI image into a valve segmentation model to obtain the corresponding target heart valve segmented image. Since the target heart valve mask in this invention is directly generated using a valve generation model based on the corresponding target heart valve annulus image, this target heart valve mask can be used as shape prior information and input together with the corresponding target heart valve region of interest image into the valve segmentation model. This not only improves the segmentation accuracy and continuity of the final segmented target heart valve image (e.g., aortic valve segmentation image), but also reduces the variability that may be caused by human factors. Furthermore, this invention enables an end-to-end algorithm flow and has strong versatility, thus better assisting doctors in improving diagnostic efficiency and reducing the risks caused by errors in using echocardiography for heart valve abnormality analysis.

[0205] It should be noted that computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0206] It should be noted that the apparatus and methods disclosed in the embodiments herein can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, program, or part of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system to perform the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0207] In addition, the functional modules in the various embodiments of this article can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0208] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for segmenting heart valve images, characterized in that, include: A target detection model was used to extract the region of interest (ROI) of the target heart valve from each frame of the acquired cardiac video to obtain the corresponding ROI image of the target heart valve. The region of interest image of the target heart valve is segmented using a valve annulus segmentation model to obtain the corresponding target heart valve annulus image; The target heart valve annulus image is input into the valve generation model to generate the corresponding target heart valve mask; The target heart valve mask and its corresponding target heart valve region of interest image are input into the valve segmentation model to obtain the corresponding target heart valve segmentation image. The step of inputting the target heart valve annulus image into the valve generation model to generate a corresponding target heart valve mask includes: The annular state classification model is used to determine the open and closed states of the target heart valve corresponding to the annular image of the target heart valve. If the target heart valve corresponding to the target heart valve annulus image is in an open state, then the target heart valve annulus image is input into the open valve generation model to generate the corresponding target heart valve mask. If the target heart valve corresponding to the target heart valve annulus image is in a closed state, then the target heart valve annulus image is input into the closed valve generation model to generate the corresponding target heart valve mask.

2. The method for segmenting heart valve images according to claim 1, characterized in that, The step involves using a target detection model to extract the region of interest (ROI) of the target heart valve from each frame of the acquired cardiac video, thereby obtaining the corresponding ROI image of the target heart valve. This includes: A target detection model was used to extract the region of interest (ROI) of the target heart valve in each frame of the acquired cardiac video to obtain the location information of the corresponding target heart valve ROI. Curve fitting is performed based on the location information of the target heart valve region of interest corresponding to each frame of cardiac images, and the location information of the target heart valve region of interest corresponding to each frame of cardiac images is corrected based on the fitting results. Based on the location information of the corrected target heart valve region of interest corresponding to each frame of the cardiac image, the corresponding target heart valve region of interest is cropped from each frame of the cardiac image to obtain the corresponding target heart valve region of interest image.

3. The method for segmenting heart valve images according to claim 2, characterized in that, The step of performing curve fitting based on the location information of the target heart valve region of interest corresponding to each frame of echocardiogram, and correcting the location information of the target heart valve region of interest corresponding to each frame of echocardiogram based on the fitting result, includes: Based on the location information of the target heart valve region of interest corresponding to each frame of cardiac images extracted by the target detection model, curve fitting is performed to obtain the correspondence between the fitted image frames and the location information of the target heart valve region of interest. Based on the correspondence between the fitted image frames and the location information of the target heart valve region of interest, the location information of the target heart valve region of interest corresponding to each frame of cardiac images is corrected to obtain the corrected location information of the target heart valve region of interest corresponding to each frame of cardiac images.

4. The method for segmenting heart valve images according to claim 3, characterized in that, The step of correcting the position information of the target heart valve region of interest corresponding to each frame of the cardiac image based on the correspondence between the fitted image frames and the position information of the target heart valve region of interest, to obtain the corrected position information of the target heart valve region of interest corresponding to each frame of the cardiac image, includes: For each frame of the heartbeat image: Based on the correspondence between the fitted image frame and the location information of the target heart valve region of interest, the location information of the fitted target heart valve region of interest corresponding to the cardiac image frame is obtained. The first positional deviation information corresponding to the cardiac image is obtained based on the absolute value of the difference between the positional information of the target heart valve region of interest corresponding to the cardiac image frame extracted by the target detection model and the positional information of the fitted target heart valve region of interest corresponding to the cardiac image frame. Based on the first positional deviation information corresponding to the frame of cardiac images and the confidence probability value of the target heart valve region of interest extracted by the target detection model, the second positional deviation information corresponding to the frame of cardiac images is obtained. Based on the second position deviation information corresponding to the frame of the heart image, it is determined whether the position information of the target heart valve region of interest corresponding to the frame of the heart image extracted by the target detection model is accurate; If so, the location information of the target heart valve region of interest corresponding to the frame of cardiac image extracted by the target detection model is used as the corrected location information of the target heart valve region of interest corresponding to the frame of cardiac image. If not, then based on the location information of the target heart valve region of interest corresponding to the previous frame of the cardiac image with accurate location information and the location information of the target heart valve region of interest corresponding to the next frame of the cardiac image with accurate location information, the corrected location information of the target heart valve region of interest corresponding to the current frame of the cardiac image is obtained.

5. The method for segmenting heart valve images according to claim 4, characterized in that, The step of determining whether the location information of the target heart valve region of interest corresponding to the frame of the cardiac image extracted by the target detection model is accurate based on the second positional deviation information corresponding to the frame of the cardiac image includes: Based on the first position deviation information corresponding to each frame of the heartbeat image, the mean first position deviation information corresponding to the heartbeat video is obtained; The average confidence probability value of the target heart valve region of interest corresponding to each frame of the heart rate image is extracted based on the target detection model, and the average confidence probability value corresponding to the heart rate video is obtained. Based on the mean first positional deviation information corresponding to the heartbeat video and the mean confidence probability corresponding to the heartbeat video, the mean second positional deviation information corresponding to the heartbeat video is obtained. The position judgment threshold is obtained based on the preset multiple threshold and the average value of the second position deviation corresponding to the heartbeat video; For each frame of the cardiac motion video, based on the second position deviation information corresponding to that frame of cardiac motion image and the position judgment threshold, it is determined whether the position information of the target heart valve region of interest corresponding to that frame of cardiac motion image extracted by the target detection model is accurate.

6. The method for segmenting heart valve images according to claim 1, characterized in that, Before segmenting the target heart valve region of interest image using a valve annulus segmentation model, the segmentation method further includes: The target side length is defined by the length dimension of the region of interest image of the target heart valve. The region of interest image of the target heart valve is filled along the width direction to adjust the width dimension of the region of interest image of the target heart valve to the target side length dimension; The target heart valve region of interest image is magnified or reduced by adjusting the width dimension to the target side length dimension, so as to adjust the size of the target heart valve region of interest image to a preset size.

7. The method for segmenting heart valve images according to claim 1, characterized in that, The valve generation model is a generator in a generative adversarial network.

8. The method for segmenting heart valve images according to claim 1, characterized in that, The valve generation model is trained through the following steps: Acquire training samples, which include images of heart valve annulus and corresponding heart valve label images; The pre-acquired generative adversarial network is trained based on the training samples to obtain a trained generative adversarial network. The generator in the trained generative adversarial network is used as the valve generation model.

9. The method for segmenting heart valve images according to claim 8, characterized in that, The step of training the pre-acquired generative adversarial network based on the training samples includes: The generator and discriminator in the generative adversarial network are trained using an alternating training method based on the training samples until the preset training termination condition is met.

10. The method for segmenting heart valve images according to claim 9, characterized in that, The preset training termination condition is that the generator and the discriminator reach equilibrium.

11. The method for segmenting heart valve images according to claim 1, characterized in that, The valve segmentation model includes a first input layer, a second input layer, a first convolutional layer, a first pooling layer, a first dense connection block, a first transition block, a second dense connection block, a second transition block, a third dense connection block, a third transition block, a fourth dense connection block, a first upward transition block, a second upward transition block, and a second convolutional layer. The first input layer is used to receive the target heart valve mask, and the second input layer is used to receive the corresponding target heart valve region of interest image. The first convolutional layer is used to extract the annular features of the target heart valve from the region of interest image of the target heart valve. The first pooling layer is used to perform pooling operations on the output of the first convolutional layer. The first dense connection block is used to extract the annular features of the target heart valve from the output of the first pooling layer or the result of adding the output of the first pooling layer to the target heart valve mask. The first transition block is used to compress the output of the first dense connection block or the result of adding the output of the first dense connection block to the target heart valve mask. The second dense connection block is used to extract the annular features of the target heart valve from the output of the first transition block or the result of adding the output of the first transition block to the target heart valve mask. The second transition block is used to compress the output of the second dense connection block or the result of adding the output of the second dense connection block to the target heart valve mask. The third dense connection block... The connecting block is used to extract the target heart valve annulus features from the output of the second transition block or the result of adding the output of the second transition block to the target heart valve mask. The third transition block is used to compress the output of the third dense connecting block or the result of adding the output of the third dense connecting block to the target heart valve mask. The fourth dense connecting block is used to extract the target heart valve annulus features from the output of the third transition block or the result of adding the output of the third transition block to the target heart valve mask. The first upward transition block is used to deconvolve the output of the fourth dense connecting block or the result of adding the output of the fourth dense connecting block to the target heart valve mask. The second upward transition block is used to deconvolve the output of the first upward transition block. The second convolutional layer is used to perform nonlinear mapping regression on the output of the second upward transition block to obtain the target heart valve segmentation result.

12. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the method of any one of claims 1 to 11.

13. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images