Left ventricle motion estimation method, system and device based on deep learning and medium

Abstract

The invention provides a left ventricle motion estimation method, system and device based on deep learning and a medium, and relates to the technical field of medical image processing. According to the heart image, carrying out segmentation processing by using an R2UNet model and carrying out feature fusion by using a multi-layer perceptron model to obtain comprehensive information of a key point set of the left ventricular intimal contour image; according to the comprehensive information of the key point set, performing feature aggregation by using a graph neural network and performing feature matching by using a Sinkhorn algorithm to obtain a diastolic-systolic deformation relationship of the left ventricular intimal contour; and according to the diastolic-systolic deformation relationship of the left ventricular intimal contour, estimating the diastolic-systolic motion of the left ventricular intimal contour to obtain a left ventricular motion estimation result. According to the method, the R2UNet model and the Sinkhorn algorithm are combined, so that the segmentation precision of the heart image and the accuracy of feature matching are improved, and the accuracy of left ventricle motion estimation is improved.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a left ventricular motion estimation method, system, device and medium based on deep learning. Background Art

[0002] Cardiac medical image processing technology plays an important role in the early diagnosis and treatment of cardiovascular diseases. It can accurately track the changes in the myocardial motion of the left ventricle and assist doctors in making more accurate medical decisions. However, existing technologies such as methods based on deformation models, methods based on image registration, and methods based on feature tracking are highly sensitive to image quality. They are not robust enough when dealing with abnormal cardiac morphology, poor image quality, and sparse features. It is difficult to obtain discriminative features in cardiac images, resulting in low matching accuracy and difficulty in obtaining accurate motion estimation results, which limits their robustness and applicability. Summary of the invention

[0003] The present application provides a left ventricular motion estimation method, system, device and medium based on deep learning to solve one or more technical problems existing in the prior art, and at least provide beneficial options or create conditions.

[0004] On the one hand, the present application provides a left ventricular motion estimation method based on deep learning, comprising the following steps: Acquire cardiac images; According to the heart image, segmentation processing is performed using the R2UNet model to obtain a key point set of the left ventricular endocardial contour image; the key point set includes an end-diastolic key point subset and an end-systolic key point subset; According to the key point set, a multi-layer perceptron model is used to perform feature fusion to obtain comprehensive information of the key point set; According to the comprehensive information of the key point set, feature aggregation is performed using a graph neural network model to obtain embedded information of the key point set; According to the embedded information and comprehensive information of the key point set, feature matching is performed in combination with the Sinkhorn algorithm to obtain the diastolic-systolic deformation relationship of the left ventricular endocardial contour; The diastolic-systolic motion of the left ventricular endocardial contour is estimated according to the diastolic-systolic deformation relationship of the left ventricular endocardial contour to obtain a left ventricular motion estimation result.

[0005] Furthermore, the segmentation process is performed using the R2UNet model according to the cardiac image to obtain a set of key points of the left ventricular endocardial contour image, including: According to the heart image, segmentation processing is performed using the R2UNet model to obtain the left ventricular endocardial contour image; According to the left ventricular endocardial contour image and in combination with a key point sampling method, a key point set of the left ventricular endocardial contour image is obtained.

[0006] Furthermore, the step of performing feature fusion based on the key point set using a multi-layer perceptron model to obtain comprehensive information of the key point set includes: Performing feature detection on the key point set to obtain coordinate information, confidence information and attribute information of the key point set; According to the coordinate information, confidence information and attribute information of the key point set, the multi-layer perceptron model is used to perform feature fusion to obtain comprehensive information of the key point set.

[0007] Furthermore, the step of performing feature aggregation using a graph neural network model based on the comprehensive information of the key point set to obtain the embedded information of the key point set includes: According to the comprehensive information of the key point set, the self-attention mechanism of the graph neural network model is used to obtain the in-graph aggregation features of the key point set; According to the comprehensive information of the key point set, the cross-graph aggregation features of the key point set are obtained by utilizing the cross-attention mechanism of the graph neural network model; Embedding information of the key point set is obtained according to the intra-graph aggregation features and cross-graph aggregation features of the key point set.

[0008] Further, the embedding information of the key point set includes end-diastolic embedding information and end-systolic embedding information; The embedding information and comprehensive information of the key point set are combined with the Sinkhorn algorithm to perform feature matching to obtain the diastolic-systolic deformation relationship of the left ventricular endocardial contour, including: Calculating the similarity between the end-diastolic embedded information and the end-systolic embedded information to obtain a score matrix; According to the score matrix, feature matching is performed in combination with the Sinkhorn algorithm to obtain a matching matrix; The diastolic-systolic deformation relationship of the left ventricular endocardial contour is obtained according to the comprehensive information of the matching matrix and the key point set.

[0009] Furthermore, the feature matching is performed according to the score matrix in combination with the Sinkhorn algorithm to obtain a matching matrix, including: Iteratively normalizing the score matrix using the Sinkhorn algorithm to obtain a converged double random matrix; According to the converged double random matrix, the matching matrix is ​​obtained; in the matching matrix, only one element in each row and each column is 1, and all other elements are 0; The matching matrix is ​​optimized by minimizing the negative log loss.

[0010] Furthermore, the diastolic-systolic deformation relationship of the left ventricular endocardial contour is obtained based on the comprehensive information of the matching matrix and the key point set, including: Obtaining a deformation function according to the comprehensive information of the matching matrix and the key point set; According to the deformation function, the diastolic-systolic deformation relationship of the left ventricular endocardium contour is obtained.

[0011] On the other hand, the present application provides a left ventricular motion estimation system based on deep learning, including: a data acquisition module, a segmentation processing module, a feature fusion module, a feature aggregation module, a feature matching module and a deformation estimation module; The data acquisition module is used to acquire a heart image; The segmentation processing module is used to perform segmentation processing based on the cardiac image using the R2UNet model to obtain a key point set of the left ventricular endocardial contour image; the key point set includes an end-diastolic key point subset and an end-systolic key point subset; The feature fusion module is used to perform feature fusion according to the key point set using a multi-layer perceptron model to obtain comprehensive information of the key point set; The feature aggregation module is used to perform feature aggregation using a graph neural network model according to the comprehensive information of the key point set to obtain embedded information of the key point set; The feature matching module is used to perform feature matching based on the embedded information and comprehensive information of the key point set in combination with the Sinkhorn algorithm to obtain the diastolic-systolic deformation relationship of the left ventricular endocardial contour; The deformation estimation module is used to estimate the diastolic-systolic motion of the left ventricular endocardial contour according to the diastolic-systolic deformation relationship of the left ventricular endocardial contour to obtain a left ventricular motion estimation result.

[0012] On the other hand, the present application provides a left ventricular motion estimation device based on deep learning, comprising: a processor and a memory; the memory is used to store a program; When the program is executed by the processor, the processor implements the aforementioned left ventricular motion estimation method based on deep learning.

[0013] On the other hand, the present application provides a computer-readable storage medium having a processor-executable program stored therein, wherein the processor-executable program is used to implement the aforementioned deep learning-based left ventricular motion estimation method when executed by the processor.

[0014] The beneficial effects of the present application are as follows: the present application provides a left ventricular motion estimation method based on deep learning, including: obtaining a cardiac image; according to the cardiac image, using the R2UNet model to perform segmentation processing to obtain a key point set of the left ventricular endocardial contour image; the key point set includes a subset of end-diastolic key points and a subset of end-systolic key points; according to the key point set, using a multi-layer perceptron model to perform feature fusion to obtain comprehensive information of the key point set; according to the comprehensive information of the key point set, using a graph neural network model to perform feature aggregation to obtain embedded information of the key point set; according to the embedded information and comprehensive information of the key point set, combining the Sinkhorn algorithm to perform feature matching to obtain the diastolic-systolic deformation relationship of the left ventricular endocardial contour; according to the diastolic-systolic deformation relationship of the left ventricular endocardial contour, the diastolic-systolic motion of the left ventricular endocardial contour is estimated to obtain a left ventricular motion estimation result. The present application improves the segmentation accuracy of cardiac image data and the accuracy of feature matching by combining the segmentation processing method and the feature matching method, thereby improving the accuracy of left ventricular motion estimation. The present application also provides corresponding devices, systems and media. The beneficial effects of the devices, systems and media are similar to those of the methods and will not be described repeatedly here.

[0015] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0017] Figure 1 is a flowchart of a left ventricular motion estimation method based on deep learning provided by the present application; Figure 2 It is a schematic diagram of the principle of determining the embedding information of a set of key points provided by the present application; Figure 3 It is a schematic diagram of the principle of performing feature matching on the embedded information and comprehensive information of the key point set provided by the present application to obtain the left ventricular motion estimation result; Figure 4 is a structural diagram of a left ventricular motion estimation system based on deep learning provided by the present application; Figure 5 This is a structural diagram of the left ventricular motion estimation device based on deep learning provided in this application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0020] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0022] Cardiovascular diseases have an effective impact on the structure and function of the heart, especially the morphology and function of the left ventricle. As the key chamber for pumping blood, the normal operation of the left ventricle is essential for maintaining heart function. However, common cardiovascular diseases such as coronary heart disease and hypertension can cause abnormalities in the structure of the left ventricle, such as myocardial hypertrophy or dilatation, which in turn affects the systolic and diastolic functions of the heart. Therefore, early and accurate estimation of left ventricular motion is of great significance for the diagnosis and treatment of heart disease. With the advancement of medical imaging technologies such as magnetic resonance imaging and computed tomography, computer-aided diagnosis has played an important role in medical imaging research. Traditionally, medical experts rely on personal experience to interpret medical images and provide diagnostic reports, but this method is easily affected by subjective factors and may lead to misdiagnosis or missed diagnosis. By introducing computer-aided technology, the error rate can be significantly reduced and help medical experts analyze the condition more accurately. In recent years, research on combining computer technology with medical imaging has become a hot area, aiming to improve diagnostic accuracy and treatment effects.

[0023] Current methods for analyzing and estimating left ventricular motion can be divided into three categories: methods based on deformation models, methods based on image registration, and methods based on feature tracking.

[0024] The deformation model-based method describes the left ventricular motion through geometric or topological models, and analyzes the left ventricular structure by calculating the strain information of different myocardial segments. Due to the complexity of cardiac motion and individual differences, it is relatively complicated to construct and parameterize such models, and it is difficult to flexibly respond to the pathological characteristics of different patients.

[0025] The image registration-based method uses cardiac images acquired at different time points to estimate the motion of the left ventricle by establishing a spatial transformation function or deformation mapping. This method performs well in terms of time series consistency, but it is highly dependent on image quality and often has difficulty in obtaining accurate motion estimation when the contours are not clear.

[0026] Feature tracking-based methods estimate left ventricular motion by extracting feature points from image sequences and tracking the displacement of these points at different time points. Traditional feature extraction methods include scale-invariant feature transform, accelerated robust features, and oriented gradient histograms. Such methods do not require complex model construction, can adapt to different image types, and are suitable for clinical applications. However, due to the homogeneity and feature sparsity of myocardial tissue, these methods still face challenges in establishing correspondence on the myocardial contour, and the accuracy of feature tracking is also highly dependent on the quality of the extracted features.

[0027] The scheme based on the fully convolutional network and graph matching method combines the image registration-based method and the feature tracking-based method. It first uses the fully convolutional network to segment the myocardial contour, and then establishes the correspondence between the feature points of the heart images at different time points through the graph matching method. Although this method performs well in the case of regular structures, its assumptions about the left ventricular morphology are too idealized and difficult to adapt to the abnormal morphology in actual applications.

[0028] Traditional feature extraction methods are based on feature tracking. They include scale-invariant feature transformation, accelerated robust features, and directional gradient histograms, which are used to extract local feature points in the image and track the changes of these points at different time points to estimate cardiac motion. This type of method does not require the construction of a complex three-dimensional model, has high computational efficiency and strong adaptability, but when dealing with blurred myocardial contours or high homogeneity, feature point extraction is insufficient, affecting matching accuracy.

[0029] The graph matching method is also based on feature tracking. It estimates motion by constructing a structured graph of the cardiac image and using the edge connections between feature points. However, the graph matching method is highly dependent on image quality. It is difficult to accurately find the correspondence between feature points in low-quality images, resulting in motion estimation deviation. In addition, the traditional graph matching method only relies on local feature points and lacks global information integration, which reduces the matching accuracy in areas with sparse or blurred features.

[0030] Although the graph matching method based on graph neural network has introduced the attention mechanism, its application in the field of cardiac image analysis is still relatively limited, especially its matching effect in the case of abnormal cardiac morphology has not been fully verified. In addition, the existing graph neural network method fails to fully combine global and local features when processing graph structures, lacks the ability to integrate multi-scale features, and affects the reliability of matching.

[0031] In response to the problems and defects in the related technologies, the embodiments of the present application provide a left ventricular motion estimation method, system, device and medium based on deep learning, aiming to improve the accuracy and robustness of left ventricular motion estimation based on deep learning, especially the performance when processing abnormal contours. The present application improves the segmentation accuracy and adaptability of cardiac image data by combining a segmentation processing method and a feature fusion method, and can effectively process left ventricular images with complex backgrounds and morphological abnormalities, and provide more accurate motion estimation. In addition, the present application extracts discriminative feature information from cardiac images by combining a feature aggregation method and a feature matching method, thereby improving the accuracy of matching, especially in the case of variable cardiac morphology or poor image quality, ensuring the high accuracy and consistency of the matching results.

[0032] First, the implementation steps of the left ventricular motion estimation method based on deep learning provided by an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0033] The left ventricular motion estimation method based on deep learning provided in the embodiment of the present application can be applied to a terminal, a server, or software running in a terminal or a server. The terminal can be a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms.

[0034] Reference Figure 1 The implementation process of the left ventricular motion estimation method based on deep learning provided in the embodiment of the present application may include but is not limited to the following steps.

[0035] Step 101, obtaining a heart image.

[0036] In step 101, cardiac images are obtained, which are the basis for all subsequent analyses. The cardiac images are usually obtained from medical imaging technologies such as magnetic resonance imaging or echocardiography, and can provide key information on cardiac structure and function.

[0037] Step 102: Perform segmentation processing using the R2UNet model based on the heart image to obtain a set of key points of the left ventricular endocardium contour image.

[0038] It should be noted that the key point set includes the end-diastolic key point subset and the end-systolic key point subset. The contour of the left ventricular endocardium can reflect the morphological changes of the heart at different stages of the cardiac cycle (such as end-diastole and end-systole). By analyzing the changes in these contours, the functional state of the heart can be evaluated, including the contractility of the myocardium and the changes in ventricular volume. Many cardiovascular diseases can cause abnormalities in cardiac structure and function, which are usually reflected in the contour of the left ventricular endocardium. For example, diseases such as myocardial infarction and cardiomyopathy can cause local or overall myocardial movement to weaken or disappear, thereby changing the normal contour of the left ventricular endocardium. Therefore, accurately identifying and matching the left ventricular endocardium contour can help to detect and diagnose these diseases at an early stage.

[0039] In step 102, by using R2UNet to perform segmentation processing, the left ventricular endocardial contour is accurately extracted from the heart image, and a subset of key points at the end of diastole and end of systole are identified. Segmentation processing is to separate the region of interest (such as the left ventricle) from the background and other structures for subsequent analysis. Effective segmentation can effectively improve the understanding of changes in myocardial motion.

[0040] R2UNet is an improved U-Net architecture. By introducing recursive residual units and jump connections, the performance of medical image segmentation is effectively improved. R2UNet adds recursive residual units to the traditional U-Net encoding-decoding structure. These units repeatedly learn attribute information through multiple convolution operations, thereby enhancing the ability to capture subtle structures and complex contours. This multi-scale feature extraction capability enables R2UNet to better handle the subtle structures of myocardial tissue, such as myocardial edges and lesion areas, and improves the accuracy and detail clarity of segmentation. In addition, the recursive residual unit reduces information loss through jump connections, retains key features, and ensures the accuracy and completeness of the segmentation results. When processing cardiac images with abnormal contours, R2UNet performs well, can more flexibly respond to the pathological characteristics of different patients, and enhances the robustness and adaptability of the model. The reason for choosing R2UNet for segmentation processing is its multi-scale feature extraction, enhanced feature learning ability, reduced information loss, and higher adaptability and segmentation accuracy. These characteristics make R2UNet particularly suitable for segmentation tasks in left ventricular motion estimation based on deep learning, providing a high-quality foundation for subsequent feature matching and motion estimation. In summary, R2UNet effectively improves segmentation accuracy and robustness through its advanced structural design, becoming an efficient and reliable medical image segmentation tool.

[0041] Step 103, based on the key point set, a multi-layer perceptron model is used to perform feature fusion to obtain comprehensive information of the key point set.

[0042] In step 103, a multi-layer perceptron model is used to perform feature fusion, combining the position information of each key point with its feature description to create a comprehensive feature representation. Feature fusion enhances the uniqueness of key points, making the matching between different time points more accurate and stable, ensuring that key points contain not only spatial position information, but also carry rich local attribute information, thereby improving matching accuracy.

[0043] Step 104, based on the comprehensive information of the key point set, feature aggregation is performed using a graph neural network model to obtain embedded information of the key point set.

[0044] It should be noted that the graph neural network model is a deep learning method specially designed to process graph structured data, and is particularly suitable for key point aggregation processing in left ventricular motion estimation based on deep learning. In left ventricular motion estimation based on deep learning, key points can be modeled as key points in the graph, and the relationship between key points is represented as edges. The graph neural network model updates the representation of each key point through information transfer and aggregation between key points, thereby capturing the spatial relationship and contextual information between key points. This method can effectively integrate local and global features and improve the description ability and matching accuracy of key points. Especially in the case of dealing with abnormal left ventricular morphology or blurred contours, the graph neural network model can better identify and match key points, enhancing the overall robustness and accuracy.

[0045] Compared with traditional feature extraction methods, the graph neural network model can not only process local key points, but also integrate global information to ensure global consistency in the matching process, thereby providing more accurate and reliable left ventricular motion estimation results based on deep learning. This global consistency and multi-scale feature integration capabilities make the graph neural network model an ideal choice for processing complex cardiac images, providing strong support for the diagnosis and treatment of heart disease.

[0046] In step 104, feature aggregation is performed using a graph neural network model to generate a higher level feature embedding based on the comprehensive information of each key point. This process is not just a simple combination of features, but rather a gradual refinement of feature representation by calculating similarity metrics and dynamically updating the relationship between key points. Feature aggregation helps capture global structural information so that key points not only retain local characteristics but also reflect their changing trends throughout the cardiac motion cycle. The feature aggregation method generates an embedding vector containing rich contextual information for each initial key point subset. This embedded information can better describe the role of the key points in cardiac motion and improve the accuracy and stability of subsequent matching.

[0047] Step 105, based on the embedded information and comprehensive information of the key point set, feature matching is performed in combination with the Sinkhorn algorithm to obtain the diastolic-systolic deformation relationship of the left ventricular endocardium contour.

[0048] In step 105, based on the embedded information and comprehensive information obtained in the previous steps, the distribution matrix between the feature points is calculated in combination with the Sinkhorn algorithm. This step ensures the one-to-one matching relationship of the feature points, avoids the problem of repeated matching or missing matching of feature points, and improves the accuracy and consistency of the estimation of the diastolic-systolic deformation relationship of the left ventricular endocardial contour.

[0049] Step 106, estimating the diastolic-systolic motion of the left ventricular endocardial contour according to the diastolic-systolic deformation relationship of the left ventricular endocardial contour, and obtaining a left ventricular motion estimation result.

[0050] In step 106, the movement of the left ventricle is estimated based on the diastolic-systolic deformation relationship of the left ventricular endocardial contour. The high-precision and robust matching results obtained in the above steps can more accurately track the changes of the left ventricle from diastolic to systolic state, and then evaluate its functional state. This is of great significance for the early diagnosis and treatment of heart disease and can provide valuable reference information for clinicians.

[0051] In some embodiments of the present application, reference Figure 2 In step 102, the R2UNet model is used to perform segmentation processing on the cardiac image to obtain a set of key points of the left ventricular endocardial contour image, including but not limited to the following steps.

[0052] Step 201, based on the heart image, segmentation processing is performed using the R2UNet model to obtain a left ventricular endocardial contour image.

[0053] In step 201, the R2UNet model is used to accurately segment the heart image to obtain a left ventricular endocardial contour image, which provides an important data basis for subsequent left ventricular motion estimation.

[0054] Optionally, the embodiments of the present application use a variety of data enhancement techniques in combination with the R2UNet model, including scaling, vertical flipping and horizontal flipping, to improve the generalization and robustness of the model. Among them, by scaling, the size of the image is changed, so that the model can adapt to cardiac images of different resolutions and sizes, and the adaptability of the model to different imaging conditions is enhanced. By vertically flipping and horizontally flipping, the direction of the image is changed to simulate the situation of the heart in different positions, thereby increasing the diversity of training data. These data enhancement techniques not only increase the number of training samples, but also improve the robustness of the model to various image changes, so that when processing left ventricular images with abnormal contours or complex backgrounds, segmentation can be performed more accurately, improving the overall segmentation accuracy and stability. In summary, these data enhancement techniques effectively improve the generalization and segmentation performance of the model.

[0055] Optionally, the implementation process of the R2UNet model includes but is not limited to the following steps.

[0056] First, the labeled heart image data is used as training data to perform preliminary training on R2UNet so that it can perform the task of segmenting the left ventricular endocardium contour.

[0057] Secondly, feature extraction is performed. Specifically, the input cardiac image data is subjected to multiple layers of convolution operations to gradually extract the feature information of the image. Each layer of convolution operation convolves the image through the convolution kernel to extract local features. Through pooling operations (such as maximum pooling or average pooling), the spatial resolution of the image is reduced, while the feature dimension is increased to generate a low-resolution, high-feature-dimension feature map. This helps capture higher-level abstract features.

[0058] Furthermore, residual connections are performed. The feature map of each layer is passed to the next layer through the residual structure of the convolutional layer. Residual connections allow the model to learn deeper feature representations and reduce information loss. Specifically, the residual unit directly transfers the features of the previous layer to the next layer through a skip connection, thereby retaining key features and avoiding the gradient vanishing problem. The residual structure enables R2UNet to better learn and retain detailed information in deep networks, improving the efficiency and accuracy of feature extraction.

[0059] Then, a jump connection is made. R2UNet retains the jump connection structure of U-Net and directly passes the low-level features of the encoder part to the decoder part. This structure enables the decoder to utilize the low-level features in the encoder during upsampling. The jump connection enables the decoder to combine high-level and low-level features, which can retain detail information to the greatest extent, making it more accurate in restoring image details and improving the refinement and boundary clarity of the segmentation results.

[0060] Finally, the Dice loss function is used to optimize R2UNet. The Dice loss function is a loss function widely used in medical image segmentation. It is based on the Dice coefficient and measures the similarity between the estimated segmentation result and the true label. Dice loss can effectively evaluate the quality of the segmentation result and guide the model to optimize the segmentation boundary and improve the segmentation accuracy. Especially when dealing with medical image segmentation tasks with unbalanced categories, Dice loss is more effective than the traditional cross entropy loss. Dice loss function Satisfies the following formula (1): (1); In formula (1), represents the true segmentation result in the dataset, represents the segmentation result estimated by the model, is the size of the overlapping area between the two.

[0061] Through the above steps, R2UNet can accurately segment the left ventricular endocardium contour in cardiac image data. Multi-layer convolution and residual structure enhance the ability of feature extraction, jump connection retains detail information, and Dice loss ensures the high quality of segmentation results. The combination of these technologies enables R2UNet to perform well in processing left ventricular images with complex backgrounds and morphological abnormalities, providing accurate and stable segmentation results.

[0062] Step 202: According to the left ventricular endocardial contour image, a key point set of the left ventricular endocardial contour image is obtained in combination with a key point sampling method.

[0063] In step 202, a key point set is extracted from the left ventricular endocardial contour image using a key point sampling method, providing an important data basis for subsequent left ventricular motion estimation.

[0064] Optionally, the key point sampling method includes uniform sampling. Uniform sampling refers to selecting key points on the segmented left ventricular endocardium contour according to a certain interval or rule. This ensures that the key points are evenly distributed on the entire contour, avoiding some areas being too dense and other areas being sparse.

[0065] Optionally, uniform sampling can be performed by calculating the length of the contour and determining the location of the sampling points according to a preset sampling interval (e.g., selecting a point every 10 pixels). More complex algorithms, such as arc-length-based uniform sampling, can also be used to ensure that the distance between each key point is roughly equal.

[0066] In some embodiments of the present application, reference Figure 2 In step 103, the process of performing feature fusion using a multi-layer perceptron model based on the key point set to obtain comprehensive information of the key point set includes but is not limited to the following steps.

[0067] Step 301: perform feature detection on a key point set to obtain coordinate information, confidence information, and attribute information of the key point set.

[0068] It should be noted that the coordinate information provides the precise location of the key point in the image, which is crucial for subsequent motion estimation. The confidence information reflects the reliability of key point detection, which helps to screen out high-quality key points and improve the accuracy of matching. The attribute information describes the local characteristics of the key point through the feature vector, such as texture, edge, etc. This information plays an important role in distinguishing different key points and their matching.

[0069] In step 301, the information of each key point is further refined by a feature detection method. Specifically, the coordinate information, confidence information, and attribute information of each key point are calculated. These detailed information helps to improve the recognition of key points and provides a richer basis for feature matching, thereby improving the accuracy and robustness of matching.

[0070] Optionally, the feature detection method includes a scale-invariant feature transformation method and an accelerated robust feature method.

[0071] The scale-invariant feature transformation method is a feature detection method used in image processing and computer vision, which can stably identify key points in images at different scales. This method detects extreme points by constructing a Gaussian difference pyramid and calculates the direction histogram of these points to obtain scale and rotation invariance. Each key point is represented by a descriptor, which has good robustness to changes in illumination and perspective. This method is widely used in fields such as object recognition and image matching. Although its computational complexity is high, it performs well in tasks that require high-precision feature matching.

[0072] The accelerated robust feature method aims to increase the speed of feature detection without effectively sacrificing accuracy. This method uses a box filter to approximate a Gaussian filter, which greatly speeds up the key point detection process. It also constructs the scale space of the image, but uses integral image technology for fast calculation. The key point descriptor of this method is determined based on the maximum value of the Hessian matrix, and the descriptor is constructed using the haar wavelet response. This method is not only computationally efficient, but also has good invariance to image scaling, rotation and brightness changes, making it suitable for real-time applications and large-scale data set processing.

[0073] Step 302, based on the coordinate information, confidence information and attribute information of the key point set, a multi-layer perceptron model is used to perform feature fusion to obtain comprehensive information of the key point set.

[0074] In step 302, by comprehensively processing the coordinate information, confidence information and attribute information of each key point to generate more comprehensive and rich comprehensive information, the representation ability of the key points can be enhanced, so that each key point contains not only its position information in the image, but also its validity and uniqueness in the image. The comprehensive information of the key point set provides a more comprehensive and reliable basis for subsequent feature matching, thereby improving the overall accuracy and robustness of left ventricular motion estimation based on deep learning.

[0075] The multilayer perceptron model is a feedforward neural network, which consists of an input layer, a hidden layer, and an output layer. Each layer processes data through linear transformation and nonlinear activation functions. In the process of feature fusion, the multilayer perceptron model receives the coordinate information, confidence information, and attribute information of each key point as input, performs multiple linear transformations and nonlinear activations in the hidden layer to extract complex features, and generates a fused feature vector in the output layer to obtain the comprehensive information of the key point set. The multilayer perceptron model has nonlinear mapping capabilities and can effectively capture the complex relationship between coordinate information, confidence information, and attribute information; at the same time, it can enhance feature representation, making the comprehensive information of the key point set richer and more distinguishable. In addition, the structure of the multilayer perceptron model is flexible and adjustable, easy to implement, and can be efficiently optimized through back propagation. It also has good generalization ability to ensure the robustness of the model to unseen data. Therefore, the multilayer perceptron model is an ideal choice for improving the accuracy of key point matching and system performance.

[0076] Optionally, the multilayer perceptron model satisfies the following formula (2): (2) In formula (2), for each key point , the key points Coordinate information of , confidence information and attribute information As the input of the multi-layer perceptron, output each key point Comprehensive information , Optionally, attribute information The dimension is 256.

[0077] In some embodiments of the present application, reference Figure 2 In step 104, based on the comprehensive information of the key point set, the graph neural network model is used to perform feature aggregation to obtain the embedded information of the key point set, including but not limited to the following steps.

[0078] Step 401, based on the comprehensive information of the key point set, the self-attention mechanism of the graph neural network model is used to obtain the in-graph aggregation features of the key point set.

[0079] Step 401, using the self-attention mechanism of the graph neural network model, generates graph aggregate features containing rich local context information for each key point. These features better reflect the structural relationship within the key point set and improve the accuracy of subsequent analysis. The self-attention mechanism allows each key point to pay attention to its own and neighboring key point information, dynamically adjusts the importance weights of different key points, thereby enhancing the uniqueness of each key point, and updates the feature representation based on local context information. This can better capture the relationship between key points, thereby improving the accuracy of subsequent matching. This process can highlight the important features within the key point set and generate more robust graph aggregate features.

[0080] Step 402, based on the comprehensive information of the key point set, the cross-graph aggregation features of the key point set are obtained by utilizing the cross-attention mechanism of the graph neural network model.

[0081] In step 402, the relationship between the end-diastolic key point subset and the end-systolic key point subset is captured through the cross-attention mechanism, and cross-graph aggregate features containing cross-time point information are generated for each key point. These features not only retain local characteristics, but also reflect their changes throughout the cardiac motion cycle, enhancing the global consistency of feature representation. The cross-attention layer allows the model to learn how to establish connections between two different time points, thereby dynamically updating the matching relationship of key points. This information exchange across time points helps to improve the consistency and accuracy of matching. This process can identify the correspondence between two time points and generate cross-graph aggregate features, thereby better understanding the overall change trend of cardiac motion.

[0082] Step 403: Obtain embedding information of the key point set according to the in-graph aggregation features and cross-graph aggregation features of the key point set.

[0083] In this step, the intra-graph aggregate features and cross-graph aggregate features are combined to form the final embedded information, which not only contains rich local structural information, but also reflects the changing trend across time points. This embedded information provides a solid foundation for subsequent feature matching and effectively improves the accuracy and reliability of left ventricular motion estimation. This process is not just a simple combination of two features, but a specific algorithm to ensure the coordination and consistency between various information dimensions, thereby enhancing the uniqueness and discrimination of feature representation.

[0084] Optionally, the graph neural network model contains three attention aggregation structures, each of which includes a self-attention layer and a cross-attention layer.

[0085] Optionally, the implementation process of the graph neural network model includes but is not limited to the following steps.

[0086] First, the comprehensive information of the key point set is used as the input of the graph neural network model to establish the internal edge set and the external edge set , where the internal edges represent the internal connections of the end-diastolic key point subset and the internal connections of the end-systolic key point subset, and the external edges represent the connections between the end-diastolic key point subset and the end-systolic key point subset. Both the internal edges and the external edges are undirected edges.

[0087] Secondly, the query vector, key vector, and value vector are calculated for each key point in the key point set. The query vector represents the type of information that the current key point wants to focus on and is used to filter out relevant information from other key points. Through the query vector, the model can determine what type of features the current key point wants to obtain from the surrounding context. The query vector satisfies the following formula (3): (3); In formula (3), Indicates key points Comprehensive information, Indicates key points The query vector is represents the first trainable weight matrix, represents the first bias term, Indicates the key points in the aggregation process No. Layer embedding representation, .

[0088] The key vector represents the type of information that each key point can provide. It is compared with the query vector to determine whether the key point contains the information sought by the query vector. The key vector enables the model to identify which key points are useful for the current key point. This helps to establish associations between key points and allows the model to adjust the delivery of information based on these associations. The key vector satisfies the following formula (4): (4); In formula (4), Indicates key points The key vector of represents the second trainable weight matrix, represents the second bias term.

[0089] The value vector contains the actual information content of each key point. Once the degree of match between the query vector and the key vector is calculated, the value vector will be weighted and summed according to the degree of match to form the final output. The value vector provides specific and useful information that is selected based on the query-key alignment results. In this way, the model can focus on the most relevant information rather than all available information, thereby improving efficiency and accuracy. The value vector satisfies the following formula (5): (5); In formula (5), Indicates key points The value vector of represents the third trainable weight matrix, represents the third bias term.

[0090] Then, according to the internal edge set and the external edge set , through the self-attention layer and the cross-attention layer, we get the key points The aggregation result satisfies the following formula (6): (6); In formula (6), Indicates key points The aggregation results of and key points There is an edge between ; In the odd-numbered layers of the network, , the self-attention mechanism is applied; in the even-numbered layer network, , a cross-attention mechanism is applied; Indicates key points and key points The attention weights between The following formula (7) is satisfied: (7); In formula (7), Indicates key points The transpose of the query vector, Indicates key points The key vector of .

[0091] Finally, the key point In the The iterative update process of the layer’s embedded representation satisfies the following formula (8): (8).

[0092] After multiple iterative updates, the embedded information of the key point set is obtained.

[0093] Through the above three steps, the transformation from the comprehensive information of the key point set to the embedded information is achieved. The self-attention mechanism enhances the structural information within the key point set, and the cross-attention mechanism captures the relationship between the end-diastolic key point subset and the end-systolic key point subset. The combination of the two forms a more comprehensive and robust embedded information. This systematic processing method effectively improves the quality of feature representation and provides strong support for subsequent motion estimation.

[0094] In some embodiments of the present application, the embedded information of the key point set includes end-diastolic embedded information and end-systolic embedded information. Figure 3 ,According to the embedded information and comprehensive information of the key point set, the Sinkhorn algorithm is combined to perform feature matching, and the implementation process of obtaining the diastolic-systolic deformation relationship of the left ventricular endocardial contour includes but is not limited to the following steps.

[0095] Step 501, calculating the similarity between the end-diastolic embedded information and the end-systolic embedded information to obtain a score matrix.

[0096] In step 501, the similarity between the end-diastolic embedded information and the end-systolic embedded information is calculated using methods such as inner product or cosine similarity to obtain a score matrix, in which each element represents the similarity between the key point subsets at the corresponding positions of the two time points. This score matrix reflects the similarity between key point pairs at different time points.

[0097] Step 502: Perform feature matching based on the score matrix in combination with the Sinkhorn algorithm to obtain a matching matrix.

[0098] It should be noted that the Sinkhorn algorithm is an iterative method based on optimal transmission theory, which is used to generate a double random matrix to ensure that each key point has a unique and optimal matching relationship at different time points. The algorithm gradually converges to a stable double random matrix by alternating the rows and columns of the normalized matrix so that the sum of each row and column is 1. The reason for choosing the Sinkhorn algorithm for feature matching is that it can ensure the uniqueness and consistency of the matching and avoid repeated or missed matching. In addition, the Sinkhorn algorithm takes into account global information, improves the accuracy and robustness of the matching, and performs well in processing low-quality images or abnormal contours. Although it involves an iterative process, it has high computational efficiency and is suitable for large-scale data processing. It can also be used in combination with a variety of similarity metrics to provide a flexible matching scheme. These characteristics make the Sinkhorn algorithm suitable for feature matching tasks in left ventricular motion estimation based on deep learning.

[0099] In step 502, the score matrix is ​​processed using the Sinkhorn algorithm to generate a matching matrix, which effectively solves the uniqueness and consistency problems of key points, avoids repeated matching or missing matching, and thus improves the accuracy and reliability of matching. The matching matrix reflects the mapping relationship between key points at different time points, which is the key to understanding cardiac motion, especially the myocardial displacement and deformation that need to be tracked when evaluating cardiac function.

[0100] Step 503: Obtain the diastolic-systolic deformation relationship of the left ventricular endocardium contour based on the comprehensive information of the matching matrix and the key point set.

[0101] It should be noted that the diastolic-systolic deformation relationship describes how the inner membrane contour of the left ventricle changes shape during a complete cardiac cycle, which is very important for diagnosing heart diseases. For example, abnormal deformation patterns may be manifestations of certain pathological conditions, such as myocardial infarction or ventricular wall motion disorder. Therefore, accurately estimating this deformation relationship is of great significance for clinical applications.

[0102] In step 503, the diastolic-systolic deformation relationship of the left ventricular endocardial contour is obtained based on the comprehensive information of the matching matrix and the key point set. The relative movement of each key point in the diastolic and systolic states can be determined, and the change trend of the entire left ventricular endocardial contour can be depicted accordingly.

[0103] In some embodiments of the present application, reference Figure 3 In step 502, feature matching is performed based on the score matrix in combination with the Sinkhorn algorithm to obtain a matching matrix, and the implementation process includes but is not limited to the following steps.

[0104] Step 601, using the Sinkhorn algorithm to iteratively normalize the score matrix to obtain a converged double random matrix.

[0105] In step 601, the score matrix is ​​converted into a double random matrix (i.e., the sum of each row and each column is 1). The Sinkhorn algorithm iteratively adjusts the element values ​​in the score matrix so that the sum of each row and each column of the final matrix is ​​close to 1. This process ensures that each end-diastolic key point matches only one end-systolic key point, and vice versa, thereby avoiding the problem of duplicate matching or missing matching.

[0106] Specifically, the Sinkhorn algorithm gradually approaches the double random condition by alternating normalization operations on the rows and columns of the score matrix. Each iteration brings the matrix closer to satisfying the double random constraint. When the iteration reaches a certain number of times or meets the preset convergence criteria, the score matrix will become a stable double random matrix, which directly reflects the optimal matching possibility between key points.

[0107] Step 602: Obtain a matching matrix based on the converged double random matrix. In the matching matrix, only one element in each row and column is 1, and all other elements are 0.

[0108] In step 602, the double random matrix is ​​converted into a matching matrix, and the probability distribution in the double random matrix is ​​interpreted as the basis for the actual matching decision. Since the elements of the double random matrix represent the possibility of matching, it is necessary to select the most likely matching relationships to construct the final matching matrix. The matching matrix is ​​a binary matrix, in which only one element in each row and column is 1, and all other elements are 0.

[0109] Optionally, a matching matrix is ​​constructed by selecting the maximum value in each row or column of a doubly random matrix to decide which elements should be set to 1.

[0110] Step 603: The matching matrix is ​​optimized by minimizing the negative logarithmic loss.

[0111] In step 603, the matching matrix is ​​further optimized to improve the quality of the matching matrix. Minimizing the negative logarithmic loss is a common optimization strategy that aims to adjust the matching matrix to better conform to the actual cardiac motion pattern.

[0112] By minimizing the negative logarithmic loss function, the matching results can be made more consistent with the expected myocardial motion model and reduce unreasonable matching situations. The optimization process may involve adjusting certain matching decisions in the matching matrix to ensure that the overall matching scheme can minimize the total loss function. This helps to improve the accuracy and reliability of key point matching, especially in the presence of noisy or abnormal data.

[0113] The above three steps together ensure that the conversion from the score matrix to the matching matrix not only satisfies the optimal transmission problem in theory, but also provides high-quality and reliable key point matching results in practice, which is crucial for the subsequent analysis of the diastolic-systolic deformation relationship of the left ventricular endocardial contour.

[0114] In some embodiments of the present application, reference Figure 3 In step 503, the process of obtaining the diastolic-systolic deformation relationship of the left ventricular endocardial contour based on the comprehensive information of the matching matrix and the key point set includes but is not limited to the following steps.

[0115] Step 701, obtaining a deformation function according to the comprehensive information of the matching matrix and the key point set.

[0116] In step 701, a mathematical model (i.e., a deformation function) is constructed based on the matching results between the key points, and the model can quantify the shape changes of the heart at different time points. The deformation function is a key tool for connecting the left ventricular endocardial contours at the end of diastole and end of systole. It not only reflects the spatial displacement of the key points, but may also include local deformations (such as stretching, compression, etc.). Specifically, the matching matrix provides information on the correspondence between each key point from the end of diastole to the end of systole. These one-to-one matching pairs are the basis for constructing the deformation function. The comprehensive information includes the location information, attribute information, and confidence information of the key points. Using this information, the deformation function can be defined more accurately to ensure that it can truly reflect the actual movement pattern of the heart endocardium.

[0117] Step 702: Obtain the diastolic-systolic deformation relationship of the left ventricular endocardium contour according to the deformation function.

[0118] In step 702, the position of the key points on the left ventricular endocardial contour is reconstructed using a deformation function. By applying the deformation function to all key points, the diastolic-systolic deformation relationship of the left ventricular endocardial contour can be obtained. Based on the diastolic-systolic deformation relationship of the left ventricular endocardial contour, a series of quantitative analyses can be performed, such as calculating the thickness change, volume change rate, strain and strain rate of the myocardial wall. These parameters are crucial for evaluating the heart's pumping efficiency and detecting potential heart disease lesions.

[0119] Specifically, first, based on the comprehensive information of the matching matrix and key points, a deformation function is constructed, which describes the spatial displacement of each key point from end-diastole to end-systole. Commonly used deformation functions include affine transformation, thin plate spline or radial basis function, which are used to accurately quantify the movement path of each key point on the left ventricular endocardial contour.

[0120] Next, the deformation function is applied to all points of the entire left ventricular endocardial contour, not just the key points, to achieve modeling of continuous deformation of the endocardial surface. For points that are not directly involved in the matching, an interpolation method is used to predict their movement paths to ensure the integrity of the deformation model. This step generates a detailed pattern of changes in the left ventricular endocardial contour during the cardiac cycle, namely the diastolic-systolic deformation relationship.

[0121] Using these deformation relationships, a series of quantitative motion parameters are calculated, such as displacement field (specific moving direction and distance of each point), strain rate (local myocardial stretching or compression), volume change (estimate the volume change of the left ventricle during the cardiac cycle to calculate the ejection fraction EF), and wall thickness change (observe the change of myocardial wall thickness over time). The above parameters are integrated to generate a comprehensive left ventricular motion estimation report. Finally, by comparing and cross-validating with clinical standard measurement results, the deformation function and its related parameters are continuously optimized to improve the accuracy of the model.

[0122] Optionally, the process of implementing feature matching based on the comprehensive information and embedded information of the key point set in combination with the Sinkhorn algorithm to obtain the diastolic-systolic deformation relationship of the left ventricular endocardial contour includes but is not limited to the following steps.

[0123] First, calculate the end-diastolic embedding information and end-systolic embedded information The similarity between them is used to obtain the score matrix . Scoring Matrix The following formula (9) is satisfied: (9); In formula (9), represents the similarity function, It can be a similarity function based on Euclidean distance or a cosine similarity function.

[0124] Second, the scoring matrix Iterative normalization is performed through the Sinkhorn algorithm, iterative After times, we get the converged double random matrix , the sum of each row and column of the doubly random matrix is ​​1, matching the matrix The iterative process of the Sinkhorn algorithm satisfies the following formula (10): (10); In formula (10), the initial matrix , Indicates the number of iterations; and are two normalization factors, respectively adjusting The rows and columns of is the normalization factor Iteration The result of the Indicated by The diagonal matrix formed by Normalization factor Iteration The result of the Indicated by The diagonal matrix formed by The iterative process satisfies the following formula (11): (11); The iterative process satisfies the following formula (12): (12).

[0125] Then, the matching matrix is ​​optimized by minimizing the negative logarithmic loss , minimizing the negative log loss The following formula (13) is satisfied: (13); Ideally, the matching matrix It should be a permutation matrix, meaning that each keypoint in the end-diastolic keypoint subset corresponds to only one keypoint in the end-systolic keypoint subset, and the matching matrix Elements Represents the key points in the end-diastolic key point subset Key points from the end-systolic key point subset Match, otherwise 0. The matching matrix obtained by the Sinkhorn algorithm After optimization, it is close to a permutation matrix, but it is not completely a strict permutation matrix, because the result of the Sinkhorn algorithm may contain some non-zero non-integer values. However, in practical applications, this approximation is usually sufficient to determine the one-to-one correspondence between key points.

[0126] Then, according to the matching matrix The coordinate information of the key point set is used to determine the matching key point pairs in the key point set and obtain the deformation function. The change of the deformation function is the result of the superposition of the changes along the x-axis and y-axis directions. Taking the x-axis direction as an example, the deformation function The following formula (14) is satisfied: (14); In formula (14), the current position Refers to the specific location in the image that needs to be deformed; From the current location To the deformed position, Indicates the number of key points; It is the key point location, yes The weight coefficient of is the effective radius, which determines the influence range of the key point and is calculated based on the Euclidean distance; is the radial basis function, which calculates the contribution of each key point to the deformation. The following formula (15) is satisfied: (15); In formula (15), Represents the normalized distance, indicating the current position arrive Distance and effective radius The ratio of hour, Forced to 0, which means that keypoints outside the effective radius of the control point will not be affected by that keypoint.

[0127] Optionally, the radial basis function It is based on Wendland's C2 function.

[0128] In summary, the embodiments of the present application can provide the following technical effects.

[0129] The left ventricular motion estimation method based on deep learning proposed in the embodiment of the present application uses a segmentation processing method to accurately extract the left ventricular endocardial contour, and combines the position information of key points with rich local features in combination with feature fusion technology to create a comprehensive feature representation. Subsequently, the self-attention and cross-attention mechanisms of the graph neural network model are used to generate in-graph aggregate features and cross-graph aggregate features, respectively, to capture the internal and cross-time relationship of the key point subset, and to form a more comprehensive and robust embedded information. This method not only enhances the accuracy of feature representation, but also improves the adaptability to morphological abnormalities.

[0130] Furthermore, the embodiment of the present application uses the Sinkhorn algorithm to optimize the score matrix and generate a matching matrix to ensure that each key point matches only one key point at another time point, avoiding the problem of duplicate or missed matching. Based on the matching results and the comprehensive information of each initial key point subset, a deformation function describing the change of heart movement from end diastole to end systole is constructed, which quantifies the movement pattern of the left ventricle and provides a more reliable heart movement analysis.

[0131] Secondly, refer to Figure 4 The left ventricular motion estimation system based on deep learning provided in an embodiment of the present application includes: a data acquisition module 801, a segmentation processing module 802, a feature fusion module 803, a feature aggregation module 804, a feature matching module 805 and a deformation estimation module 806.

[0132] The data acquisition module 801 is used to acquire cardiac images.

[0133] The segmentation processing module 802 is used to perform segmentation processing based on the heart image using the R2UNet model to obtain a key point set of the left ventricular endocardial contour image. The key point set includes an end-diastolic key point subset and an end-systolic key point subset.

[0134] The feature fusion module 803 is used to perform feature fusion based on the key point set using a multi-layer perceptron model to obtain comprehensive information of the key point set.

[0135] The feature aggregation module 804 is used to perform feature aggregation based on the comprehensive information of the key point set using a graph neural network model to obtain embedded information of the key point set.

[0136] The feature matching module 805 is used to perform feature matching based on the embedded information and comprehensive information of the key point set in combination with the Sinkhorn algorithm to obtain the diastolic-systolic deformation relationship of the left ventricular endocardium contour.

[0137] The deformation estimation module 806 is used to estimate the diastolic-systolic motion of the left ventricular endocardial contour according to the diastolic-systolic deformation relationship of the left ventricular endocardial contour to obtain a left ventricular motion estimation result.

[0138] On the other hand, refer to Figure 5 The left ventricular motion estimation device based on deep learning provided by the embodiment of the present application includes: a processor and a memory. The memory is used to store a program. When the program is executed by the processor, the processor implements the aforementioned left ventricular motion estimation method based on deep learning.

[0139] In addition, an embodiment of the present application provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement the aforementioned deep learning-based left ventricular motion estimation method.

[0140] Similarly, the contents of the above method embodiments are all applicable to system embodiments, device embodiments and medium embodiments. The functions specifically implemented by the system embodiments, device embodiments and medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0141] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0142] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A left ventricular motion estimation method based on deep learning, characterized in that: The steps include: Acquire cardiac images; According to the heart image, segmentation processing is performed using the R2UNet model to obtain a key point set of the left ventricular endocardial contour image; the key point set includes an end-diastolic key point subset and an end-systolic key point subset; According to the key point set, a multi-layer perceptron model is used to perform feature fusion to obtain comprehensive information of the key point set; According to the comprehensive information of the key point set, feature aggregation is performed using a graph neural network model to obtain embedded information of the key point set; According to the embedded information and comprehensive information of the key point set, feature matching is performed in combination with the Sinkhorn algorithm to obtain the diastolic-systolic deformation relationship of the left ventricular endocardial contour; The diastolic-systolic motion of the left ventricular endocardial contour is estimated according to the diastolic-systolic deformation relationship of the left ventricular endocardial contour to obtain a left ventricular motion estimation result.

2. The method for estimating left ventricular motion based on deep learning according to claim 1, characterized in that: The segmentation process is performed using the R2UNet model according to the heart image to obtain a set of key points of the left ventricular endocardial contour image, including: According to the heart image, segmentation processing is performed using the R2UNet model to obtain the left ventricular endocardial contour image; According to the left ventricular endocardial contour image and in combination with a key point sampling method, a key point set of the left ventricular endocardial contour image is obtained.

3. The method for estimating left ventricular motion based on deep learning according to claim 1, characterized in that: The step of performing feature fusion using a multi-layer perceptron model according to the key point set to obtain comprehensive information of the key point set includes: Performing feature detection on the key point set to obtain coordinate information, confidence information and attribute information of the key point set; According to the coordinate information, confidence information and attribute information of the key point set, the multi-layer perceptron model is used to perform feature fusion to obtain comprehensive information of the key point set.

4. The method for estimating left ventricular motion based on deep learning according to claim 1, characterized in that: The step of performing feature aggregation using a graph neural network model based on the comprehensive information of the key point set to obtain embedded information of the key point set includes: According to the comprehensive information of the key point set, the self-attention mechanism of the graph neural network model is used to obtain the in-graph aggregation features of the key point set; According to the comprehensive information of the key point set, the cross-graph aggregation features of the key point set are obtained by utilizing the cross-attention mechanism of the graph neural network model; Embedding information of the key point set is obtained according to the intra-graph aggregation features and cross-graph aggregation features of the key point set.

5. The method for estimating left ventricular motion based on deep learning according to claim 1, characterized in that: The embedding information of the key point set includes end-diastolic embedding information and end-systolic embedding information; The embedding information and comprehensive information of the key point set are combined with the Sinkhorn algorithm to perform feature matching to obtain the diastolic-systolic deformation relationship of the left ventricular endocardial contour, including: Calculating the similarity between the end-diastolic embedded information and the end-systolic embedded information to obtain a score matrix; According to the score matrix, feature matching is performed in combination with the Sinkhorn algorithm to obtain a matching matrix; The diastolic-systolic deformation relationship of the left ventricular endocardial contour is obtained according to the comprehensive information of the matching matrix and the key point set.

6. The method for estimating left ventricular motion based on deep learning according to claim 5, characterized in that: The step of performing feature matching according to the score matrix and in combination with the Sinkhorn algorithm to obtain a matching matrix includes: Iteratively normalizing the score matrix using the Sinkhorn algorithm to obtain a converged double random matrix; According to the converged double random matrix, the matching matrix is ​​obtained; in the matching matrix, only one element in each row and each column is 1, and all other elements are 0; The matching matrix is ​​optimized by minimizing the negative log loss.

7. The method for estimating left ventricular motion based on deep learning according to claim 5, characterized in that: The step of obtaining the diastolic-systolic deformation relationship of the left ventricular endocardial contour according to the comprehensive information of the matching matrix and the key point set includes: Obtaining a deformation function according to the comprehensive information of the matching matrix and the key point set; According to the deformation function, the diastolic-systolic deformation relationship of the left ventricular endocardium contour is obtained.

8. Left ventricular motion estimation system based on deep learning, characterized in that: include: Data acquisition module, segmentation processing module, feature fusion module, feature aggregation module, feature matching module and deformation estimation module; The data acquisition module is used to acquire a heart image; The segmentation processing module is used to perform segmentation processing based on the cardiac image using the R2UNet model to obtain a key point set of the left ventricular endocardial contour image; the key point set includes an end-diastolic key point subset and an end-systolic key point subset; The feature fusion module is used to perform feature fusion according to the key point set using a multi-layer perceptron model to obtain comprehensive information of the key point set; The feature aggregation module is used to perform feature aggregation using a graph neural network model according to the comprehensive information of the key point set to obtain embedded information of the key point set; The feature matching module is used to perform feature matching based on the embedded information and comprehensive information of the key point set in combination with the Sinkhorn algorithm to obtain the diastolic-systolic deformation relationship of the left ventricular endocardial contour; The deformation estimation module is used to estimate the diastolic-systolic motion of the left ventricular endocardial contour according to the diastolic-systolic deformation relationship of the left ventricular endocardial contour to obtain a left ventricular motion estimation result.

9. A left ventricular motion estimation device based on deep learning, characterized in that: include: Processor and memory; The memory is used to store programs; When the program is executed by the processor, the processor implements the left ventricular motion estimation method based on deep learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the left ventricular motion estimation method based on deep learning as described in any one of claims 1 to 7 when executed by the processor.

Images