Myocardial strain value acquisition method and myocardial ventricle segmentation method

Abstract

The invention discloses a myocardial strain value acquisition method and a myocardial ventricle segmentation method. The myocardial strain value acquisition method comprises the following steps: step (1), acquiring heart MRI (Magnetic Resonance Imaging) video data consisting of multiple frames of heart images; (2) inputting a heart image to be segmented into the trained image segmentation model to segment the cardiac muscle and the ventricle to obtain a segmentation result; (3) inputting the paired heart images obtained in the step (1) into the trained myocardial motion estimation network to obtain motion estimation; and (4) calculating the segmentation result obtained in the step (2) and the motion estimation obtained in the step (3) to obtain a strain value. According to the method, efficient learning and prediction can be realized under the condition of fewer labeled samples, so that the cardiac muscle and the ventricle can be segmented, and then the strain value is calculated according to the segmentation result and motion estimation. According to the method, efficient learning and prediction can be realized under the condition that only a small number of labeled samples exist.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and particularly relates to a method for obtaining myocardial strain values and a method for segmenting myocardial ventricles. Background Art

[0002] In recent years, myocarditis, as a serious cardiovascular disease, has received extensive attention. Its early identification and diagnosis are crucial for improving the prognosis of patients. Traditional diagnosis of myocarditis mainly relies on clinical symptoms, electrocardiogram, and imaging examinations. However, these methods often have certain limitations, such as insufficient sensitivity, high false negative rate, and long diagnostic process. With the development of medical imaging technology and the rapid progress of artificial intelligence (AI) technology, using AI for medical image analysis has become a current research hotspot.

[0003] In the method for judging myocarditis, cardiac magnetic resonance imaging (CMR) and echocardiography are the main imaging examination methods. Currently, AI technology, especially deep learning algorithms, has been applied to the automatic analysis of medical images and can extract features from complex image data and classify them. In cardiac image analysis, deep learning models such as convolutional neural networks (CNNs) are widely used and can effectively identify changes in cardiac structure and function.

[0004] Although many studies have explored the application of AI in myocarditis, the existing technology still faces many challenges in practical applications. For example: 1. Existing AI models often rely on a large amount of labeled data for training, and it is relatively difficult to obtain a high-quality labeled data set. 2. Most classification models directly obtain classification results through image data, with low clinical interpretability, and their universality and robustness still need to be verified in applications among different populations and devices.

[0005] Myocardial strain value is an important means to evaluate cardiac function and can provide information about myocardial mechanical properties. By measuring the degree of deformation of the myocardium during contraction and relaxation, the functional state of the heart can be more accurately reflected. Traditional methods for measuring myocardial strain usually rely on echocardiography or magnetic resonance imaging, but these methods often require complex operations and professional interpretations.

[0006] Therefore, aiming at the deficiencies of the existing technology, it is very necessary to provide a method for obtaining myocardial strain values and a method for segmenting myocardial ventricles to solve the deficiencies of the existing technology. Summary of the Invention

[0007] The first object of the present invention is to avoid the deficiencies of the existing technology and provide a method for obtaining myocardial strain values. This method for obtaining myocardial strain values can achieve efficient learning and prediction with fewer labeled samples, so as to be able to segment the myocardium and ventricles, and then calculate the strain value according to the segmentation result and motion estimation.

[0008] The above object of the present invention is achieved by the following technical measures:

[0009] A method for obtaining myocardial strain values is provided, which is carried out by the following steps:

[0010] Step (1), obtaining cardiac MRI video data composed of multiple frames of cardiac images;

[0011] Step (2), inputting the cardiac image to be segmented into the trained image segmentation model to segment the myocardium and ventricles, and obtaining the segmentation result;

[0012] Step (3), inputting the paired cardiac images obtained in step (1) into the trained myocardial motion estimation network to obtain the motion estimation;

[0013] Step (4), calculating the strain value from the segmentation result obtained in step (2) and the motion estimation obtained in step (3).

[0014] In the training of the image segmentation model, the training data is the labeled cardiac images and the unlabeled cardiac images, and the segmentation results of the labeled cardiac images have been obtained in advance.

[0015] Preferably, the training process of the above image segmentation model is as follows:

[0016] A1. Respectively extract the features of the labeled cardiac images in the training data and the features of the unlabeled cardiac images in the training data through the feature extractor in the image segmentation model;

[0017] A2. Send the features of the labeled cardiac images and the features of the unlabeled cardiac images extracted in A1 into the pyramid progressive mapping module in the image segmentation model, so as to map the features of the unlabeled cardiac images into the feature distribution of the labeled cardiac images, and enter A3;

[0018] A3. Use the two convolutional network branches and the edge repair module in the image segmentation model as a whole, and then respectively extract the features of the labeled cardiac images and the features of the unlabeled cardiac images;

[0019] A4. Selectively arrange the features of the unlabeled cardiac images extracted in A3 with the edges of the labeled cardiac images to obtain the segmentation result of the labeled cardiac images;

[0020] A5. Calculate the loss value between the segmentation result obtained in A4 and the labeled cardiac images. When the segmentation result obtained in A4 converges, enter A7; otherwise, enter A6;

[0021] A6. Calculate the gradient according to the loss value obtained in A5 to update the parameters of the image segmentation model, and return to A1;

[0022] A7. When the training is completed, the current image segmentation model is used as the trained image segmentation model.

[0023] Preferably, the extraction method of the above feature extractor is as follows:

[0024] When training the image segmentation model, in the labeled heart image, a feature x1 with a resolution of 8a×8a is extracted, where a is a positive integer; in the unlabeled heart image, a feature y1 with a resolution of 4a×4a, a feature y2 with a resolution of 3a×3a, a feature y3 with a resolution of 2a×2a, and a feature y4 with a resolution of a×a are extracted;

[0025] When segmenting the image segmentation model, in the heart image to be segmented, a feature x1 with a resolution of 8a×8a is extracted, where a is a positive integer; in other auxiliary frame heart images in the same heart sequence as the heart image to be segmented, a feature y1 with a resolution of 4a×4a, a feature y2 with a resolution of 3a×3a, a feature y3 with a resolution of 2a×2a, and a feature y4 with a resolution of a×a are extracted.

[0026] Preferably, the mapping method of the above pyramid progressive mapping module is carried out by the following steps:

[0027] B1. First, calculate the similarity between the position features of each pixel point in the feature x1 and the window range features at the corresponding positions in the feature y4, and use the cosine similarity to determine the most matching feature within the window range to obtain the mapping matrix w1. The mapping matrix w1 is the corresponding position of each pixel point in the feature x1 at the corresponding position in the feature y4; then the mapping matrix w1 transforms the feature y3, making each 2×2 region in the feature y3 correspond to the feature y4; finally, according to the mapping matrix w1, each 2×2 region in the feature y3 is mapped, and the position of the mapping matrix w1 is redistributed to form the mapped feature

[0028] B2. First, calculate the similarity between the position features of each pixel point in the feature x1 and the window range features at the corresponding positions in the, and use the cosine similarity to determine the most matching feature within the window range to obtain the mapping matrix w2; then each 4×4 region of the feature y2 is made to correspond to each position of the feature y4, and by mapping with the mapping matrix w1, each 4×4 region of the feature y2 is made to correspond to the 2×2 region of the ; finally, through mapping with the mapping matrix w2, each 2×2 region of the feature y2 is made to correspond to each position of the feature x1 to form the mapped feature

[0029] B3. First, calculate the similarity between the position features of each pixel point in the feature x1 and the The similarity of the window range features at the corresponding positions in [feature y1], and use cosine similarity to determine the most matching features within the window range to obtain the mapping matrix w3; then, each 8×8 region of feature y1 is corresponding to each position of feature y4, and each 8×8 region of feature y1 is mapped to the 2×2 regions of [feature y2] through the mapping matrix w1, and then each 4×4 region of feature y1 is rearranged through the mapping matrix w2 to be mapped to the 2×2 regions of [feature y3]; finally, each 2×2 region of feature y1 is rearranged through the mapping matrix w3 to be corresponding to each position of feature x1, forming the mapped feature of 2×2 regions, and then each 4×4 region of feature y1 is rearranged through the mapping matrix w2 to be mapped to the 2×2 regions of ; finally, each 2×2 region of feature y1 is rearranged through the mapping matrix w3 to be corresponding to each position of feature x1, forming the mapped feature

[0030] B4. First, calculate the similarity of the feature at each pixel position in feature x1 and the window range features at the corresponding positions in [feature y1], and use cosine similarity to determine the most matching features within the window range to obtain the mapping matrix w4; then, the mapping matrix w4 is used to map and rearrange feature y1 to form the mapped feature ; finally, each 2×2 region of feature y1 is rearranged through the mapping matrix w3 to be corresponding to each position of feature x1, forming the mapped feature

[0031] In the image segmentation model, two convolutional network branches are respectively defined as the first convolutional network branch and the second convolutional network branch.

[0032] Preferably, the above-mentioned first convolutional network branch includes 7 first convolutional blocks, the 7 first convolutional blocks are connected from front to back, and the resolutions of these 7 first convolutional blocks are 128, 64, 32, 16, 32, 64, 128 from front to back.

[0033] Preferably, the above-mentioned second convolutional network branch includes 9 second convolutional blocks, the 9 second convolutional blocks are connected from front to back, and the resolutions of these 9 second convolutional blocks are 256, 128, 64, 32, 16, 32, 64, 128, 256 from front to back.

[0034] During training, the first convolutional network branch is used to process the features of unlabeled heart images, and the second convolutional network branch is used to process the features of labeled heart images.

[0035] During segmentation, the first convolutional network branch is used to process the features of other auxiliary frame heart images in the same heart sequence as the heart image to be segmented, and the second convolutional network branch is used to process the features of the heart image to be segmented.

[0036] In the image segmentation model, there are 7 edge repair modules, each edge repair module is respectively connected to the first convolutional network branch and the second convolutional network branch, and each edge repair module is respectively connected to the first convolutional block and the second convolutional block with the same resolution.

[0037] Preferably, the repair of the above-mentioned edge repair module is carried out in the following steps:

[0038] C1. For each position feature P in the feature Fx input through the second convolutional block, take the 9 position features within the 3×3 window corresponding to the position of Fy in the feature input through the first convolutional block, and define the 9 position features as P1, P2, P3, P4, P5, P6, P7, P8, and P9 respectively; x Take the 9 position features within the 3×3 window corresponding to the position of Fy in the feature input through the first convolutional block for each position feature P in the feature Fx input through the second convolutional block, and define the 9 position features as P1, P2, P3, P4, P5, P6, P7, P8, and P9 respectively;

[0039] C2. Calculate the cosine similarity between P and each block in P1, P2, P3, P4, P5, P6, P7, P8, and P9 through cosine similarity calculation to obtain the similarity matrix M x ; cos ;

[0040] C3. Select the most similar feature to P in the similarity matrix M cos , and then map the most similar feature to the position of P in the labeled heart image to obtain P x ; x ; y .

[0041] Preferably, the training of the above-mentioned myocardial motion estimation network is carried out in the following steps:

[0042] D1. Input a pair of sequential heart image pairs Image ES and Image ED into the motion estimation network, and predict the motion field M ES from Image ED to Image ES→ED , and predict the motion field -M ED from Image ES to Image ES→ED ;

[0043] D2. According to the motion field M ES→ED obtained in D1, use the image deformation function W to map Image ES to the space of Image ED to obtain the predicted image Image ED_pre ; According to the motion field -M ES→ED obtained in D1, use the image deformation function W to map Image ED to the space of Image ES to obtain the predicted image Image ES_pre ;

[0044] D3. Compare the reconstruction loss L ED_pre between the predicted image Image ED and Image rec according to Equation (1):

[0045] L rec = ||Image ED - Image ED_pre || 2 + ||Image ES - Image ES_pre || 2

[0046] …… Equation (1);

[0047] Calculate the smooth loss L of the square of the change amount between adjacent points in the sports field according to Equation (2) smooth ;

[0048]

[0049] Where M is the sports field, and M is M ES→ED , -M ES→ED , M ES→ED Or -M ES→ED ;

[0050] According to the smooth loss L of Equation (3) smooth , the reconstruction loss L rec Calculate the total loss L;

[0051] L = δ rec L rec + δ smooth L smooth …… Equation (3);

[0052] Where δ rec and δ smooth Are weight parameters respectively;

[0053] D4. Determine whether the total loss L has converged. When it has converged, enter D5; when not, update the parameters of the motion estimation network according to the total loss L and the optimizer, and return to D1;

[0054] D5. The training is over, and the current motion estimation network is used as the trained motion estimation network.

[0055] Preferably, the above motion estimation network is Unet.

[0056] Preferably, the above optimizer is Adam or SGD.

[0057] The second object of the present invention is to provide a myocardial ventricle segmentation method to avoid the deficiencies of the prior art. This myocardial ventricle segmentation method can segment the myocardium and ventricles.

[0058] The above object of the present invention is achieved by the following technical measures:

[0059] A myocardial ventricular segmentation method is provided, which is carried out by using steps (1) and (2) in the above-mentioned myocardial strain value acquisition method.

[0060] A method for obtaining myocardial strain values and a method for segmenting myocardial ventricles according to the present invention. The method for obtaining myocardial strain values is carried out by the following steps: Step (1), obtaining cardiac MRI video data composed of multiple frames of cardiac images; Step (2), inputting the cardiac image to be segmented into a trained image segmentation model to segment the myocardium and ventricles, and obtaining a segmentation result; Step (3), inputting the paired cardiac images obtained in Step (1) into a trained myocardial motion estimation network to obtain a motion estimation; Step (4), calculating the strain value by using the segmentation result obtained in Step (2) and the motion estimation obtained in Step (3). The present invention can achieve efficient learning and prediction with fewer labeled samples, so as to segment the myocardium and ventricles, and then calculate the strain value according to the segmentation result and the motion estimation. The present invention can achieve efficient learning and prediction with only a small number of labeled samples. Compared with the traditional deep learning models in the prior art that rely on a large amount of labeled data, the present invention can effectively cope with the problem of missing labeled data, is not limited by the number of labeled samples, is more flexible, can make full use of limited data resources for training and inference, and obtains better results. Description of the Drawings

[0061] The present invention is further described with reference to the accompanying drawings, but the content in the drawings does not constitute any limitation to the present invention.

[0062] Figure 1 It is a flow chart of a method for obtaining myocardial strain values.

[0063] Figure 2 It is a schematic diagram of a cardiac image.

[0064] Figure 3 It is a schematic diagram of the segmentation result.

[0065] Figure 4 It is a schematic diagram of the motion estimation.

[0066] Figure 5 It is a schematic diagram of the strain value.

[0067] Figure 6 It is a training flow chart of the image segmentation model.

[0068] Figure 7 It is a structural diagram of the pyramid progressive mapping module.

[0069] Figure 8 It is a repair flow chart of the edge repair module.

[0070] Figure 9 It is an estimation flow chart of the myocardial motion estimation network. Detailed implementation manners

[0071] The technical solution of the present invention will be further described in conjunction with the following embodiments.

[0072] Embodiment 1

[0073] A method for obtaining myocardial strain values, as Figure 1 shown, is carried out by the following steps:

[0074] Step (1): Obtain cardiac MRI video data composed of multiple frames of cardiac images;

[0075] Step (2): Input the cardiac image to be segmented (such as Figure 2 ) into the trained image segmentation model to segment the myocardium and ventricles, and obtain the segmentation result (such as Figure 3 );

[0076] Step (3): Input the paired cardiac images obtained in step (1) into the trained myocardial motion estimation network to obtain the motion estimation (such as Figure 4 );

[0077] Step (4): Calculate the strain value from the segmentation result obtained in step (2) and the motion estimation obtained in step (3) (such as Figure 5 ), where Figure 5 is the strain value at different positions of cardiac images of different objects.

[0078] In the training of the image segmentation model, the training data is the labeled cardiac images and the unlabeled cardiac images, where the labeled cardiac images have pre-obtained segmentation results.

[0079] As Figure 6 , the training process of the image segmentation model of the present invention is as follows:

[0080] A1: Respectively extract the features of the labeled cardiac images in the training data and the features of the unlabeled cardiac images in the training data through the feature extractor in the image segmentation model;

[0081] A2: Send the features of the labeled cardiac images and the features of the unlabeled cardiac images extracted in A1 into the pyramid progressive mapping module in the image segmentation model, so as to map the features of the unlabeled cardiac images into the feature distribution of the labeled cardiac images, and enter A3;

[0082] A3: Use the two convolutional network branches and the edge repair module in the image segmentation model as a whole, and then extract the features of the labeled cardiac images and the features of the unlabeled cardiac images correspondingly;

[0083] A4. Selectively arrange the features of the unlabeled cardiac images extracted in A3 and the edges of the labeled cardiac images to obtain the segmentation result of the labeled cardiac images;

[0084] A5. Calculate the loss value between the segmentation result obtained in A4 and the labeled cardiac images. When the segmentation result obtained in A4 converges, proceed to A7; otherwise, proceed to A6;

[0085] A6. Calculate the gradient based on the loss value obtained in A5 to update the parameters of the image segmentation model, and return to A1;

[0086] A7. The training is completed, and the current image segmentation model is used as the trained image segmentation model.

[0087] The extraction method of the feature extractor of the present invention is as follows:

[0088] During the training of the image segmentation model, extract the feature x1 with a resolution of 8a×8a from the labeled cardiac images, where a is a positive integer; in the unlabeled cardiac images, extract the feature y1 with a resolution of 4a×4a, the feature y2 with a resolution of 3a×3a, the feature y3 with a resolution of 2a×2a, and the feature y4 with a resolution of a×a, where a is 2;

[0089] During the segmentation of the image segmentation model, in the cardiac image to be segmented, extract the feature x1 with a resolution of 8a×8a; in other auxiliary frame cardiac images in the same cardiac sequence as the cardiac image to be segmented, extract the feature y1 with a resolution of 4a×4a, the feature y2 with a resolution of 3a×3a, the feature y3 with a resolution of 2a×2a, and the feature y4 with a resolution of a×a.

[0090] The purpose of the pyramid progressive mapping module is to accurately register the features of the unlabeled cardiac images and the features of the labeled cardiac images during training to ensure that each pixel point can find a corresponding relationship; during segmentation, it is to accurately register the features of the cardiac image to be segmented and the features of other auxiliary frame cardiac images in the same cardiac sequence as the cardiac image to be segmented.

[0091] Such as Figure 7 , the registration process of the pyramid progressive mapping module of the present invention starts from the feature y4 with the lowest resolution. Therefore, the mapping method of the pyramid progressive mapping module is carried out by the following steps:

[0092] B1. First, calculate the similarity between the position features of each pixel in feature x1 and the window range features at the corresponding positions in feature y4, and use cosine similarity to determine the most matching feature within the window range to obtain the mapping matrix w1. The mapping matrix w1 is the corresponding position of each pixel in feature x1 in feature y4. Through cosine similarity, the mapping matrix w1 is obtained, and this mapping matrix indicates the corresponding position of each pixel in feature x1 in y4. Then, the mapping matrix w1 transforms feature y3 to correspond each 2×2 region in feature y3 with feature y4. Finally, according to the mapping matrix w1, each 2×2 region in feature y3 is mapped, and the positions of the mapping matrix w1 are redistributed to form the mapped feature. After being rearranged according to the registration information w1 between x1 and y4 in this way, each position feature of x1 corresponds to the corresponding 2×2 region at the corresponding position in a one-to-one registration manner.

[0093] B2. First, calculate the similarity between the position features of each pixel in feature x1 and the window range features at the corresponding positions in feature to obtain the mapping matrix w2 by using cosine similarity to determine the most matching feature within the window range. This process helps to lock the specific position of each pixel in x1 in , thus obtaining the corresponding relationship between each position of x1 and . And since only the registration of 4 features within the 2×2 range at the corresponding position needs to be calculated for x1, the computational complexity is greatly reduced. Then, each 4×4 region of feature y2 is corresponded to each position of feature y4. Through the mapping of the mapping matrix w1, each 4×4 region of feature y2 is corresponded to the 2×2 region of . Finally, through the mapping of the mapping matrix w2, each 2×2 region of feature y2 is corresponded to each position of feature x1 to form the mapped feature This process is similar to the process of obtaining the feature through mapping . Since the resolution of y2 is 64×64, first, each 4×4 region of y2 is corresponded to each position of y4. Through the mapping of w1, each 4×4 region can be corresponded to the 2×2 region of .

[0094] B3. First, calculate the similarity between the position features of each pixel in feature x1 and the window range features at the corresponding positions in feature to obtain the mapping matrix w3 by using cosine similarity to determine the most matching feature within the window range. This process helps to lock the specific position of each pixel in x1 in . Then, each 8×8 region of feature y1 is corresponded to each position of y4. Through the mapping of the mapping matrix w1, each 8×8 region in feature y1 is corresponded to the feature corresponds to the 2×2 region, and then through the mapping matrix w2, the rearrangement mapping is performed to make each 4×4 region of the feature y1 correspond to the 2×2 region of the feature ; finally, through the mapping matrix w3, the rearrangement mapping is performed to make each 2×2 region of the feature y1 correspond to each position of the feature x1, forming the mapped feature

[0095] B4. First, calculate the similarity between the feature of each pixel position in the feature x1 and the feature within the window range at the corresponding position in the feature , and use the cosine similarity to determine the most matching feature within the window range to obtain the mapping matrix w4; then, the mapping matrix w4 performs the rearrangement mapping on the feature y1 to form the mapped feature The process of obtaining the mapping matrix w4 is used to lock the specific position of each pixel in x1 in , and then through w4, a final rearrangement mapping can be performed on the basis of to obtain the feature This is also to rearrange y1 based on w 1～4 to complete the precise mapping of the features from the unlabeled frame to the ED / ES frame. This process ensures that each pixel can find its matching feature between the unlabeled frame and the ED / ES frame.

[0096] For example Figure 7 , taking the feature x1(i,j) at the (i,j) position in the ED / ES frame as an example in the present invention, first obtain the features within the 5×5 window centered at the corresponding position of x1(i,j) in y4 (adjust the window size according to the deformation range from the ED / ES frame to the unlabeled frame to ensure that its deformed position is included). First, through the first similarity calculation, locate which feature in this 5×5 window is more similar to x1(i,j) to obtain the matching position of x1(i,j) therein; then, according to the located position, x1(i,j) can be locked to the 2×2 window at the corresponding position in ; then, similarly, calculate the matching position of x1(i,j) in the 2×2 window in , and x1(i,j) can be locked to the 2×2 window at the corresponding position in . By analogy, obtain the position of x1(i,j) in , and based on the final relationship, can be arranged to obtain

[0097] Similarly, the mapping process is described taking the feature x1(i, j) at the (i, j) position in the ED / ES frame as an example; for the positional relationship w1(i, j) between the initially obtained x1(i, j) and y4, we can lock the 8×8 region in y1, and then with the help of w2(i, j) (i.e., the corresponding relationship between x1(i, j) and ), we can lock to the 4×4 region from this 8×8 region; then with the help of w3(i, j), lock to the 2×2 region from the 4×4 region; finally, we can lock to the corresponding position feature in y1 at the same resolution through w4(i, j). Based on this process, the feature at each position in y1 can be rearranged according to the distribution in the corresponding x1, so that the unlabeled frame information is mapped to the distribution of the ED / ES frame, completing the mapping from the unlabeled frame to the labeled frame.

[0098] In the image segmentation model, the two convolutional network branches are respectively defined as the first convolutional network branch and the second convolutional network branch. Among them, the first convolutional network branch includes 7 first convolutional blocks, and the 7 first convolutional blocks are connected from front to back, and the resolutions of these 7 first convolutional blocks are 128, 64, 32, 16, 32, 64, 128 from front to back. The second convolutional network branch includes 9 second convolutional blocks, and the 9 second convolutional blocks are connected from front to back, and the resolutions of these 9 second convolutional blocks are 256, 128, 64, 32, 16, 32, 64, 128, 256 from front to back.

[0099] During training, the first convolutional network branch is used to process the features of the unlabeled heart images, and the second convolutional network branch is used to process the features of the labeled heart images;

[0100] During segmentation, the first convolutional network branch is used to process the features of other auxiliary frame heart images in the same heart sequence as the heart image to be segmented, and the second convolutional network branch is used to process the features of the heart image to be segmented.

[0101] In the image segmentation model, there are 7 edge repair modules, each edge repair module is respectively connected to the first convolutional network branch and the second convolutional network branch, and each edge repair module is respectively connected to the first convolutional block and the second convolutional block with the same resolution.

[0102] The seven first convolutional blocks of the present invention are respectively defined as the first convolutional block 1, the first convolutional block 2, the first convolutional block 3, the first convolutional block 4, the first convolutional block 5, the first convolutional block 6, and the first convolutional block 7 from front to back. The nine second convolutional blocks are respectively defined as the second convolutional block 1, the second convolutional block 2, the second convolutional block 3, the second convolutional block 4, the second convolutional block 5, the second convolutional block 6, the second convolutional block 7, the second convolutional block 8, and the second convolutional block 9 from front to back. The seven edge repair modules are respectively defined as the edge repair module 1, the edge repair module 2, the edge repair module 3, the edge repair module 4, the edge repair module 5, the edge repair module 6, and the edge repair module 7 from front to back.

[0103] Among them, the feature x1 is input into the first convolutional block 1, and the feature is input into the second convolutional block 1. There are skip connections between the first convolutional block 1 and the first convolutional block 7, between the first convolutional block 2 and the first convolutional block 6, and between the first convolutional block 3 and the first convolutional block 5. There are skip connections between the second convolutional block 1 and the second convolutional block 9, between the second convolutional block 2 and the second convolutional block 8, between the second convolutional block 3 and the second convolutional block 7, and between the second convolutional block 4 and the second convolutional block 6. The first convolutional block 1 and the second convolutional block 2 are respectively connected to the edge repair module 1; the first convolutional block 2 and the second convolutional block 3 are respectively connected to the edge repair module 2; the first convolutional block 3 and the second convolutional block 4 are respectively connected to the edge repair module 3; the first convolutional block 4 and the second convolutional block 5 are respectively connected to the edge repair module 4; the first convolutional block 5 and the second convolutional block 6 are respectively connected to the edge repair module 5; the first convolutional block 6 and the second convolutional block 7 are respectively connected to the edge repair module 6; the first convolutional block 7 and the second convolutional block 8 are respectively connected to the edge repair module 7.

[0104] The upsampling output by the first convolutional block 7 and the upsampling output by the second convolutional block 9 constitute the segmentation result.

[0105] Due to the possible accuracy loss caused by layer-by-layer mapping, the present invention repairs it. The repair of the edge repair module is carried out by the following steps, as Figure 8 shown:

[0106] C1. For each position feature P x in the feature Fx input through the second convolutional block, take the 9 position features within the 3×3 window at the corresponding position in the feature Fy input through the first convolutional block. The 9 position features are respectively defined as P1, P2, P3, P4, P5, P6, P7, P8, and P9;

[0107] C2. Calculate the cosine similarity between P x and each block in P1, P2, P3, P4, P5, P6, P7, P8, and P9 through cosine similarity to obtain the similarity matrix Mcos ;

[0108] C3. In the similarity matrix M cos Select the feature most similar to P x Then map the most similar feature to the position of P in the labeled heart image to obtain P x of. y .

[0109] The labeled heart image of the present invention is described by taking the ED / ES frame heart image as an example. The present invention sends the ED / ES frame heart image into the pyramid progressive mapping module to map the feature information of the unlabeled frame into the distribution of the ED / ES frame. Through the layer-by-layer registration mapping from low resolution to high resolution, the pyramid progressive mapping module gradually guides the information of the unlabeled frame to the high-resolution ED / ES frame. In this process, by matching and mapping the spatial information of the unlabeled frame and the ED / ES frame, the unlabeled frame can adapt to the shape of the ED / ES frame, thereby enhancing the accuracy of the segmentation task.

[0110] After the mapping of the pyramid progressive mapping module is completed, the feature information of the ED / ES frame and the unlabeled frame is extracted through two convolutional network branches respectively.

[0111] In each layer of the convolutional network branch, in order to cope with the possible accuracy loss caused by layer-by-layer mapping, the present invention also adds an edge repair module. The edge repair module performs secondary selection and arrangement of the edge information of the unlabeled frame based on the information of the ED / ES frame through window scanning to ensure that its edge can better match the ED / ES frame. This repair process can effectively improve the detail performance of the segmentation result and ensure the accuracy of the segmentation. By mapping the information of the unlabeled frame to the distribution of the ED / ES frame, the present invention breaks through the limitation of the unlabeled frame. In this process, the annotation of the ED / ES frame can be used to supervise the unlabeled frame and guide the extraction of its segmentation information. This method not only improves the utilization efficiency of the unlabeled data, but also effectively reduces the dependence on the labeled data and improves the robustness of the model. Finally, through the combination of the information of the unlabeled frame with the same distribution as the ED / ES frame, the accurate segmentation of the ED / ES frame is completed.

[0112] As Figure 9 shown, the training of the myocardial motion estimation network is carried out by the following steps:

[0113] D1. Input a pair of sequential heart image pairs Image ES and Image ED into the motion estimation network, and predict the motion field M ES from Image ED to Image ES→ED , to predict from ImageED to Image ES 's sports field - M ES→ED ; A pair of sequential cardiac image pairs are illustrated by taking the ED frame and ES frame as examples;

[0114] D2. The motion field M obtained according to D1 ES→ED , using the image deformation function W, map Image ES to the space of Image ED to obtain the predicted image Image ED_pre ; The motion field - M obtained according to D1 ES→ED , using the image deformation function W, map Image ED to the space of Image ES to obtain the predicted image Image ES_pre ;

[0115] D3. Compare the reconstruction loss L ED_pre of the predicted image Image ED with Image rec :

[0116] L rec = ||Image ED - Image ED_pre || 2 + ||Image ES - Image ES_pre || 2

[0117] …… Equation (1);

[0118] Calculate the smooth loss L of the square of the change amount of adjacent points in the motion field according to Equation (2) smooth ;

[0119]

[0120] where M is the motion field, and M is M ES→ED , -M ES→ED , M ES→ED or -M ES→ED ;

[0121] Calculate the total loss L according to the smooth loss L smooth and the reconstruction loss L rec ;

[0122] L = δ rec L rec + δ smooth L smooth …… Equation (3);

[0123] where δrec and δ smooth are weight parameters respectively;

[0124] D4. Determine whether the total loss L has converged. When it converges, go to D5; otherwise, update the parameters of the motion estimation network according to the total loss L and the optimizer, and return to D1;

[0125] D5. The training is over, and the current motion estimation network is used as the trained motion estimation network. The motion estimation network of the present invention is Unet; the optimizer is Adam or SGD.

[0126] It should be noted that since there is a lack of annotation in the present invention, the network is obtained by unsupervised training using data in cardiac MRI. The training method is to map ES according to the predicted myocardial motion estimation field and then compare it with ED. The training convergence of the present invention is judged according to the value of the loss function. After each round of training is completed, the average loss of this round will be output. Loss convergence means that after multiple trainings, the loss value of each round no longer decreases, which is loss convergence.

[0127] This method for obtaining myocardial strain values can achieve efficient learning and prediction with fewer labeled samples, so as to be able to segment the myocardium and ventricles, and then calculate the strain values according to the segmentation results and motion estimation. This method for obtaining myocardial strain values can achieve efficient learning and prediction with only a small number of labeled samples. Compared with traditional deep learning models in the prior art that rely on a large amount of labeled data, the present invention can effectively cope with the problem of missing labeled data, is not limited by the number of labeled samples, and thus is more flexible, can make full use of limited data resources for training and inference, and obtain better results.

[0128] Embodiment 2

[0129] A method for obtaining myocardial strain values as in Embodiment 1, and the specific experimental conditions are as follows:

[0130] Experimental data: The segmented part of the data consists of 121 cases of 4D short-axis cardiac cine-MRIs data collected by the Southern University of Science and Technology Hospital in Shenzhen, and experts segmented and labeled the ED and ES of the left ventricle (LV) and left ventricular myocardium (LVM). In the experiment, through five-fold cross-validation, the image size was unified to 256*256.

[0131] The strain calculation and diagnostic data consist of 75 cases of 4D short-axis cardiac cine-MRIs data collected by the Southern University of Science and Technology Hospital in Shenzhen, including 26 cases in the control group and 49 cases in the myocarditis group (22 cases of rEF and 27 cases of pEF)

[0132] Experimental environment: The model was trained using the PyTorch framework on 4 RTX 4090 GPUs with 24GB of memory. During training, the initial learning rate was set to 10 -3 , and a learning rate decay mechanism with a decay coefficient of 0.98 was utilized. The present invention set 200 batches, and in each batch, the batch size was set to 16.

[0133] Through the above experiments, the myocardial strain value obtained by the myocardial strain value acquisition method of the present invention can intuitively reflect its myocardial function status. The segmentation method proposed by the present invention addresses the problem that most current methods rely on a large amount of labeled data, ensuring as high a segmentation result as possible with a small amount of labeling, and ensuring the accuracy of myocardial strain value calculation.

[0134] Example 3

[0135] A myocardial ventricle segmentation method is carried out by adopting steps (1) and (2) in the myocardial strain value acquisition method of Example 1.

[0136] This myocardial ventricle segmentation method can achieve efficient learning and prediction with fewer labeled samples, thereby being able to segment the myocardium and ventricles. The present invention can achieve efficient learning and prediction with only a small number of labeled samples. Compared with traditional deep learning models in the prior art that rely on a large amount of labeled data, the present invention can effectively address the problem of missing labeled data and is not limited by the number of labeled samples, thus being more flexible and able to make full use of limited data resources for training and inference to obtain better results.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the protection scope of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A method for obtaining myocardial strain value, characterized in that: The following steps are performed: Step (1), obtaining cardiac MRI video data consisting of multiple frames of cardiac images; Step (2), inputting the heart image to be segmented into the trained image segmentation model to segment the myocardium and ventricle, and obtaining a segmentation result; Step (3), inputting the paired cardiac images obtained in step (1) into the trained myocardial motion estimation network to obtain motion estimation; Step (4), calculating the strain value by combining the segmentation result obtained in step (2) and the motion estimation obtained in step (3).

2. The method for obtaining myocardial strain value according to claim 1, characterized in that: In the training of the image segmentation model, the training data are labeled heart images and unlabeled heart images, wherein the labeled heart images have already obtained segmentation results in advance; The training process of the image segmentation model is: A1. Extracting features of the labeled cardiac images in the training data and features of the unlabeled cardiac images in the training data respectively by a feature extractor in the image segmentation model; A2, sending the features of the labeled heart image and the features of the unlabeled heart image extracted by A1 to the pyramid progressive mapping module in the image segmentation model, so as to map the features of the unlabeled heart image to the feature distribution of the labeled heart image, and then entering A3; A3, using the two convolutional network branches and the edge restoration module in the image segmentation model as a whole, and then extracting the features of the labeled heart image and the features of the unlabeled heart image accordingly; A4, selecting and arranging the features of the unlabeled heart image extracted in A3 and the edges of the labeled heart image to obtain a segmentation result of the labeled heart image; A5, calculate the loss value between the segmentation result obtained in A4 and the annotated heart image. If the segmentation result obtained in A4 converges, enter A7; otherwise, enter A6; A6, calculates the gradient based on the loss value obtained in A5 to update the parameters of the image segmentation model, and returns to A1; A7. After the training is completed, the current image segmentation model is used as the trained image segmentation model.

3. The method for obtaining myocardial strain value according to claim 2, characterized in that: The extraction method of the feature extractor is: When the image segmentation model is trained, a feature x1 with a resolution of 8a×8a is extracted from the labeled heart image, where a is a positive integer; and a feature y1 with a resolution of 4a×4a, a feature y2 with a resolution of 3a×3a, a feature y3 with a resolution of 2a×2a, and a feature y4 with a resolution of a×a are extracted from the unlabeled heart image; When the image segmentation model is segmenting, in the heart image to be segmented, feature x1 with a resolution of 8a×8a is extracted; in other auxiliary frame heart images in the same heart sequence as the heart image to be segmented, feature y1 with a resolution of 4a×4a, feature y2 with a resolution of 3a×3a, feature y3 with a resolution of 2a×2a, and feature y4 with a resolution of a×a are extracted.

4. The method for obtaining myocardial strain value according to claim 3, characterized in that: The mapping method of the pyramid progressive mapping module is performed by the following steps: B1. First, calculate the similarity between the position feature of each pixel point in feature x1 and the window range feature in the corresponding position in feature y4, and use cosine similarity to determine the most matching feature in the window range to obtain the mapping matrix w1. The mapping matrix w1 is the corresponding position of each pixel point in feature x1 in the feature y4; then the mapping matrix w1 transforms feature y3, and corresponds each 2×2 area in feature y3 to feature y4; finally, map each 2×2 area of ​​feature y3 according to the mapping matrix w1, redistribute the position of the mapping matrix w1, and form the mapped feature B2. First, calculate the position feature and feature of each pixel in feature x1 The window range feature similarity in the corresponding position in , and the cosine similarity is used to determine the best matching feature in the window range to obtain the mapping matrix w2; then each 4×4 region of feature y2 corresponds to each position of feature y4, and each 4×4 region of feature y2 is mapped to feature y4 through the mapping matrix w1. Finally, the mapping matrix w2 is used to map each 2×2 region of feature y2 to each position of feature x1, forming the mapped feature B3. First, calculate the position feature and feature of each pixel in feature x1 The window range feature similarity in the corresponding position in , and the cosine similarity is used to determine the best matching feature in the window range to obtain the mapping matrix w3; then each 8×8 area of ​​feature y1 corresponds to each position of feature y4, and each 8×8 area in feature y1 is mapped to feature y4 through the mapping matrix w1. The 2×2 region of feature y1 is mapped and rearranged by mapping matrix w2 to correspond to each 4×4 region of feature y1. 2×2 area; finally, the mapping matrix w3 is used to map and rearrange each 2×2 area of ​​feature y1 to correspond to each position of feature x1, forming the mapped feature B4. First, calculate the position feature and feature of each pixel in feature x1 The window range feature similarity in the corresponding position in , and the cosine similarity is used to determine the best matching feature in the window range to obtain the mapping matrix w4; then the mapping matrix w4 maps and rearranges the feature y1 to form the mapped feature 5. The method for obtaining myocardial strain value according to claim 4, characterized in that: The two convolutional network branches in the image segmentation model are respectively defined as a first convolutional network branch and a second convolutional network branch; The first convolutional network branch includes 7 first convolutional blocks, the 7 first convolutional blocks are connected from front to back, and the resolutions of the 7 first convolutional blocks are 128, 64, 32, 16, 32, 64, and 128 from front to back; The second convolutional network branch includes 9 second convolutional blocks, the 9 second convolutional blocks are connected from front to back, and the resolutions of the 9 second convolutional blocks are 256, 128, 64, 32, 16, 32, 64, 128, and 256 from front to back; During training, the first convolutional network branch is used to process features of unlabeled cardiac images, and the second convolutional network branch is used to process features of labeled cardiac images; During segmentation, the first convolutional network branch is used to process features of other auxiliary frame cardiac images in the same cardiac sequence as the cardiac image to be segmented, and the second convolutional network branch is used to process features of the cardiac image to be segmented.

6. The method for obtaining myocardial strain value according to claim 5, characterized in that: The image segmentation model includes 7 edge repair modules, each of which is connected to the first convolutional network branch and the second convolutional network branch respectively, and each edge repair module is connected to the first convolutional block and the second convolutional block of the same resolution.

7. The method for obtaining myocardial strain value according to claim 6, characterized in that: The repair of the edge repair module is performed by the following steps: C1, each position feature P in the feature Fx input by the second convolution block x , take the 9 position features in the 3×3 window corresponding to the position of Fy in the features input by the first convolution block, and define the 9 position features as P1, P2, P3, P4, P5, P6, P7, P8 and P9 respectively; C2. Calculate P by cosine similarity x The cosine similarity with each block in P1, P2, P3, P4, P5, P6, P7, P8 and P9 gives the similarity matrix M cos ; C3, in the similarity matrix M cos Select and P x The most similar feature is then mapped to the labeled heart image P x The position of P y .

8. The method for obtaining myocardial strain value according to any one of claims 1 to 7, characterized in that: The training of the myocardial motion estimation network is carried out by the following steps: D1. Input a pair of time-series cardiac images Image ES and Image ED Input motion estimation network, predict from Image ES to Image ED Sports Field M ES→ED , to predict from Image ED to Image ES Sports Field-M ES→ED ; D2, the sports field M obtained according to D1 ES→ED , use the image deformation function W to transform Image ES Mapping to Image ED space, and get the predicted image Image ED_pre ; The motion field obtained from D1 - M ES→ED , use the image deformation function W to transform Image ED Mapping to Image ES space, and get the predicted image Image ES_pre ; D3. Compare the predicted image Image according to formula (1) ED_pre With Image ED The reconstruction loss L rec : L rec = ||Image ED - Image ED_pre || 2 + ||Image ES - Image ES_pre || 2 …… Equation (1); According to formula (2), the smooth loss L of the square of the change of adjacent points in the motion field is calculated smooth ; Where M is the sports field and M is M ES→ED , -M ES→ED 、M ES→ED or -M ES→ED ; According to the smoothing loss L of formula (3) smooth , reconstruction loss L rec Calculate the total loss L; L=δ rec L rec +d smooth L smooth ……expression(3); where δ rec and δ smooth are weight parameters respectively; D4, determine whether the total loss L has reached convergence, and enter D5 if it has converged; if not, update the parameters of the motion estimation network according to the total loss L and the optimizer, and return to D1; D5. The training is completed and the current motion estimation network is used as the post-training motion estimation network.

9. The method for obtaining myocardial strain value according to claim 8, characterized in that: The motion estimation network is Unet; The optimizer is Adam or SGD.

10. A myocardial ventricle segmentation method, characterized in that: The method is performed by using steps (1) and (2) of the method for obtaining myocardial strain values ​​as described in any one of claims 1 to 9.

Images