An automated method for measuring the long axis of the aorta in left ventricular ultrasound images

Abstract

This invention discloses an automatic method for measuring the long axis of the aorta in left ventricular ultrasound images, relating to the field of medical image processing. The method involves collecting left ventricular ultrasound images, converting the original images into grayscale images, and processing each grayscale image using an improved semi-blind deconvolution algorithm. Key points are sequentially labeled with their serial numbers and coordinate positions on each processed image. A training set and a validation set are then divided from the labeled image set. An improved HRNet model is trained using the training set to obtain a key point detection model. Images from the validation set are input into the key point detection model, detecting six key points in each input image that correspond to the endpoints of the aortic long axis. Straight lines are drawn using each pair of key points as endpoints. The length of each straight line is calculated based on the coordinates of its endpoints and multiplied by the image scaling factor to obtain the aortic long axis measurement result.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and specifically to an automatic method for measuring the long axis of the aorta in left ventricular ultrasound images. Background Technology

[0002] Left ventricular ultrasound images are cardiac images obtained through ultrasound examination and used to assess cardiac structure and function. Measurement of the aortic long axis is crucial for assessing aortic size and morphological abnormalities. In medical practice, the aorta is the largest artery in the human body, responsible for transporting oxygen and blood from the heart to the entire body. The morphology and size of the aorta are essential to the function and health of the cardiovascular system. Therefore, accurate measurement of the aortic long axis is important for assessing heart health and predicting the risk of cardiovascular disease.

[0003] In the past, measuring the long axis of the aorta typically required manual operation by physicians, a method that was time-consuming and susceptible to subjective bias. With advancements in medical technology and computer vision, automated methods for measuring the long axis of the aorta have gained increasing attention. Using computer algorithms and image processing techniques, the aorta can be automatically identified and measured in left ventricular ultrasound images. Automated measurement of the aortic long axis provides physicians with more accurate and objective aortic measurement results. Compared to manual measurement, automated methods reduce the impact of subjective errors and improve measurement consistency and repeatability. This is crucial for the accuracy of clinical diagnosis and treatment, especially in the diagnosis and management of cardiovascular diseases. Secondly, automated measurement of the aortic long axis can improve work efficiency and save medical resources. Traditional manual measurement methods require physicians to expend significant time and effort, while automated methods can complete measurements quickly, reducing the workload of physicians and improving diagnostic efficiency.

[0004] Chinese patent application CN117788550A, entitled "An Automatic Measurement Method for Fetal Cranial Ultrasound Images Based on Edge Guidance," provides an automatic measurement method for ultrasound image parameters. This method first inputs the fetal cranial ultrasound image into an edge-guided segmentation model to obtain the segmentation results of the fetal cranial region. Then, connected component labeling is performed on the segmentation results, and the connected component with the largest pixel area is selected. Next, the Canny edge detection algorithm is used to detect the contour of this connected component, thereby obtaining the contour of the fetal cranial region. Subsequently, ellipse fitting is performed on the contour of the fetal cranial region to obtain the fitting ellipse parameters. Finally, the fetal head circumference is calculated based on the fitting parameters. This method requires edge detection of the image when implementing edge-guided segmentation. However, ultrasound images of the aortic long axis may be affected by various factors, such as tissue structure and blood flow velocity, resulting in unclear edges or indistinct contours. Therefore, it is not suitable for the automatic measurement of the aortic long axis in left ventricular ultrasound images. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the present invention provides an automatic measurement method for the long axis of the aorta in left ventricular ultrasound images, aiming to achieve efficient and accurate automatic measurement of the long axis of the aorta in left ventricular ultrasound images.

[0006] The technical solution of this invention is:

[0007] An automated method for measuring the long axis of the aorta in left ventricular ultrasound images, the method comprising the following steps:

[0008] Step 1: Collect left ventricular ultrasound images and record the relevant information corresponding to each image; the relevant information includes the aortic valve annulus diameter AVAD, the aortic sinus diameter AS, the aortic sinus junction diameter St Jn, and the image scaling ratio S;

[0009] Step 2: Convert the original left ventricular ultrasound image into a grayscale image;

[0010] Step 3: Improve the semi-blind deconvolution algorithm and use the improved semi-blind deconvolution algorithm to process each grayscale image;

[0011] Step 4: On each image after the improved semi-blind deconvolution processing, the upper and lower endpoints of the key points, namely AVAD, AS, and St Jn, are sequentially labeled with their numbers and coordinate positions. Then, the training set and validation set are divided from the labeled image set.

[0012] Step 5: Improve the HRNet model by training the improved HRNet model using the training set to obtain the key point detection model.

[0013] Step 6: Input the images in the validation set into the keypoint detection model to detect 6 keypoints P1, P2, P3, P4, P5, P6 in each input image, which correspond to the endpoints of the long axis of the aorta. Label these 6 keypoints on the image. Draw straight lines for each pair of labeled P1, P2; P3, P4; P5, P6 as endpoints. Calculate the length of each line based on the coordinates of the two endpoints of the lines, obtaining the lengths of the three lines as D1, D2, and D3. Then, D1*S, D2*S, and D3*S are the final measurement results of AVAD, AS, and St Jn, respectively.

[0014] Furthermore, step 3 includes the following steps:

[0015] Step 3.1: Divide the M×N grayscale image I into multiple pixel matrices of size p×q. Then the grayscale image I has (M / p)×(N / q) pixel matrices. Define I... i,j This represents the pixel matrix at the i-th row and j-th column in image I;

[0016] Step 3.2: Let the fuzzy kernel K be a k×k matrix, and use the uniform fuzzy kernel as the initial fuzzy kernel K. 0 K 0 Each value in the array is equal to 1 / k 2 ;

[0017] Step 3.3: Perform an adaptive deconvolution operation for each pixel matrix, and use Nesterov acceleration gradient optimization to minimize the minimization problem in the adaptive deconvolution operation to estimate the pixel matrix obtained after performing the adaptive deconvolution operation on each pixel matrix. Merging pixel matrix Build image Then combine grayscale image I and image Optimize the fuzzy kernel to obtain the fuzzy kernel K. t Alternating iterative calculation of the pixel matrix and fuzzy kernel K t The result image B is obtained after processing each grayscale image using the improved semi-blind deconvolution algorithm.

[0018] Further, step 3.3 includes the following steps:

[0019] Step 3.3.1: Let pixel matrix I i,j The initial estimation matrix is Pixel Matrix I i,j The initial momentum variable matrix is The gradient descent step size of adaptive deconvolution is α, and the momentum factor is β;

[0020] Step 3.3.2: Iteratively calculate the matrix I for each pixel. i,j The pixel matrix obtained after performing adaptive deconvolution operation

[0021] In the t-th iteration, the calculation is first performed according to formula (1):

[0022]

[0023] in, For I i,j The actual estimated matrix of the pixel matrix obtained after performing the t-th adaptive deconvolution operation; For I i,j The result obtained after performing the (t-1)th adaptive deconvolution operation The prediction estimation matrix; for The gradient of is calculated using the following formula:

[0024]

[0025] Among them, K t-1 The fuzzy kernel is obtained after performing the (t-1)th fuzzy kernel optimization; λ is the regularization parameter; * denotes the convolution operation; (K t-1 ) T K represents t-1 The transpose of the matrix;

[0026] Then calculate the iteration result according to formula (3):

[0027]

[0028] in, The result obtained after performing the t-th adaptive deconvolution operation on the pixel matrix The prediction estimation matrix; For I i,j The actual estimated matrix of the pixel matrix obtained after performing the (t+1)th adaptive deconvolution operation;

[0029] Step 3.3.3: Iteration results of all pixel matrices Merge and construct images

[0030] Step 3.3.4: Utilizing images Iterative optimization of the fuzzy kernel: Let the step size of the gradient descent for fuzzy kernel optimization be δ. In the t-th iteration, calculate according to formula (4):

[0031]

[0032] Among them, K t The actual estimated matrix of K obtained after performing the t-th fuzzy kernel optimization on K; For K t-1 The gradient of is calculated using the following formula:

[0033]

[0034] in The image is constructed by merging all pixel matrices after t-1 iterations of computation. for The transpose of ; γ is the regularization parameter;

[0035] Step 3.3.5: Repeat steps 3.3.2 to 3.3.4 until the preset stopping condition is met.

[0036] Furthermore, for each pixel matrix, the initial estimation matrix... By using the initial fuzzy kernel K 0 As a convolution kernel combined with deblurring method for I i,jObtained by performing deblurring.

[0037] Furthermore, at t=1: K t-1 =K 0 .

[0038] Furthermore, step 5 includes the following steps:

[0039] Step 5.1: Use the feature extraction module in the HRNet model to extract the feature map of the input image;

[0040] Step 5.2: Generate a real key point heatmap based on the annotation information and the extracted feature map, and generate a predicted key point heatmap based on the extracted feature map. Predict preliminary candidate key points from the predicted key point heatmap.

[0041] Step 5.3: Filter the preliminary candidate key points to obtain the 6 most significant key points corresponding to the upper and lower endpoints of AVAD, AS, and St Jn.

[0042] Furthermore, six empty matrices are created for each feature map, with the same size as the feature map. The coordinates of key points in the annotation information are scaled proportionally to the size of the feature map and the original image. A two-dimensional Gaussian distribution is generated in the empty matrix with the scaled key point coordinates as the center to fill the entire matrix, generating a real key point heatmap.

[0043] Furthermore, feature maps of different resolutions obtained from the feature extraction module are input into a 1x1 convolutional layer. Each feature map generates 6 predicted key point heatmaps. Each predicted key point heatmap generated from the same feature map corresponds to the location of a key point in a left ventricular ultrasound image.

[0044] Furthermore, the method for predicting preliminary candidate key points from the predicted key point heatmap is as follows: assuming the maximum pixel value in the predicted key point heatmap is E, then all pixels in the predicted key point heatmap with a pixel value greater than E*ρ are considered as preliminary candidate key points, where 0.5≤ρ≤1.

[0045] Furthermore, the improvement to the HRNet model is as follows: Structural similarity loss is added to the loss calculation process of the HRNet model, specifically: first, the structural similarity SSIM(X,Y) between the predicted keypoint heatmap and the actual keypoint heatmap is calculated; then, the SSIM loss is calculated according to the following formula:

[0046] SSIM Loss=1-SSIM(X,Y) (7);

[0047] Subsequently, the SSIM loss is combined with the Cross Entropy Loss function to construct the total loss function of the HRNet model:

[0048] L=α1×Cross Entropy Loss+β1×SSIM Loss (8).

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] This invention provides an efficient and accurate automated method for measuring the long axis of the aorta in left ventricular ultrasound images. It addresses the problems of cumbersome manual operation, subjective errors, and inconsistent accuracy and consistency in existing technologies. By automating the measurement process, subjective errors caused by human intervention are reduced, improving the accuracy and consistency of aortic long axis measurements, ensuring the reliability of the results, and filling a gap in existing computer-aided diagnostic technologies in this field. Attached Figure Description

[0051] Figure 1 This is a flowchart of the automatic measurement method for the long axis of the aorta in left ventricular ultrasound images according to this embodiment;

[0052] Figure 2 This is a flowchart of the feature extraction module of the HRNet model in this embodiment;

[0053] Figure 3 This is a visualization of the implementation method. Detailed Implementation

[0054] To facilitate understanding of this application, a more comprehensive description of this application will be provided below with reference to the accompanying drawings.

[0055] The core idea of ​​this invention is as follows: Combining the characteristics of left ventricular ultrasound images, this invention proposes an automatic measurement method for the long axis of the aorta in left ventricular ultrasound images based on an improved semi-blind deconvolution method and an improved HRNet (High-Resolution Net) model: 1. By alternately optimizing the blur kernel and processing the image globally using a semi-blind deconvolution method, the blurring effect in the image is removed, restoring the image details. This invention improves upon existing semi-blind deconvolution methods, mainly by introducing local adaptive deconvolution and the Nesterov accelerated gradient method. Local adaptive deconvolution aims to perform deconvolution operations and optimizations individually for each local image patch, which is very effective for processing ultrasound images with complex and non-uniform blur. The Nesterov accelerated gradient method is an optimization acceleration method that can significantly improve the convergence speed and stability of the optimization process. The improved semi-blind deconvolution method can effectively handle common blurring problems in ultrasound images, ensuring the accuracy of subsequent keypoint detection. 2. The HRNet model can perform accurate keypoint detection while maintaining high-resolution feature maps. This invention improves the HRNet model by introducing Structural Similarity (SSIM) loss into the loss function to calculate the structural similarity between the predicted keypoint heatmap and the actual keypoint heatmap. Typically, the relative positions of the aortic long axis in left ventricular ultrasound images are fixed. Therefore, adding SSIM loss to the loss function calculation step allows the model to better preserve structural information in the left ventricular ultrasound image during training, thereby improving the accuracy and robustness of keypoint detection. 3. This invention utilizes the improved HRNet model to perform keypoint detection on ultrasound images processed by improved semi-blind deconvolution, accurately locating the six endpoints of the aortic long axis, i.e., the start and end points of the three long axes.

[0056] Figure 1 This is a flowchart of the automatic measurement method for the long axis of the aorta in left ventricular ultrasound images according to this embodiment. Figure 1 As shown, the automatic measurement method for the long axis of the aorta in the left ventricular ultrasound image includes the following steps:

[0057] Step 1: Collect left ventricular ultrasound images and record the relevant information for each image to construct a left ventricular ultrasound image dataset;

[0058] In this embodiment, DICOM files of left ventricular ultrasound images are first collected, and then converted to PNG format. Relevant information for each image is recorded for subsequent HRNet model training and metric calculation. This relevant information includes the aortic annulus diameter (AVAD), aortic sinus diameter (AS), aortic sinus junction diameter (StJn), and the scaling factor S of the left ventricular ultrasound image.

[0059] Step 2: Convert each original left ventricular ultrasound image in the left ventricular ultrasound image dataset into a grayscale image;

[0060] Step 3: Improve the semi-blind deconvolution algorithm and use the improved semi-blind deconvolution algorithm to process each grayscale image to improve the detail quality of the image;

[0061] In this embodiment, the restoration quality and processing efficiency of ultrasound images are improved by introducing local adaptive deconvolution and Nesterov accelerated gradient method (NAG). This method utilizes local adaptive deconvolution to more accurately capture the blur characteristics of different regions in the image, while NAG accelerates the optimization process, achieving faster convergence and higher image restoration accuracy.

[0062] Step 3.1: Divide each grayscale image into pixel matrices: Divide the M×N grayscale image I into multiple pixel matrices of size p×q (e.g., 50×50). Then, the grayscale image I has (M / p)×(N / q) pixel matrices. Define I... i,j This represents the pixel matrix at the i-th row and j-th column in image I;

[0063] Step 3.2: Let the fuzzy kernel K be a k×k (e.g., 5×5) matrix, representing the range of fuzzy diffusion; use a uniform fuzzy kernel as the initial fuzzy kernel K. 0 K 0 Each value in the array is equal to 1 / k 2 ;

[0064] Step 3.3: Perform an adaptive deconvolution operation for each pixel matrix, and use Nesterov acceleration gradient optimization to minimize the minimization problem in the adaptive deconvolution operation to estimate the pixel matrix obtained after performing the adaptive deconvolution operation on each pixel matrix. Merging pixel matrix Build image Then combine grayscale image I and image Optimize the fuzzy kernel to obtain the fuzzy kernel K. t Alternating iterative calculation of the pixel matrix and fuzzy kernel K t The result image B is obtained after processing each grayscale image using the improved semi-blind deconvolution algorithm.

[0065] Step 3.3.1: Initialize variables: Let the pixel matrix I i,j The initial estimation matrix is Pixel Matrix I i,j The initial momentum variable matrix is The gradient descent step size for adaptive deconvolution is α (e.g., 0.01), and the momentum factor is β (e.g., 0.9); for each pixel matrix, the initial estimation matrix... This can be achieved by using the initial fuzzy kernel K. 0 As a convolution kernel combined with deblurring methods (such as Wiener filtering) for I i,j Obtained by performing deblurring.

[0066] Step 3.3.2: Iteratively calculate the matrix I for each pixel. i,j The pixel matrix obtained after performing adaptive deconvolution operation

[0067] In the th iteration, the formula is first calculated according to formula (1):

[0068]

[0069] in, For I i,j The actual estimated matrix of the pixel matrix obtained after performing the t-th adaptive deconvolution operation; For I i,j The result obtained after performing the (t-1)th adaptive deconvolution operation The prediction estimation matrix, it should be noted that when t=1, for The gradient of the image consists of two parts: a data fidelity term and a regularization term. The data fidelity term ensures that the resulting image is as close as possible to the grayscale image, while the regularization term constrains the smoothness of the resulting image. The formula for calculating the gradient is:

[0070]

[0071] Among them, K t-1 To obtain the fuzzy kernel after performing the (t-1)th fuzzy kernel optimization, it should be noted that when t=1, K t-1 =K 0 ; λ is the regularization parameter (e.g., 0.01); * indicates the convolution operation; (K t-1 ) T K represents t-1 The transpose of .

[0072] The iteration results are then calculated according to formula (3):

[0073]

[0074] in The result obtained after performing the t-th adaptive deconvolution operation on the pixel matrix The prediction estimation matrix; For I i,j The actual estimated pixel matrix obtained after performing the (t+1)th adaptive deconvolution operation. A momentum term is added during the iteration. The calculation helps to accelerate the convergence speed and reduce oscillations that occur during the optimization process.

[0075] Step 3.3.3: Iteration results of all pixel matrices Merge and construct images

[0076] Step 3.3.4: Utilizing images Perform fuzzy kernel optimization: Let the step size of the gradient descent in fuzzy kernel optimization be δ (e.g., 0.01). In the t-th iteration, the calculation formula is:

[0077]

[0078] Among them, K t The actual estimated matrix of K obtained after performing the t-th fuzzy kernel optimization on K; For K t-1 The gradient of is calculated using the following formula:

[0079]

[0080] in The image is constructed by merging all pixel matrices after t-1 iterations. It should be noted that when t=1... for The merged result; for The transpose of the fuzzy kernel; γ is the regularization parameter (e.g., 0.01), which constrains the complexity of the fuzzy kernel and prevents it from overfitting.

[0081] Step 3.3.5: Repeat steps 3.3.2 to 3.3.4, iteratively calculating the pixel matrix. and fuzzy kernel K t The iteration continues until the stopping condition is met. Given a maximum number of iterations E (e.g., 50) and a convergence threshold ∈ (e.g., 10), the iteration continues until the stopping condition is met. -5 Each iteration checks whether t equals E or whether... in For K t The gradient is calculated, and iteration stops if the condition is met. Suppose that the stopping condition is met after the Qth alternating iteration, and the image is obtained.

[0082] Step 4: On each image after the improved semi-blind deconvolution processing, the upper and lower endpoints of the key points, namely AVAD, AS, and St Jn, are sequentially labeled with their numbers and coordinate positions. Then, the training set and validation set are divided from the labeled image set.

[0083] In this embodiment, the labelme tool was used to sequentially label the upper and lower endpoints of key points (AVAD, AS, St Jn) on each image after the improved semi-blind deconvolution processing, generating corresponding JSON files. A total of 240 clear left ventricular ultrasound images were obtained. 168 non-overlapping images were randomly selected to construct a training set for training, 48 images were selected to construct a test set for testing, and 24 images were selected to construct a validation set for validation. Simultaneously, the labeled JSON files were integrated according to their respective tasks to generate three JSON files: training set, test set, and validation set.

[0084] Step 5: Improve the HRNet model by training the improved HRNet model using the training set partitioned in Step 4 to obtain the key point detection model.

[0085] Using the HRNet model can improve detection accuracy through high-resolution feature representation, especially significantly enhancing its ability to handle small targets and complex backgrounds. The process of training the improved HRNet model using the training set partitioned in step 4 includes the following steps:

[0086] Step 5.1: Use the feature extraction module in the HRNet model to extract the feature map of the input image.

[0087] The model first extracts image features through a feature extraction module. The feature extraction module process is as follows: Figure 2 As shown in the diagram, in the feature extraction module, taking a 256x192 pixel input image as an example, the input image first passes through a 3x3 convolutional layer to reduce the resolution to 128x96 pixels. Next, the image enters the first stage, maintaining the 128x96 resolution through multiple 3x3 convolutional layers. In the second stage, the feature map is divided into two branches: one continues to maintain 128x96 pixels, and the other is downsampled to 64x48 pixels through a 3x3 convolutional layer. In the third stage, the feature map is further divided into three branches, maintaining 128x96, 64x48, and 32x24 pixels respectively, with the third branch obtained through downsampling. The fourth stage generates four feature maps: 128x96, 64x48, 32x24, and 16x12 pixels, with the lowest resolution feature map obtained through further downsampling. After each stage, multi-scale feature fusion is performed. Upsampling and downsampling are used to bring feature maps of different resolutions to the same resolution, and then they are fused to finally generate four feature maps of different resolutions.

[0088] Step 5.2: Generate a real key point heatmap based on the annotation information and the extracted feature map, and generate a predicted key point heatmap based on the extracted feature map to predict the initial candidate key points.

[0089] Based on the resolution of the four feature maps, six empty matrices are created for each feature map, with the matrix size being the same as that of the feature map. The coordinates of key points in the annotation information are scaled proportionally to the size of the feature map and the original image. A two-dimensional Gaussian distribution is generated to fill the entire empty matrix with the scaled key point coordinates as the center, generating a realistic key point heatmap.

[0090] Four feature maps of different resolutions obtained from the feature extraction module are input into a 1x1 convolutional layer. Each feature map generates six predicted keypoint heatmaps, and each predicted keypoint heatmap generated from the same feature map corresponds to the location of a keypoint in the left ventricular ultrasound image. The 1x1 convolutional layer maps each pixel to the probability value of a keypoint. The keypoint heatmaps are two-dimensional Gaussian distributions, with peak positions corresponding to the precise locations of the keypoints.

[0091] Finally, the position of each preliminary candidate key point is detected from the predicted key point heatmap: assuming the maximum pixel value in the predicted key point heatmap is E, then pixels with pixel values ​​greater than E*ρ (0.5≤ρ≤1) in the predicted key point heatmap are all taken as preliminary candidate key points.

[0092] Step 5.3: Filter the initial candidate key points to obtain the 6 most significant key points.

[0093] The Non-Maximum Suppression (NMS) algorithm is used to filter the heatmaps of each predicted keypoint. The position of the maximum pixel value E in each heatmap after filtering is retained as the predicted coordinate of the keypoint, while other preliminary candidate keypoints are suppressed by setting their pixel values ​​to 0. This step effectively reduces redundant noise points, ensuring that only the most significant keypoints are retained. Subsequently, the four heatmaps corresponding to each keypoint are summed pixel by pixel to generate a fused heatmap of that keypoint. The position of the maximum pixel value in the fused heatmap of that keypoint is used as the predicted position of that keypoint. Finally, the final six keypoints P1, P2, P3, P4, P5, and P6 are obtained and output in the order of the upper and lower endpoints of AVAD, AS, and StJn as marked in step 4.

[0094] The batch size was set to 210 and the batch size to 32. The improved HRNet model was trained at a learning rate of 0.001. After training, the key point detection model was obtained.

[0095] The main improvement of the HRNet model in this invention lies in considering structural similarity loss. Structural similarity loss is added to the loss calculation process of the existing HRNet model. Specifically:

[0096] First, the structural similarity between the predicted keypoint heatmap X and the ground truth keypoint heatmap Y is calculated using the SSIM formula. The SSIM formula is as follows:

[0097]

[0098] Where, μ X and μ Y These are the pixel mean values ​​of the predicted keypoint heatmap and the actual keypoint heatmap, respectively. and These are the variances of the pixel values ​​in the two heatmaps, σ and σ. X,Y It is the covariance of the pixel values ​​of the two heatmaps, where c1 and c2 are constants used to stabilize the denominator. Let c1 = (k1L). 2 c2=(k2L) 2 L is the upper limit of the pixel value range of the two heatmaps, L = 1; k1 and k2 are small constants (e.g., 0.01 and 0.03).

[0099] After obtaining the SSIM value between the predicted keypoint heatmap X and the ground truth keypoint heatmap Y, the SSIM loss is calculated using the following formula:

[0100] SSIM Loss = 1 - SSIM(X,Y) (7)

[0101] Subsequently, the SSIM loss is combined with the Cross Entropy Loss function to form the total loss function:

[0102] L=α1×Cross Entropy Loss+β1×SSIM Loss (8)

[0103] Here, α1 and β1 are weighting coefficients, α1+β1=1, used to balance the importance of different loss terms (e.g. 0.7 and 0.3).

[0104] Step 6: Input the images from the validation set into the keypoint detection model. Detect six keypoints P1, P2, P3, P4, P5, and P6 in each input image, corresponding to the endpoints of the aortic long axis. Label these six keypoints on the image. P1, P2; P3, P4; P5, P6 are paired as endpoints to draw straight lines, such as... Figure 3As shown, the length of each line is calculated based on the coordinates of the two endpoints of the line, and the lengths of the three lines are obtained as D1, D2, and D3. Then, D1*S, D2*S, and D3*S are the final measurement results of AVAD, AS, and St Jn, respectively.

[0105] Using the method of this invention, the mean relative error (MRE) was used to test the results on 24 verification images. The test results showed that the MRE for AVAD was 9.31%, for AS it was 8.28%, and for St Jn it was 11.63%. The results indicate that the method proposed in this invention has a small error and can effectively complete the task of automatic measurement of the aortic long axis. The method of this invention is simple to implement, accurate, and effective, meeting the application requirements.

[0106] It should be understood that, inspired by the technical concept of this invention, those skilled in the art can make various improvements or modifications based on the above content without departing from the scope of this invention, and these modifications still fall within the protection scope of this invention.

Claims

1. An automatic method for measuring the long axis of the aorta in left ventricular ultrasound images, characterized in that, The method includes the following steps: Step 1: Collect left ventricular ultrasound images and record the relevant information corresponding to each image; the relevant information includes the aortic valve annulus diameter AVAD, the aortic sinus diameter AS, the aortic sinus junction diameter St Jn, and the image scaling ratio S; Step 2: Convert the original left ventricular ultrasound image into a grayscale image; Step 3: Improve the semi-blind deconvolution algorithm and use the improved semi-blind deconvolution algorithm to process each grayscale image; Step 4: On each image after the improved semi-blind deconvolution processing, the upper and lower endpoints of the key points, namely AVAD, AS, and St Jn, are sequentially labeled with their numbers and coordinate positions. Then, the training set and validation set are divided from the labeled image set. Step 5: Improve the HRNet model by training the improved HRNet model using the training set to obtain the key point detection model; the improvement of the HRNet model is to consider structural similarity loss and add structural similarity loss to the loss calculation process of the HRNet model. Step 6: Input the images in the validation set into the keypoint detection model to detect 6 keypoints in each input image. Corresponding to the endpoints of the long axis of the aorta, these six key points are marked on the image. ; ; Draw straight lines using each pair of endpoints. Calculate the length of each line based on the coordinates of its endpoints, resulting in the lengths of the three lines. , The final measurement results for AVAD, AS, and St Jn are respectively. Step 3 includes the following steps: Step 3.1: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require Grayscale image of size Divide into multiple sizes The pixel matrix, then the grayscale image have A pixel matrix, defined Representing an image The Middle line, number The pixel matrix at each column; Step 3.2: Set the fuzzy kernel For one The matrix is ​​obtained by using a uniform fuzzy kernel as the initial fuzzy kernel. , Each value in is equal to ; Step 3.3: Perform an adaptive deconvolution operation on each pixel matrix, and use Nesterov acceleration gradient optimization to minimize the minimization problem in the adaptive deconvolution operation to estimate the pixel matrix obtained after performing the adaptive deconvolution operation on each pixel matrix. Merge pixel matrix Build image Combined with grayscale images and images Optimize the fuzzy kernel to obtain the fuzzy kernel. Alternating iterative calculation of the pixel matrix and fuzzy kernel The resulting image is obtained after processing each grayscale image using the improved semi-blind deconvolution algorithm. .

2. The automatic measurement method for the long axis of the aorta in left ventricular ultrasound images according to claim 1, characterized in that, Step 3.3 includes the following steps: Step 3.3.1: Set the pixel matrix The initial estimation matrix is pixel matrix The initial momentum variable matrix is =, the step size of gradient descent in adaptive deconvolution is The momentum factor is ; Step 3.3.2: Iteratively calculate the matrix for each pixel. The pixel matrix obtained after performing adaptive deconvolution operation ; In the In the next iteration, the calculation is first performed according to formula (1): - （1）； in, for Execute the The actual estimated matrix of the pixel matrix obtained after the adaptive deconvolution operation; for Execute the The result obtained after the subadaptive deconvolution operation The prediction estimation matrix; for The gradient of is calculated using the following formula: （2）； in, To execute the first The fuzzy kernel obtained after secondary fuzzy kernel optimization; It is a regularization parameter; Indicates the convolution operation; express The transpose of the matrix; Then calculate the iteration result according to formula (3): = （3）； in, The result obtained after performing the i-th adaptive deconvolution operation on the pixel matrix The prediction estimation matrix; for Execute the The actual estimated matrix of the pixel matrix obtained after the adaptive deconvolution operation; Step 3.3.3: Iteration results of all pixel matrices Merge and construct images ; Step 3.3.4: Utilizing images Iterative optimization of the fuzzy kernel: Let the step size of the gradient descent for fuzzy kernel optimization be... In the In the next iteration, the calculation is performed according to formula (4): （4）； in, for Execute the The result obtained after sub-fuzzy kernel optimization The actual estimated matrix; for The gradient of is calculated using the following formula: = -2( ) （5）； in For all pixel matrices through The image constructed by merging the results of each iteration; for The transpose of the matrix; It is a regularization parameter; Step 3.3.5: Repeat steps 3.3.2 to 3.3.4 until the preset stopping condition is met.

3. The automatic measurement method for the long axis of the aorta in left ventricular ultrasound images according to claim 2, characterized in that, For each pixel matrix, the initial estimation matrix By using the initial fuzzy kernel As a convolution kernel combined with deblurring method Obtained by performing deblurring.

4. The automatic measurement method for the long axis of the aorta in left ventricular ultrasound images according to claim 2, characterized in that, hour: ; = .

5. The automatic measurement method for the long axis of the aorta in left ventricular ultrasound images according to claim 1, characterized in that, Step 5 includes the following steps: Step 5.1: Use the feature extraction module in the HRNet model to extract the feature map of the input image; Step 5.2: Generate a real key point heatmap based on the annotation information and the extracted feature map, and generate a predicted key point heatmap based on the extracted feature map. Predict preliminary candidate key points from the predicted key point heatmap. Step 5.3: Filter the preliminary candidate key points to obtain the 6 most significant key points corresponding to the upper and lower endpoints of AVAD, AS, and St Jn.

6. The automatic measurement method for the long axis of the aorta in left ventricular ultrasound images according to claim 5, characterized in that, For each feature map, create 6 empty matrices with the same size as the feature map. The coordinates of key points in the annotation information are scaled proportionally to the size of the feature map and the original image. A two-dimensional Gaussian distribution is generated in the empty matrix with the scaled key point coordinates as the center to fill the entire matrix, generating a real key point heatmap.

7. The automatic measurement method for the long axis of the aorta in left ventricular ultrasound images according to claim 5, characterized in that, The feature maps of different resolutions obtained from the feature extraction module are input into a 1x1 convolutional layer. Each feature map generates 6 predicted key point heatmaps. Each predicted key point heatmap generated from the same feature map corresponds to the location of a key point in the left ventricular ultrasound image.

8. The automatic measurement method for the long axis of the aorta in left ventricular ultrasound images according to claim 7, characterized in that, The method for predicting preliminary candidate keypoints from the predicted keypoint heatmap is as follows: Assuming the maximum pixel value in the predicted keypoint heatmap is E, then a pixel value greater than E in the predicted keypoint heatmap... All pixels were used as preliminary candidate keypoints, among which .

9. The automatic measurement method for the long axis of the aorta in left ventricular ultrasound images according to claim 6, characterized in that, The improvement to the HRNet model specifically involves: firstly, calculating the structural similarity between the predicted keypoint heatmap and the actual keypoint heatmap. Then calculate the SSIM loss using the following formula: （7）； Subsequently, the SSIM loss and the cross-entropy loss function were compared. Combined, construct the total loss function for the HRNet model: （8）。

Images