A method for positioning the left atrial appendage opening plane of 3D-tee images based on a multi-scale information guided diffusion model

Abstract

The application discloses a 3D-TEE image left atrial appendage opening plane positioning method based on a multi-scale information guided diffusion model, wherein first, a 3D-TEE image to be predicted x is input into a ResNet image encoder to obtain image features p(x); meanwhile, the 3D-TEE image to be predicted x is input into a trained multi-scale information guided module to generate multi-scale prior features y prior After 6 steps, finally, a predicted positioning result is obtained. The 3D-TEE image left atrial appendage opening plane positioning method based on the multi-scale information guided diffusion model utilizes HRNet to extract high-resolution multi-scale features, and designs multi-scale attention blocks to fuse the multi-scale features, fully utilizes the multi-scale features, and improves the perception of the model to the image spatial structure. According to the target of the task, the application also decouples the plane positioning parameters into a rotation component and a translation component, adopts different loss functions for different components, and enhances the prediction accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the field of medical artificial intelligence algorithm technology, and in particular to a method for locating the left atrial appendage opening plane in a 3D-TEE image based on a multi-scale information-guided diffusion model. Background Art

[0002] Left atrial appendage occlusion is a minimally invasive cardiac interventional procedure used to prevent stroke in patients with non-valvular atrial fibrillation. The device is delivered to the left atrial appendage ostium via a catheter delivery system and then released, isolating the blood flow between the left atrial appendage and the left atrium, thereby preventing the formation of left atrial appendage thrombus. Before performing left atrial appendage occlusion, imaging tests are required to exclude left atrial appendage thrombus and measure left atrial appendage parameters to determine the type and size of the occluder. The left atrial appendage ostium plane is an important reference plane used by clinicians to determine the size of the occluder and plan the occluder implantation location during left atrial appendage occlusion. However, the anatomical structure of the left atrial appendage varies greatly from person to person. When clinicians use preoperative ultrasound images to determine the left atrial appendage ostium plane and plan the occluder size and release location, they are greatly influenced by personal subjectivity, which often leads to discrepancies between preoperative judgments and intraoperative judgments.

[0003] At present, in the relevant research on the positioning of the left atrial appendage opening plane, the method based on the deep reinforcement learning model is mainly used. It constructs the center line of the left atrial appendage and uses the deep reinforcement model to search for the left atrial appendage opening plane in the center line of the left atrial appendage, thereby locating the left atrial appendage opening plane. This method is applicable to CT images, and lacks the consideration of the posture parameters of the left atrial appendage opening plane. It only searches the plane determined by the center line and cannot locate the correct direction of the left atrial appendage opening plane. Secondly, the anatomical variability of the left atrial appendage is large. For the "cauliflower-shaped" and "cactus-shaped" left atrial appendages with multi-leaf structures, it is difficult to represent them with a center line in the TEE image. Moreover, the deep reinforcement learning model is mainly applicable to structures with smaller anatomical structures and obvious feature changes. For structures with large anatomical variability such as the left atrial appendage, the current model is difficult to accurately complete the motion prediction of locating the left atrial appendage opening plane.

[0004] Therefore, in view of the shortcomings of the existing technology, it is necessary to provide a 3D-TEE image left atrial appendage opening plane positioning method based on a multi-scale information guided diffusion model to solve the shortcomings of the existing technology. Summary of the Invention

[0005] The present invention aims to overcome the shortcomings of the prior art and provide a method for locating the left atrial appendage ostium plane in 3D-TEE images based on a multi-scale information-guided diffusion model. This method can improve the model's perception of the spatial structure of the image and enhance the model's prediction accuracy.

[0006] The above-mentioned purpose of the present invention is achieved through the following technical measures:

[0007] A method for locating the left atrial appendage opening plane in 3D-TEE images based on a multi-scale information-guided diffusion model is provided, comprising the following steps:

[0008] S1. Input the 3D-TEE image x to be predicted into the ResNet image encoder to obtain the image feature ρ(x). At the same time, input the 3D-TEE image x to be predicted into the trained multi-scale information guidance module to generate the multi-scale prior feature y. prior ;

[0009] S2, at time T, the multi-scale prior feature y obtained in S1 prior Add a random Gaussian noise to get the noise variable y T , T is a positive integer;

[0010] S3, the image feature ρ(x) obtained in S1 and the multi-scale prior feature y prior , and the noise variable y T Splicing is performed, and the spliced ​​image features ρ(x) and multi-scale prior features y prior and the noise variable y T Input the trained denoising model ∈ θ , predict the noise at time T-1 and get the noise variable y T-1 ;

[0011] S4, determine whether T-1 reaches 0, if yes, go to S6, if no, go to S5;

[0012] S5. Let T-1 = T, noise variable y T-1 = noise variable y T , return to S3;

[0013] S6. Set the noise variable y T-1 Defined as predicted positioning results The predicted positioning result It includes a rotation component and a translation component. The rotation component is the rotation transformation from the target plane to the xoy plane. The rotation transformation is represented by a quaternion R: [w, x, y, z]. The translation component is the Euclidean distance from the target plane to the center of the 3D-TEE image.

[0014] Preferably, the above denoising model ∈ θ The training method is carried out in the following steps:

[0015] A1. Obtain the 3D-TEE training image x, then obtain the true label y0 based on the 3D-TEE training image x, input the 3D-TEE training image x into the post-training multi-scale information guidance module to obtain the multi-scale prior feature yprior ;

[0016] A2, 3D-TEE training image x is input into ResNet image encoder to obtain image feature ρ(x), and then the image feature ρ(x) and the multi-scale prior feature y obtained by A1 are combined. prior The true label y0 obtained by A1 is diffused according to a randomly generated time step t to generate a noise variable y t , there exists t<T, t is a positive integer;

[0017] A3, the image feature ρ(x) obtained in A1 and the multi-scale prior feature y prior , and A2 obtains the noise variable y t Splicing is performed, and the spliced ​​image features ρ(x) and multi-scale prior features y prior and the noise variable y t Input denoising model ∈ θ , get the noise distribution prediction result;

[0018] A4. Calculate the loss function based on the mean square error. The noise distribution prediction results of A3 and the noise variable y of A1 t The loss result is then used to perform gradient update on the loss result to determine the denoising model ∈ θ , enter A5;

[0019] A5, judge the current training times, when the current training times are less than the maximum training times, return to A2; when the current training times are equal to the maximum training times, use the currently updated denoising model ∈ θ As the trained denoising model ∈ θ .

[0020] Preferably, the above-mentioned multi-scale information guidance model consists of a backbone, a channel mapping layer and a multi-scale attention block, wherein the backbone is HRNet, and the channel mapping layer is added to the output head structure of the HRNet to form a fully connected layer, and the output of the channel mapping layer serves as the input of the multi-scale attention block.

[0021] Preferably, the training process of the multi-scale information guidance module is carried out by the following steps:

[0022] B1. Input the 3D-TEE training image x into the multi-scale information guidance model, extract the multi-scale features, and then calculate the attention of the multi-scale features to obtain the multi-scale prior features y prior ;

[0023] B2, the multi-scale prior feature y obtained from B1 priorThe maximum pooling operation and the full connection layer are applied, and quaternion and translation value results are output, so as to optimize the parameters of the multi-scale information guided model B1 according to the quaternion and translation value results and the pose loss function;

[0024] B3, determining the current training number, when the current training number is less than the target training number, entering B4; when the current training number is equal to the target training number, entering B10;

[0025] B4, determining whether the early stop mechanism is met, when it is not met, entering B1; when it is met, entering B5, and the satisfaction condition of the early stop mechanism is that the loss value of the current training is equal to or greater than the loss value of the previous training;

[0026] B5, determining the difference between the current training number and the target training number, when the training number difference is greater than or equal to the early stop training number, entering B6; when the training number difference is less than the early stop training number, returning to B1;

[0027] B6, setting the current training number as 1, and entering B7;

[0028] B7, inputting the 3D-TEE training image x into the optimized multi-scale information guided model, extracting the multi-scale features, and then calculating the attention of the multi-scale features to obtain the multi-scale prior feature y prior ;

[0029] B8, the multi-scale prior feature y prior obtained in B7 is applied to the maximum pooling operation and the full connection layer, and quaternion and translation value results are output, so as to optimize the parameters of the multi-scale information guided model B7 according to the quaternion and translation value results and the pose loss function;

[0030] B9, determining whether the current training number reaches the early stop training number, when the current training number is less than the early stop training number, entering B7; when the current training number is equal to the early stop training number, entering B10;

[0031] B10, taking the current optimized multi-scale information guided model as the trained multi-scale information guided model.

[0032] Preferably, the above-mentioned pose loss function is represented by formula (1) to formula (3):

[0033]

[0034] wherein, is the pose loss value, is the quaternion loss; is the translation loss; q gt is the quaternion real label of the target plane; q pred is the quaternion prediction positioning result of the target plane; Tgt is the true label of the translation distance from the target plane to the center point of the image; T pred Predict the positioning result as the translation distance from the target plane to the center point of the image.

[0035] Preferably, the target number of training sessions is 300; and the early stopping training session is 10.

[0036] Preferably, the multi-scale attention block calculates the multi-scale attention in a serial superposition manner, and adjusts the order of the layer normalization operations in the scale attention, performs self-attention calculation on the features of each scale, and superimposes the self-attention calculation results of the previous scale, and then sends them to the multi-layer perceptron MLP, and applies layer normalization Layer Norm after each operation.

[0037] The multi-scale attention series superposition method is calculated by equations (4) and (5):

[0038]

[0039] y′ i =y i +y′ i-1 ...Formula (5);

[0040] where y i and y′ i are the input and output features of the i-th multi-scale self-attention respectively; y′ i-1 is the output feature of the i-1th multi-scale attention; Attn(·) is the self-attention, Q, K, V are the query, key and value in the self-attention respectively; W i Q 、W i K and W i V are the learnable parameters of the linear mapping layer.

[0041] In A2, the noise variable y t It is calculated by the diffusion formula, specifically formula (6):

[0042]

[0043] Where ∈~N(0,I), N(0,I) is Gaussian distribution, I is the unit matrix, α t =1-β t , {β t} t＝1：T ∈(0, 1) T , α t Preserve the ratio for the signal, is α t The cumulative product of β t is the noise intensity.

[0044] In the A6, the loss result is obtained by formula (7):

[0045]

[0046] in As a result of loss.

[0047] Preferably, the above-mentioned predicted positioning result It is expressed by formula (8):

[0048]

[0049] in, is a normal distribution, I is the unit matrix, y 0：T-1 is the noise variable y t , and t∈[0,...,T-1].

[0050] Preferably, the above T is 1000.

[0051] Preferably, the above denoising model ∈ θ It consists of multiple fully connected layers embedded with time codes. In each fully connected layer, the Hadamard product operation is performed on the output vector of the fully connected layer and the time code, and the Softplus activation function is applied to the output to embed the time step. The fully connected layer embedded with time code is expressed by formula (8):

[0052] l i =σ(g(l i-1 )⊙g(t))......Formula (8);

[0053] where l i For the denoising model ∈ θ The output of the i-th layer, l i-1 For the denoising model ∈ θ The i-1th layer output of , σ is the Softplus activation function, g(·) is the fully connected layer, and ⊙ is the Hadamard product.

[0054] The present invention provides a 3D-TEE image left atrial appendage opening plane positioning method based on a multi-scale information guided diffusion model, comprising the following steps: S1, inputting the 3D-TEE image x to be predicted into a ResNet image encoder to obtain image features ρ(x), and at the same time inputting the 3D-TEE image x to be predicted into a trained multi-scale information guided module to generate a multi-scale prior feature y prior ; S2, at time T, the multi-scale prior feature y obtained in S1prior Add a random Gaussian noise to get the noise variable y T , T is a positive integer; S3, the image feature ρ(x) obtained by S1, the multi-scale prior feature y prior , and the noise variable y T Splicing is performed, and the spliced ​​image features ρ(x) and multi-scale prior features y prior and the noise variable y T Input the trained denoising model ∈ θ , predict the noise at time T-1 and get the noise variable y T-1 ; S4, determine whether T-1 reaches 0, if yes, go to S6, if not, go to S5; S5, let T-1 = T, noise variable y T-1 = noise variable y T , return to S3; S6, the noise variable y T-1 Defined as predicted positioning results The predicted positioning result It includes a rotation component and a translation component, the rotation component is the rotation transformation from the target plane to the xoy plane, the rotation transformation is represented by a quaternion of R: [w, x, y, z], and the translation component is the Euclidean distance from the target plane to the center of the 3D-TEE image. This 3D-TEE image left atrial appendage opening plane positioning method based on a multi-scale information guided diffusion model utilizes HRNet to extract high-resolution multi-scale features, and designs a multi-scale attention block to fuse multi-scale features, making full use of multi-scale features to improve the model's perception of the image space structure. The present invention also decouples the plane positioning parameters according to the objectives of the task, decomposes them into rotation and translation components, and adopts different loss functions for different components to enhance the prediction accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present invention is further described with reference to the accompanying drawings, but the contents in the accompanying drawings do not constitute any limitation to the present invention.

[0056] Figure 1 This is a flowchart of a method for locating the left atrial appendage opening plane in 3D-TEE images based on a multi-scale information-guided diffusion model.

[0057] Figure 2 Schematic diagram of the structure of the multi-scale information guidance model.

[0058] Figure 3 Schematic diagram of the denoising model.

[0059] Figure 4 This is a comparison chart of the predicted positioning results of the present invention and the predicted positioning results of the prior art. DETAILED DESCRIPTION

[0060] The technical solution of the present invention is further described with reference to the following examples.

[0061] Example 1

[0062] A method for locating the left atrial appendage opening plane in 3D-TEE images based on a multi-scale information-guided diffusion model. Figure 1 As shown, the following steps are included:

[0063] S1. Input the 3D-TEE image x to be predicted into the ResNet image encoder to obtain the image feature ρ(x). At the same time, input the 3D-TEE image x to be predicted into the trained multi-scale information guidance module to generate the multi-scale prior feature y. prior ;

[0064] S2, at time T, the multi-scale prior feature y obtained in S1 prior Add a random Gaussian noise to get the noise variable y T , T is a positive integer;

[0065] S3, the image feature ρ(x) obtained in S1 and the multi-scale prior feature y prior , and the noise variable y T Splicing is performed, and the spliced ​​image features ρ(x) and multi-scale prior features y prior and the noise variable y T Input the trained denoising model ε θ , predict the noise at time T-1 and get the noise variable y T-1 ;

[0066] S4, determine whether T-1 reaches 0, if yes, go to S6, if no, go to S5;

[0067] S5. Let T-1 = T, the noise variable y T-1 = noise variable y T , return to S3;

[0068] S6. Set the noise variable y T-1 Defined as predicted positioning results Predicted positioning results It includes rotation and translation components. The rotation component is the rotation transformation from the target plane to the xoy plane. The rotation transformation is represented by the quaternion R: [w, x, y, z]. The translation component is the Euclidean distance from the target plane to the center of the 3D-TEE image.

[0069] like Figure 2As shown in the figure, the multi-scale information guidance model consists of a backbone, a channel mapping layer and a multi-scale attention block, where the backbone is HRNet. The channel mapping layer is added to the output head structure of HRNet to form a fully connected layer, and the output of the channel mapping layer serves as the input of the multi-scale attention block.

[0070] The training process of the multi-scale information guidance module is carried out in the following steps:

[0071] B1. Input the 3D-TEE training image x into the multi-scale information guidance model, extract the multi-scale features, and then calculate the attention of the multi-scale features to obtain the multi-scale prior features y prior ;

[0072] B2, the multi-scale prior feature y obtained from B1 prior Apply the maximum pooling operation and the fully connected layer to output the quaternion and translation value results. The quaternion and translation value results and the parameters of the multi-scale information guidance model of the attitude loss function B1 are obtained;

[0073] B3. Determine the current number of training sessions. If the current number of training sessions is less than the target number of training sessions, proceed to B4. If the current number of training sessions is equal to the target number of training sessions, proceed to B10.

[0074] B4: Determine whether the early stopping mechanism is satisfied. If not, proceed to B1. If satisfied, proceed to B5. The early stopping mechanism is satisfied when the loss value of the current training is equal to or greater than the loss value of the previous training.

[0075] B5. Determine the difference between the current number of training sessions and the target number of training sessions. If the difference is greater than or equal to the number of early-stop training sessions, proceed to B6. If the difference is less than the number of early-stop training sessions, return to B1.

[0076] B6. Set the current number of training times to 1 and proceed to B7;

[0077] B7. Input the optimized multi-scale information guidance model into the 3D-TEE training image x, extract the multi-scale features, and then perform attention calculation on the multi-scale features to obtain the multi-scale prior features y prior ;

[0078] B8, multi-scale prior features y obtained in B7 prior Apply the maximum pooling operation and the fully connected layer to output the quaternion and translation value results, and use the quaternion and translation value results and the attitude loss function to optimize the parameters of the multi-scale information guidance model of B7;

[0079] B9. Determine whether the current number of training times reaches the number of early-stop training times. If the current number of training times is less than the number of early-stop training times, proceed to B7. If the current number of training times is equal to the number of early-stop training times, proceed to B10.

[0080] B10. Use the currently optimized multi-scale information guidance model as the trained multi-scale information guidance model.

[0081] It should be noted that the training process of the multi-scale information guidance module of the present invention adopts the Early Stop training strategy and adopts posture loss as the loss function for training. When the early stopping mechanism is triggered, the parameters of the multi-scale information guidance conditional model are frozen, and no gradient update is performed in the subsequent regression diffusion model training. Among them, the target number of training times, early stop training times, maximum number of training times and T of the present invention are determined by actual conditions. In this embodiment, the target number of training times is 300; the early stop training times are 10, the maximum number of training times is 350, and T is 1000 as an example for explanation.

[0082] The posture loss function is expressed by equations (1) to (3):

[0083]

[0084] in, is the posture loss value, is the quaternion loss; is the translation loss; q gt is the quaternion true label of the target plane; q pred is the quaternion prediction positioning result of the target plane; T gt is the true label of the translation distance from the target plane to the center point of the image; T pred Predict the positioning result as the translation distance from the target plane to the center point of the image.

[0085] It should be noted that in the task definition, the present invention decomposes the plane positioning parameters into quaternion components and translation components. Therefore, the present invention designs a posture loss function to train the multi-scale information guidance model. The posture loss value is composed of quaternion loss and translation loss, where quaternion loss is the sum of the two components. Used to measure the similarity between the target quaternion and the predicted quaternion; translation loss Used to measure the distance between the target plane and the predicted plane.

[0086] The multi-scale attention block calculates multi-scale attention in a serial superposition manner and adjusts the order of layer normalization operations in the scale attention. It calculates self-attention for the features of each scale and superimposes the self-attention calculation results of the previous scale. It then feeds them into the multi-layer perceptron (MLP) and applies layer normalization after each operation.

[0087] The serial superposition calculation of multi-scale attention is expressed by equations (4) and (5):

[0088]

[0089] y′ i =y i +y′ i-1 ...Formula (5);

[0090] where y i and y′ i are the input and output features of the i-th multi-scale self-attention respectively; y′ i-1 is the output feature of the i-1th multi-scale attention; Attn(·) is the self-attention, Q, K, V are the query, key and value in the self-attention respectively; W i Q 、W i K and W i V are the learnable parameters of the linear mapping layer.

[0091] The denoising model ∈ θ The training method is carried out in the following steps:

[0092] A1. Obtain the 3D-TEE training image x, then obtain the true label y0 based on the 3D-TEE training image x, input the 3D-TEE training image x into the post-training multi-scale information guidance module to obtain the multi-scale prior feature y prior ;

[0093] A2, 3D-TEE training image x is input into ResNet image encoder to obtain image feature ρ(x), and then the image feature ρ(x) and the multi-scale prior feature y obtained by A1 are combined. prior The true label y0 obtained by A1 is diffused according to a randomly generated time step t to generate a noise variable y t , there exists t<T, t is a positive integer;

[0094] A3, the image feature ρ(x) obtained in A1 and the multi-scale prior feature y prior , and A2 obtains the noise variable y t Splicing is performed, and the spliced ​​image features ρ(x) and multi-scale prior features y prior and the noise variable y t Input denoising model ∈ θ , get the noise distribution prediction result;

[0095] A4. Calculate the loss function based on the mean square error. The noise distribution prediction results of A3 and the noise variable y of A1 t The loss result is then used to perform gradient update on the loss result to determine the denoising model ∈ θ , enter A5;

[0096] A5, judge the current training times, when the current training times are less than the maximum training times, return to A2; when the current training times are equal to the maximum training times, use the currently updated denoising model ∈ θ As the trained denoising model ∈ θ .

[0097] It should be noted that S1-S6 of the present invention can also be called the inference stage of the model, or the reverse process, and A1-A5 can also be called the training stage of the model, or the forward process. The forward process is to continuously add noise to the true label according to the time step t (also known as diffusion), that is, y0→…→y t The reverse process is to denoise the noise continuously according to the time step T until the true label is restored (the so-called denoising process), that is,

[0098] The denoising model ∈ θ The pseudo code for training is as follows:

[0099]

[0100] The inference calculation plane prediction result of the present invention is first input into the test set image into the multi-scale information guidance model and image encoder to obtain the multi-scale prior feature y prior And image features ρ(x), sample random Gaussian noise z. Then reverse the diffusion process, according to And the following formula I and formula II, from t = 1000 to 1, infer and reconstruct the predicted positioning result

[0101]

[0102] The pseudo code of the reasoning process is as follows:

[0103]

[0104]

[0105] In the denoising model ∈ θ In the training, the noise variable is calculated by the diffusion formula, specifically formula (6):

[0106]

[0107] Where ∈~N(0,I), N(0,I) is Gaussian distribution, I is the unit matrix, α t =1-β t , {β t} t＝1：T∈(0, 1) T , α t Preserve the ratio for the signal, is α t The cumulative product of β t is the noise intensity. In the denoising model ∈ θ In the training of t , multi-scale prior features y prior And the image feature ρ(x) is input to the denoising model ∈ θ (ρ(x), y t ,y prior , t), the noise distribution is learned.

[0108] In A6, the loss result is obtained by formula (7):

[0109]

[0110] in As a result of loss.

[0111] The denoising model ∈ θ The role of is to use the deep neural network as a function approximator of the parameterized reverse denoising process to learn the noise distribution in the forward diffusion process, ∈ θ The prediction result (i.e. output) is the noise variable y generated after sampling t time steps from the true label y0 t The added real noise distribution ∈, its training can be expressed as the noise estimation loss by formula (7) minimize.

[0112] Predicted positioning results It is expressed by formula (8):

[0113]

[0114] in, is a normal distribution, I is the unit matrix, y 0：T-1 is the noise variable y t , and t∈[0,...,T-1].

[0115] It should be noted that, in the reverse denoising process, the present invention uses the denoising model ∈ trained in the training phase θ , for random Gaussian noise variable distribution p θ (y T ) to carry out the real noise variable distribution p θ The prediction of (y0) is gradually denoised to finally generate the predicted positioning result The predicted positioning results are the quaternion and translation value, which together define the spatial position of the left atrial appendage opening plane.

[0116] like Figure 3 As shown, the denoising model ∈ θ It consists of multiple fully connected layers embedded with time codes. In each fully connected layer, the Hadamard product operation is performed on the output vector of the fully connected layer and the time code, and the Softplus activation function is applied to the output to embed the time step. The fully connected layer embedded with time code is expressed by formula (8):

[0117] l i =σ(g(l i-1 )⊙g(t))......Formula (8);

[0118] where l i For the denoising model ∈ θ The output of the i-th layer, l i-1 For the denoising model ∈ θ The i-1th layer output of , σ is the Softplus activation function, g(·) is the fully connected layer, and ⊙ is the Hadamard product.

[0119] This method for locating the left atrial appendage opening plane in 3D-TEE images, based on a multiscale information-guided diffusion model, utilizes HRNet to extract high-resolution multiscale features and designs a multiscale attention block to fuse these features, fully utilizing them to improve the model's perception of the image's spatial structure. Based on the task's objectives, the method also performs posture decoupling on the plane positioning parameters, decomposing them into rotational and translational components. Different loss functions are applied to these components to enhance the model's prediction accuracy.

[0120] Example 2

[0121] A method for locating the left atrial appendage opening plane of a 3D-TEE image based on a multi-scale information-guided diffusion model, with other features being the same as those in Example 1. The data set used in this embodiment was collected from the Department of Cardiology of Zhujiang Hospital, containing a total of 407 3D-TEE image data. The data was pre-processed by resampling, padding, and cropping to ensure that the size of the input image was 128*128*128. The model was then trained and inferred according to the steps of Example 1. The angle Angle (°) and Euclidean distance (mm) between the target plane and the prediction plane were used as evaluation indicators.

[0122] The quantitative results of the present invention and the prior art are shown in Table 1:

[0123] Table 1. Quantitative results of the present invention and prior art

[0124] Diffusion MSGM Pose-Loss Angle(°) Distance(mm) ResNet — — — 23.65 4.44 M.1 √ — — 16.59 6.3 M.2 √ √ — 13.57 8.11 The present invention √ √ √ 13.86 7.95

[0125] Where √ represents the model to which this technology is applied, M1 represents the classic regression diffusion model, and M2 represents the regression diffusion model with the addition of a multi-scale information guidance model.

[0126] As can be seen from Table 1, compared with the simple convolutional neural network ResNet, the technology of the present invention has been greatly improved in the prediction of the rotation component, but the prediction error of the translation component has become larger. According to M1 compared with M2, when multi-scale information guidance is added, the rotation component is further improved, but the prediction of the translation component is not improved, which shows that multi-scale information has an improvement for the spatial structure information of the left atrial appendage perceived by the model. Comparing M2 with the technology of the present invention, it can be seen that the prediction of the translation component is improved, while the prediction of the rotation component is slightly reduced, which shows that posture loss has an improvement effect on the perception of the model.

[0127] Figure 4 This is the qualitative result of the present invention and the prior art. Figure 4 The following are visualization results of two examples of LAA opening plane positioning. The purple plane is the ground truth label result, and the blue plane is the model prediction result. Compared with other models, the proposed method has better LAA opening plane positioning effect.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A method for locating the left atrial appendage opening plane in 3D-TEE images based on a multi-scale information-guided diffusion model, characterized in that: The steps include: S1. Input the 3D-TEE image x to be predicted into the ResNet image encoder to obtain image features At the same time, the 3D-TEE image to be predicted x is input into the trained multi-scale information guidance module to generate multi-scale prior features ; S2, in T At time t, the multi-scale prior features obtained in S1 Add a random Gaussian noise to get the noise variable y T , T is a positive integer; S3, the image features obtained by S1 , multi-scale prior features , and the noise variables y T Perform stitching and make the stitched image features , multi-scale prior features and noise variables y T Input the trained denoising model ,predict T -1 moment noise, get the noise variable y T-1 ; S4. Judgment T -1 reaches 0, if yes, go to S6, if no, go to S5; S5. Order T -1= T , noise variable y T-1 = noise variable y T , return to S3; S6. Set the noise variable y T-1 Defined as predicted positioning results , the predicted positioning result It includes a rotation component and a translation component. The rotation component is the rotation transformation from the target plane to the xoy plane. The rotation transformation is represented by a quaternion R: [w, x, y, z]. The translation component is the Euclidean distance from the target plane to the center of the 3D-TEE image. The multi-scale information guidance model consists of a backbone, a channel mapping layer, and a multi-scale attention block, wherein the backbone is HRNet. The channel mapping layer is added to the output head structure of the HRNet to form a fully connected layer, and the output of the channel mapping layer serves as the input of the multi-scale attention block. The posture loss function is expressed by equations (1) to (3): ...Formula (1); ...Formula (2); ...Formula (3); in, is the posture loss value, is the quaternion loss; is the translation loss; is the quaternion true label of the target plane; Quaternion prediction positioning result for the target plane; is the true label of the translation distance from the target plane to the center point of the image; Predict the positioning result based on the translation distance from the target plane to the center point of the image; The multi-scale attention block calculates multi-scale attention in a serial superposition manner, adjusts the order of layer normalization operations in the multi-scale attention, calculates self-attention for the features of each scale, and superimposes the self-attention calculation results of the previous scale, and then sends them to the multi-layer perceptron MLP, and applies layer normalization LayerNorm after each operation; The multi-scale attention series superposition method is calculated by equations (4) and (5): ...Formula (4); ...Formula (5); in and Respectively i Input features and output features of multi-scale self-attention; For the i-1 Output features of multi-scale attention; For self-attention, Q 、 K 、 V The query, key and value in the self-attention respectively; 、 and are the learnable parameters of the linear mapping layer.

2. The method for locating the left atrial appendage opening plane in 3D-TEE images based on a multi-scale information-guided diffusion model according to claim 1, characterized in that: The denoising model The training method is carried out in the following steps: A1. Obtain 3D-TEE training images x , and then based on the 3D-TEE training image x Get the true label y 0 , the 3D-TEE training image x Input the trained multi-scale information guidance module to obtain multi-scale prior features ; A2. 3D-TEE training images Input ResNet image encoder to get image features , and then the image feature , multi-scale prior features obtained by A1 And the true label obtained by A1 y 0 Generate a time step randomly t Diffusion is performed to generate noise variables y t ,exist t<T , t is a positive integer; A3, the image features obtained by A1 and multi-scale prior features , and A2 to obtain the noise variable y t Perform stitching and make the stitched image features , multi-scale prior features and noise variables y t Input denoising model , get the noise distribution prediction result; A4. Calculation of loss function based on mean square error The noise distribution prediction results of A3 and the noise variables of A1 y t The loss result is then used to perform gradient update on the loss result to denoise the model. , enter A5; A5. Determine the current number of training times. If the current number of training times is less than the maximum number of training times, return to A2. When the current number of training times is equal to the maximum number of training times, the denoising model currently updated is used. As a post-training denoising model .

3. The method for locating the left atrial appendage opening plane in 3D-TEE images based on a multi-scale information-guided diffusion model according to claim 2, characterized in that: The training process of the multi-scale information guidance module is carried out by the following steps: B1. 3D-TEE training image Input multi-scale information to guide the model, extract multi-scale features, and then calculate the attention of the multi-scale features to obtain multi-scale prior features ; B2, multi-scale prior features obtained from B1 Apply the maximum pooling operation and the fully connected layer to output the quaternion and translation value results. The quaternion and translation value results and the parameters of the multi-scale information guidance model of the attitude loss function B1 are obtained; B3. Determine the current number of training sessions. If the current number of training sessions is less than the target number of training sessions, proceed to B4. If the current number of training sessions is equal to the target number of training sessions, proceed to B10. B4. Determine whether the early stopping mechanism is satisfied. If not, proceed to B1. If satisfied, proceed to B5. The early stopping mechanism is satisfied when the loss value of the current training is equal to or greater than the loss value of the previous training. B5. Determine the difference between the current number of training sessions and the target number of training sessions. If the difference is greater than or equal to the number of early-stop training sessions, proceed to B6. When the difference in training times is less than the number of early stopping training times, return to B1; B6. Set the current number of training times to 1 and proceed to B7; B7. 3D-TEE training image Input the optimized multi-scale information guidance model, extract the multi-scale features, and then calculate the attention of the multi-scale features to obtain the multi-scale prior features ; B8, multi-scale prior features obtained from B7 Apply the maximum pooling operation and the fully connected layer to output the quaternion and translation value results, and use the quaternion and translation value results and the attitude loss function to optimize the parameters of the multi-scale information guidance model of B7; B9. Determine whether the current number of training times reaches the number of early-stop training times. If the current number of training times is less than the number of early-stop training times, proceed to B7. When the current number of training times is equal to the number of early stop training times, enter B10; B10. Use the currently optimized multi-scale information guidance model as the trained multi-scale information guidance model.

4. The method for locating the left atrial appendage opening plane in 3D-TEE images based on a multi-scale information-guided diffusion model according to claim 3, characterized in that: In A2, the noise variable y t Calculated by the diffusion formula, the specific formula (6) is: ...Formula (6); in , N(0, I ) is a Gaussian distribution, I is the unit matrix, , , , Preserve the ratio for the signal, for The cumulative result of is the noise intensity.

5. The method for locating the left atrial appendage opening plane in 3D-TEE images based on a multi-scale information-guided diffusion model according to claim 4, characterized in that: In A4, the loss result is obtained by formula (7): ...Formula (7); in As a result of loss.

6. The method for locating the left atrial appendage opening plane in 3D-TEE images based on a multi-scale information-guided diffusion model according to any one of claims 1 to 2, characterized in that: The predicted positioning result It is expressed by formula (8): ...Formula (8); in, is a normal distribution, is the unit matrix, is the noise variable y t ,and ; described T is 1000.

7. The method for locating the left atrial appendage opening plane in 3D-TEE images based on a multi-scale information-guided diffusion model according to claim 2, characterized in that: The denoising model It consists of multiple fully connected layers embedded with time codes, and in each fully connected layer, the Hadamard product operation is performed on the output vector of the fully connected layer and the time code, and the Softplus activation function is applied to the output to embed the time step. The fully connected layer embedded with time code is expressed by formula (9): ...Formula (9); in For the denoising model No. i Layer output, l i-1 For the denoising model No. i-1 Layer output, is the Softplus activation function, is the fully connected layer, is the Hadamard product.

Images