Image measurement method and system for left atrial appendage, electronic device and storage medium

Abstract

The application provides an image measurement method and system of a left atrial appendage, an electronic device and a storage medium, and relates to the technical field of image measurement of a left atrial appendage. The image measurement method comprises: transforming three-dimensional heart image data according to a spatial position transformation relationship to determine transformed three-dimensional heart image data; transforming the transformed three-dimensional heart image data according to a left atrial appendage motion change function to determine four-dimensional heart image data; and analyzing the four-dimensional heart image data based on a preset AI algorithm to determine a target dynamic parameter of the left atrial appendage in a heartbeat cycle. The application introduces a dynamic parameter change curve of the left atrial appendage in the entire heartbeat cycle (including the systole and diastole) based on the spatial position transformation relationship and the left atrial appendage motion change function, provides dynamic change information of the left atrial appendage morphology in the heartbeat cycle, and thus makes the image measurement of the left atrial appendage more reliable and stable, and improves the measurement accuracy.

Description

Technical Field

[0001] This application relates to the technical field of left atrial appendage image measurement, and more specifically, to a left atrial appendage image measurement method and system, electronic device and storage medium. Background Technology

[0002] In existing technologies, the anatomical structures of the left atrial appendage are generally measured using ultrasound images. However, the inventors of this application have discovered that current methods for measuring the left atrial appendage using imaging techniques still have at least the following problems:

[0003] 1. It relies heavily on transesophageal echocardiography, which has a narrow imaging field of view, providing only a limited two-dimensional cross-sectional view. It cannot fully display the spatial relationship between the left atrial appendage and surrounding key anatomical structures, and lacks the ability to observe and measure in three-dimensional space.

[0004] 2. Although existing technologies have introduced three-dimensional CT images to compensate for three-dimensional spatial information, conventional three-dimensional CT images are static images, which can only capture the state of the left atrial appendage at a certain moment and lack information on the dynamic changes in the morphology of the left atrial appendage during the heartbeat cycle.

[0005] 3. Using traditional algorithms to directly register 3D CT images with 2D ultrasound images results in poor registration accuracy and stability due to the different imaging methods and significant dimensional differences between the two images. The registered images have low reliability and cannot provide a stable and reliable basis for dynamic measurements.

[0006] 4. Reliance on manual or traditional semi-automatic segmentation methods. The left atrial appendage has a complex morphology, and manual segmentation is time-consuming, subjective, and has poor repeatability. Furthermore, traditional segmentation algorithms have low accuracy in regions with blurred boundaries. Errors in the segmentation results directly propagate to subsequent parameter measurements (such as opening diameter and depth), leading to measurement errors.

[0007] The content in the background section is merely technology known to the public and does not necessarily represent existing technology in this field. Summary of the Invention

[0008] This application provides an image measurement method and system for the left atrial appendage, an electronic device, and a storage medium, which aim to solve at least one of the above-mentioned technical problems.

[0009] According to one aspect of this application, an image measurement method for the left atrial appendage (LAA) is provided, comprising: acquiring three-dimensional cardiac image data and two-dimensional cardiac dynamic video data from a user; performing temporal registration on the two-dimensional cardiac dynamic video data to determine the motion change function of the LAA at different times in the heartbeat cycle; performing spatial registration on the three-dimensional cardiac image data and the two-dimensional cardiac dynamic video data to determine the spatial position transformation relationship; transforming the three-dimensional cardiac image data according to the spatial position transformation relationship to determine transformed three-dimensional cardiac image data; transforming the transformed three-dimensional cardiac image data according to the LAA motion change function to determine four-dimensional cardiac image data; and analyzing the four-dimensional cardiac image data based on a preset AI algorithm to determine the target dynamic parameters of the LAA during the heartbeat cycle.

[0010] According to some embodiments of this application, temporal registration of two-dimensional cardiac dynamic video data to determine the left atrial appendage (LAA) motion change function at different moments in the heartbeat cycle includes: preprocessing the two-dimensional cardiac dynamic video data to obtain preprocessed two-dimensional cardiac dynamic video data; extracting feature points from consecutive frames of the preprocessed two-dimensional cardiac dynamic video data to determine target feature points; tracking the motion vectors of the target feature points in consecutive video frames based on a preset tracking algorithm; constructing a dynamic contour change model of the left atrial appendage based on the motion vectors; determining the dynamic motion trajectory based on the dynamic contour change model; and fitting the left atrial appendage motion change function based on the dynamic motion trajectory.

[0011] According to some embodiments of this application, spatial registration of three-dimensional cardiac image data and two-dimensional cardiac dynamic video data to determine the spatial position transformation relationship includes: resampling the three-dimensional cardiac image data to determine standardized three-dimensional data; determining three-dimensional key points in the standardized three-dimensional data; determining two-dimensional key points in the two-dimensional cardiac dynamic video data; determining the feature matching relationship between the three-dimensional key points and the two-dimensional key points to obtain an initial transformation matrix from the three-dimensional key points to the two-dimensional key points; and optimizing the initial transformation matrix based on the iterative nearest point algorithm to obtain the spatial position transformation relationship.

[0012] According to some embodiments of this application, the target dynamic parameters include one or more of the opening diameter and depth distance values ​​of the left atrial appendage during the systolic phase of the heartbeat cycle; and / or, the target dynamic parameters include one or more of the opening diameter and depth distance values ​​of the left atrial appendage during the diastolic phase of the heartbeat cycle.

[0013] According to another aspect of this application, an image measurement system for the left atrial appendage (LAA) is provided, comprising an image acquisition module, a temporal registration module, a spatial registration module, a model reconstruction module, and a parameter analysis module. The image acquisition module acquires three-dimensional cardiac image data and two-dimensional cardiac dynamic video data from the user; the temporal registration module performs temporal registration on the two-dimensional cardiac dynamic video data to determine the left atrial appendage motion change function at different times during the heartbeat cycle; the spatial registration module performs spatial registration on the three-dimensional cardiac image data and the two-dimensional cardiac dynamic video data to determine the spatial position transformation relationship; the model reconstruction module transforms the three-dimensional cardiac image data according to the spatial position transformation relationship to determine transformed three-dimensional cardiac image data, and transforms the transformed three-dimensional cardiac image data according to the left atrial appendage motion change function to determine four-dimensional cardiac image data; the parameter analysis module analyzes the four-dimensional cardiac image data based on a preset AI algorithm to determine the target dynamic parameters of the left atrial appendage during the heartbeat cycle.

[0014] According to some embodiments of this application, the temporal registration module preprocesses the two-dimensional cardiac dynamic video data to obtain preprocessed two-dimensional cardiac dynamic video data; the temporal registration module extracts feature points from consecutive frames of the preprocessed two-dimensional cardiac dynamic video data to determine target feature points; the temporal registration module tracks the motion vectors of the target feature points in consecutive video frames based on a preset tracking algorithm; the temporal registration module constructs a dynamic contour change model of the left atrial appendage based on the motion vectors; the temporal registration module determines the dynamic motion trajectory based on the dynamic contour change model; and the temporal registration module fits the left atrial appendage motion change function based on the dynamic motion trajectory.

[0015] According to some embodiments of this application, the spatial registration module resamples the three-dimensional cardiac image data to determine standardized three-dimensional data; the spatial registration module determines the three-dimensional key points in the standardized three-dimensional data; the spatial registration module determines the two-dimensional key points in the two-dimensional cardiac dynamic video data; the spatial registration module determines the feature matching relationship between the three-dimensional key points and the two-dimensional key points to obtain an initial transformation matrix from the three-dimensional key points to the two-dimensional key points; the spatial registration module optimizes the initial transformation matrix based on the iterative nearest point algorithm to obtain the spatial position transformation relationship.

[0016] According to some embodiments of this application, the target dynamic parameters include one or more of the opening diameter and depth distance values ​​of the left atrial appendage during the systolic phase of the heartbeat cycle; and / or, the target dynamic parameters include one or more of the opening diameter and depth distance values ​​of the left atrial appendage during the diastolic phase of the heartbeat cycle.

[0017] According to another aspect of this application, an electronic device is also provided. The electronic device includes: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement the image measurement method as described above.

[0018] According to another aspect of this application, a non-volatile computer-readable storage medium is also provided. This storage medium stores a computer program that, when executed by a processor, can implement the image measurement method described above.

[0019] According to another aspect of this application, this application also provides a computer program product. The computer program product includes: a computer program stored on a computer-readable storage medium; the computer program includes program instructions that, when executed by a computer, cause the computer to perform the image measurement method as described above.

[0020] Beneficial effects

[0021] This application performs temporal registration on two-dimensional cardiac dynamic video data to determine the left atrial appendage (LAA) motion variation function at different moments in the heartbeat cycle. It then performs spatial registration on three-dimensional cardiac image data and two-dimensional cardiac dynamic video data to determine the spatial transformation relationship. Subsequently, the three-dimensional cardiac image data is transformed according to the spatial transformation relationship to determine the transformed three-dimensional cardiac image data. The transformed three-dimensional cardiac image data is then transformed according to the LA motion variation function to determine the four-dimensional cardiac image data. Finally, a preset AI algorithm is used to analyze the four-dimensional cardiac image data to determine the target dynamic parameters of the LA during the heartbeat cycle.

[0022] This application provides a three-dimensional spatial information with a large imaging area by spatially registering three-dimensional cardiac image data and two-dimensional cardiac dynamic video data. Based on spatial position transformation relationships and left atrial appendage motion change functions, this application introduces dynamic parameter change curves of the left atrial appendage throughout the entire cardiac cycle (including systole and diastole), providing dynamic change information of the left atrial appendage morphology during the cardiac cycle. This allows for the modeling of the three-dimensional left atrial appendage structure to a four-dimensional left atrial appendage structure, thereby making the image measurement of the left atrial appendage more reliable and stable, and improving measurement accuracy.

[0023] This application achieves image registration between video frames of two-dimensional cardiac dynamic video data through temporal and spatial registration. By analyzing the motion between the same modality (ultrasound-ultrasound) video sequences, the differences in dimensionality and grayscale distribution between images of different modalities can be reduced, thereby improving the image registration accuracy.

[0024] This application employs AI algorithms based on deep learning (such as the U-Net architecture) for automatic segmentation of four-dimensional cardiac image data. This addresses the problems of long processing times, high subjectivity, poor repeatability, and inaccurate boundary segmentation associated with manual or traditional semi-automatic segmentation methods on complex left atrial appendage structures. The AI ​​model can quickly and accurately identify the left atrial appendage boundary and further smooth and optimize the segmentation results by combining with fitting algorithms. This reduces measurement errors introduced by human operation and achieves automated, objective, and standardized measurement of left atrial appendage dynamic parameters. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic flowchart of an image measurement method for the left atrial appendage according to an embodiment of this application is shown.

[0027] Figure 2 This is another schematic flowchart illustrating the image measurement method for the left atrial appendage according to an embodiment of this application;

[0028] Figure 3 This is another schematic flowchart illustrating the image measurement method for the left atrial appendage according to an embodiment of this application;

[0029] Figure 4 A schematic diagram of the structure of the left atrial appendage image measurement system according to an embodiment of this application is shown.

[0030] Explanation of reference numerals in the attached figures:

[0031] Image measurement system 1; image acquisition module 10; temporal registration module 20; spatial registration module 30; model reconstruction module 40; parameter analysis module 50. Detailed Implementation

[0032] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] According to one aspect of this application, an image measurement method for the left atrial appendage is provided. Figure 1 A schematic flowchart illustrating an image measurement method for the left atrial appendage according to an embodiment of this application is shown.

[0034] According to the example embodiment, such as Figure 1 As shown, the image measurement method may include steps S100-S600. Exemplarily, the image measurement method may be performed by an image measurement system with computing capabilities.

[0035] In step S100, the image measurement system acquires the user's three-dimensional cardiac image data and two-dimensional cardiac dynamic video data.

[0036] For example, three-dimensional cardiac image data can be 3D-CTA (3-Dimensional Computed Tomography Angiography) of the user's heart. Two-dimensional cardiac dynamic video data can be transesophageal echocardiography video data of the user's heart.

[0037] In step S200, the image measurement system performs time-series registration on the two-dimensional cardiac dynamic video data to determine the left atrial appendage motion change function S(t) at different times of the heartbeat cycle.

[0038] For example, temporal registration involves aligning and normalizing video frames from different time points in two-dimensional cardiac dynamic video data along the time axis, mapping them to a standardized heartbeat cycle. The left atrial appendage motion variation function S(t) characterizes the changes in the morphology (e.g., opening size, area, shape) and / or position (e.g., movement, rotation) of the left atrial appendage over a standardized heartbeat cycle (typically from 0% to 100%, where 0% represents the end of diastole and 100% represents the next end of diastole). By performing temporal registration on two-dimensional cardiac dynamic video data, the image measurement system can transform the video frames of the two-dimensional cardiac dynamic video data into a standard motion variation model (i.e., the left atrial appendage motion variation function S(t)).

[0039] In step S300, the image measurement system performs spatial registration on the three-dimensional cardiac image data and the two-dimensional cardiac dynamic video data to determine the spatial position transformation relationship T(CTA, US).

[0040] For example, due to the different imaging methods of 3D CT images and 2D ultrasound images, there are various differences between 3D cardiac image data and 2D cardiac dynamic video data, such as dimensional differences, modal differences, and spatial coordinate system differences. Image measurement systems can determine the spatial transformation relationship T(CTA, US) between 3D cardiac image data and 2D cardiac dynamic video data through spatial registration.

[0041] In step S400, the image measurement system transforms the three-dimensional cardiac image data according to the spatial position transformation relationship to determine the transformed three-dimensional cardiac image data.

[0042] For example, an image measurement system can establish a geometric correspondence between three-dimensional cardiac image data and two-dimensional cardiac dynamic video data through this spatial position transformation relationship T(CTA, US), thereby transforming both into the same three-dimensional coordinate system.

[0043] For example, the image measurement system transforms the three-dimensional cardiac image data according to the spatial position transformation relationship T(CTA, US) to obtain the transformed three-dimensional cardiac image data (i.e., the transformed three-dimensional cardiac image data). The central plane of this transformed three-dimensional cardiac image data and the two-dimensional cardiac dynamic video data can be on the same plane.

[0044] In step S500, the image measurement system transforms the three-dimensional cardiac image data according to the left atrial appendage motion change function to determine the four-dimensional cardiac image data.

[0045] For example, an image measurement system can transform newly obtained transformed 3D cardiac image data using the left atrial appendage motion change function S(t), resulting in 3D cardiac image data corresponding to each video frame of the 2D cardiac dynamic video data. A series of 3D cardiac image data forms 4D cardiac image data (such as 4D-CTA-LAA, LAA: Left Atrial Appendix, left atrial appendage). This 4D cardiac image data can include the 3D anatomical image of the left atrial appendage corresponding to the image space of each video frame of the 2D cardiac dynamic video data, and can include the continuous and dynamic 3D morphological changes of the left atrial appendage within one or more complete cardiac cycles.

[0046] In step S600, the image measurement system analyzes four-dimensional cardiac image data based on a preset AI algorithm to determine the target dynamic parameters of the left atrial appendage during the heartbeat cycle.

[0047] For example, an image measurement system can perform image segmentation on four-dimensional heart image data based on a preset AI algorithm (such as a convolutional neural network based on the U-Net architecture, where U-Net is an end-to-end fully convolutional encoder-decoder structure) to obtain the left atrial appendage segmentation result.

[0048] Image measurement systems can analyze the segmentation results of the left atrial appendage using analytical geometric algorithms, thereby determining the target dynamic parameters of the left atrial appendage during the heartbeat cycle.

[0049] The image measurement system can calculate the center point of the left atrial appendage opening based on the segmented left atrial appendage image. Using this center point as a reference, multiple line segments intersecting the boundary of the left atrial appendage region are determined along 180° in preset angles (e.g., each degree). The opening diameter can be determined by calculating the length of the line segments (e.g., the longest line segment can be determined as the maximum opening diameter, and the shortest line segment can be determined as the minimum opening diameter).

[0050] Optionally, the target dynamic parameters include, but are not limited to, the opening diameter and depth distance of the left atrial appendage during the systolic phase of the heartbeat cycle, and the opening diameter and depth distance during the diastolic phase of the heartbeat cycle.

[0051] Through the above embodiments, this application can perform temporal registration on two-dimensional cardiac dynamic video data to determine the left atrial appendage (LAA) motion change function at different moments in the heartbeat cycle. It can also perform spatial registration on three-dimensional cardiac image data and two-dimensional cardiac dynamic video data to determine the spatial position transformation relationship. Then, based on the spatial position transformation relationship, the three-dimensional cardiac image data is transformed to determine the transformed three-dimensional cardiac image data. Finally, based on the LAA motion change function, the transformed three-dimensional cardiac image data is transformed to determine the four-dimensional cardiac image data. Finally, a preset AI algorithm can be used to analyze the four-dimensional cardiac image data to determine the target dynamic parameters of the LAA during the heartbeat cycle.

[0052] This application provides a three-dimensional spatial information with a large imaging area by spatially registering three-dimensional cardiac image data and two-dimensional cardiac dynamic video data. Based on spatial position transformation relationships and left atrial appendage motion change functions, this application introduces dynamic parameter change curves of the left atrial appendage throughout the entire cardiac cycle (including systole and diastole), providing dynamic change information of the left atrial appendage morphology during the cardiac cycle. This allows for the modeling of the three-dimensional left atrial appendage structure to a four-dimensional left atrial appendage structure, thereby making the image measurement of the left atrial appendage more reliable and stable, and improving measurement accuracy.

[0053] This application achieves image registration between video frames of two-dimensional cardiac dynamic video data through temporal and spatial registration. By analyzing the motion between the same modality (ultrasound-ultrasound) video sequences, the differences in dimensionality and grayscale distribution between images of different modalities can be reduced, thereby improving the image registration accuracy.

[0054] This application employs AI algorithms based on deep learning (such as the U-Net architecture) for automatic segmentation of four-dimensional cardiac image data. This addresses the problems of long processing times, high subjectivity, poor repeatability, and inaccurate boundary segmentation associated with manual or traditional semi-automatic segmentation methods on complex left atrial appendage structures. The AI ​​model can quickly and accurately identify the left atrial appendage boundary and further smooth and optimize the segmentation results by combining with fitting algorithms. This reduces measurement errors introduced by human operation and achieves automated, objective, and standardized measurement of left atrial appendage dynamic parameters.

[0055] Figure 2 This is another schematic flowchart illustrating the image measurement method for the left atrial appendage according to an embodiment of this application.

[0056] Optionally, such as Figure 2 As shown, step S200 may also include steps S210-S260.

[0057] In step S210, the image measurement system preprocesses the two-dimensional cardiac dynamic video data to obtain preprocessed two-dimensional cardiac dynamic video data.

[0058] For example, the image measurement system can perform noise reduction processing on each frame of the two-dimensional cardiac dynamic video data to reduce speckle noise in the ultrasound images. Furthermore, the image measurement system can perform image enhancement processing to improve the visibility of the endocardial boundary and left atrial appendage tissue structures. With these settings, the image measurement system can obtain preprocessed two-dimensional cardiac dynamic video data with a higher signal-to-noise ratio and clearer tissue boundaries.

[0059] In step S220, the image measurement system extracts feature points from consecutive frames of the preprocessed two-dimensional cardiac dynamic video data to determine the target feature points.

[0060] In step S230, the image measurement system tracks the motion vectors of target feature points in consecutive video frames based on a preset tracking algorithm.

[0061] For example, an image measurement system can extract stable target feature points from consecutive video frames based on feature point extraction techniques. The system can also track the motion vectors of these target feature points within consecutive video frames using pre-defined tracking algorithms (such as optical flow or AI tracking algorithms).

[0062] For example, the optical flow method can estimate the displacement vector of each target feature point between two consecutive video frames by calculating the spatiotemporal gradient of the image sequence, thereby outputting the motion trajectory data of each target feature point.

[0063] In step S240, the image measurement system constructs a dynamic contour change model of the left atrial appendage based on the motion vector.

[0064] For example, the image measurement system constructs a dynamic contour change model of the left atrial appendage based on the motion vectors of the obtained target feature points. For example, the image measurement system generates a unified model (i.e., a dynamic contour change model) that can reflect the deformation process of the overall contour of the left atrial appendage (including the opening edge, body and tip) in time through scaling, displacement and rotation by using the motion trajectory of the target feature points as constraints.

[0065] In step S250, the image measurement system determines the dynamic motion trajectory based on the dynamic contour change model.

[0066] For example, an image measurement system can extract a preliminary trajectory from a constructed dynamic contour change model, representing the overall motion of the left atrial appendage (e.g., displacement of the opening center point, changes in opening area, etc.) as it changes over time. Subsequently, the image measurement system uses a Kalman filter algorithm to smooth the preliminary trajectory and remove abnormal data points caused by tracking errors or image noise, finally outputting an optimized, smoothed dynamic motion trajectory.

[0067] In step S260, the image measurement system obtains the left atrial appendage motion change function by fitting the dynamic motion trajectory.

[0068] For example, the image measurement system generates the left atrial appendage motion change function S(t) based on the optimized dynamic motion trajectory using a mathematical fitting algorithm.

[0069] For example, the left atrial appendage motion change function S(t) can be a quantitative parameter value of the left atrial appendage as a whole obtained through multiple time points, in order to fit the motion amplitude under the heartbeat cycle.

[0070] For example, the image measurement system can use the least squares method to fit and calculate the left atrial appendage motion change function S(t) from the dynamic motion trajectory. As an embodiment, if the opening area of ​​the left atrial appendage changes with time, the image measurement system can calculate the value of the left atrial appendage opening area at any time using the least squares method based on the values ​​of the left atrial appendage opening area at multiple times (e.g., 10 times) and the corresponding times, thereby fitting the left atrial appendage motion change function S(t).

[0071] Through the above embodiments, this application can transform unstructured two-dimensional ultrasound video data into an accurate and quantifiable motion change model (dynamic contour change model) through temporal registration, which can provide temporal driving information for subsequent four-dimensional image data reconstruction.

[0072] Figure 3 This is another schematic flowchart illustrating the image measurement method for the left atrial appendage according to an embodiment of this application.

[0073] Optionally, such as Figure 3 As shown, step S300 may also include steps S310-S350.

[0074] In step S310, the image measurement system resamples the three-dimensional cardiac image data to determine standardized three-dimensional data.

[0075] In step S320, the image measurement system determines the three-dimensional key points in the standardized three-dimensional data.

[0076] In step S330, the image measurement system determines the two-dimensional key points in the two-dimensional cardiac dynamic video data.

[0077] For example, an image measurement system resamples input three-dimensional cardiac image data to obtain standardized three-dimensional data. The system then extracts three-dimensional anatomical keypoints (i.e., 3D keypoints) from the standardized 3D data and two-dimensional anatomical keypoints (i.e., 2D keypoints) from the two-dimensional cardiac motion video data. Exemplarily, the image measurement system can employ deep learning techniques (such as convolutional neural networks) for keypoint detection.

[0078] In step S340, the image measurement system determines the feature matching relationship between the three-dimensional key points and the two-dimensional key points to obtain the initial transformation matrix from the three-dimensional key points to the two-dimensional key points.

[0079] In step S350, the image measurement system optimizes the initial transformation matrix based on the iterative nearest point algorithm to obtain the spatial position transformation relationship.

[0080] For example, an image measurement system can establish the correspondence between 3D keypoints and 2D keypoints to obtain an initial transformation matrix containing rotation, translation, or projection parameters. Then, an improved iterative nearest-point algorithm is used to refine and optimize the initial transformation matrix.

[0081] It is understandable that traditional iterative closest point algorithms optimize the transformation matrix through a process of "matching point pairs - solving the transformation - iterative convergence," but they suffer from sensitivity to noise and outliers. This application's improved iterative closest point algorithm reduces outlier weights based on Mahalanobis distance / kernel function, preventing outliers from dominating the transformation solution. Furthermore, during iteration, point pair weights can be dynamically adjusted based on the residuals, with smaller residuals resulting in higher weights. Additionally, this improved iterative closest point algorithm incorporates a damping factor when solving the transformation matrix, preventing matrix singularities and improving iterative stability. Convergence is determined by the rate of change of the residuals, reducing ineffective iterations.

[0082] Through the above embodiments, this application can solve the modal difference problem caused by different imaging principles based on deep learning technology and iterative nearest point algorithm, and can provide a precise spatial constraint basis for subsequent four-dimensional image data reconstruction.

[0083] According to another aspect of this application, an imaging measurement system for the left atrial appendage is also provided. Figure 4 A schematic diagram of the left atrial appendage imaging measurement system according to an embodiment of this application is shown. Figure 4 As shown, the image measurement system 1 may include an image acquisition module 10, a temporal registration module 20, a spatial registration module 30, a model reconstruction module 40, and a parameter analysis module 50.

[0084] According to an example embodiment, the image acquisition module 10 acquires the user's three-dimensional heart image data and two-dimensional heart dynamic video data.

[0085] For example, three-dimensional cardiac image data can be 3D-CTA (3-Dimensional Computed Tomography Angiography) of the user's heart. Two-dimensional cardiac dynamic video data can be transesophageal echocardiography video data of the user's heart.

[0086] The temporal registration module 20 performs temporal registration on the two-dimensional cardiac dynamic video data to determine the left atrial appendage motion change function S(t) at different times of the heartbeat cycle.

[0087] For example, temporal registration involves aligning and normalizing video frames from different time points in two-dimensional cardiac dynamic video data along the time axis, mapping them to a standardized heartbeat cycle. The left atrial appendage motion variation function S(t) characterizes the changes in the morphology (such as opening size, area, and shape) and / or position (such as movement and rotation) of the left atrial appendage over a standardized heartbeat cycle (typically from 0% to 100%, where 0% represents the end of diastole and 100% represents the next end of diastole). The temporal registration module 20, through temporal registration of the two-dimensional cardiac dynamic video data, can convert the video frames of the two-dimensional cardiac dynamic video data into a standard motion variation model (i.e., the left atrial appendage motion variation function S(t)).

[0088] The spatial registration module 30 performs spatial registration on the three-dimensional cardiac image data and the two-dimensional cardiac dynamic video data to determine the spatial position transformation relationship T(CTA, US).

[0089] For example, due to the different imaging methods of 3D CT images and 2D ultrasound images, there are various differences between 3D cardiac image data and 2D cardiac dynamic video data, such as dimensional differences, modal differences, and spatial coordinate system differences. The spatial registration module 30 can determine the spatial position transformation relationship T(CTA, US) between the 3D cardiac image data and the 2D cardiac dynamic video data through spatial registration.

[0090] The model reconstruction module 40 transforms the three-dimensional heart image data according to the spatial position transformation relationship to determine the transformed three-dimensional heart image data.

[0091] For example, the model reconstruction module 40 can establish a geometric correspondence between three-dimensional cardiac image data and two-dimensional cardiac dynamic video data through the spatial position transformation relationship T(CTA, US), thereby transforming the two into the same three-dimensional coordinate system.

[0092] For example, the model reconstruction module 40 transforms the three-dimensional cardiac image data according to the spatial position transformation relationship T(CTA, US) to obtain the transformed three-dimensional cardiac image data (i.e., the transformed three-dimensional cardiac image data). The central plane of the transformed three-dimensional cardiac image data and the two-dimensional cardiac dynamic video data can be on the same plane.

[0093] The model reconstruction module 40 transforms the three-dimensional heart image data according to the left atrial appendage motion change function to determine the four-dimensional heart image data.

[0094] For example, the model reconstruction module 40 transforms the newly obtained transformed three-dimensional cardiac image data using the left atrial appendage motion change function S(t), which yields three-dimensional cardiac image data corresponding to each video frame of the two-dimensional cardiac dynamic video data. A series of three-dimensional cardiac image data forms four-dimensional cardiac image data (such as 4D-CTA-LAA, LAA: Left Atrial Appendix, left atrial appendage). This four-dimensional cardiac image data can include the three-dimensional anatomical image of the left atrial appendage corresponding to the image space of each video frame of the two-dimensional cardiac dynamic video data, and can include the continuous and dynamic three-dimensional morphological changes of the left atrial appendage within one or more complete cardiac cycles.

[0095] The parameter analysis module 50 analyzes four-dimensional cardiac image data based on a preset AI algorithm to determine the target dynamic parameters of the left atrial appendage during the heartbeat cycle.

[0096] For example, the parameter analysis module 50 can perform image segmentation on four-dimensional heart image data based on a preset AI algorithm (such as a convolutional neural network based on the U-Net architecture, U-Net: an end-to-end fully convolutional encoder-decoder structure) to obtain the left atrial appendage segmentation result.

[0097] The parameter analysis module 50 can analyze the left atrial appendage segmentation results through analytical geometric algorithms, and then determine the target dynamic parameters of the left atrial appendage during the heartbeat cycle through geometric algorithms.

[0098] The parameter analysis module 50 can calculate the center point of the left atrial appendage opening based on the segmented left atrial appendage image, and use the center point as a reference to determine multiple line segments that intersect the boundary of the left atrial appendage region along a 180° angle (e.g., each degree). By calculating the length of the line segments, the opening diameter can be determined (e.g., the longest line segment can be determined as the maximum opening diameter, and the shortest line segment can be determined as the minimum opening diameter, etc.).

[0099] Optionally, the target dynamic parameters include, but are not limited to, the opening diameter and depth distance of the left atrial appendage during the systolic phase of the heartbeat cycle, and the opening diameter and depth distance during the diastolic phase of the heartbeat cycle.

[0100] Through the above embodiments, this application can perform temporal registration on two-dimensional cardiac dynamic video data to determine the left atrial appendage (LAA) motion change function at different moments in the heartbeat cycle. It can also perform spatial registration on three-dimensional cardiac image data and two-dimensional cardiac dynamic video data to determine the spatial position transformation relationship. Then, based on the spatial position transformation relationship, the three-dimensional cardiac image data is transformed to determine the transformed three-dimensional cardiac image data. Finally, based on the LAA motion change function, the transformed three-dimensional cardiac image data is transformed to determine the four-dimensional cardiac image data. Finally, a preset AI algorithm can be used to analyze the four-dimensional cardiac image data to determine the target dynamic parameters of the LAA during the heartbeat cycle.

[0101] This application provides a three-dimensional spatial information with a large imaging area by spatially registering three-dimensional cardiac image data and two-dimensional cardiac dynamic video data. Based on spatial position transformation relationships and left atrial appendage motion change functions, this application introduces dynamic parameter change curves of the left atrial appendage throughout the entire cardiac cycle (including systole and diastole), providing dynamic change information of the left atrial appendage morphology during the cardiac cycle. This allows for the modeling of the three-dimensional left atrial appendage structure to a four-dimensional left atrial appendage structure, thereby making the image measurement of the left atrial appendage more reliable and stable, and improving measurement accuracy.

[0102] This application achieves image registration between video frames of two-dimensional cardiac dynamic video data through temporal and spatial registration. By analyzing the motion between the same modality (ultrasound-ultrasound) video sequences, the dimensional and grayscale distribution differences between images of different modalities can be reduced, thereby improving the image registration accuracy.

[0103] This application employs AI algorithms based on deep learning (such as the U-Net architecture) for automatic segmentation of four-dimensional cardiac image data. This addresses the problems of long processing times, high subjectivity, poor repeatability, and inaccurate boundary segmentation associated with manual or traditional semi-automatic segmentation methods on complex left atrial appendage structures. The AI ​​model can quickly and accurately identify the left atrial appendage boundary and further smooth and optimize the segmentation results by combining with fitting algorithms. This reduces measurement errors introduced by human operation and achieves automated, objective, and standardized measurement of left atrial appendage dynamic parameters.

[0104] Optionally, the timing registration module 20 preprocesses the two-dimensional cardiac dynamic video data to obtain preprocessed two-dimensional cardiac dynamic video data.

[0105] For example, the temporal registration module 20 can perform noise reduction processing on each frame of the two-dimensional cardiac dynamic video data to reduce speckle noise in the ultrasound images. Furthermore, the temporal registration module 20 can also perform image enhancement processing to improve the visibility of the endocardial boundary and left atrial appendage tissue structures. With this configuration, the temporal registration module 20 can obtain preprocessed two-dimensional cardiac dynamic video data with a higher signal-to-noise ratio and clearer tissue boundaries.

[0106] The temporal registration module 20 extracts feature points from consecutive frames of preprocessed two-dimensional cardiac dynamic video data to determine the target feature points.

[0107] The timing registration module 20 tracks the motion vectors of target feature points in consecutive video frames based on a preset tracking algorithm.

[0108] For example, the temporal registration module 20 can extract stable target feature points in consecutive video frames based on feature point extraction technology. The temporal registration module 20 can also track the motion vectors of the target feature points in consecutive video frames based on a preset tracking algorithm (such as optical flow, AI tracking algorithm, etc.).

[0109] For example, the optical flow method can estimate the displacement vector of each target feature point between two consecutive video frames by calculating the spatiotemporal gradient of the image sequence, thereby outputting the motion trajectory data of each target feature point.

[0110] The temporal registration module 20 constructs a dynamic contour change model of the left atrial appendage based on the motion vector.

[0111] For example, the temporal registration module 20 constructs a dynamic contour change model of the left atrial appendage based on the obtained motion vectors of the target feature points. For example, the temporal registration module 20 generates a unified model (i.e., a dynamic contour change model) that can reflect the deformation process of the overall contour of the left atrial appendage (including the opening edge, body and tip) in time through scaling, displacement and rotation by using the motion trajectory of the target feature points as a constraint.

[0112] The timing registration module 20 determines the dynamic motion trajectory based on the dynamic contour change model.

[0113] For example, the temporal registration module 20 can extract the preliminary trajectory of key geometric parameters (such as the displacement of the opening center point, the change of the opening area, etc.) representing the overall motion of the left atrial appendage from the constructed dynamic contour change model. Subsequently, the temporal registration module 20 uses the Kalman filter algorithm to smooth the preliminary trajectory and remove abnormal data points caused by tracking errors or image noise, and finally outputs the optimized and smoothed dynamic motion trajectory.

[0114] The timing registration module 20 obtains the left atrial appendage motion change function by fitting the dynamic motion trajectory.

[0115] For example, the timing registration module 20 generates the left atrial appendage motion change function S(t) based on the optimized dynamic motion trajectory using a mathematical fitting algorithm.

[0116] For example, the left atrial appendage motion change function S(t) can be a quantitative parameter value of the left atrial appendage as a whole obtained through multiple time points, in order to fit the motion amplitude under the heartbeat cycle.

[0117] For example, the time registration module 20 can use the least squares method to fit and calculate the left atrial appendage motion change function S(t) from the dynamic motion trajectory. As an embodiment, if the opening area of ​​the left atrial appendage changes with time, the time registration module 20 can calculate the value of the left atrial appendage opening area at any time using the least squares method based on the values ​​of the left atrial appendage opening area at multiple times (e.g., 10 times) and the corresponding times, thereby fitting the left atrial appendage motion change function S(t).

[0118] Through the above embodiments, this application can transform unstructured two-dimensional ultrasound video data into an accurate and quantifiable motion change model (dynamic contour change model) through temporal registration, which can provide temporal driving information for subsequent four-dimensional image data reconstruction.

[0119] Optionally, the spatial registration module 30 resamples the three-dimensional cardiac image data to determine standardized three-dimensional data.

[0120] The spatial registration module 30 identifies the three-dimensional key points in the standardized three-dimensional data.

[0121] The spatial registration module 30 determines the two-dimensional key points in the two-dimensional cardiac dynamic video data.

[0122] For example, the spatial registration module 30 resamples the input three-dimensional cardiac image data to obtain standardized three-dimensional data. Then, the spatial registration module 30 extracts three-dimensional anatomical keypoints (i.e., three-dimensional keypoints) from the standardized three-dimensional data and extracts two-dimensional anatomical keypoints (i.e., two-dimensional keypoints) from the two-dimensional cardiac dynamic video data. Exemplarily, the spatial registration module 30 can employ deep learning techniques (such as convolutional neural networks) for keypoint detection.

[0123] The spatial registration module 30 determines the feature matching relationship between the three-dimensional key points and the two-dimensional key points to obtain the initial transformation matrix from the three-dimensional key points to the two-dimensional key points.

[0124] The spatial registration module 30 optimizes the initial transformation matrix based on the iterative nearest point algorithm to obtain the spatial position transformation relationship.

[0125] For example, the spatial registration module 30 can establish the correspondence between 3D keypoints and 2D keypoints to obtain an initial transformation matrix containing rotation, translation, or projection parameters. Then, an improved iterative nearest-point algorithm is used to refine and optimize the initial transformation matrix.

[0126] It is understandable that traditional iterative closest point algorithms optimize the transformation matrix through a process of "matching point pairs - solving the transformation - iterative convergence," but they suffer from sensitivity to noise and outliers. This application's improved iterative closest point algorithm reduces outlier weights based on Mahalanobis distance / kernel function, preventing outliers from dominating the transformation solution. Furthermore, during iteration, point pair weights can be dynamically adjusted based on the residuals, with smaller residuals resulting in higher weights. Additionally, this improved iterative closest point algorithm incorporates a damping factor when solving the transformation matrix, preventing matrix singularities and improving iterative stability. Convergence is determined by the rate of change of the residuals, reducing ineffective iterations.

[0127] Through the above embodiments, this application can solve the modal difference problem caused by different imaging principles based on deep learning technology and iterative nearest point algorithm, and can provide a precise spatial constraint basis for subsequent four-dimensional image data reconstruction.

[0128] According to another aspect of this application, an electronic device is also provided. The electronic device includes: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, enable the one or more processors to perform the methods described above.

[0129] According to another aspect of this application, a non-volatile computer-readable storage medium is also provided. This storage medium stores a computer program that, when executed by a processor, can perform the methods described above.

[0130] According to another aspect of this application, this application also provides a computer program product. The computer program product includes: a computer program stored on a computer-readable storage medium; the computer program includes program instructions that, when executed by a computer, cause the computer to perform the methods described above.

[0131] Finally, it should be noted that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions of the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for measuring the image of the left atrial appendage, characterized in that, include: Acquire the user's 3D cardiac image data and 2D dynamic cardiac video data; The two-dimensional cardiac dynamic video data is time-series registered to determine the left atrial appendage motion change function at different times of the heartbeat cycle; Spatial registration is performed on the three-dimensional cardiac image data and the two-dimensional cardiac dynamic video data to determine the spatial position transformation relationship; The three-dimensional cardiac image data is transformed according to the spatial position transformation relationship to determine the transformed three-dimensional cardiac image data; The three-dimensional cardiac image data is transformed according to the left atrial appendage motion change function to determine the four-dimensional cardiac image data; The four-dimensional heart image data is analyzed based on a preset AI algorithm to determine the target dynamic parameters of the left atrial appendage during the heartbeat cycle.

2. The image measurement method according to claim 1, characterized in that, The step of performing time-series registration on the two-dimensional cardiac dynamic video data to determine the left atrial appendage motion change function at different moments in the cardiac cycle includes: The two-dimensional cardiac dynamic video data is preprocessed to obtain preprocessed two-dimensional cardiac dynamic video data; Feature points are extracted from consecutive frames of the preprocessed two-dimensional cardiac dynamic video data to determine the target feature points; The motion vectors of the target feature points in the consecutive frames of the video are tracked based on a preset tracking algorithm; A dynamic contour change model of the left atrial appendage is constructed based on the motion vectors; The dynamic motion trajectory is determined based on the dynamic contour change model. The left atrial appendage motion change function is obtained by fitting the dynamic motion trajectory.

3. The image measurement method according to claim 1, characterized in that, The step of spatially registering the three-dimensional cardiac image data and the two-dimensional cardiac dynamic video data to determine the spatial position transformation relationship includes: The three-dimensional cardiac image data is resampled to determine standardized three-dimensional data; Identify the three-dimensional key points in the standardized three-dimensional data; Identify the two-dimensional key points in the two-dimensional cardiac dynamic video data; The feature matching relationship between the three-dimensional key points and the two-dimensional key points is determined to obtain the initial transformation matrix from the three-dimensional key points to the two-dimensional key points; The initial transformation matrix is ​​optimized based on the iterative nearest point algorithm to obtain the spatial position transformation relationship.

4. The image measurement method according to claim 1, characterized in that, The target dynamic parameters include one or more of the opening diameter and depth distance values ​​of the left atrial appendage during the systolic phase of the cardiac cycle; and / or The target dynamic parameters include one or more of the opening diameter and depth distance values ​​of the left atrial appendage during the diastolic phase of the heartbeat cycle.

5. An image measurement system for the left atrial appendage, characterized in that, include: The image acquisition module acquires the user's three-dimensional heart image data and two-dimensional dynamic heart video data; The temporal registration module performs temporal registration on the two-dimensional cardiac dynamic video data to determine the left atrial appendage motion change function at different times in the heartbeat cycle; The spatial registration module performs spatial registration on the three-dimensional cardiac image data and the two-dimensional cardiac dynamic video data to determine the spatial position transformation relationship; The model reconstruction module transforms the three-dimensional heart image data according to the spatial position transformation relationship to determine the transformed three-dimensional heart image data, and transforms the transformed three-dimensional heart image data according to the left atrial appendage motion change function to determine the four-dimensional heart image data. The parameter analysis module analyzes the four-dimensional heart image data based on a preset AI algorithm to determine the target dynamic parameters of the left atrial appendage during the heartbeat cycle.

6. The image measurement system according to claim 5, characterized in that, The time-series registration module preprocesses the two-dimensional cardiac dynamic video data to obtain preprocessed two-dimensional cardiac dynamic video data. The temporal registration module extracts feature points from consecutive frames of the preprocessed two-dimensional cardiac dynamic video data to determine the target feature points. The temporal registration module tracks the motion vectors of the target feature points in the consecutive frames of the video based on a preset tracking algorithm; The temporal registration module constructs a dynamic contour change model of the left atrial appendage based on the motion vector; The temporal registration module determines the dynamic motion trajectory based on the dynamic contour change model; The time-series registration module obtains the left atrial appendage motion change function by fitting the dynamic motion trajectory.

7. The image measurement system according to claim 5, characterized in that, The spatial registration module resamples the three-dimensional cardiac image data to determine standardized three-dimensional data; The spatial registration module determines the three-dimensional key points in the standardized three-dimensional data; The spatial registration module determines the two-dimensional key points in the two-dimensional cardiac dynamic video data; The spatial registration module determines the feature matching relationship between the three-dimensional key points and the two-dimensional key points to obtain the initial transformation matrix from the three-dimensional key points to the two-dimensional key points; The spatial registration module optimizes the initial transformation matrix based on the iterative nearest point algorithm to obtain the spatial position transformation relationship.

8. The image measurement system according to claim 5, characterized in that, The target dynamic parameters include one or more of the opening diameter and depth distance values ​​of the left atrial appendage during the systolic phase of the cardiac cycle; and / or The target dynamic parameters include one or more of the opening diameter and depth distance values ​​of the left atrial appendage during the diastolic phase of the heartbeat cycle.

9. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image measurement method as described in any one of claims 1-4.

10. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image measurement method as described in any one of claims 1-4.

Images