Intracardiac ultrasound image intracardiac structure three-dimensional reconstruction system and method

Abstract

This application provides a system and method for three-dimensional reconstruction of cardiac cavity structures from intracardiac ultrasound images. The method involves determining edge enhancement maps at different time phases; then determining segmentation masks for the cavity structures at different time phases; further determining the contour gradients of the inner and outer membrane boundaries of the cavity between adjacent time phases based on the contour difference features between the segmentation masks of all cavity structures and the edge enhancement maps at adjacent time phases; further determining the contour mask map of the cavity structure at each time phase; and then determining the spatial distribution constraints of the cavity thickness within each contour mask map; finally, fusing and reconstructing each cavity image using all spatial distribution constraints to obtain a reconstructed feature map of the cardiac cavity structure; and finally, performing image segmentation on the reconstructed feature map based on all contour gradients to obtain a segmentation feature map of the cardiac cavity structure. Using the scheme of this application, accurate segmentation of the cardiac cavity structure can be achieved even when the contour drift of the cardiac membrane structure is caused by nonlinear motion.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction technology, and more specifically, to a system and method for three-dimensional reconstruction of cardiac cavity structure from intracardiac ultrasound images. Background Technology

[0002] 3D reconstruction is a key computational method that restores two-dimensional image data into a three-dimensional form with spatial structure. It is widely used in medical imaging, computer vision, robot navigation and other fields. In the field of medical image processing, 3D reconstruction can reconstruct three-dimensional models of organs, tissues or lesions based on multimodal image data such as computed tomography (CT), magnetic resonance imaging (MRI), and ultrasound. It provides intuitive and reliable structural information for clinical diagnosis, preoperative planning, intraoperative navigation and disease progression assessment. 3D reconstruction usually includes multiple stages such as image preprocessing, key structure segmentation, spatial registration, shape modeling and visualization. Among them, image segmentation is one of the core basic steps of 3D reconstruction. Especially in the 3D modeling of the heart, the accurate segmentation of the cardiac chamber structure directly determines the boundary accuracy and geometric integrity of the reconstructed model. Cardiac chamber image segmentation extracts the target area from the complex background by identifying the edge contours of internal structures such as atria and ventricles, providing high-quality data input for subsequent spatial relocation, thickness analysis and dynamic modeling.

[0003] In medical image analysis, with the development of 3D reconstruction and dynamic modeling technologies, clinical practice has placed higher demands on the accuracy of identifying internal cardiac structures. During the circulatory process of systole and diastole, the cardiac cavity membrane structure undergoes complex and significant nonlinear deformation, causing the boundaries of the intima and epicardium to not only drift continuously in spatial position but also to undergo local distortion of morphology and dynamic changes in thickness. This non-rigid change in the boundary is commonly referred to as contour drift, making it difficult to accurately correspond and register the boundaries of cardiac cavity images at different time phases. Traditional methods based on single-frame image segmentation or simple temporal processing are unable to capture this dynamic and complex boundary change, resulting in a lack of continuity in the segmentation results in the time series and difficulty in maintaining consistency in the spatial structure, thus affecting the accuracy of 3D reconstruction and functional assessment of the cardiac cavity. Therefore, how to accurately segment the cardiac cavity structure under the condition of contour drift caused by nonlinear motion of the cardiac membrane structure has become a challenge faced by the industry. Summary of the Invention

[0004] This application provides a system and method for three-dimensional reconstruction of cardiac cavity structure from intracardiac ultrasound images, which can achieve precise segmentation of cardiac cavity structure when the contour of the cardiac membrane structure is drifted due to nonlinear motion.

[0005] In a first aspect, this application provides a method for segmenting cardiac chamber images, applied to a three-dimensional reconstruction system of cardiac chamber structure from intracardiac ultrasound images. The method includes the following steps:

[0006] The cavity images of the heart chambers are acquired at different time phases during the cardiac cycle, and then edge enhancement is performed on each cavity image to obtain edge enhancement maps at different time phases.

[0007] The segmentation mask of the cavity structure under different time phases is determined, and then the contour gradient of the cavity inner membrane boundary and the cavity outer membrane boundary between adjacent time phases is determined based on the contour difference features between the segmentation mask of all cavity structures and the edge enhancement map under adjacent time phases. The contour gradient represents the parameter that guides the recognition of the heart cavity structure contour during image segmentation.

[0008] For each edge enhancement map, deformable convolution is performed to detect the cavity contour, resulting in a contour mask map of the cavity structure in each time phase, thereby determining the spatial distribution constraints of the cavity thickness within each contour mask map;

[0009] By fusing and reconstructing cavity images at each time phase using spatial distribution constraints of all cavity thicknesses, a reconstructed feature map of the cardiac cavity structure is obtained. Then, image segmentation is performed on the reconstructed feature map based on all contour gradients to obtain a segmentation feature map of the cardiac cavity structure.

[0010] In some embodiments, determining the segmentation mask for the cavity structure at different time phases specifically includes:

[0011] Adaptive threshold segmentation is performed on the edge enhancement map for each time phase to obtain the binary map of the cavity region for each time phase;

[0012] The cavity mask for each time phase is obtained by filling the void regions in the binary image of the cavity region under each time phase through morphological closing operations;

[0013] The boundary of the cavity mask in each time phase is corrected at the sub-pixel level based on the gradient information of the edge enhancement map in each time phase, so as to obtain the segmentation mask of the cavity structure in each time phase.

[0014] In some embodiments, determining the contour gradient of the inner membrane boundary and outer membrane boundary of the cavity between adjacent time phases based on the contour difference features between the segmentation mask of all cavity structures and the lower edge enhancement map of adjacent time phases specifically includes:

[0015] The overlapping area between the segmentation masks of the cavity structure in adjacent time phases is determined, and then the displacement change characteristics of the inner membrane boundary of the cavity between adjacent time phases are extracted;

[0016] The deformation amplitude of the extracorporeal membrane boundary between adjacent time phases is determined by the contour difference features between the lower edge enhancement maps of adjacent time phases;

[0017] The contour gradients of the inner and outer membrane boundaries of the cavity between adjacent time phases are determined based on the displacement variation characteristics of the inner membrane boundary of the cavity between adjacent time phases and the deformation amplitude of the outer membrane boundary of the cavity between adjacent time phases.

[0018] In some embodiments, cavity contour detection by deformable convolution of each edge enhancement map to obtain a contour mask map of the cavity structure in each temporal phase specifically includes:

[0019] Construct deformable convolutional network models;

[0020] The contour probability distribution of the cavity structure in each edge enhancement image is extracted using the deformable convolutional network model.

[0021] Non-maximum suppression is applied to the contour probability distribution of the cavity structure in each edge enhancement image to obtain multiple confidence edge points of the cavity structure in each edge enhancement image;

[0022] Construct a contour mask map of the cavity structure corresponding to each edge enhancement map at a given time based on multiple confidence edge points of the cavity structure within each edge enhancement map.

[0023] In some embodiments, determining the spatial distribution constraints of the cavity thickness within each contour mask specifically includes:

[0024] For each contour mask;

[0025] The radial distance between the inner membrane boundary and the outer membrane boundary of the cavity is sampled along the contour normal direction of the contour mask image, thereby generating the thickness distribution sequence of the cavity structure;

[0026] The thickness constraint weights of the cavity structure are determined based on the thickness distribution sequence.

[0027] The spatial distribution constraint of the cavity thickness in the contour mask is determined by the thickness constraint weight and the spatial gradient characteristics of the cavity thickness in the contour mask.

[0028] In some embodiments, fusing and reconstructing cavity images at each time phase using spatial distribution constraints of all cavity thicknesses to obtain a reconstructed feature map of the cardiac cavity structure specifically includes:

[0029] Based on the spatial distribution constraints of all cavity thicknesses, the cavity structure in each time phase image is spatially relocated to obtain the relocation vector of the cavity structure in each cavity image.

[0030] A reconstructed feature map of the cardiac cavity structure is constructed based on the relocation vector of the cavity structure within each cavity image and the feature response intensity of the edge of the cavity structure within each cavity image.

[0031] In some embodiments, image segmentation is performed on the reconstructed feature map based on all contour gradients to obtain a segmentation feature map of the cardiac cavity structure, specifically including:

[0032] The pixel segmentation regions of the internal cavity structure in the reconstructed feature map are determined based on all contour gradients;

[0033] The segmentation feature map of the cardiac cavity structure is generated by segmenting the pixel regions.

[0034] In some embodiments, the cardiac cycle specifically includes atrial systole, ventricular systole, and diastole.

[0035] In some embodiments, cavity images are acquired using a high-frequency ultrasonic probe.

[0036] Secondly, this application provides a three-dimensional reconstruction system for cardiac cavity structure from intracardiac ultrasound images. This system includes a cardiac cavity image segmentation unit, which comprises:

[0037] The acquisition module is used to acquire images of the heart chambers at different time phases during the cardiac cycle, and then perform edge enhancement on each chamber image to obtain edge enhancement maps at different time phases;

[0038] The processing module is used to determine the segmentation mask of the cavity structure at different time phases, and then determine the contour gradient of the cavity inner membrane boundary and the cavity outer membrane boundary between adjacent time phases based on the contour difference features between the segmentation mask of all cavity structures and the edge enhancement map at adjacent time phases. The contour gradient represents the parameters that guide the recognition of the heart cavity structure contour during image segmentation.

[0039] The processing module is also used to perform deformable convolution cavity contour detection on each edge enhancement map to obtain a contour mask map of the cavity structure in each phase, and then determine the spatial distribution constraint of the cavity thickness in each contour mask map.

[0040] The execution module is used to fuse and reconstruct the cavity images at each time phase by constraining the spatial distribution of all cavity thicknesses to obtain a reconstructed feature map of the cardiac cavity structure, and then perform image segmentation on the reconstructed feature map according to all contour gradients to obtain a segmentation feature map of the cardiac cavity structure.

[0041] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0042] The system and method for three-dimensional reconstruction of cardiac cavity structure using intracardiac ultrasound images provided in this application acquires cavity images at different time phases during the cardiac cycle, and then performs edge enhancement on each cavity image to obtain edge enhancement maps at different time phases; determines segmentation masks for the cavity structure at different time phases, and then determines the contour gradients of the inner and outer membrane boundaries of the cavity between adjacent time phases based on the contour difference features between the segmentation masks of all cavity structures and the edge enhancement maps at adjacent time phases; performs deformable convolution cavity contour detection on each edge enhancement map to obtain a contour mask map of the cavity structure at each time phase, and then determines the spatial distribution constraints of the cavity thickness within each contour mask map; fuses and reconstructs the cavity images at each time phase using the spatial distribution constraints of all cavity thicknesses to obtain a reconstructed feature map of the cardiac cavity structure, and then performs image segmentation on the reconstructed feature map based on all contour gradients to obtain a segmentation feature map of the cardiac cavity structure.

[0043] Therefore, in this application, the reconstructed feature map can be segmented based on all contour gradients to obtain a segmentation feature map of the cardiac cavity structure. Firstly, by independently segmenting the cavity image for each cardiac phase, the actual morphology and positional changes of the intramural and extramural boundaries of the cardiac cavity in each phase can be accurately reflected. This further determines the segmentation mask for the cavity structure in different phases. The segmentation mask effectively captures and adjusts boundary drift caused by dynamic motion, avoiding segmentation errors due to boundary mismatch, thus ensuring the spatiotemporal consistency of the cardiac cavity structure throughout the cardiac cycle. Secondly, through quantization... The contour gradients of the intramural and extramural boundaries of the cardiac cavity between adjacent time phases can capture boundary drift and morphological distortion of the cardiac cavity membrane layer between adjacent time phases. These contour gradients provide dynamically adjustable gradient information for the segmentation algorithm, enabling it to adaptively adjust the boundary segmentation strategy, avoid error accumulation due to boundary drift, and enhance the robustness of the segmentation algorithm to non-rigid deformations. Furthermore, deformable convolutional cavity contour detection on each edge enhancement image can adaptively capture the complex morphological changes and local distortions of the cardiac cavity membrane layer boundary under different cardiac time phases. Then, the empty space of the cavity thickness within each contour mask image is determined. Spatial distribution constraints can effectively control the thickness variation of the cardiac cavity membrane, thereby avoiding unreasonable deformation caused by boundary drift due to nonlinear motion. By introducing spatial distribution constraints on cavity thickness, the segmentation process is not limited to identifying contour boundaries, but also ensures the continuity and consistency of the internal cavity structure, making the thickness variation of the cardiac cavity membrane conform to physiological reality. Furthermore, by fusing the spatial distribution constraints of cavity thickness across all time phases and reconstructing the cavity image for each time phase, the thickness and morphological information of the dynamic cardiac cavity membrane can be effectively integrated, eliminating inconsistencies caused by boundary drift and local deformation due to nonlinear motion. Finally, image segmentation of the reconstructed feature map based on all contour gradients can fully utilize the subtle information of boundary changes between different time phases, effectively guiding the segmentation process to accurately locate the endocardium and epicardium edges. Contour gradients, as a key driving factor for dynamic adjustment, help the segmentation model adapt to contour drift and morphological distortion of the cardiac membrane caused by nonlinear motion, ensuring the continuity and consistency of the segmentation boundary in time and space. In summary, the solution of this application can achieve accurate segmentation of the cardiac cavity structure even when the cardiac membrane structure experiences contour drift due to nonlinear motion. Attached Figure Description

[0044] Figure 1 This is an exemplary flowchart of a cardiac chamber image segmentation method according to some embodiments of this application;

[0045] Figure 2 This is a schematic flowchart illustrating the process of determining the contour gradient according to some embodiments of this application;

[0046] Figure 3This is a schematic flowchart illustrating the process of determining a contour mask according to some embodiments of this application;

[0047] Figure 4 This is a schematic diagram of the structure of a cardiac chamber image segmentation unit according to some embodiments of this application;

[0048] Figure 5 This is a schematic diagram of the structure of a computer device for implementing a cardiac cavity image segmentation method according to some embodiments of this application. Detailed Implementation

[0049] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] refer to Figure 1 The figure is an exemplary flowchart of a cardiac cavity image segmentation method according to some embodiments of this application. The cardiac cavity image segmentation method 100 mainly includes the following steps:

[0051] In step 101, cavity images of the heart chambers are acquired at different time phases during the cardiac cycle, and edge enhancement is performed on each cavity image to obtain edge enhancement maps at different time phases.

[0052] It should be noted that, in this application, the cardiac cycle represents the entire process of the heart completing one contraction and relaxation, specifically including the atrial systole, ventricular systole, and cardiac diastole, reflecting the complete mechanical and hemodynamic process of the heart pumping blood. The phase represents a specific point in time within the cardiac cycle, representing different physiological states of the heart within a cardiac cycle. By dividing the complete cardiac cycle into several discrete phases, the dynamic changes in cardiac structure and function can be observed and analyzed in stages.

[0053] In practice, firstly, a high-frequency ultrasound probe is used to continuously image the heart chambers of the subject, acquiring ultrasound images of the heart chambers at different phases within the cardiac cycle, and all the obtained ultrasound images are used as cavity images. Then, the cavity images at different phases are filtered (e.g., Gaussian filtering). Further, an edge detection operator (e.g., Canny operator) is used to perform edge detection on the filtered cavity images. Then, an adaptive histogram equalization technique is used to locally enhance the contrast of the edge regions of the edge-detected cavity images, and the locally enhanced cavity images are used as edge enhancement maps, thus obtaining edge enhancement maps at different phases. The high-frequency ultrasound probe is a medical imaging device that can provide high spatial resolution imaging, usually operating above 10MHz. It achieves clear imaging of the heart cavity structure by emitting high-frequency ultrasound waves and receiving the echo signals reflected at the tissue interface.

[0054] It should be noted that the edge enhancement map described in this application refers to an image after the edge information in a cardiac ultrasound image has been enhanced.

[0055] In step 102, segmentation masks for cavity structures at different time phases are determined, and then the contour gradients of the inner and outer membrane boundaries of the cavity between adjacent time phases are determined based on the contour difference features between the segmentation masks of all cavity structures and the edge enhancement maps of adjacent time phases. The contour gradients represent parameters that guide the identification of the contours of cardiac cavity structures during image segmentation.

[0056] In some embodiments, determining the segmentation mask for the cavity structure at different time phases can be achieved using the following steps:

[0057] Adaptive threshold segmentation is performed on the edge enhancement map for each time phase to obtain the binary map of the cavity region for each time phase;

[0058] The cavity mask for each time phase is obtained by filling the void regions in the binary image of the cavity region under each time phase through morphological closing operations;

[0059] The boundary of the cavity mask in each time phase is corrected at the sub-pixel level based on the gradient information of the edge enhancement map in each time phase, so as to obtain the segmentation mask of the cavity structure in each time phase.

[0060] It should be noted that the cavity region binary image described in this application represents a binary image obtained through preliminary threshold segmentation, used to identify the approximate region of the cardiac cavity structure; the cavity mask represents a binary image marking the spatial location of the cardiac cavity structure.

[0061] In specific implementation, adaptive thresholding segmentation is performed on the edge enhancement map at each time phase to obtain the cavity region binary map at each time phase. This can be achieved in the following way: an adaptive thresholding segmentation algorithm (such as the Otsu thresholding method) is used to perform thresholding segmentation on the edge enhancement map at each time phase, separating the cavity region from the background in the edge enhancement map, and the image obtained after thresholding segmentation is used as the cavity region binary map at the corresponding time phase, thus obtaining the cavity region binary map at each time phase; the cavity mask at each time phase is obtained by filling the hole regions in the cavity region binary map at each time phase through morphological closing operations, which can be achieved in the following way: morphological closing operations (i.e., dilation followed by erosion) are used to fill the hole regions in the cavity region binary map at each time phase, and the image obtained after filling is used as the cavity mask at the corresponding time phase, thus obtaining the cavity mask at each time phase. Here, the hole regions in the cavity region binary map refer to the regions in the cavity region binary map that have small holes or breaks.

[0062] It should be noted that the gradient information of the edge enhancement map described in this application represents the gradient direction and gradient magnitude of the pixel value change at each pixel point in the edge enhancement map, and the segmentation mask represents an image that accurately marks the boundary region of the cardiac cavity structure contour.

[0063] In specific implementation, the boundary of the cavity mask in each time phase is corrected at the sub-pixel level according to the gradient information of the edge enhancement map in each time phase. The segmentation mask of the cavity structure in each time phase can be implemented in the following way: For the cavity mask in each time phase, all boundary pixels of the cavity mask are obtained by an edge detection algorithm (such as the Canny edge detection algorithm). At the same time, the gradient magnitude of the boundary pixels and their neighboring pixels in the edge enhancement map in the corresponding time phase is obtained. Then, the gradient magnitude is subjected to a second interpolation fitting along the gradient direction of the boundary pixels. By solving the maximum point of the second interpolation fitting curve, the pixel offset at the peak position of the gradient magnitude of all boundary pixels is obtained. Then, the obtained sub-pixel offset is added to the integer pixel coordinates of the boundary pixels, and the added value is used as the sub-pixel coordinates of the boundary pixels. Then, the region formed by the sub-pixel coordinates of all boundary pixels is used as the segmentation mask of the cavity structure, thereby obtaining the segmentation mask of the cavity structure in each time phase.

[0064] In some embodiments, reference Figure 2 As shown in the figure, this is a flowchart illustrating the process of determining the contour gradient in some embodiments of this application. In this embodiment, the contour gradient of the inner membrane boundary and outer membrane boundary of the cavity between adjacent time phases can be determined based on the contour difference characteristics between the segmentation mask of all cavity structures and the lower edge enhancement map of adjacent time phases using the following steps:

[0065] The overlapping area between the segmentation masks of the cavity structure in adjacent time phases is determined, and then the displacement change characteristics of the inner membrane boundary of the cavity between adjacent time phases are extracted;

[0066] The deformation amplitude of the extracorporeal membrane boundary between adjacent time phases is determined by the contour difference features between the lower edge enhancement maps of adjacent time phases;

[0067] The contour gradients of the inner and outer membrane boundaries of the cavity between adjacent time phases are determined based on the displacement variation characteristics of the inner membrane boundary of the cavity between adjacent time phases and the deformation amplitude of the outer membrane boundary of the cavity between adjacent time phases.

[0068] It should be noted that the displacement change characteristics described in this application refer to the spatial position change characteristics of the inner membrane boundary of the cavity between adjacent time phases.

[0069] In specific implementation, determining the overlapping region between the segmentation masks of the cavity structure in adjacent time phases, and then extracting the displacement change features of the cavity inner membrane boundary between adjacent time phases, can be achieved in the following way: First, based on the shape matching method between images (e.g., shape overlap ratio), compare the segmentation masks of the cavity structure in adjacent time phases to extract the overlapping region between the segmentation masks of the cavity structure in adjacent time phases. Then, for each boundary pixel in the overlapping region, calculate the Euclidean distance between the boundary pixel and the segmentation masks of the cavity structure in adjacent time phases, and use the Euclidean distance between all boundary pixels and the segmentation masks of the cavity structure in adjacent time phases as the displacement change features of the cavity inner membrane boundary between adjacent time phases.

[0070] It should be noted that, in this application, the contour difference feature refers to the feature of the difference in contour gradient amplitude between adjacent time phases of the edge enhancement map, and the deformation amplitude refers to the intensity of deformation of the outer membrane boundary of the cavity between adjacent time phases.

[0071] In specific implementation, the deformation amplitude of the extracorporeal membrane boundary between adjacent time phases can be determined by the contour difference features between edge enhancement maps of adjacent time phases in the following way: For edge enhancement maps of adjacent time phases, the gradient magnitude of each pixel in the edge enhancement map is calculated using a gradient operator (such as the Sobel operator), and all gradient magnitudes are combined into a matrix according to the spatial position of the pixel in the edge enhancement map. The resulting matrix is ​​used as the gradient field of the edge enhancement map. Then, the absolute difference of the gradient magnitudes at corresponding positions in the gradient field of the edge enhancement maps of adjacent time phases is calculated, and the matrix composed of all the values ​​obtained after the absolute difference is used as the contour difference feature between the edge enhancement maps of adjacent time phases. Then, multiple feature values ​​are extracted from the contour difference feature using the NumPy library in Python, and the sum of all feature values ​​is used as the deformation amplitude of the extracorporeal membrane boundary between adjacent time phases.

[0072] In practice, the contour gradients of the inner and outer membrane boundaries of the cavity between adjacent time phases can be determined by the following method based on the displacement change characteristics of the inner membrane boundary of the cavity between adjacent time phases and the deformation amplitude of the outer membrane boundary of the cavity between adjacent time phases: the displacement change characteristics of the inner membrane boundary of the cavity between adjacent time phases are divided by the deformation amplitude of the outer membrane boundary of the cavity between adjacent time phases, and the value obtained by division is used as the contour gradient of the inner and outer membrane boundaries of the cavity between adjacent time phases.

[0073] It should be noted that the contour gradient mentioned in this application represents the parameters that guide the recognition of the cardiac cavity structure contour during image segmentation, which will not be elaborated here.

[0074] In step 103, deformable convolution cavity contour detection is performed on each edge enhancement map to obtain the contour mask map of the cavity structure in each phase, and then the spatial distribution constraint of the cavity thickness in each contour mask map is determined.

[0075] In some embodiments, reference Figure 3 As shown in the figure, this is a schematic flowchart of the process for determining the contour mask map in some embodiments of this application. In this embodiment, the cavity contour detection by deformable convolution of each edge enhancement map can be achieved by the following steps to obtain the contour mask map of the cavity structure in each phase:

[0076] First, in step 1031, a deformable convolutional network model is constructed;

[0077] Secondly, in step 1032, the contour probability distribution of the cavity structure in each edge enhancement image is extracted through the deformable convolutional network model;

[0078] Then, in step 1033, non-maximum suppression is performed on the contour probability distribution of the cavity structure in each edge enhancement image to obtain multiple confidence edge points of the cavity structure in each edge enhancement image;

[0079] Finally, in step 1034, a contour mask map of the cavity structure corresponding to each edge enhancement map is constructed based on multiple confidence edge points of the cavity structure within each edge enhancement map.

[0080] It should be noted that the deformable convolutional network described in this application is an improved convolutional neural network structure. By introducing learnable spatial offsets into traditional convolution operations, the convolutional kernels can dynamically adjust their sampling positions according to the local geometric deformations of the input image, thereby capturing complex and irregular features more flexibly. This mechanism enables the network to significantly improve feature representation capabilities and recognition accuracy when processing image tasks with shape changes and rich details, and it is widely used in visual tasks such as object detection, semantic segmentation, and contour extraction. In the specific training process, a training dataset with heart cavity contour annotations is first prepared. The annotations are usually based on the heart cavity contour. The cavity contour mask is used, and the input image is then fed into a network model containing deformable convolutional layers. This network enhances its ability to capture deformation and details by dynamically learning the sampling offset of the convolutional kernels. Next, a loss function suitable for contour segmentation is used to compare the contour predictions output by the network with the ground truth annotations, and the error is calculated. Through the backpropagation algorithm, the weights and offset parameters of the deformable convolutional layers and other layers in the network are adjusted to gradually improve the model's ability to recognize and segment the features of the heart cavity contour. After training iterations until the loss converges, the model can effectively segment the accurate heart cavity contour in new images, achieving the goal of high-precision contour segmentation.

[0081] It should be noted that the contour probability distribution described in this application represents the spatial distribution of the probability values ​​of each pixel in the edge enhancement image being predicted as belonging to the contour of the cardiac cavity structure; the confidence edge point represents the pixel with the highest edge response intensity in the local neighborhood of the contour probability distribution.

[0082] In specific implementation, the contour probability distribution of the cavity structure within each edge enhancement image can be extracted using the deformable convolutional network model in the following manner: First, each edge enhancement image is input into the deformable convolutional network model, and forward computation is performed through the deformable convolutional network model. The result obtained after the forward computation is used as the contour probability distribution of the corresponding edge enhancement image, thus obtaining the contour probability distribution of the cavity structure within each edge enhancement image. Second, non-maximum suppression is applied to the contour probability distribution of the cavity structure within each edge enhancement image to obtain multiple confidence edge points of the cavity structure within each edge enhancement image. This can be achieved in the following manner: Non-maximum suppression is applied to the contour probability distribution of the cavity structure within each edge enhancement image, and the pixels obtained after non-maximum suppression are used as the confidence edge points of the cavity structure within the corresponding edge enhancement image, thus obtaining multiple confidence edge points of the cavity structure within each edge enhancement image.

[0083] It should be noted that the contour mask image described in this application represents a binary image describing the contour lines of the cardiac cavity structure. The contour mask image typically uses white pixels (pixel value 255) to represent the contour lines of the cardiac cavity structure and black pixels (pixel value 0) to represent the background area.

[0084] In specific implementation, the contour mask map of the cavity structure corresponding to each edge enhancement map can be constructed based on multiple confidence edge points of the cavity structure in each edge enhancement map in the following way: For each edge enhancement map, a polygon fitting algorithm (such as B-spline curve fitting algorithm) is used to perform curve fitting on all confidence edge points of the cavity structure in the edge enhancement map to obtain a contour fitting line. The area enclosed by the contour fitting line is filled as the foreground by a scan line filling algorithm, and the image obtained after filling is used as the contour mask map of the cavity structure corresponding to the edge enhancement map in the corresponding phase.

[0085] In some embodiments, determining the spatial distribution constraints of the cavity thickness within each contour mask image can be achieved using the following steps:

[0086] For each contour mask;

[0087] The radial distance between the inner membrane boundary and the outer membrane boundary of the cavity is sampled along the contour normal direction of the contour mask image, thereby generating the thickness distribution sequence of the cavity structure;

[0088] The thickness constraint weights of the cavity structure are determined based on the thickness distribution sequence.

[0089] The spatial distribution constraint of the cavity thickness in the contour mask is determined by the thickness constraint weight and the spatial gradient characteristics of the cavity thickness in the contour mask.

[0090] It should be noted that the thickness distribution sequence described in this application represents an ordered sequence of local thicknesses of the cardiac cavity structure obtained by sequentially sampling along the contour normal direction of the contour mask image.

[0091] In a specific implementation, the radial distance between the inner membrane boundary and the outer membrane boundary of the cavity is sampled along the contour normal direction of the contour mask image, and then the thickness distribution sequence of the cavity structure is generated. This can be achieved in the following way: the radial distance between the inner membrane boundary and the outer membrane boundary of the cavity is sampled along the contour normal direction of the contour mask image using a contour normal sampling algorithm, and the sequence composed of all the obtained radial distances is used as the thickness distribution sequence of the cavity structure.

[0092] It should be noted that the thickness constraint weight described in this application represents the weight parameter for constraining the thickness fluctuation of the cavity structure between adjacent time phases.

[0093] In specific implementation, the thickness constraint weight of the cavity structure can be determined according to the thickness distribution sequence in the following way: perform a difference operation on the thickness distribution sequence, calculate the mean and variance of the values ​​obtained after the difference operation, divide the variance by the mean, and use the value obtained by the division as the thickness constraint weight of the cavity structure.

[0094] It should be noted that the spatial gradient feature described in this application represents the degree of fluctuation in the thickness of the cavity within the contour mask image in space, and the spatial distribution constraint represents the conditional parameter that constrains the change of cavity thickness in the image space during cardiac cavity image segmentation. The spatial distribution can identify the trend of change of the cavity tissue boundary in the image space.

[0095] In specific implementation, the spatial distribution constraint of the thickness of the cavity in the contour mask can be determined by the thickness constraint weight and the spatial gradient feature of the cavity thickness in the contour mask in the following way: First, the gradient magnitude of each pixel in the contour mask is calculated using a gradient operator (e.g., Sobel operator). Then, the gradient magnitudes of all gradient values ​​are differentially calculated. Then, the values ​​obtained from all differential calculations are summed, and the summed value is used as the spatial gradient feature of the cavity thickness in the contour mask. Finally, the product between the thickness constraint weight and the spatial gradient feature of the cavity thickness in the contour mask is used as the spatial distribution constraint of the cavity thickness in the contour mask.

[0096] In step 104, the cavity images at each time phase are fused and reconstructed by the spatial distribution constraints of all cavity thicknesses to obtain a reconstructed feature map of the cardiac cavity structure. Then, the reconstructed feature map is segmented according to all contour gradients to obtain a segmented feature map of the cardiac cavity structure.

[0097] In some embodiments, fusing and reconstructing cavity images at each time phase based on spatial distribution constraints of all cavity thicknesses to obtain a reconstructed feature map of the cardiac cavity structure can be achieved using the following steps:

[0098] Based on the spatial distribution constraints of all cavity thicknesses, the cavity structure in each time phase image is spatially relocated to obtain the relocation vector of the cavity structure in each cavity image.

[0099] A reconstructed feature map of the cardiac cavity structure is constructed based on the relocation vector of the cavity structure within each cavity image and the feature response intensity of the edge of the cavity structure within each cavity image.

[0100] It should be noted that the relocation vector described in this application represents the vector representation of the spatial position offset of the cavity structure within the cavity image relative to the reference time phase.

[0101] In specific implementation, the cavity structure within each time phase of the cavity image is spatially relocated based on the spatial distribution constraints of all cavity thicknesses. The relocation vector of the cavity structure within each cavity image can be obtained in the following way: First, select the time phase corresponding to the smallest spatial distribution constraint from all cavity thicknesses and use this time phase as the reference time phase. Then, select the cavity image under the reference time phase as the reference template. Construct a regularization term for the deformation field based on the spatial distribution constraints of all cavity thicknesses. Then, use a multi-scale pyramid strategy to perform image registration layer by layer, gradually adjusting the position of control points from coarse to fine to adapt to local nonlinear deformation. During the registration process, a similarity metric function (e.g., mutual information) is used to evaluate the deformation effect. Then, the control point coordinates are iteratively optimized through an optimization algorithm (e.g., gradient descent algorithm). Finally, after registration is completed, extract the spatial positioning vector of the cavity structure in each time phase relative to the reference time phase through the displacement vector of the control points in the deformation field, and use all the obtained spatial positioning vectors as the relocation vectors of the cavity structure within the cavity image in the corresponding time phase.

[0102] It should be noted that the feature response intensity described in this application reflects the variation range of the edge intensity of the cavity structure within the cavity image, and the reconstructed feature map represents the reconstructed image of the overall morphology of the heart cavity after spatial alignment of heart cavity images from multiple time phases.

[0103] In specific implementation, the reconstruction feature map of the heart cavity structure can be constructed based on the relocation vector of the cavity structure in each cavity image and the feature response intensity of the cavity structure edge in each cavity image in the following manner: First, the gradient magnitude of each pixel in each cavity image is calculated using a gradient operator (such as the Sobel operator), and the variance of all gradient magnitudes is used as the feature response intensity of the cavity structure edge in the corresponding cavity image; then, based on the relocation vector of the cavity structure in each cavity image, an affine transformation algorithm is used to align the cavity structure of each cavity image to the reference template; next, in all cavity images registered to the coordinate system of the reference template, for each pixel position, pixel values ​​from different time phases are aggregated, and the feature response intensity at the corresponding position is used as a weighting coefficient, and a weighted average is used for fusion, and the fused image is used as the reconstruction feature map of the heart cavity structure.

[0104] In some embodiments, image segmentation of the reconstructed feature map based on all contour gradients to obtain a segmentation feature map of the cardiac cavity structure can be achieved by the following steps:

[0105] The pixel segmentation regions of the internal cavity structure in the reconstructed feature map are determined based on all contour gradients;

[0106] The segmentation feature map of the cardiac cavity structure is generated by segmenting the pixel regions.

[0107] It should be noted that the segmentation feature map described in this application represents an image of the morphological features of the cardiac cavity structure.

[0108] In specific implementation, determining the pixel segmentation region of the inner cavity structure of the reconstructed feature map based on all contour gradients can be achieved in the following way: First, the reconstructed feature map is regarded as a weighted graph structure, and each pixel corresponds to a node in the weighted graph structure. The contour gradients of all nodes are set as the weights of the edges between nodes. Then, using the contour gradients as edge weight parameters in the energy function, the graph cut algorithm achieves the globally optimal segmentation of the reconstructed feature map by minimizing this energy function, accurately distinguishing the pixels of the inner cavity structure into the foreground region, and using the obtained foreground region as the pixel segmentation region of the inner cavity structure in the reconstructed feature map. Generating the segmentation feature map of the inner cavity structure through the pixel segmentation region can be achieved in the following way: Morphological operations (such as dilation, erosion, etc.) are used to process the pixel segmentation region to eliminate noise and fill holes, improve the integrity and continuity of the pixel segmentation region, and the processed image is used as the segmentation feature map of the inner cavity structure.

[0109] Furthermore, in another aspect of this application, in some embodiments, this application provides a three-dimensional reconstruction system for cardiac cavity structure from intracardiac ultrasound images. This system includes a cardiac cavity image segmentation unit, with reference to... Figure 4The figure is a schematic diagram of the structure of a cardiac cavity image segmentation unit 400 according to some embodiments of this application. The cardiac cavity image segmentation unit 400 includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described below:

[0110] The acquisition module 401 in this application is mainly used to acquire cavity images of the heart chambers at different time phases during the cardiac cycle, and then perform edge enhancement on each cavity image to obtain edge enhancement maps at different time phases.

[0111] Processing module 402, in this application, is used to determine the segmentation mask of the cavity structure under different time phases, and then determine the contour gradient of the inner membrane boundary and outer membrane boundary of the cavity between adjacent time phases based on the contour difference characteristics between the segmentation mask of all cavity structures and the edge enhancement map under adjacent time phases.

[0112] It should be noted that the processing module 402 described in this application is also used to perform deformable convolution cavity contour detection on each edge enhancement map to obtain a contour mask map of the cavity structure in each phase, and then determine the spatial distribution constraint of the cavity thickness in each contour mask map.

[0113] The execution module 403 in this application is mainly used to fuse and reconstruct the cavity images in each time phase by constraining the spatial distribution of all cavity thicknesses to obtain the reconstructed feature map of the heart cavity structure, and then perform image segmentation on the reconstructed feature map according to all contour gradients to obtain the segmentation feature map of the heart cavity structure.

[0114] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described cardiac cavity image segmentation method.

[0115] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device implementing a cardiac cavity image segmentation method according to some embodiments of this application. The cardiac cavity image segmentation method in the above embodiments can be implemented by... Figure 5 The computer device shown is used to implement this, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0116] The processor 501 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more devices used to control the execution of the cardiac cavity image segmentation method in this application.

[0117] The communication bus 502 can be used to transmit information between the aforementioned components.

[0118] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 503 may exist independently and be connected to processor 501 via communication bus 502. Memory 503 may also be integrated with processor 501.

[0119] The memory 503 stores program code for executing the scheme of this application, and its execution is controlled by the processor 501. The processor 501 executes the program code stored in the memory 503. The program code may include one or more software modules. The method described in the above method embodiments can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0120] Communication interface 504 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0121] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0122] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0123] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described cardiac cavity image segmentation method.

[0124] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0125] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for segmenting cardiac chamber images, applied to a three-dimensional reconstruction system of cardiac chamber structure from intracardiac ultrasound images, characterized in that, The method includes the following steps: The cavity images of the heart chambers are acquired at different time phases during the cardiac cycle, and then edge enhancement is performed on each cavity image to obtain edge enhancement maps at different time phases. The segmentation mask of the cavity structure under different time phases is determined, and then the contour gradient of the cavity inner membrane boundary and the cavity outer membrane boundary between adjacent time phases is determined based on the contour difference features between the segmentation mask of all cavity structures and the edge enhancement map under adjacent time phases. The contour gradient represents the parameter that guides the recognition of the heart cavity structure contour during image segmentation. For each edge enhancement map, deformable convolution is performed to detect the cavity contour, resulting in a contour mask map of the cavity structure in each time phase, thereby determining the spatial distribution constraints of the cavity thickness within each contour mask map; By fusing and reconstructing cavity images at each time phase using spatial distribution constraints of all cavity thicknesses, a reconstructed feature map of the cardiac cavity structure is obtained. Then, image segmentation is performed on the reconstructed feature map based on all contour gradients to obtain a segmentation feature map of the cardiac cavity structure.

2. The method as described in claim 1, characterized in that, The specific steps for determining the segmentation mask for the cavity structure at different time phases include: Adaptive threshold segmentation is performed on the edge enhancement map for each time phase to obtain the binary map of the cavity region for each time phase; The cavity mask for each time phase is obtained by filling the void regions in the binary image of the cavity region under each time phase through morphological closing operations; The boundary of the cavity mask in each time phase is corrected at the sub-pixel level based on the gradient information of the edge enhancement map in each time phase, so as to obtain the segmentation mask of the cavity structure in each time phase.

3. The method as described in claim 1, characterized in that, Based on the contour difference features between the segmentation mask of all cavity structures and the lower edge enhancement map of adjacent temporal phases, the contour gradients of the inner and outer membrane boundaries of the cavity between adjacent temporal phases are determined, specifically including: The overlapping area between the segmentation masks of the cavity structure in adjacent time phases is determined, and then the displacement change characteristics of the inner membrane boundary of the cavity between adjacent time phases are extracted; The deformation amplitude of the extracorporeal membrane boundary between adjacent time phases is determined by the contour difference features between the lower edge enhancement maps of adjacent time phases; The contour gradients of the inner and outer membrane boundaries of the cavity between adjacent time phases are determined based on the displacement variation characteristics of the inner membrane boundary of the cavity between adjacent time phases and the deformation amplitude of the outer membrane boundary of the cavity between adjacent time phases.

4. The method as described in claim 1, characterized in that, For each edge enhancement map, deformable convolution is performed to detect the cavity contour, resulting in a contour mask map of the cavity structure for each temporal phase, specifically including: Construct deformable convolutional network models; The contour probability distribution of the cavity structure in each edge enhancement image is extracted using the deformable convolutional network model. Non-maximum suppression is applied to the contour probability distribution of the cavity structure in each edge enhancement image to obtain multiple confidence edge points of the cavity structure in each edge enhancement image; Construct a contour mask map of the cavity structure corresponding to each edge enhancement map at a given time based on multiple confidence edge points of the cavity structure within each edge enhancement map.

5. The method as described in claim 1, characterized in that, Determining the spatial distribution constraints of the cavity thickness within each contour mask specifically includes: For each contour mask; The radial distance between the inner membrane boundary and the outer membrane boundary of the cavity is sampled along the contour normal direction of the contour mask image, thereby generating the thickness distribution sequence of the cavity structure; The thickness constraint weights of the cavity structure are determined based on the thickness distribution sequence. The spatial distribution constraint of the cavity thickness in the contour mask is determined by the thickness constraint weight and the spatial gradient characteristics of the cavity thickness in the contour mask.

6. The method as described in claim 1, characterized in that, By fusing and reconstructing the cavity images at each time phase using spatial distribution constraints on the thickness of all cavities, the reconstructed feature map of the cardiac cavity structure is obtained, specifically including: Based on the spatial distribution constraints of all cavity thicknesses, the cavity structure in each time phase image is spatially relocated to obtain the relocation vector of the cavity structure in each cavity image. A reconstructed feature map of the cardiac cavity structure is constructed based on the relocation vector of the cavity structure within each cavity image and the feature response intensity of the edge of the cavity structure within each cavity image.

7. The method as described in claim 1, characterized in that, Image segmentation is performed on the reconstructed feature map based on all contour gradients to obtain a segmentation feature map of the cardiac cavity structure, specifically including: The pixel segmentation regions of the internal cavity structure in the reconstructed feature map are determined based on all contour gradients; The segmentation feature map of the cardiac cavity structure is generated by segmenting the pixel regions.

8. The method as described in claim 1, characterized in that, The cardiac cycle specifically includes atrial systole, ventricular systole, and diastole.

9. The method as described in claim 1, characterized in that, Images of the cavity are acquired using a high-frequency ultrasonic probe.

10. A three-dimensional reconstruction system for cardiac cavity structure from intracardiac ultrasound images, the system comprising a cardiac cavity image segmentation unit, characterized in that, The cardiac chamber image segmentation unit includes: The acquisition module is used to acquire images of the heart chambers at different time phases during the cardiac cycle, and then perform edge enhancement on each chamber image to obtain edge enhancement maps at different time phases; The processing module is used to determine the segmentation mask of the cavity structure at different time phases, and then determine the contour gradient of the cavity inner membrane boundary and the cavity outer membrane boundary between adjacent time phases based on the contour difference features between the segmentation mask of all cavity structures and the edge enhancement map at adjacent time phases. The contour gradient represents the parameters that guide the recognition of the heart cavity structure contour during image segmentation. The processing module is also used to perform deformable convolution cavity contour detection on each edge enhancement map to obtain a contour mask map of the cavity structure in each phase, and then determine the spatial distribution constraint of the cavity thickness in each contour mask map. The execution module is used to fuse and reconstruct the cavity images at each time phase by constraining the spatial distribution of all cavity thicknesses to obtain a reconstructed feature map of the cardiac cavity structure, and then perform image segmentation on the reconstructed feature map according to all contour gradients to obtain a segmentation feature map of the cardiac cavity structure.

Images