A View Localization Method in 3D Cardiac Imaging Based on Relative Position Regression

Abstract

This invention belongs to the interdisciplinary field of medicine and computer science, and discloses a view localization method in three-dimensional cardiac images based on relative position regression. The method resamples, aligns, extracts the region of interest (ROI) of the heart, and standardizes the intensity of the input three-dimensional cardiac CT / MRI data, automatically generating two-dimensional slices with arbitrary orientation and position and their rigid pose parameters. A self-supervised pre-training encoder-decoder network is performed based on a random occlusion reconstruction task, outputting a low-dimensional representation vector for any slice. Within the same three-dimensional cardiac image, three-point labels with relative displacement are constructed for any slice pair, and a relative position regression mapping network is trained to learn relative geometric relationships in a unified latent coordinate space. The method receives reference view prompts, performs multi-step navigation and multi-initial position integration in the target three-dimensional cardiac image, outputs the planar parameters, DICOM coordinates, and quality score of the target view, and supports integration with clinical workstations or robot navigation systems.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of medicine and computer science, and specifically relates to a view localization method in three-dimensional cardiac images based on relative position regression. Background Technology

[0002] As the resolution and coverage of 3D imaging data continue to improve, how to quickly and stably locate a target 2D view within massive 3D cardiac body data, and maintain the consistency and comparability of views across different individuals and modalities, has become a key issue in computer-assisted surgery and intelligent navigation systems. Achieving high-quality view localization requires simultaneously considering the uncertainties brought about by individual anatomical differences, cardiac motion and rhythm irregularities, as well as the strong modal and noise variations caused by cross-device and cross-protocol issues.

[0003] Currently, existing technologies for automatic localization of standard views or specific sections in three-dimensional cardiac images mainly include the following categories of methods:

[0004] View planning methods based on template registration and rule definition: These methods typically first align the cardiac body data to be analyzed to a pre-constructed cardiac template or standard anatomical space using rigid or non-rigid registration. Then, standard plane parameters are calculated in the template space according to predefined rules. Finally, the data is transformed back to the patient space to obtain the target view. Its advantages include clear logic, strong interpretability, and ease of integration with traditional workstations. However, in the presence of significant cardiac anomalies, enlarged cardiac chambers, postoperative implanted devices, or strong noise artifacts, global registration is often unstable, and the computational load for registration optimization is large. This makes it difficult to meet timeliness requirements in scenarios requiring multiple interactions or real-time navigation, and it also struggles to support flexible positioning of arbitrary non-predefined views.

[0005] View localization methods based on anatomical structure segmentation and landmark inference: These methods rely on the automatic segmentation of multiple structures such as cardiac chambers, myocardium, and valves, or on the automatic detection of several key anatomical landmarks (such as the apex, mitral annulus, and aortic valve orifice), and then derive standard section equations based on empirical formulas or geometric configurations. Their advantage lies in their close integration with clinical knowledge; once the segmentation or landmarks are reliable, the view geometry derivation is relatively stable. However, the robustness of segmentation and landmark detection significantly decreases in the presence of contrast agent inhomogeneity, excessively fast heart rate, metal artifacts, or local motion abnormalities. Furthermore, these methods typically only target a few fixed standard views, making it difficult to achieve direct mapping from any query view to the target body in the entire volume data, which is detrimental to complex surgical planning and individualized view design.

[0006] Direct view regression and navigation methods based on end-to-end learning: Some methods that have emerged in recent years attempt to use convolutional neural networks or deep reinforcement learning to directly regress the center point, normal vector, and rotation angle of the target plane from 3D volume data or its projected view, or to train an agent to move the observation plane in volume data until reaching the standard view. These methods can achieve good results on specific datasets, but they generally suffer from the following problems: First, most still rely on explicit or implicit fixed physical coordinate spaces, binding view parameters to a certain global coordinate system, making it difficult to maintain stable relative geometric relationships between different individuals, different devices, and multimodal data; Second, reinforcement learning-based navigation often requires a large amount of interactive environment sampling and policy evaluation, resulting in high training costs, sensitivity to reward design, and a tendency to convergence instability or get stuck in local optima in complex cardiac anatomy and noisy environments; Third, existing networks mostly focus on single predictions or regression based on absolute coordinates, rarely explicitly modeling the relative positional relationships between different slices within the same cardiac volume data, making it difficult to build a universal cross-individual similar view navigation mechanism between arbitrary source views and target volumes.

[0007] While the aforementioned methods have promoted the development of automatic planning and localization of 3D cardiac images to some extent, they still have significant shortcomings: First, existing methods mostly rely on global registration or fixed template coordinates, which have limited adaptability to cardiac anatomical variations, postoperative structural reconstruction, and cross-center and cross-modal differences, making it difficult to construct a unified and stable view space without the need for physical templates. Second, most methods are geared towards a limited number of predefined standard views and lack a universal mapping mechanism from any queried view to the most similar view in the target 3D cardiac body data, failing to support rich individualized and task-oriented view requirements. Third, existing end-to-end networks typically only learn a unidirectional mapping from image to absolute pose, lacking systematic modeling of the relative positions between any two slices within the same body data, making it difficult to reuse prior internal geometric relationships during cross-individual migration. Fourth, the multi-step navigation and iterative localization process lacks an effective robustness enhancement mechanism, is sensitive to random initialization, local noise, and local optima, and the stability and confidence of the localization results are difficult to self-consistently evaluate.

[0008] Therefore, existing technologies cannot effectively solve the core challenge of rapid, accurate, and robust localization of arbitrary views in 3D cardiac images under multimodal and multi-body conditions. There is an urgent need for a new method that can fully explore the structural and spatial layout priors of arbitrary cardiac slices under self-supervised pre-training, explicitly model the 3D geometric relationships between arbitrary slice pairs within the same body data through relative position regression, and build an online view localization framework that can navigate in multiple steps, integrate multiple initial positions, and evaluate the confidence of results. This would enable accurate and automatic localization of arbitrary views across individuals and modalities without the need for explicit template registration. Summary of the Invention

[0009] To overcome the shortcomings of existing technologies in providing fast, accurate, and robust localization of arbitrary views in 3D cardiac images, especially the limitations of relying on fixed physical template coordinates, explicit registration, and a small number of predefined standard views, which make it difficult to adapt to cross-individual and cross-modal anatomical differences and strong noise disturbances, this invention provides a view localization method in 3D cardiac images based on relative position regression. This method comprehensively utilizes volumetric data from 3D cardiac CT (Computed Tomography) / MRI (Magnetic Resonance Imaging) to construct a unified technical system encompassing 3D cardiac image preprocessing and arbitrary slice construction, occlusion reconstruction pre-training and slice representation learning, mapping network training based on relative position regression, and online view localization and result robustness enhancement based on multi-step navigation. It learns prior knowledge of the anatomical structure and spatial layout of arbitrary slices through self-supervised learning, explicitly models the 3D relative positional relationship between any two slices within the same cardiac volumetric data, and performs multi-step navigation and multi-initial position integration in the target volumetric data using query views as prompts during the online phase. Without constructing a global physical template or performing explicit image registration, it achieves rapid and automatic localization of arbitrary views in multimodal, multi-volume 3D cardiac images, significantly improving the accuracy, robustness, and cross-scene transferability of view localization, providing a unified arbitrary view acquisition capability for intelligent diagnosis and navigation of cardiovascular diseases.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0011] A view localization method in three-dimensional cardiac images based on relative position regression includes the following steps:

[0012] Based on the input 3D heart body data, a set of 2D slices covering all directions and positions within the 3D heart bounding box is constructed. Each 2D slice contains a 2D slice image and rigid pose parameters.

[0013] Each two-dimensional slice image in the two-dimensional slice set is randomly occluded and then input into the encoder. The decoder reconstructs the two-dimensional slice image based on the encoder's output. The encoder and decoder are then trained and updated based on the reconstruction error to obtain a pre-trained encoder.

[0014] Based on the set of two-dimensional slices, two slices belonging to the same three-dimensional heart body data are taken as slice pairs. The relative displacement of the slice pairs is used as labels to generate training samples. Training samples with relative displacement within the coarse displacement range constitute the coarse displacement training set, and training samples with relative displacement within the fine displacement range constitute the fine displacement training set.

[0015] The relative position regression mapping network is trained and updated using coarse displacement training sets and fine displacement training sets respectively to obtain coarse localization model and fine localization model. The relative position regression mapping network includes a regression encoder and a regression mapping head. The regression encoder shares weights with the pre-trained encoder and freezes the weights during training and updating.

[0016] For the target 3D heart body data, a proxy slice is obtained by random initialization. Based on the proxy slice and the query slice image, a coarse localization model is used to obtain the coarse localization result. Based on the coarse localization result and the query slice image, a fine localization model is used to obtain the fine localization result. Based on the fine localization result, the target view slice image is obtained.

[0017] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0018] Preferably, the construction of a set of two-dimensional slices covering the three-dimensional heart bounding box space in all directions and at all locations includes:

[0019] The input 3D heart body data is preprocessed to obtain normalized volume data and 3D heart bounding box in a unified 3D physical coordinate system.

[0020] Sampling reference points are uniformly distributed within the three-dimensional heart bounding box; polar angles are uniformly sampled within the polar angle range; and azimuth angles are uniformly sampled within the azimuth angle range.

[0021] A rotation operator is constructed using any polar angle and any azimuth angle. The rotation operator is then multiplied by the initial three-dimensional vector to obtain the normal vector.

[0022] By combining all normal vectors with all reference points one by one, we obtain a set of normal vector and reference point combinations that cover the three-dimensional heart bounding box;

[0023] For each combination of normal vector and reference point, a set of orthogonal basis vectors is constructed in a plane orthogonal to the normal vector. A local two-dimensional coordinate system is established with the reference point as the origin and the orthogonal basis vectors as the basis. Trilinear interpolation sampling is performed on the local two-dimensional coordinate system at a preset resolution to obtain a two-dimensional slice image.

[0024] A random rotation angle is applied to the local two-dimensional coordinate system, and the random rotation angle, normal vector, and reference point are used as rigid pose parameters of the corresponding two-dimensional slice image.

[0025] Preferably, the preprocessing of the input three-dimensional heart body data to obtain normalized volume data and a three-dimensional heart bounding box in a unified three-dimensional physical coordinate system includes:

[0026] Obtain the voxel spacing and spatial orientation matrix from the input 3D heart body data;

[0027] Based on the voxel spacing, the anisotropic voxel spacing is mapped to the preset isotropic voxel spacing by the resampling operator to obtain standard spatial volume data;

[0028] Based on the human body position and spatial orientation matrix corresponding to the three-dimensional heart body data, the standard spatial volume data is aligned to a unified three-dimensional physical coordinate system.

[0029] Under a unified three-dimensional physical coordinate system, a three-dimensional heart bounding box is extracted based on standard spatial volume data;

[0030] The standard spatial volume data is spatially clipped according to the three-dimensional heart bounding box to obtain the volume data of the region of interest of the heart.

[0031] The coordinates of the volumetric data of the region of interest in the heart are scaled and the intensity of the volumetric data of the region of interest in the heart is normalized by grayscale, so as to obtain normalized volumetric data in a unified three-dimensional physical coordinate system.

[0032] Preferably, the step of randomly occluding each two-dimensional slice image in the two-dimensional slice set includes:

[0033] The three-dimensional heart bounding box is divided into several sub-regions, and the unit normal sphere is divided into several orientation intervals. In the combined space of each sub-region and orientation interval, two-dimensional slices are randomly extracted from the two-dimensional slice set. All extracted two-dimensional slices are aggregated to form a training slice set.

[0034] For the training slice set, each two-dimensional slice is divided into multiple non-overlapping rectangular patch units according to a preset patch size. An occlusion ratio is randomly generated from the occlusion ratio range. The number of occlusion units is generated according to the occlusion ratio and the number of patch units.

[0035] Without replacement, a number of patch cells are randomly selected from the set of patch cells as occlusion positions. An occlusion operator is used to occlude the occlusion positions. Patch cells outside the occlusion positions retain their original grayscale, generating a two-dimensional slice after random region occlusion.

[0036] Preferably, the step of taking two slices belonging to the same three-dimensional cardiac body data as a slice pair based on a two-dimensional slice set, and using the relative displacement of the slice pair as a label to generate training samples includes:

[0037] Multi-scale and multi-relative displacement slice sampling is performed within the same three-dimensional cardiac body data;

[0038] For any pair of slices, select three non-collinear standard reference points in each slice plane, unify the mapping values ​​of the standard reference points in the three-dimensional physical coordinate system, and obtain the three-dimensional marker points located on the slice plane.

[0039] Calculate the coordinate difference between corresponding 3D marker points in the two slices within a slice pair, and combine the coordinate differences of the three 3D marker points to form a label. Combine the slice pair and the label to form a training sample.

[0040] Preferably, the relative position regression mapping network processes the following steps:

[0041] For any slice within a slice pair in the training samples, pass it sequentially through a regression encoder, global average pooling, and a linear mapping layer to obtain the corresponding initial slice representation vector;

[0042] Construct dual input features based on the initial slice representation vector. , Indicates the first Within the three-dimensional cardiac body data, the first The initial slice representation vector of the slice. Indicates the first Within the three-dimensional cardiac body data, the first The initial slice representation vector of the slice. Represents the absolute value of each element. Represents element-wise product;

[0043] The dual input features are input into the regression mapping head, and then passed through a fully connected layer, a normalization layer, a nonlinear activation unit, and another fully connected layer in sequence to obtain the forward output.

[0044] The hyperbolic tangent function is used to perform element-wise bounded transformation on the forward output, and after weighting by hyperparameters within the displacement range, the relative displacement prediction values ​​of slice pairs in the training samples are obtained.

[0045] Preferably, the process of randomly initializing the target three-dimensional cardiac body data to obtain proxy slices includes:

[0046] Preprocessing the target 3D heart body data yields normalized volume data and the target 3D heart bounding box in a unified 3D physical coordinate system.

[0047] Random sampling is performed within the target's three-dimensional heart bounding box using a uniform distribution. Identify a proxy reference point and construct the rigid pose parameters of the proxy reference point;

[0048] Based on the rigid pose parameters of the surrogate reference point, a set of orthogonal basis vectors is constructed in a plane orthogonal to the normal vector. A local two-dimensional coordinate system is established with the surrogate reference point as the origin and the orthogonal basis vectors as the basis. Trilinear interpolation sampling is performed on the local two-dimensional coordinate system at a preset resolution to obtain the surrogate slice image.

[0049] Preferably, the step of obtaining coarse localization results based on proxy slices and query slice images using a coarse localization model includes:

[0050] Using a proxy reference point as a navigation trajectory, for the first... The navigation trajectory is in the first The proxy slice image and query slice image of the first step are used to construct the second step. The dual input features of the step;

[0051] The first The dual input features of the step are input into the coarse localization model, and the coarse relative displacement prediction value is obtained based on the coarse displacement range hyperparameter and the large-scale normalization factor of the normalization layer.

[0052] The position of the surrogate reference point is updated using the coarse relative displacement prediction value, and new rigid pose parameters are obtained;

[0053] Based on the updated rigid pose parameters of the proxy reference point, the first... The proxy slice image of the step;

[0054] The algorithm iteratively obtains coarse relative displacement predictions based on the surrogate slice image and the query slice image until a stopping condition is met or the maximum number of steps is reached. It then outputs the final surrogate slice image and rigid pose parameters as the first step. Coarse positioning results of the navigation trajectory.

[0055] Preferably, the step of obtaining fine localization results based on coarse localization results and query slice images using a fine localization model includes:

[0056] A local three-dimensional search window is constructed centered on the coarse positioning result of each navigation trajectory;

[0057] Regarding the first The navigation trajectory is in the first The proxy slice image and query slice image of the first step are used to construct the second step. The dual input features of the step;

[0058] The first The dual input features of the step are input into the fine localization model. Based on the fine displacement range hyperparameter and the small-scale normalization factor of the normalization layer, the fine relative displacement prediction value is obtained. The fine displacement range hyperparameter is smaller than the coarse displacement range hyperparameter, and the small-scale normalization factor is smaller than the large-scale normalization factor.

[0059] The position of the surrogate reference point is updated using the predicted values ​​of fine relative displacement, and new rigid pose parameters are obtained;

[0060] Based on the updated rigid pose parameters of the surrogate reference point, the first... (The sentence is incomplete and requires more context to be translated.) The proxy slice image of the step;

[0061] The algorithm iteratively obtains fine relative displacement predictions based on the surrogate slice image and the query slice image until a stopping condition is met or the maximum number of steps is reached. It then outputs the final surrogate slice image and rigid pose parameters as the first... Detailed positioning results of the navigation trajectory.

[0062] Preferably, obtaining the target view slice image based on the fine positioning result includes:

[0063] Calculate the similarity score between the proxy slice image and the query slice image for each navigation trajectory, and normalize or temperature-weight the similarity score to obtain the normalized weight.

[0064] Based on normalized weights, the rigid pose parameters of all navigation trajectories are weighted and fused to obtain the rigid pose parameters of the target view reference point.

[0065] The rigid pose parameters of the target view reference point are back-projected into the unified three-dimensional physical coordinate system and the medical digital imaging and communication standard coordinate system of the target three-dimensional heart body data, and the output is complete view geometric parameters including plane equation, position of target view reference point, slice orientation, pixel spacing and imaging matrix.

[0066] Extract target view slice images from the target 3D heart body data based on the complete view geometry parameters;

[0067] If the quality score of the target view slice image is lower than the quality threshold, the target view slice image is re-extracted; otherwise, the target view slice image and quality score are output.

[0068] The advantages of this invention compared to existing technologies are as follows: This invention, through a unified technical system of arbitrary slice construction, occlusion reconstruction self-supervised pre-training, relative position regression mapping, multi-step navigation, and result integration, eliminates the dependence on fixed template coordinates or global image registration. It explicitly learns the relative geometric relationships between arbitrary slice pairs within the same 3D cardiac image and transfers these relationships to cross-individual and cross-modal arbitrary view localization tasks. Occlusion reconstruction pre-training fully exploits structural priors in unlabeled 3D cardiac data, significantly reducing label dependence. The construction of a unified latent coordinate space through relative displacement regression of three-point markers and symmetry constraints improves the stability and interpretability of view localization under anatomical distortion and morphological variation conditions. Multi-initial position, multi-step coarse-grained navigation, and weighted result fusion effectively suppress the influence of random initialization, local optima, and noise artifacts, enhancing the robustness and confidence assessment capabilities of online localization results. Therefore, this invention can achieve rapid, accurate, and robust automatic localization of arbitrary views on multimodal data such as 3D cardiac CT / MRI, providing a unified basis for arbitrary view acquisition for intelligent diagnosis of cardiovascular diseases, preoperative planning, and robot navigation. Attached Figure Description

[0069] Figure 1 This is a block diagram of the view localization method in three-dimensional cardiac images based on relative position regression according to the present invention;

[0070] Figure 2 This is a flowchart of the view localization method in three-dimensional cardiac images based on relative position regression according to the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0073] Example 1: As Figure 1As shown, this embodiment provides a view localization method for 3D cardiac images based on relative position regression, which addresses the technical problem of existing view localization methods relying on fixed physical coordinate spaces or iterative registration, making it difficult to efficiently and accurately locate arbitrary two-dimensional slices in different individual and multimodal 3D cardiac images. The method can be divided into two parts: a training phase and an inference phase. In the training phase, arbitrary slices are first sampled using 3D cardiac CT / MRI volume data, and a pre-training task of occlusion reconstruction is constructed through random occlusion. With the help of an encoder-decoder network, slice representations that can characterize the prior anatomical structure and spatial layout of the heart are learned in a self-supervised manner. Subsequently, in the relative position regression fine-tuning phase, arbitrary slice pairs within the same 3D cardiac image are input into a shared encoder, and the 3D relative displacement between the two slices is regressed through a fully connected mapping network, thereby explicitly modeling the relative positional relationship of arbitrary slice pairs in a unified latent coordinate space. During the inference phase, the given query view serves as a prompt. A proxy position and initial slice are randomly initialized within the target 3D cardiac body data. A trained relative position regression mapping network is used to perform multi-step navigation for both coarse and fine localization. The slice positions are iteratively updated until convergence to the target view that best matches the anatomical structure of the query view, thus achieving automatic and accurate localization of any view within a 3D cardiac image. Compared to existing methods that rely on fixed template coordinates or single-step absolute pose regression, this invention significantly improves the accuracy, robustness, and transferability of view localization across multimodal and multi-volume data by combining relative position regression with coarse and fine-grained navigation.

[0074] like Figure 2 As shown, the view localization method in 3D cardiac images based on relative position regression in this embodiment includes a 3D cardiac image preprocessing and arbitrary slice construction stage, an occlusion reconstruction pre-training and slice representation learning stage, a mapping network training stage based on relative position regression, and an online view localization and result robustness enhancement stage based on multi-step navigation. Specifically, it includes the following steps:

[0075] Step 1: Based on the input 3D heart body data, construct a set of 2D slices covering all directions and positions within the 3D heart bounding box. Each 2D slice contains a 2D slice image and rigid pose parameters.

[0076] In the 3D cardiac image preprocessing and arbitrary slice construction stage: the acquired 3D cardiac CT or MRI volume data are resampled, registered, and intensity normalized to establish a unified 3D physical coordinate system and cardiac voxel mesh; under the unified 3D physical coordinate system, the rigid transformation of the slice plane is decomposed into initial 3D vectors. Rotate to target 3D vector The combination of spatial rotation and two-dimensional rotation within the slice is used to generate a normal vector uniformly distributed on the unit sphere through inverse transformation sampling. and in the three-dimensional heart surround box The translation amount is uniformly sampled to automatically construct a large-scale set of two-dimensional slices covering the entire three-dimensional cardiac imaging space with arbitrary orientation and position, and the rigid pose parameters of each slice in physical space are recorded. For the minimum and maximum values ​​of the 3D heart bounding box on the X-axis, For the minimum and maximum values ​​of the 3D heart bounding box on the Y-axis, These are the minimum and maximum values ​​of the three-dimensional heart bounding box on the Z-axis.

[0077] Step 1.1: Preprocess the input 3D heart body data to obtain normalized volume data and 3D heart bounding box in a unified 3D physical coordinate system.

[0078] Step 1.1.1: Obtain the voxel spacing and spatial orientation matrix from the input 3D cardiac body data. This is for the input raw 3D cardiac CT or MRI body data. Read the voxel spacing in DICOM (Digital Imaging and Communications in Medicine) standard. Spatial direction matrix .

[0079] Step 1.1.2: Based on the voxel spacing, the anisotropic voxel spacing is mapped to the preset isotropic voxel spacing by the resampling operator to obtain the standard space volume data.

[0080] Through resampling operator Anisotropic voxel spacing Mapped to a preset isotropic voxel spacing Standard space volume data is obtained:

[0081]

[0082] in, For standard space volume data, These are voxel points in standard spatial volume data. The resampling operator can be a nearest neighbor interpolation algorithm.

[0083] Step 1.1.3: Based on the human body position and spatial orientation matrix corresponding to the 3D heart body data, align the standard spatial volume data to a unified 3D physical coordinate system. This embodiment is based on patient position labels and spatial orientation matrices. Aligning standard spatial volume data to a unified three-dimensional physical coordinate system ensures that the three-dimensional cardiac images of different individuals have consistent spatial orientation, providing a unified reference system for the construction of subsequent arbitrary slices.

[0084] Step 1.1.4: Extract the three-dimensional heart bounding box based on the standard spatial volume data under a unified three-dimensional physical coordinate system; perform spatial clipping on the standard spatial volume data according to the three-dimensional heart bounding box to obtain the heart region of interest volume data; normalize the scale of the coordinates of the heart region of interest volume data, and normalize the grayscale of the heart region of interest volume data to obtain the normalized volume data under the unified three-dimensional physical coordinate system.

[0085] Under a unified three-dimensional physical coordinate system, standard spatial volume data Using coarse segmentation function Or based on intensity threshold Extraction function of Ω-shaped bounding box of the heart with connected component analysis The heart region is automatically located, resulting in a 3D bounding box for the heart.

[0086]

[0087] And standard space volume data According to the three-dimensional heart surround box Spatial clipping is performed to obtain volumetric data of the region of interest in the heart. Simultaneously, the cropped coordinates are scale-normalized to ensure that the spatial scale and center position of different individual hearts within the coordinate system remain within a preset range, thereby reducing the impact of inter-individual morphological differences on subsequent arbitrary slice sampling and relative position regression. The coarse segmentation function... For example, a coarse segmentation function based on an intensity threshold can be used to extract the function. For example, a coarse segmentation result can be obtained by using a coarse segmentation function based on an intensity threshold, and then connected component analysis can be performed to extract the three-dimensional heart bounding box.

[0088] For scale-normalized cardiac region of interest volume data Statistical analysis of the mean voxel intensity with standard deviation z-score normalization was used to normalize the grayscale, resulting in normalized volume data. :

[0089]

[0090] In other embodiments, extreme intensity values ​​can be truncated or linearly compressed by combining window adjustment and histogram stretching, so that cardiac images under different scanning protocols, different devices and different contrast agent conditions are unified to a comparable range in intensity distribution, thereby improving the stability of arbitrary slice representation and relative position learning on cross-modal and cross-center data.

[0091] Step 1.2: Uniformly distribute sampling reference points within the 3D heart bounding box, uniformly sample polar angles within the polar angle interval, and uniformly sample azimuth angles within the azimuth angle interval; construct a rotation operator using any polar angle and any azimuth angle, multiply the rotation operator by the initial 3D vector to obtain the normal vector; combine all normal vectors with all reference points one by one to obtain the set of normal vectors and reference points covering the 3D heart bounding box.

[0092] Within a unified three-dimensional physical coordinate system and a three-dimensional heart-bound box Ω, any slice plane is represented by its normal vector. With a reference point on the plane Perform parameterization; set an initial three-dimensional vector. By rotation operator The initial three-dimensional vector Rotate to the target normal vector:

[0093]

[0094] polar angle Azimuth Uniform or quasi-uniform random sampling is performed within the corresponding interval to ensure that the normal vector is approximately uniformly distributed on the unit sphere; reference point Samples are taken uniformly within the three-dimensional cardiac bounding box Ω, denoted as . , To define a uniform distribution on the three-dimensional bounding box of the heart, thereby forming a volume covering the heart. The combined set provides a parameter basis for the automatic construction of large-scale slices in arbitrary directions and positions.

[0095] Step 1.3: For each combination of normal vector and reference point, construct a set of orthogonal basis vectors in a plane orthogonal to the normal vector, establish a local two-dimensional coordinate system with the reference point as the origin and the orthogonal basis vectors as the basis, perform trilinear interpolation sampling on the local two-dimensional coordinate system at a preset resolution to obtain a two-dimensional slice image; apply a random rotation angle to the local two-dimensional coordinate system, and use the random rotation angle, normal vector and reference point as the rigid pose parameters of the corresponding two-dimensional slice image.

[0096] For each sampled normal vector and reference point combination In relation to the normal vector Construct a set of orthogonal basis vectors in an orthogonal plane. Establish with reference points Origin, orthonormal basis vectors Local two-dimensional coordinate system based on At a preset resolution, for the local two-dimensional coordinate system The grid is sampled using trilinear interpolation to obtain the corresponding two-dimensional slice image:

[0097]

[0098] At the same time, a random rotation angle is applied to the local coordinate system within the slice plane. , normal vector The location of the reference point (three-dimensional coordinates) and in-plane rotation angle The combined rigid pose parameters of this slice :

[0099]

[0100] and the corresponding two-dimensional slice image Each signal is bound to a storage container and used as a supervision signal during the training of the mapping network based on relative position regression, as well as as the basis for inverse calculation and verification of the physical position of the target view during online arbitrary view localization. Local two-dimensional coordinate system The following two-dimensional slice image, This is a trilinear interpolation sampling function.

[0101] Step 2: After randomly occluding each 2D slice image in the 2D slice set, input it into the encoder. The decoder reconstructs the 2D slice image based on the encoder's output, and trains and updates the encoder and decoder based on the reconstruction error to obtain a pre-trained encoder. The encoder-decoder network is used to learn the prior anatomical structure and spatial layout of the slices in a self-supervised manner.

[0102] In the occlusion reconstruction pre-training and slice representation learning stages: a random occlusion strategy is introduced to construct a self-supervised pre-training task. An encoder-decoder network is used to perform occlusion reconstruction pre-training on the above arbitrary slice images. Each original slice image is input into the encoder after random region occlusion to obtain multi-scale feature representations. Then, the decoder performs pixel-level reconstruction of the occluded regions and optimizes the reconstruction error. This allows the encoder to learn the local texture features and global spatial layout priors of the cardiac anatomy structure in an unsupervised or self-supervised manner. By traversing three-dimensional cardiac CT / MRI data of different individuals and different modalities for pre-training, pre-training weights that can uniformly represent any cardiac slice are obtained and used as the initialization for the subsequent relative position regression mapping network.

[0103] Step 2.1: Divide the 3D heart bounding box into several sub-regions, and divide the unit normal sphere into several orientation intervals. In the combined space of each sub-region and orientation interval, randomly extract two-dimensional slices from the two-dimensional slice set, and gather all the extracted two-dimensional slices to form a training slice set.

[0104] After completing the three-dimensional cardiac image preprocessing and arbitrary slice construction stages, a set of two-dimensional slices covering the entire three-dimensional cardiac region is obtained. ,in Indicates the first A two-dimensional slice image, For the first Rigid pose parameters of a two-dimensional slice image.

[0105] To ensure the uniformity of the self-supervised representation in three-dimensional space, the three-dimensional heart bounding box Ω and the unit normal sphere are divided into several sub-regions and orientation intervals. For each sub-region-orientation region combination, a set of two-dimensional slice images is selected according to a preset ratio. A fixed number of slices are randomly selected as pre-training samples to form a training slice set that is approximately uniformly distributed in position and orientation. This provides rich and balanced input data for occlusion reconstruction pre-training. This represents the total number of pre-training samples in the training slice set.

[0106] Step 2.2, for the training slice set Each 2D slice is divided into sections according to a preset patch size. The patch unit is a set of non-overlapping rectangular patches, denoted by patch index set . From the range of occlusion ratios Randomly generated occlusion ratio The number of occlusions is generated based on the occlusion ratio and the number of patch units. , This indicates taking values ​​downwards; randomly selecting a number of occluded patch units without replacement from the patch unit set as the occlusion position, a subset of patch indices. And employ occlusion operator Block the obstructed area (for areas belonging to) The patch areas are zeroed out, filled with the mean, or filled with noise. Patch cells outside the occluded positions retain their original grayscale. The occluded areas only retain their position indices and are not directly observed, generating two-dimensional slices after random area occlusion. By randomly changing the occlusion ratio on different slices and the location of the obstruction This forces the model to infer the shape and spatial layout of the occluded heart structure from the visible area during the pre-training phase.

[0107] Step 2.3: Construct an encoder-decoder structure based on a convolutional neural network, using the encoder... With decoder As the main body, among which , These are the learnable parameters; each occluded slice As input, it is fed into the encoder to extract multi-scale feature maps. :

[0108]

[0109] The encoder consists of multiple layers of convolutional, normalized, and nonlinear activation units, and can optionally employ residual connections and pyramid feature aggregation structures to simultaneously capture local texture and global heart morphology; subsequently, multi-scale feature mapping is performed. Input Decoder By progressively upsampling and convolutional reconstruction of the occluded region, the corresponding reconstructed slices are obtained. :

[0110]

[0111] The decoder is structurally symmetrical or semi-symmetrical with the encoder, and its output size is consistent with the original slice to ensure pixel-level reconstruction capability for occluded patches.

[0112] This embodiment provides a suitable existing codec network or framework and parameters as follows: Occlusion reconstruction pre-training can adopt the SparK (Sparse and Hierarchical Masked Modeling) self-supervised pre-training framework as the training paradigm; the encoder backbone network uses ResNet50 to achieve robust feature extraction of two-dimensional heart slices; the input slices are uniformly scaled to... and with Random occlusion is applied to non-overlapping patches, with an occlusion ratio set to 60%, only obscuring unoccluded areas. The data is fed into the encoder for feature extraction and then into the decoder to reconstruct the occluded region; the reconstruction loss uses the mean squared error loss calculated only in the occluded region. Pre-training optimization uses the AdamW optimizer with a weight decay coefficient of . The initial learning rate is The learning rate is scheduled using cosine annealing; the effective batch size is increased to 2048 through gradient accumulation, and the number of pre-training rounds is 300, so as to obtain a transferable slice representation encoder for subsequent relative position regression network training and online localization inference.

[0113] Step 2.4: For each pair of original slices With reconstructed slices Only in its obstructed area The mean squared error reconstruction loss is calculated internally, and the single-sample reconstruction loss is defined as:

[0114]

[0115] in This means that only at the location of the occluded patch, the average of the squared errors of the original pixel and the reconstructed pixel is taken; within a pre-training batch, for The total batch loss is obtained by summing or averaging the reconstruction losses of the training slices.

[0116]

[0117] The encoder parameters are optimized using stochastic gradient descent or Adam optimization algorithms. With decoder parameters Perform iterative updates to reduce the total batch loss. Minimize; after pre-training, retain the optimized encoder. As a representation extraction network for any heart slice, for any unobstructed slice Calculate its feature mapping And a low-dimensional slice representation vector is obtained through global average pooling and linear mapping:

[0118]

[0119] in, For global average pooling, For linear mapping layers, low-dimensional slices represent vectors. As input features for subsequent relative position regression-based mapping networks, these features are used to characterize the anatomical structure and spatial layout priors of arbitrary slices, achieving knowledge transfer from the self-supervised task of occlusion reconstruction to the task of relative position regression. Through repeated pre-training on slices of 3D cardiac images from different individuals and modalities, the encoder learns both local cardiac texture and global spatial layout priors simultaneously in an unsupervised or self-supervised manner, resulting in a unified representation network that can be reused on any cardiac slice, providing stable slice feature inputs for subsequent relative position regression mapping networks.

[0120] Step 3: Based on the 2D slice set, select two slices belonging to the same 3D heart body data as slice pairs. Use the relative displacement of the slice pair as labels to generate training samples. Training samples with relative displacement within the coarse displacement range constitute the coarse displacement training set, and training samples with relative displacement within the fine displacement range constitute the fine displacement training set. Use the coarse and fine displacement training sets respectively to train and update the relative position regression mapping network, obtaining the coarse localization model and the fine localization model. The relative position regression mapping network includes a regression encoder and a regression mapping head. The regression encoder shares weights with the pre-trained encoder, and the weights are frozen during training and updates.

[0121] During the training phase of the mapping network based on relative position regression: within the same 3D cardiac image, for any two slices and Read its true pose in physical space and The rigid transformation is represented by three non-collinear three-dimensional points on the slice plane using a three-point notation method. and will from arrive The true relative displacement is expressed as the difference between the coordinates of three points. A MapNet network with shared weights for relative position regression is constructed. Two slice features obtained from a pre-trained encoder are input into a fully connected regression head, which outputs the corresponding predicted displacements of three points. The MapNet is trained end-to-end using the regression loss between predicted displacement and actual displacement as a supervision signal. Through this relative position regression process, any slice is mapped to a unified three-dimensional latent coordinate space to obtain a relative position representation with anatomical consistency, thereby achieving accurate modeling of the relative position relationship between any two slices within the same three-dimensional cardiac image.

[0122] Step 3.1, regarding the first Three-dimensional heart body data After completing the preprocessing and arbitrary slice construction stages, a set of two-dimensional slices corresponding to the volume data is obtained. ,in For the first A two-dimensional slice image, Its rigid pose parameters include the normal vector, the planar reference point, and the in-plane rotation angle. For three-dimensional heart body data The total number of two-dimensional slices in the corresponding two-dimensional slice set; in each Based on the preset spatial spacing distribution, from the index set Randomly sample slices of the index according to a hierarchical or segmented distance strategy. Construct training slice pairs and from The corresponding real pose is read from the data and used for subsequent calculation of relative position labels. By sampling slices of the same volume data at multiple scales and distances, the training process includes both fine-grained local relative displacements that are spatially close and long-distance relative displacements that cross the heart cavity and cross the plane, thereby improving the mapping network's ability to model spatial relationships at different scales.

[0123] Step 3.2, for data of the same entity Any pair of slices Select three non-collinear standard reference points within their respective slice planes. For example, taking the slice center and two fixed offset points along the local coordinate axis as references, the planar parameters obtained through the preprocessing stage are... and local orthogonal basis vectors Mapping to a unified three-dimensional physical coordinate system yields three-dimensional marker points located on the slice plane:

[0124]

[0125] in, Representation data No. Within the slice image, the first Three-dimensional marker points, Representation data No. The position corresponding to the slice image Representation data No. The normal vector corresponding to the slice image. Similarly, the th slice image can be obtained. slice image 3D marker points From arrive The true relative displacement is expressed as the difference in coordinates of three points:

[0126]

[0127] The three displacements are then connected in a fixed order or stacked to form a relative position regression label vector:

[0128]

[0129] in, For body data The Middle slice image To the slice image The The coordinate difference of the three-dimensional marker points For body data The Middle slice image With the slice image The labels between them are used to obtain training samples. Therefore, without explicitly solving for the rigid body registration matrix, the three-dimensional relative displacement marked at three points is used as a monitoring signal to characterize the rigid relative positional relationship between two arbitrary slices.

[0130] To cover displacement ranges at different scales, two MapNet models were trained: a coarse localization model and a fine localization model. The coarse localization model randomly selected two training samples within the effective region (approximately 66.67% of the effective region of the volume data center). The fine localization model first randomly selected one training sample and then selected another training sample within the neighborhood of the prediction error range of the coarse localization model.

[0131] The coarse displacement range is taken as the effective region, and training samples whose relative positions are within the coarse displacement range are included in the coarse displacement training set; the fine displacement range is taken as the neighborhood of the prediction error range of the coarse localization model, and training samples whose relative positions are within the fine displacement range are included in the fine displacement training set. Thus, in this embodiment, coarse displacement training set and fine displacement training set are constructed for training the coarse localization model and the fine localization model, respectively.

[0132] Step 3.3: Construct a relative position regression mapping network ,in These are the parameters to be learned. It consists of two parts: a shared slice feature extraction module and a relative displacement regression head.

[0133] In the shared slice feature extraction module, slice pairs are processed. Regression encoders that are consistent with the structure and weights of the pre-trained encoder. Features are extracted and subjected to global average pooling and linear mapping to obtain the initial slice representation vector. ,in It can load pre-trained parameters and freeze or fine-tune them with a small learning rate during the training phase of the mapping network, and construct dual input features including single-slice representations, difference representations, and element-wise products:

[0134]

[0135] in, For slice pairs The corresponding dual input features, Represents the absolute value of each element. Represents element-wise product. The feature dimension is defined by combining single-slice representation, differential representation, and interactive representation. This enables the subsequent mapping network to simultaneously perceive the anatomical features of the two slices and their relative differences, providing geometrically rich input features for accurate regression of three-dimensional relative displacement.

[0136] In the two parts of the relative displacement regression, the dual input features are... Input regression mapping head Passing through fully connected layers in sequence Normalization layer, nonlinear activation unit and fully connected layer Then, the forward output is obtained; where The output dimension is 9; and in and Normalization layers (batch normalization or layer normalization) and nonlinear activation functions (ReLU or GELU) are set between the layers, and residual or skip connection structures can be optionally added to enhance the expressive power of deep networks; the forward output of the regression head for:

[0137]

[0138] To constrain the displacement prediction range and improve training stability, the hyperbolic tangent function can be used. Element-wise boundedness is performed, and a displacement range hyperparameter is introduced. The relative displacement prediction values ​​of slice pairs in the training samples are obtained. :

[0139]

[0140] Then Decomposed into three sets of three-dimensional predicted relative displacements The three-dimensional coordinate differences corresponding to the three marked points are as follows:

[0141]

[0142] in, For body data The Middle slice image To the slice image The The predicted value of the coordinate difference of three-dimensional marker points.

[0143] During the reasoning process, Perform the aforementioned shared encoding, dual feature fusion, and forward computation of the regression head output on the input slice pair to output the predicted value. As a 3D relative displacement estimate between arbitrary slice pairs, it is used for pose update and multistep navigation in subsequent view localization; and At least regression encoder parameters With regression mapping head parameters (Include The weights and biases, as well as the normalized layer parameters, are used to achieve an end-to-end regression mapping from slice pair representations to nine-dimensional three-point displacements.

[0144] Step 3.4: For a set of slice pairs within a training batch ,in, This represents the set of training batch indices obtained by sampling in a single parameter update iteration. This specifies the number of slice pairs included in the training batch. In this embodiment, the true relative displacements of the three points are first scaled, and the label vectors are scaled according to a preset length. Scaling is performed to obtain normalized labels. For the predicted vector The main regression loss is defined using the mean squared error. And proposes a symmetric regularization term. :

[0145]

[0146]

[0147] in, For training samples The corresponding predicted relative displacement value, For body data medium slice image With sliced ​​images The labels between them.

[0148] and with To predetermine the weights, the symmetric regularization term is combined with the principal regression loss in a weighted manner to obtain the overall training objective function of the mapping network. By jointly optimizing the encoder and mapping network parameters on multi-case, multi-modal datasets, the relative positional relationships of any pair of slices in a unified latent coordinate space are explicitly learned.

[0149]

[0150] On a multi-case 3D cardiac CT / MRI dataset, the pre-trained encoder was tested. The parameters are partially frozen or fine-tuned with a small learning rate to map the network parameters. Iterative training is performed using stochastic gradient descent or Adam optimization algorithms, so that... Minimize; during training, all slices from the same 3D cardiac image are treated as discrete sampling points in a unified 3D latent coordinate space, with the predicted relative displacement between each pair of them being minimized. The directed edge set that constitutes this latent space minimizes the regression error and symmetry constraints in the global scope, so that the relative position prediction between any pair of slices in the same body data gradually approaches the real geometric relationship in the graph structure. Thus, without explicitly building a physical template or global registration, the process of embedding each slice into a unified three-dimensional latent coordinate space is implicitly completed, providing a stable and consistent relative position regression basis for subsequent online arbitrary view positioning based on multi-step navigation.

[0151] This embodiment provides a training parameter setting as follows: For each 3D cardiac image, the cardiac region is first segmented and cropped to the cardiac region; then, a slice sampling strategy is used to generate approximately [number missing] samples within each 3D volumetric data. Group slices and location-based samples, and limit the sampled planar parameters to approximately the volume data center. Within the effective region, to ensure that the sampled slices contain sufficient cardiac anatomical information; to cover different scale displacement ranges, two MapNet models, a coarse localization model and a fine localization model, were trained respectively: the coarse localization model randomly selects two sets of the above-generated sample combinations within the effective region, and uses the coarse displacement range hyperparameter... Constrain the predicted displacement boundary; the fine positioning model first randomly selects a set of the above-generated sample combinations, and then selects a set of the above-generated sample combinations within the neighborhood of the coarse model's prediction error range, using the fine displacement range hyperparameter. Constrain fine-grained displacement boundaries; during fine-tuning training, the input slices are uniformly adjusted to... Gradient accumulation is used to make the effective batch size... The optimizer chosen is AdamW with a weight decay factor of 1. The learning rate is scheduled using cosine annealing, and the initial learning rate during the fine-tuning phase is set to... The number of training rounds is The system employs a round of data augmentation with random intensity transformation to enhance robustness across modalities and cases.

[0152] Step 4: Randomly initialize the target 3D heart body data to obtain proxy slices. Based on the proxy slices and query slice images, use a coarse localization model to obtain coarse localization results. Based on the coarse localization results and query slice images, use a fine localization model to obtain fine localization results. Obtain the target view slice image based on the fine localization results.

[0153] Online view localization and result robustness enhancement stage based on multi-step navigation: In the online inference stage, for query slice images in the reference 3D cardiac image. Using this as the target view prompt input, one or more proxy slice locations are randomly initialized in the three-dimensional volume data of the target 3D cardiac image V. Capture the proxy slice image at the current location. , with query slice image By using the MapNet network, a relative position regression mapping network, as the common input, the predicted relative displacement from the current tile to the target tile is obtained. And update the proxy location accordingly. This completes the coarse localization; then, within the vicinity of the coarse localization result, sampling or interpolation is performed within a preset small range to generate local candidate slices. A fine-grained MapNet trained with a small displacement range is then used to process the query slice image. Perform multi-step iterative prediction and position update with the current candidate slice until the displacement amplitude is below a threshold or the similarity converges, in order to obtain the finely localized target view position. and corresponding slices To further suppress the influence of random initialization and local optima, the above multi-step navigation process is repeated for multiple different initial positions. The multiple positioning results are integrated using strategies such as position averaging, weighted fusion based on structural similarity measurement, or majority voting. The final target view that is most similar to the anatomical structure of the query slice is output, so as to realize the rapid and automatic positioning of any view of three-dimensional cardiac images without the need to construct physical template coordinates or explicit image registration.

[0154] Step 4.1: Randomly initialize the target 3D heart body data to obtain proxy slices.

[0155] During the online inference phase, the reference 3D cardiac image was used. The target view is pre-annotated or interactively selected by the operator, and its corresponding two-dimensional slice image is used. As a query view prompt; when the query view comes from a separately acquired 2D echocardiogram or X-ray projection, it is first mapped to the same spatial resolution and intensity distribution domain as the 2D slices in the training phase through geometric correction, size resampling, and grayscale normalization, thus obtaining a standardized query slice image. ; Call the pre-trained encoder obtained during the training phase Global average pooling and linear mapping layers are used to obtain the query view representation vector. :

[0156]

[0157] The vector Used to compare with any candidate slice representation within the target 3D cardiac image during subsequent navigation, serving as a fixed target representation input for multi-step navigation.

[0158] Target 3D cardiac image First, obtain the normalized volume data in a unified three-dimensional physical coordinate system using the method in step 1.1 of the training phase. and target 3D heart bounding box Then, within the target's three-dimensional heart bounding box... Random sampling according to uniform distribution Individual agent reference points And the plane normal vector of the reference volume data where the queried slice image is located. As the initialization direction, Alternatively, small perturbation sampling can be performed in the vicinity to form a proxy reference point. Initial rigid pose parameters , The data is obtained through random initialization; referring to step 1.3, the local 2D coordinate construction and slice interpolation sampling module is called to extract proxy slice images as current candidate slices under each initial rigid pose parameter:

[0159]

[0160] And its representation is computed using a pre-trained encoder:

[0161]

[0162] in, In a local two-dimensional coordinate system Downstream Agent Reference Point The corresponding proxy slice image, For agent reference point The corresponding quasi-orthogonal basis vectors, For agent reference point The corresponding low-dimensional representation vector. This forms within the target volume data. The initial state of a navigation trajectory starting from different initial positions.

[0163] Step 4.2: Based on the proxy slice and query slice images, obtain the coarse localization result using the coarse localization model.

[0164] Take coarse positioning model With fine positioning model The former uses coarse displacement range hyperparameters and a larger scale normalization factor during the training phase. The latter employs a fine-range displacement hyperparameter and a small-scale normalization factor for learning. Learn the fine local relative displacements.

[0165] This embodiment provides a and The parameter range or parameter value is set as follows: for coarse-grained navigation networks Setting coarse displacement range hyperparameters during the training phase The maximum possible displacement between any two slices within the effective area of ​​the same three-dimensional cardiac image is determined, and a larger-scale normalization factor is taken accordingly. This is to achieve stable normalization learning of large-scale displacement monitoring signals. For fine-grained navigation networks... Setting fine-range hyperparameters during the training phase A second slice is sampled within the coarse positioning error neighborhood to focus on local fine displacement; a corresponding smaller-scale normalization factor is taken. It is used to perform more detailed normalized regression learning on small-scale displacement monitoring signals.

[0166] During the online phase, for each navigation trajectory... Step, use With larger step size Complete the coarse positioning that quickly converges to the vicinity of the query view, and sum the relative position differences. Reduce from an initial large value to a preset coarse positioning threshold. The following or the number of iterations reached Once the coarse positioning conditions are met, a local 3D search window is constructed near the current position. This limits subsequent navigation steps and position updates within this local window, and then switches to using... With smaller step size The fine-grained navigation mode iterates over each trajectory until the displacement norm is reached. Or the number of iterations reaches This enables a multi-scale navigation process that proceeds from coarse to fine, balancing convergence speed and final positioning accuracy.

[0167] Specifically, for the coarse positioning process: a proxy reference point is used as a navigation trajectory, for the first... The navigation trajectory is in the first Step's agent reference point low-dimensional representation vector With query view representation vector Construct the first Dual input features of the step :

[0168]

[0169] Input it based on the coarse localization model Based on the coarse displacement range hyperparameter and the large-scale normalization factor of the normalization layer, the coarse relative displacement prediction value is obtained. :

[0170]

[0171] Each of them For the first Three-dimensional displacement prediction at three reference points; the arithmetic mean of the three displacements is used to obtain a composite displacement estimate from the current position to the target position:

[0172]

[0173] And according to the preset coarse step size factor Updated proxy reference point location:

[0174]

[0175] in, No. The location of the agent reference point for the step. For the first The coarse step size factor is a preset value. It can be the same in every time step or it can vary according to the time step.

[0176] Using three non-collinear reference points within the current slice plane as anchor points, let the first... The 3D reference point of the proxy slice image of the step is The updated three-dimensional reference points are constructed based on the predicted displacement. :

[0177]

[0178] From the reference point Construct principal direction vectors in the plane, and define the current principal direction vector. With the updated principal direction vector for:

[0179]

[0180] in This means projecting a vector onto the normal vector. Within the slice plane; let the first... The orthogonal basis in the slice plane is and define Based on planar coordinates Calculate the arctangent function of its azimuth angle, then the in-plane rotation increment. Determine using the following formula:

[0181]

[0182] To suppress angular oscillations caused by noise, define To be Cut off to interval If the limiting function is applied within the plane, the in-plane rotation angle is updated using the following formula:

[0183]

[0184] in This is the preset maximum in-plane rotation angle per step; when in-plane rotation updates are not enabled, it is taken as... As a special case.

[0185] While maintaining the plane normal vector With the parameters remaining unchanged or slightly adjusted according to the predicted displacement direction, new plane parameters are obtained through updating. Based on this, the next candidate proxy slice image is re-extracted. and the corresponding low-dimensional representation vector The coarse localization process is repeated until the stopping condition is met or the maximum number of steps is reached, outputting the final surrogate slice image and rigid pose parameters as the first... Coarse positioning results of the navigation trajectory.

[0186] Step 4.3: Based on the coarse localization result and the query slice image, obtain the fine localization result using the fine localization model. In this embodiment, the fine localization process is similar to the coarse localization process, executed based on the coarse localization result and the fine localization model (using a preset fine step size factor), as follows:

[0187] A local three-dimensional search window is constructed centered on the coarse positioning result of each navigation trajectory;

[0188] Regarding the first The navigation trajectory is in the first The proxy slice image and query slice image of the first step are used to construct the second step. The dual input features of the step;

[0189] The first The dual input features of the step are input into the fine localization model. Based on the fine displacement range hyperparameter and the small-scale normalization factor of the normalization layer, the fine relative displacement prediction value is obtained. The fine displacement range hyperparameter is smaller than the coarse displacement range hyperparameter, and the small-scale normalization factor is smaller than the large-scale normalization factor.

[0190] The position of the surrogate reference point is updated using the predicted values ​​of fine relative displacement, and new rigid pose parameters are obtained;

[0191] Based on the updated rigid pose parameters of the surrogate reference point, the first... (The sentence is incomplete and requires more context to be translated.) The proxy slice image of the step;

[0192] The algorithm iteratively obtains fine relative displacement predictions based on the surrogate slice image and the query slice image until a stopping condition is met or the maximum number of steps is reached. It then outputs the final surrogate slice image and rigid pose parameters as the first... Detailed positioning results of the navigation trajectory.

[0193] The same content in the fine positioning process as in the coarse positioning process will not be repeated in this embodiment; please refer to the specific limitations of the coarse positioning process.

[0194] Step 4.4: Obtain the target view slice image based on the fine positioning results.

[0195] Targeted 3D cardiac imaging In The navigation trajectory, through a multi-step navigation process combining coarse and fine localization, converges in one step to the target view most similar to the anatomical structure of the query slice, thus obtaining the final surrogate reference point position for each trajectory. Planar parameters and the corresponding final slice image Define a view similarity metric function based on pixels or features. It can be achieved by normalized cross-correlation, structural similarity (SSIM), or feature space cosine similarity, calculating the similarity score between the final proxy slice image and the query slice image for each trajectory. :

[0196]

[0197] The scores are then normalized or temperature-weighted to obtain the normalized weights. :

[0198]

[0199] in The temperature coefficient is used as the basis for weighted fusion of the final reference point position, in-plane rotation angle, and normal vector of all trajectories to obtain the integrated target view reference point position. In-plane rotation angle and plane normal vector :

[0200]

[0201]

[0202]

[0203] Anomaly removal strategies based on clustering or distance thresholds can be used to remove abnormal trajectories that are far from the main cluster or have low similarity. Then, weighted fusion is performed on the remaining trajectories to achieve robust suppression of random initialization and local optima, and obtain a stable final view position estimate.

[0204] After obtaining the integrated target view reference point With the corresponding plane normal vector In-plane rotation angle Then, it is back-projected onto the target three-dimensional cardiac image. In the unified 3D physical coordinate system and DICOM coordinate system, the output includes plane equations. Complete view geometry parameters, including the three-dimensional coordinates of the target view reference point, slice orientation, pixel spacing, and imaging matrix, are used for direct access in clinical workstations or robot navigation systems. This is also equivalent to extracting target view slice images from the target three-dimensional heart body data based on the complete view geometry parameters.

[0205] Simultaneously based on the similarity distribution of multiple navigation trajectories Spatial variance between trajectories and the final normalized weights Construct view quality and uncertainty scoring indicators, for example:

[0206]

[0207] in, To be based on similarity distribution Obtain the mean similarity score. The spatial variance penalty coefficient; when the quality score When the score is below a preset threshold, a low confidence flag is returned to the upper-level system or a re-initialization and navigation process is triggered (i.e., step 4 is re-executed). When the threshold requirement is met, the final complete view geometry parameters, target view slice image and its quality score are output, realizing automatic confidence assessment and interpretability enhancement of online view localization results.

[0208] This embodiment iteratively executes a closed-loop process along the navigation trajectory, including capturing the current slice, feature extraction, constructing a dual input with the query representation, predicting relative displacement using MapNet, and updating the proxy reference point. In the coarse localization stage, a larger step size and a coarse-grained network are used to quickly shorten the distance to the target view position. In the fine localization stage, a smaller step size and a fine-grained network are used for fine adjustment within a local 3D window. For the final view of all navigation trajectories, weights are calculated based on pixel-level or feature-level similarity with the query view. Abnormal trajectories with significant spatial deviations or low similarity are eliminated. The view positions and normal vectors of the remaining trajectories are weighted and averaged or clustered to obtain the geometric parameters of the final target view. A quality score and confidence index are constructed using the mean similarity and trajectory variance to achieve robust suppression of multiple initial positions, local optima, and noise disturbances.

[0209] To improve robustness, a strategy of multiple random initializations and result averaging is employed during the inference phase to suppress local optima and noise effects. Compared with existing view localization methods based on manual rules or traditional registration optimization, this invention eliminates the need to construct physical templates or perform explicit registration. It enables rapid and automatic localization of any view within 3D cardiac CT / MRI data, offering advantages such as high localization accuracy, high computational efficiency, and strong cross-individual and cross-modal generalization capabilities. This provides a reliable view acquisition foundation for cardiovascular disease screening and diagnosis, preoperative planning, and robotic ultrasound or intelligent navigation systems.

[0210] Example 2: This example provides a view localization system for 3D cardiac images based on relative position regression, including a 3D cardiac image preprocessing and arbitrary slice construction module, an occlusion reconstruction pre-training and slice representation learning module, a mapping network training module based on relative position regression, and an online view localization and result robustness enhancement module based on multi-step navigation. The 3D cardiac image preprocessing and arbitrary slice construction module is configured to resample, coordinate-align, extract cardiac regions of interest, standardize intensity, and automatically generate 2D slices with arbitrary orientation and position and their rigid pose parameters from the input 3D cardiac CT / MRI data. The occlusion reconstruction pre-training and slice representation learning module is configured to... To enable self-supervised pre-training of the encoder-decoder network for random occlusion reconstruction tasks and output low-dimensional representation vectors for arbitrary slices, a relative position regression-based mapping network training module is configured to construct three-point labels for relative displacement tags on arbitrary slice pairs within the same 3D cardiac image, and train the relative position regression mapping network to learn relative geometric relationships in a unified latent coordinate space. A multi-step navigation-based online view localization and result robustness enhancement module is configured to receive reference view cues, perform multi-step navigation and multi-initial position integration within the target 3D cardiac image, output the target view's planar parameters, DICOM coordinates, and quality score, and support integration with clinical workstations or robot navigation systems.

[0211] The limitations of the view localization system in three-dimensional cardiac images based on relative position regression in this embodiment can be found in the specific limitations of the view localization method in three-dimensional cardiac images based on relative position regression in Embodiment 1, which will not be repeated in this embodiment.

[0212] Example 3: This example provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the view localization method in three-dimensional cardiac images based on relative position regression in Example 1.

[0213] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0214] Example 4: This example provides an electronic device (including a computer, server, smartphone, etc.), including a processor and a memory storing a number of computer instructions. When the computer instructions are executed by the processor, they implement the steps of the view localization method in three-dimensional cardiac images based on relative position regression in Example 1.

[0215] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, which implements the method of the present invention by running the computer program stored in the memory.

[0216] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores the program, and the processor executes the program upon receiving an execution instruction.

[0217] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0218] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0219] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A view localization method in three-dimensional cardiac images based on relative position regression, characterized in that, Includes the following steps: Based on the input 3D heart body data, a set of 2D slices covering all directions and positions within the 3D heart bounding box is constructed. Each 2D slice contains a 2D slice image and rigid pose parameters. Each two-dimensional slice image in the two-dimensional slice set is randomly occluded and then input into the encoder. The decoder reconstructs the two-dimensional slice image based on the encoder's output. The encoder and decoder are then trained and updated based on the reconstruction error to obtain a pre-trained encoder. Based on the set of two-dimensional slices, two slices belonging to the same three-dimensional heart body data are taken as slice pairs. The relative displacement of the slice pairs is used as labels to generate training samples. Training samples with relative displacement within the coarse displacement range constitute the coarse displacement training set, and training samples with relative displacement within the fine displacement range constitute the fine displacement training set. The relative position regression mapping network is trained and updated using coarse displacement training sets and fine displacement training sets respectively to obtain coarse localization model and fine localization model. The relative position regression mapping network includes a regression encoder and a regression mapping head. The regression encoder shares weights with the pre-trained encoder and freezes the weights during training and updating. For the target 3D heart body data, a proxy slice is obtained by random initialization. Based on the proxy slice and the query slice image, a coarse localization model is used to obtain the coarse localization result. Based on the coarse localization result and the query slice image, a fine localization model is used to obtain the fine localization result. Based on the fine localization result, the target view slice image is obtained.

2. The view localization method in three-dimensional cardiac images based on relative position regression according to claim 1, characterized in that, The construction of a set of two-dimensional slices covering the three-dimensional bounding box space of the heart at all directions and locations includes: The input 3D heart body data is preprocessed to obtain normalized volume data and 3D heart bounding box in a unified 3D physical coordinate system. Sampling reference points are uniformly distributed within the three-dimensional heart bounding box; polar angles are uniformly sampled within the polar angle range; and azimuth angles are uniformly sampled within the azimuth angle range. A rotation operator is constructed using any polar angle and any azimuth angle. The rotation operator is then multiplied by the initial three-dimensional vector to obtain the normal vector. By combining all normal vectors with all reference points one by one, we obtain a set of normal vector and reference point combinations that cover the three-dimensional heart bounding box; For each combination of normal vector and reference point, a set of orthogonal basis vectors is constructed in a plane orthogonal to the normal vector. A local two-dimensional coordinate system is established with the reference point as the origin and the orthogonal basis vectors as the basis. Trilinear interpolation sampling is performed on the local two-dimensional coordinate system at a preset resolution to obtain a two-dimensional slice image. A random rotation angle is applied to the local two-dimensional coordinate system, and the random rotation angle, normal vector, and reference point are used as rigid pose parameters of the corresponding two-dimensional slice image.

3. The view localization method in three-dimensional cardiac images based on relative position regression according to claim 2, characterized in that, The preprocessing of the input 3D heart body data to obtain normalized volume data and a 3D heart bounding box in a unified 3D physical coordinate system includes: Obtain the voxel spacing and spatial orientation matrix from the input 3D heart body data; Based on the voxel spacing, the anisotropic voxel spacing is mapped to the preset isotropic voxel spacing by the resampling operator to obtain standard spatial volume data; Based on the human body position and spatial orientation matrix corresponding to the three-dimensional heart body data, the standard spatial volume data is aligned to a unified three-dimensional physical coordinate system. Under a unified three-dimensional physical coordinate system, a three-dimensional heart bounding box is extracted based on standard spatial volume data; The standard spatial volume data is spatially clipped according to the three-dimensional heart bounding box to obtain the volume data of the region of interest of the heart. The coordinates of the volumetric data of the region of interest in the heart are scaled and the intensity of the volumetric data of the region of interest in the heart is normalized by grayscale, so as to obtain normalized volumetric data in a unified three-dimensional physical coordinate system.

4. The view localization method in three-dimensional cardiac images based on relative position regression according to claim 1, characterized in that, The step of randomly occluding each two-dimensional slice image in the two-dimensional slice set includes: The three-dimensional heart bounding box is divided into several sub-regions, and the unit normal sphere is divided into several orientation intervals. In the combined space of each sub-region and orientation interval, two-dimensional slices are randomly extracted from the two-dimensional slice set. All extracted two-dimensional slices are aggregated to form a training slice set. For the training slice set, each two-dimensional slice is divided into multiple non-overlapping rectangular patch units according to a preset patch size. An occlusion ratio is randomly generated from the occlusion ratio range. The number of occlusion units is generated according to the occlusion ratio and the number of patch units. Without replacement, a number of patch cells are randomly selected from the set of patch cells as occlusion positions. An occlusion operator is used to occlude the occlusion positions. Patch cells outside the occlusion positions retain their original grayscale, generating a two-dimensional slice after random region occlusion.

5. The view localization method in three-dimensional cardiac images based on relative position regression according to claim 1, characterized in that, The method based on a set of two-dimensional slices, taking two slices belonging to the same three-dimensional cardiac body data as a slice pair, and using the relative displacement of the slice pair as a label to generate training samples, includes: Multi-scale and multi-relative displacement slice sampling is performed within the same three-dimensional cardiac body data; For any pair of slices, select three non-collinear standard reference points in each slice plane, unify the mapping values ​​of the standard reference points in the three-dimensional physical coordinate system, and obtain the three-dimensional marker points located on the slice plane. Calculate the coordinate difference between corresponding 3D marker points in the two slices within a slice pair, and combine the coordinate differences of the three 3D marker points to form a label. Combine the slice pair and the label to form a training sample.

6. The view localization method in three-dimensional cardiac images based on relative position regression according to claim 1, characterized in that, The relative position regression mapping network processes the data as follows: For any slice within a slice pair in the training samples, pass it sequentially through a regression encoder, global average pooling, and a linear mapping layer to obtain the corresponding initial slice representation vector; Construct dual input features based on the initial slice representation vector. , Indicates the first Within the three-dimensional cardiac body data, the first The initial slice representation vector of the slice, Indicates the first Within the three-dimensional cardiac body data, the first The initial slice representation vector of the slice, Represents the absolute value of each element. Represents element-wise product; The dual input features are input into the regression mapping head, and then passed through a fully connected layer, a normalization layer, a nonlinear activation unit, and another fully connected layer in sequence to obtain the forward output. The hyperbolic tangent function is used to perform element-wise bounded transformation on the forward output, and after weighting by hyperparameters within the displacement range, the relative displacement prediction values ​​of slice pairs in the training samples are obtained.

7. The view localization method in three-dimensional cardiac images based on relative position regression according to claim 1, characterized in that, The process of randomly initializing the target three-dimensional cardiac body data to obtain proxy slices includes: Preprocessing the target 3D heart body data yields normalized volume data and the target 3D heart bounding box in a unified 3D physical coordinate system. Random sampling is performed within the target's three-dimensional heart bounding box using a uniform distribution. Identify a proxy reference point and construct the rigid pose parameters of the proxy reference point; Based on the rigid pose parameters of the surrogate reference point, a set of orthogonal basis vectors is constructed in a plane orthogonal to the normal vector. A local two-dimensional coordinate system is established with the surrogate reference point as the origin and the orthogonal basis vectors as the basis. Trilinear interpolation sampling is performed on the local two-dimensional coordinate system at a preset resolution to obtain the surrogate slice image.

8. The view localization method in three-dimensional cardiac images based on relative position regression according to claim 7, characterized in that, The process of obtaining coarse localization results based on proxy slices and query slice images using a coarse localization model includes: Using a proxy reference point as a navigation trajectory, for the first... The navigation trajectory is in the first The proxy slice image and query slice image of the first step are used to construct the second step. The dual input features of the step; The first The dual input features of the step are input into the coarse localization model, and the coarse relative displacement prediction value is obtained based on the coarse displacement range hyperparameter and the large-scale normalization factor of the normalization layer. The position of the surrogate reference point is updated using the coarse relative displacement prediction value, and new rigid pose parameters are obtained; Based on the updated rigid pose parameters of the proxy reference point, the first... The proxy slice image of the step; The algorithm iteratively obtains coarse relative displacement predictions based on the surrogate slice image and the query slice image until a stopping condition is met or the maximum number of steps is reached. It then outputs the final surrogate slice image and rigid pose parameters as the first step. Coarse positioning results of the navigation trajectory.

9. The view localization method in three-dimensional cardiac images based on relative position regression according to claim 8, characterized in that, The process of obtaining fine localization results based on coarse localization results and query slice images using a fine localization model includes: A local three-dimensional search window is constructed centered on the coarse positioning result of each navigation trajectory; Regarding the first The navigation trajectory is in the first The proxy slice image and query slice image of the first step are used to construct the second step. The dual input features of the step; The first The dual input features of the step are input into the fine localization model. Based on the fine displacement range hyperparameter and the small-scale normalization factor of the normalization layer, the fine relative displacement prediction value is obtained. The fine displacement range hyperparameter is smaller than the coarse displacement range hyperparameter, and the small-scale normalization factor is smaller than the large-scale normalization factor. The position of the surrogate reference point is updated using the predicted values ​​of fine relative displacement, and new rigid pose parameters are obtained; Based on the updated rigid pose parameters of the surrogate reference point, the first... (The sentence is incomplete and requires more context to be translated.) The proxy slice image of the step; The algorithm iteratively obtains fine relative displacement predictions based on the surrogate slice image and the query slice image until a stopping condition is met or the maximum number of steps is reached. It then outputs the final surrogate slice image and rigid pose parameters as the first... Detailed positioning results of the navigation trajectory.

10. The view localization method in three-dimensional cardiac images based on relative position regression according to claim 9, characterized in that, The step of obtaining the target view slice image based on the fine positioning result includes: Calculate the similarity score between the proxy slice image and the query slice image for each navigation trajectory, and normalize or temperature-weight the similarity score to obtain the normalized weight. Based on normalized weights, the rigid pose parameters of all navigation trajectories are weighted and fused to obtain the rigid pose parameters of the target view reference point. The rigid pose parameters of the target view reference point are back-projected into the unified three-dimensional physical coordinate system and the medical digital imaging and communication standard coordinate system of the target three-dimensional heart body data, and the output is complete view geometric parameters including plane equation, position of target view reference point, slice orientation, pixel spacing and imaging matrix. Extract target view slice images from the target 3D heart body data based on the complete view geometry parameters; If the quality score of the target view slice image is lower than the quality threshold, the target view slice image is re-extracted; otherwise, the target view slice image and quality score are output.

Images