Method for constructing dynamic three-dimensional shape of heart

Abstract

The present application discloses a method for constructing a dynamic three-dimensional shape of the heart, including: segmenting the original heart image to obtain mask image information; preprocessing the mask image information to obtain a cloud of voxel coordinate points; constructing a preset loss function based on the cloud of voxel coordinate points; training the established target neural network model based on the preset loss function, and denoting the trained model as the first neural network model; obtaining the signed distance function value of each frame based on the first neural network model; and obtaining the dynamic three-dimensional shape of the heart based on the signed distance function values of all frames. This method can quickly and accurately reconstruct the dynamic three-dimensional shape of the heart from a single heart image, can reconstruct a natural ventricular shape from out-of-plane low-resolution images without a dataset, has a fast calculation speed and a high fitting accuracy, and the proposed method has good generalization and can adapt to cardiac cine magnetic resonance images of the entire time series, various pathologies, and various resolutions.

Description

Technical Field

[0001] The present disclosure relates to the field of medical technologies, and in particular, to a method for constructing a dynamic three-dimensional shape of the heart. Background Art

[0002] In existing cardiac magnetic resonance imaging technologies, cine imaging is the most important imaging modality, which can visualize the cardiac chamber structure and detect ventricular motion function. Compared with traditional methods that only refer to two-dimensional image information, the construction of a three-dimensional model of the heart can provide a more accurate and comprehensive understanding of the cardiac structure and function. However, the short-axis images obtained by fast-scan cine imaging usually exhibit anisotropic spatial resolution, resulting in low resolution between slices, making it challenging to accurately and efficiently reconstruct the shape of the heart.

[0003] So far, some research results have been achieved in medical three-dimensional reconstruction technologies, mainly including: 1) methods based on statistical shape models, which however require a large amount of mesh data as input, rely on data distribution and cannot fully fit the contour corresponding to the image; 2) deep learning methods based on template mesh deformation, which are only applicable to isotropic data and specific populations; methods for obtaining the cardiac shape based on high-resolution segmentation do not utilize geometric prior knowledge and often require a large number of nuclear magnetic resonance images, and there is no research on the construction of the dynamic three-dimensional shape of the heart. Summary of the Invention

[0004] In view of this, embodiments of the present disclosure provide a method for constructing a dynamic three-dimensional shape of the heart, which can solve problems such as the inability to accurately and efficiently reconstruct the cardiac shape in the prior art.

[0005] In a first aspect, embodiments of the present disclosure provide a method for constructing a dynamic three-dimensional shape of the heart, specifically including the following:

[0006] Segment the original cardiac image to obtain mask image information;

[0007] Preprocess the mask image information to obtain a cloud of voxel coordinate points;

[0008] Establish a target neural network model;

[0009] Construct a preset loss function based on the cloud of voxel coordinate points;

[0010] Train the target neural network model based on the preset loss function, and denote the trained model as the first neural network model;

[0011] Obtain the signed distance function value for each frame based on the first neural network model;

[0012] Obtain the dynamic three-dimensional shape of the heart based on the signed distance function values of all frames.

[0013] Optionally, segmenting the original cardiac image to obtain mask image information includes: segmenting the original cardiac image based on a deep learning segmentation algorithm to obtain a mask image;

[0014] The mask image includes a number of frames of three-dimensional original images and voxel dimension information.

[0015] Optionally, preprocessing the mask image information to obtain a voxel coordinate point cloud set, including:

[0016] Processing each frame of the three-dimensional original image based on a cropping strategy to obtain a three-dimensional cropped image for each frame;

[0017] Merging the three-dimensional cropped images of all frames to obtain a reconstructed cropped image;

[0018] Creating a grid coordinate matrix based on the size of the reconstructed cropped image and a preset resolution;

[0019] Expanding the coordinates of the grid coordinate matrix to obtain a first array;

[0020] Obtaining a set of target pixel points and a set of background pixel points in the reconstructed cropped image;

[0021] Performing conversion processing on the set of target pixel points and the set of background pixel points based on the voxel dimension information to obtain a second array and a third array;

[0022] Obtaining an inner boundary point set and an outer boundary point set of the set of target pixel points;

[0023] Performing conversion processing on the inner boundary point set and the outer boundary point set based on the voxel dimension information to obtain a fourth array and a fifth array;

[0024] Normalizing the three-dimensional coordinates and time of the first array, the second array, the third array, the fourth array, and the fifth array;

[0025] Performing random sampling on the first array, the second array, the third array, the fourth array, and the fifth array after normalization processing to obtain a first sampling array, a second sampling array, a third sampling array, a fourth sampling array, and a fifth sampling array;

[0026] The first sampling array, the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array constitute the voxel coordinate point cloud set.

[0027] Optionally, the processing each frame of the three-dimensional original image based on a cropping strategy to obtain a three-dimensional cropped image for each frame includes:

[0028] Determine the cardiac reconstruction substructure;

[0029] Traverse the three-dimensional original images of all time frames, and determine the minimum coordinate information and the maximum coordinate information of the cardiac reconstruction substructure;

[0030] If the target bounding box formed by the minimum coordinate information and the maximum coordinate information is smaller than the preset image size, expand the target bounding box formed by the minimum coordinate information and the maximum coordinate information to obtain an expanded bounding box, and crop the three-dimensional original image based on the expanded bounding box to obtain the three-dimensional cropped image;

[0031] If the target bounding box formed by the minimum coordinate information and the maximum coordinate information is not smaller than the preset image size, crop the three-dimensional original image based on the target bounding box, and fill the background pixels of the cropped image to obtain the three-dimensional cropped image.

[0032] Optionally, the establishing the target neural network model based on the implicit neural representation includes:

[0033] Establish an initial model, where the initial model includes an implicit neural representation with a periodic activation function;

[0034] Initialize the weights of the initial model using a multi-frequency geometric initialization method;

[0035] Replace the linear layer in the initialized initial model with a quadratic layer to obtain the target neural network model.

[0036] Optionally, the constructing the preset loss function based on the voxel coordinate point cloud set includes:

[0037] Determine a first loss function based on the first sampling array;

[0038] Determine a second loss function based on the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array;

[0039] Determine a third loss function based on the first sampling array;

[0040] Determine the preset loss function based on the first loss function, the second loss function, and the third loss function.

[0041] Optionally, the first loss function is L eik :

[0042]

[0043] Wherein, Denote the gradient of the function \(f\) at the midpoint \(x\) of the first array, \(\|\cdot\|_2\) represents the 2-norm of the gradient, \(N\) i is the number of sample points in the first array, and \(x\) grid is the coordinate of the \(i\)-th point in the first array; i

[0044] The second loss function is \(L\) mask :

[0045]

[0046] where \(f\) in (x i ) is the predicted value of the signed distance function of the points in the second sampling array or the fourth sampling array, and \(f\) out (x i ) is the predicted value of the signed distance function of the points in the third sampling array or the fifth sampling array, \(N\) bou is the number of points in the fourth sampling array or the fifth sampling array, and \(N\) mask is the number of points in the second sampling array or the third sampling array;

[0047] The third loss function is \(L\) hes : where \(H\) f (x i ) represents the Hessian matrix of the function \(f\) at the \(i\)-th point in the first sampling array, and \(\|\cdot\|\) F represents the Frobenius norm;

[0048] The preset loss function is \(L: L = f_0L\) eik +\(\lambda_1L\) mask +\(\lambda_2L\) hes ; where \(\lambda_0\), \(\lambda_1\), and \(\lambda_2\) are the weights of the \(L\) eik loss term, \(L\) mask loss term, and \(L\) hes loss term, respectively.

[0049] Optionally, obtaining the signed distance function value for each frame based on the first neural network model includes:

[0050] Creating a grid coordinate matrix based on a preset numerical range, a preset quantity of spatial dimension, and a preset quantity of time dimension; the preset numerical range is the range from 0 to 1; the preset quantity of spatial dimension \(\geq 128\), and the preset quantity of time dimension \(\geq\) the number of time frames in the masked image;

[0051] Unfolding the coordinates of the grid coordinate matrix to obtain a uniform point set;

[0052] Input the uniform point set into the first neural network model to obtain the signed distance function value of each point.

[0053] Optionally, obtain the dynamic three-dimensional shape of the heart based on the signed distance function values of all frames.

[0054] The dynamic three-dimensional shape includes:

[0055] Extract the signed distance function values of the same frame according to the time frames of the uniform point set, and record them as the target time frame set.

[0056] Extract the zero isosurface from the target time frame set to obtain the dynamic three-dimensional shape of the heart.

[0057] Optionally, after the zero isosurface extraction, it further includes:

[0058] Based on the extracted zero isosurface, obtain the triangular mesh of each frame.

[0059] Render all the triangular meshes to obtain the rendered sub-images.

[0060] Merge all the rendered sub-images according to the time frames to generate a video of the dynamic three-dimensional shape of the heart.

[0061] In a second aspect, the embodiments of the present disclosure further provide a system for constructing a dynamic three-dimensional shape of the heart, specifically including the following:

[0062] A segmentation module for segmenting the original heart image to obtain mask image information.

[0063] A preprocessing module for preprocessing the mask image information to obtain a cloud of voxel coordinate points.

[0064] A model establishment module for establishing a target neural network model.

[0065] A training module for training the target neural network model based on a preset loss function, and denoting the trained model as the first neural network model.

[0066] A signed distance function value acquisition module for obtaining the signed distance function value of each frame based on the first neural network model.

[0067] A construction module for obtaining the dynamic three-dimensional shape of the heart based on the signed distance function values of all frames.

[0068] The method for constructing the dynamic three-dimensional shape of the heart disclosed in this application can obtain mask image information through image segmentation, effectively avoiding background noise and enabling subsequent processing to focus more on the heart region. By preprocessing the mask image information, a cloud of voxel coordinate points is obtained, enabling the subsequent neural network model to efficiently process three-dimensional data, enhancing the flexibility and accuracy of processing. A target neural network model is established and trained, capable of learning and optimizing the complex representation of the heart shape. Trained through a preset loss function, the trained first neural network model calculates the signed distance function value for each frame of the image, providing an accurate surface representation, which helps to construct an accurate three-dimensional shape. By integrating the signed distance function values of all frames, the dynamic three-dimensional shape of the heart can be obtained. This method can not only reflect the static structure of the heart but also capture its motion and changes, providing a more comprehensive and detailed analysis. The method for constructing the dynamic three-dimensional shape of the heart disclosed in this application can quickly and accurately reconstruct the dynamic three-dimensional shape of the heart from a single cardiac cine magnetic resonance image, reconstruct the natural ventricular shape from out-of-plane low-resolution images without the need for a dataset, with a fast calculation speed and a high fitting accuracy. The proposed method has good generalization ability and can adapt to cardiac cine magnetic resonance images of the entire time series, various pathologies, and various resolutions.

[0069] The above description is only an overview of the technical solution of this disclosure. In order to understand the technical means of this disclosure more clearly, it can be implemented according to the content of the specification. And to make the above and other purposes, features, and advantages of this disclosure more obvious and understandable, the following specific preferred embodiments are given and described in detail in conjunction with the accompanying drawings as follows. Brief Description of the Drawings

[0070] In order to more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings required for use in the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of this disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0071] Figure 1 It is a schematic flowchart of the method for constructing the dynamic three-dimensional shape of the heart provided by the embodiment of this disclosure.

[0072] Figure 2 It is a schematic flowchart of the method for obtaining the cloud of voxel coordinate points provided by the embodiment of this disclosure.

[0073] Figure 3 It is a schematic flowchart of the method for obtaining the three-dimensional cropped image provided by the embodiment of this disclosure.

[0074] Figure 4 It is a schematic flowchart of the method for establishing the target neural network model provided by the embodiment of this disclosure.

[0075] Figure 5 A flowchart of a method for constructing a preset loss function based on a voxel coordinate point cloud set provided by an embodiment of the present disclosure.

[0076] Figure 6 A flowchart of a method for obtaining a signed distance function value provided by an embodiment of the present disclosure.

[0077] Figure 7 A flowchart of a method for obtaining a dynamic three-dimensional shape of a heart based on signed distance function values of all frames provided by an embodiment of the present disclosure.

[0078] Figure 8 A flowchart of a video acquisition method provided by an embodiment of the present disclosure. Detailed implementation manners

[0079] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0080] It should be clear that the following uses specific specific examples to illustrate the implementation manners of the present disclosure, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific implementation manners, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts fall within the protection scope of the present disclosure.

[0081] It should be noted that the following describes various aspects of embodiments within the scope of the appended claims. It should be obvious that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on the present disclosure, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement a device and / or practice a method. In addition, this device and / or this method can be implemented using other structures and / or functions in addition to one or more of the aspects described herein.

[0082] It should also be noted that the illustrations provided in the following embodiments only schematically illustrate the basic concept of the present disclosure. The diagrams only show the components related to the present disclosure, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and proportion of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0083] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0084] Refer to Figure 1 , the present application discloses a method for constructing a dynamic three-dimensional shape of the heart, including:

[0085] S100, segment the original heart image to obtain mask image information;

[0086] Preprocess the mask image information to obtain a cloud of voxel coordinate points.

[0087] Among them, segmenting the original heart image specifically includes: segmenting the original heart image based on a deep learning segmentation algorithm to obtain a mask image labeled with tags; the mask image includes several frames of three-dimensional original images and voxel dimension information.

[0088] The original heart image includes any one of CT images and magnetic resonance images; further, the original heart image is preferably a cardiac cine magnetic resonance image.

[0089] Further, the format of the three-dimensional original image is NIFTI.

[0090] S200, establish a target neural network model.

[0091] S300, construct a preset loss function based on the cloud of voxel coordinate points.

[0092] S400, train the target neural network model based on the preset loss function, and record the trained model as the first neural network model.

[0093] Through training, the model can adaptively learn the characteristics of the heart shape and improve the reconstruction accuracy.

[0094] S500, obtain the signed distance function value of each frame based on the first neural network model;

[0095] Based on the signed distance function values of all frames, obtain the dynamic three-dimensional shape of the heart.

[0096] The method for constructing the dynamic three-dimensional shape of the heart disclosed in this application can obtain mask image information through image segmentation, effectively avoiding background noise and enabling subsequent processing to focus more on the heart region; by preprocessing the mask image information, a cloud of voxel coordinate points is obtained, enabling the subsequent neural network model to efficiently process three-dimensional data, enhancing the flexibility and accuracy of processing; establishing and training a target neural network model that can learn and optimize the complex representation of the heart shape, training through a preset loss function, and using the trained first neural network model to calculate the signed distance function values of each frame of the image, which can provide an accurate surface representation and contribute to the construction of an accurate three-dimensional shape. By integrating the signed distance function values of all frames, the dynamic three-dimensional shape of the heart can be obtained. This method can not only reflect the static structure of the heart but also capture its movement and changes, providing a more comprehensive and detailed analysis. The method for constructing the dynamic three-dimensional shape of the heart disclosed in this application can quickly and accurately reconstruct the dynamic three-dimensional shape of the heart from a single cardiac cine magnetic resonance image, can reconstruct a natural ventricular shape from out-of-plane low-resolution images without a dataset, has a fast calculation speed and a high fitting accuracy, and the proposed method has good generalization and can adapt to cardiac cine magnetic resonance images of the entire time series, various pathologies, and various resolutions.

[0097] The dynamic three-dimensional heart model generated by the method for constructing the dynamic three-dimensional shape of the heart disclosed in this application can be used for clinical diagnosis and treatment planning, helping doctors better understand the structure and function of the heart, improving the accuracy of diagnosis and the treatment effect; the entire process has a high degree of automation, reducing human intervention and errors, and improving the processing efficiency, which is particularly important for the processing of a large amount of medical image data, and has broad application prospects and important clinical value.

[0098] Refer to Figure 2 , the method for obtaining the cloud of voxel coordinate points specifically includes:

[0099] A100, process each frame of the three-dimensional original image based on a cropping strategy to obtain the three-dimensional cropped image of each frame.

[0100] Specifically refer to Figure 3 , the method for obtaining the three-dimensional cropped image specifically includes:

[0101] A110, determine the cardiac reconstruction substructure.

[0102] Specifically, the cardiac substructure to be reconstructed (i.e., the cardiac reconstruction substructure) can be labeled as 1, and the rest are labeled as 0; determining the cardiac reconstruction substructure ensures that the cropping area is concentrated on the key parts of the heart, thereby improving the accuracy of cardiac reconstruction.

[0103] A120, traverse the three-dimensional original images of all time frames to determine the YY+242313P of the cardiac reconstruction substructure

[0104] Minimum coordinate information and maximum coordinate information.

[0105] Specifically, find the global minimum and maximum x, y, and z coordinate ranges of the pixels with label 1, that is, the minimum coordinate information and maximum coordinate information of the cardiac reconstruction substructure.

[0106] A130. If the target bounding box formed by the minimum coordinate information and the maximum coordinate information is smaller than the preset image size, expand the target bounding box formed by the minimum coordinate information and the maximum coordinate information to obtain an expanded bounding box, and crop the three-dimensional original image based on the expanded bounding box to obtain a three-dimensional cropped image.

[0107] Among them, the target bounding box formed by expanding the minimum coordinate information and the maximum coordinate information includes: adding one pixel on the basis of the minimum coordinate information and the maximum coordinate information, that is, expanding the minimum coordinate information inward by one pixel and expanding the maximum coordinate information outward by one pixel. This can avoid the boundary being too close to the cardiac reconstruction substructure during cropping, effectively prevent the loss of some information of the structure, and ensure that the cropped image contains sufficient background information.

[0108] In this step, the judgment and expansion of the target bounding box ensure that the cropping area can cover the cardiac substructure and will not lose important information due to the too small bounding box. This adjustment makes the cropped image more stable and applicable.

[0109] A140. If the target bounding box formed by the minimum coordinate information and the maximum coordinate information is not smaller than the preset image size, crop the three-dimensional original image based on the target bounding box, and fill the background pixels of the cropped image to obtain a three-dimensional cropped image.

[0110] Among them, filling the background pixels of the cropped image includes: filling a circle of pixels with label 0 around the cropped image.

[0111] For the case where the target bounding box is not smaller than the preset image size, filling the background pixels of the cropped image ensures the consistency of the size and format of the cropped image, facilitating subsequent image processing and analysis.

[0112] Whether it is cropping by expanding the bounding box or filling the background pixels, a three-dimensional cropped image that meets the preset size can be finally obtained, ensuring the consistency and comparability of the output images; this strategy can adapt to cardiac substructures of different sizes and shapes. Whether it is a large heart or a small heart, a suitable cropped image can be obtained, enhancing the generality and applicability of the algorithm.

[0113] A200. Merge the three-dimensional cropped images of all frames to obtain a reconstructed cropped image.

[0114] Merging all frames can generate a continuous three-dimensional image, facilitating overall analysis and processing. Through merging, more useful information can be retained, avoiding the loss of information in single-frame images.

[0115] A300, create a grid coordinate matrix based on the size of the reconstructed and cropped image and a preset resolution.

[0116] A400, expand the coordinates of the grid coordinate matrix to obtain a first array.

[0117] Among them, the first array is a two-dimensional array in the form of N grid ×(t, x, y, z).

[0118] A500, obtain a set of target pixel points and a set of background pixel points in the reconstructed and cropped image;

[0119] Based on the voxel dimension information, perform conversion processing on the set of target pixel points and the set of background pixel points to obtain a second array and a third array.

[0120] Among them, the set of target pixel points is a set of pixel points corresponding to the sub-structures of the heart reconstruction, and the set of background pixel points is a set of all background pixel points.

[0121] The second array and the third array are also both two-dimensional arrays in the form of N grid ×(t, x, y, z).

[0122] A600, obtain an inner boundary point set and an outer boundary point set of the set of target pixel points;

[0123] Based on the voxel dimension information, perform conversion processing on the inner boundary point set and the outer boundary point set to obtain a fourth array and a fifth array.

[0124] Among them, the fourth array and the fifth array are also both two-dimensional arrays in the form of N grid ×(t, x, y, z); further, N grid can preferably be 20000;

[0125] Specifically, the methods for obtaining the inner boundary point set and the outer boundary point set include: First, define a 3×3 2D convolution kernel for detecting neighborhood changes. For each two-dimensional image in the traversed and cropped image, perform convolution first. If the two-dimensional pixel point is 0 but there is at least one pixel point with a label of 1 in the neighborhood, then it is an outer boundary point; if the two-dimensional pixel point is 1 but the sum of the label values of the neighborhood pixel points is less than 8, indicating that there is at least one pixel point with a label of 0, then it is an inner boundary point.

[0126] The shape and structure of the target can be better understood through the inner and outer boundary point sets, and the boundary point sets can be used as important feature information to improve the classification and segmentation capabilities of the model.

[0127] A700, normalize the three-dimensional coordinates and time of the first array, second array, third array, fourth array, and fifth array.

[0128] Specifically, normalize the three-dimensional coordinates of the first array, second array, third array, fourth array, and fifth array, that is, calculate the maximum values in the x, y, and z directions, perform normalization, and then normalize the time t of the time frame.

[0129] Normalization can eliminate the dimensional differences brought by different coordinates and time scales, making the data more consistent. Normalized data can improve the training speed and stability of the model and reduce the risk of overfitting.

[0130] A800, randomly sample the first array, second array, third array, fourth array, and fifth array after normalization to obtain the first sampled array, second sampled array, third sampled array, fourth sampled array, and fifth sampled array;

[0131] The first sampled array, second sampled array, third sampled array, fourth sampled array, and fifth sampled array form a voxel coordinate point cloud set.

[0132] The specific method of random sampling includes: 1) First, sample in the time dimension: retain the key frames and randomly select some of the remaining frames; 2) Then, sample in the space dimension: specifically, for the data in the first array, second array, and third array, randomly sample points to a fixed number. For the fourth array and fifth array, since there is little data within the frame, no sampling is performed.

[0133] Furthermore, for the selection of time frames, retain the ED frame and the adjacent frame, and the ES frame and the adjacent frame. For the remaining frames, randomly select 7 frames each time an iteration is performed.

[0134] In addition, since the heart beating is a cyclic process, therefore, for step A400, step A500, and step A600, add one more frame at the end, and the data is the same as the first frame.

[0135] Through the above preprocessing steps, the mask image information can be effectively converted into a voxel coordinate point cloud set. This process not only improves the efficiency and accuracy of data processing but also enhances the multi-dimensional feature representation ability of the data, providing a solid foundation for subsequent three-dimensional analysis and machine learning tasks.

[0136] Refer to Figure 4 , the method for establishing the target neural network model specifically includes:

[0137] S210, establish an initial model, where the initial model includes an implicit neural representation with a periodic activation function.

[0138] Among them, the initial model includes five hidden layers with a dimension of 128, the feature dimension of the network input is 4, the output dimension is 1, and the spherical initialization parameter is [1.6, 1.0].

[0139] The implicit neural representation parameterizes the signal as a continuous function, mapping the domain of the signal to the value of the attribute on this coordinate.

[0140] S220, initialize the weights of the initial model using a multi-frequency geometric initialization method.

[0141] The multi-frequency geometric initialization method can accelerate the convergence speed of the model and reduce the training time by initializing the weights at different frequencies; appropriate weight initialization can avoid the problems of gradient disappearance and gradient explosion and improve the training stability of the model.

[0142] S230, replace the linear layer in the initialized initial model with a quadratic layer to obtain the target neural network model.

[0143] The quadratic layer can introduce more non-linear characteristics and enhance the non-linear fitting ability of the model; by increasing the non-linearity, the model can better fit complex data distributions and improve the prediction accuracy and generalization ability of the model.

[0144] In this embodiment, the periodic activation function and the implicit neural representation enhance the model's expression ability for periodicity and complex geometric shapes; the multi-frequency geometric initialization method can accelerate the convergence speed of the model and reduce the training time; appropriate weight initialization can improve the training stability of the model and avoid gradient problems; replacing the linear layer with a quadratic layer can increase the non-linear characteristics of the model and improve the non-linear fitting ability and prediction accuracy of the model.

[0145] Refer to Figure 5 , the method for constructing a preset loss function based on a voxel coordinate point cloud set specifically includes:

[0146] S310, determine a first loss function based on a first sampling array.

[0147] The first loss function is L eik :

[0148] Among them, represents the gradient of the function f at the point x i in the first array, ||·||2 represents the 2-norm of the gradient, N grid is the number of sample points in the first array, x iis the coordinate of the i-th point in the first array.

[0149] Furthermore, the loss function L eik is used to constrain the norm of the gradient so that its modulus is close to 1, ensuring that the modulus of the gradient at each point of the generated field is close to 1, thereby keeping the "flow" of the field smooth and avoiding mutations and discontinuities.

[0150] S320. Determine the second loss function based on the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array.

[0151] The second loss function is L mask :

[0152]

[0153] where f in (x i ) is the predicted value of the signed distance function of the point in the second sampling array or the fourth sampling array, and f out (x i ) is the predicted value of the signed distance function of the point in the third sampling array or the fifth sampling array, N bou is the number of points in the fourth sampling array or the fifth sampling array, and N mask is the number of points in the second sampling array or the third sampling array, and λ bou , λ mask are the corresponding coefficients.

[0154] Preferably, the number of points in the second sampling array is equal to the number of points in the third sampling array, and the number of points in the fourth sampling array is equal to the number of points in the fifth sampling array.

[0155] The loss function L mask is used to ensure the sign constraint between points and boundaries, ensuring that the values of internal points are negative and the values of external points are positive, ensuring the sign consistency of the generated field in the boundary and internal regions, and guaranteeing the continuity of the field and the differences between regions.

[0156] S330. Determine the third loss function based on the first sampling array.

[0157] The third loss function is L hes : where H f (x i ) represents the Hessian matrix of the function f at the i-th point in the first sampling array, and ||·|| F represents the Frobenius norm.

[0158] The loss function L hesUsed to constrain the modulus (Frobenius norm) of the second derivative to ensure that the second-order variation of the generated field is small. By constraining the second derivative, it is ensured that the generated field is locally smooth and does not change violently, which helps to generate more natural shapes.

[0159] S340, determine the preset loss function based on the first loss function, the second loss function, and the third loss function.

[0160] The preset loss function is L: L = λ0L eik + λ1L mask + λ2L hes ; where λ0, λ1, and λ2 are the weights of the L eik loss term, L mask loss term, L hes loss term, respectively.

[0161] Furthermore, a training method for training the target neural network model based on the preset loss function specifically includes:

[0162] B100, at the beginning of training, 100 ≤ λ2 ≤ 200, 5000 ≤ λ0 ≤ 10000, 500 ≤ λ1 ≤ 10000.

[0163] During the training process, λ2 is the annealing factor, which is 100 at the beginning of training. By increasing the loss function with a smoothing effect, it is used to promote the network to converge quickly and robustly.

[0164] B200, in the middle of training, λ2 gradually decreases to 1 according to the first preset interval, and λ0 and λ1 remain unchanged;

[0165] where the first preset interval is the quotient of the first difference and the number of iterations in the middle of training, and the first difference is the difference between λ2 at the beginning of training and 1.

[0166] B300, at the end of training, λ2 gradually decreases to 0 according to the second preset interval, and λ0 and λ1 remain unchanged;

[0167] where the second preset interval is the quotient of 1 and the number of iterations at the end of training.

[0168] It gradually decreases to 0 at the end of training to avoid affecting the fitting effect of the image data.

[0169] Among them, the number of iterations at the beginning of training is 20% of the total number of iterations, the number of iterations in the middle of training is 40% of the total number of iterations, and the number of iterations at the end of training is 40% of the total number of iterations.

[0170] During the training process, the optimizer preferably uses Adam, and the learning rate is preferably 5 × 10 -5, preferably, the CosineAnnealingLR learning rate scheduler is adopted, and a total of 4000 epochs are iterated.

[0171] Refer to Figure 6 , the method for obtaining the signed distance function value specifically includes:

[0172] C100, create a grid coordinate matrix based on a preset numerical range, a preset quantity of spatial dimensions, and a preset quantity of time dimensions.

[0173] Among them, the preset numerical range is the range from 0 to 1.

[0174] The preset quantity of spatial dimensions ≥ 128; the preset quantity of time dimensions ≥ the number of time frames in the masked image.

[0175] By creating a grid coordinate matrix, the spatial and time dimensions are standardized, ensuring that all input data is processed under the same standard. The grid coordinate matrix facilitates efficient parallel computing and indexing, improving the overall computing efficiency. The preset numerical range is from 0 to 1, ensuring that all input data is within a consistent numerical range and avoiding errors caused by inconsistent numerical ranges.

[0176] C200, expand the coordinates of the grid coordinate matrix to obtain a uniform point set.

[0177] Expanding the grid coordinate matrix can facilitate subsequent array operations and data processing, simplifying the computational complexity; the expanded uniform point set ensures that the distribution of each point is uniform, which helps to improve the training effect and prediction accuracy of the model.

[0178] Among them, the uniform point set is a two-dimensional array in the form of N test ×(t,x,y,z), and at this time, the resolutions of the time t and the spatial coordinates (x,y,z) can be considered as arbitrary values and are no longer restricted by the shape of the original image.

[0179] C300, input the uniform point set into the first neural network model to obtain the signed distance function value of each point.

[0180] By obtaining the signed distance function value of each point through the first neural network model, the distance and its direction from the point to the object surface can be accurately represented, which is applicable to three-dimensional reconstruction and shape representation tasks.

[0181] Among them, the method for obtaining the uniform point set specifically includes: creating a grid coordinate matrix within the range from 0 to 1, which means that all coordinate values are within this range; generating a uniform point set according to the required resolution, and the resolution can be set arbitrarily and is no longer restricted by the shape of the original image.

[0182] Assume that the spatial coordinate is (x,y,z), the time coordinate is t, and the resolution is Nx , N y , N z , N t 。

[0183] Then the point sets on each coordinate dimension can be expressed as:

[0184]

[0185] Expand these coordinates and combine them into a two-dimensional array in the form of (i, j, k, l).

[0186] Refer to Figure 7 , and based on the signed distance function values of all frames, obtain the dynamic three-dimensional shape of the heart, including:

[0187] D100: Extract the signed distance function values of the same frames according to the time frames of the uniform point set, and denote them as the target time frame set.

[0188] By extracting the signed distance function values of the same time frames, it ensures that all data are aligned in the time dimension, avoiding errors caused by time misalignment; grouping the data of all frames according to the time frames simplifies the subsequent processing steps and improves the calculation efficiency.

[0189] D200: Extract the zero isosurface from the target time frame set to obtain the dynamic three-dimensional shape of the heart.

[0190] Specifically, use the marching cubes algorithm to extract the zero isosurface from the target time frame set to obtain the dynamic three-dimensional shape of the heart, that is, obtain the dynamic three-dimensional shape composed of the triangular meshes corresponding to all frames.

[0191] The zero isosurface (i.e., the surface where the signed distance function value is zero) represents the exact position of the object surface and can accurately reconstruct the three-dimensional shape of the heart; by extracting the zero isosurfaces at different time frames, the dynamic three-dimensional shape of the heart at different time points can be obtained, showing the dynamic changes of the heart; the dynamic three-dimensional shape of the heart can be used for high-quality visualization to help doctors and researchers more intuitively understand the structure and function of the heart.

[0192] Refer to Figure 8 , the video acquisition method further includes:

[0193] E100: Based on the extracted zero isosurface, obtain the triangular mesh of each frame.

[0194] Among them, the triangular mesh is an efficient geometric representation method suitable for the reconstruction and rendering of three-dimensional shapes; the triangular mesh extracted through the zero isosurface can accurately capture the three-dimensional shape of the heart and improve the accuracy of shape representation.

[0195] E200 renders all triangular meshes to obtain sub-rendered images.

[0196] Specifically, the triangular network of each frame can be fed into the rendering software Blender for rendering to obtain sub-rendered images.

[0197] The rendering process can generate high-quality images, showing the three-dimensional shape and details of the heart, facilitating visual analysis by doctors and researchers; rendering can introduce visual effects such as lighting and shadows to enhance the realism and visual effects of the images.

[0198] E300 merges all sub-rendered images according to time frames to generate a video of the dynamic three-dimensional shape of the heart.

[0199] By merging the sub-rendered images of all frames, a video of the dynamic three-dimensional shape of the heart at different time points can be generated, showing the dynamic changes and movement trajectories of the heart; the dynamic video can more intuitively display the function and structure of the heart, helping doctors to make more accurate diagnoses and evaluations; the dynamic video can be used for doctor-patient communication and medical education to improve patient understanding and doctor teaching effects.

[0200] In the solution of the method for constructing the dynamic three-dimensional shape of the heart disclosed in this application, implicit neural representation is a method of representing complex geometric shapes through a neural network. In this method, implicit neural representation learns a continuous function to map points in space to their signed distance function values. This means that for any given point in space, the neural network can output the distance between this point and the heart surface, whether it is inside or outside; in the preprocessing stage, the original heart image is segmented and converted into a cloud of voxel coordinate points, and these clouds of points contain the geometric information of the heart, including shape, size, and movement over time. Through this information, the neural network learns the geometric features of the heart during the training process; during the training process, the target neural network model is trained through a preset loss function obtained from the cloud of voxel coordinate points. The loss function ensures that the signed distance function values output by the model can accurately reflect the geometric shape and dynamic changes of the heart. Therefore, the trained model already has the ability to infer the dynamic three-dimensional shape of the heart from the input point set (uniform point set); the uniform point set is generated within a preset numerical range, which covers the spatial area that the heart may occupy. These point sets are input into the trained model, and the model will output the signed distance function value for each point, and these values reflect the distance between the heart surface and the point, thus implicitly defining the shape of the heart; finally, by extracting the zero isosurface (i.e., the surface composed of points with a signed distance function value of zero), the dynamic three-dimensional shape of the heart can be obtained because in the implicit representation, the zero isosurface exactly corresponds to the geometric surface of the heart.

[0201] In this application, the uniform point set itself does not directly contain the image information of the heart, but through the implicit neural representation and the training of the model, it actually inherits the geometric features of the heart. Therefore, when the uniform point set is input into the trained model, the model can utilize the geometric features it has learned to accurately output the signed distance function value of each point, thereby restoring the dynamic three-dimensional shape of the heart.

[0202] In a second aspect, the embodiments of the present disclosure also provide a system for constructing a dynamic three-dimensional shape of the heart, specifically including the following:

[0203] A segmentation module for segmenting the original heart image to obtain mask image information;

[0204] A preprocessing module for preprocessing the mask image information to obtain a cloud of voxel coordinate points;

[0205] A model establishment module for establishing a target neural network model;

[0206] A training module for training the target neural network model based on a preset loss function, and denoting the trained model as the first neural network model;

[0207] A signed distance function value acquisition module for obtaining the signed distance function value of each frame based on the first neural network model;

[0208] A construction module for obtaining the dynamic three-dimensional shape of the heart based on the signed distance function values of all frames.

[0209] It should be noted that the solutions in the method for constructing a dynamic three-dimensional shape of the heart disclosed in the first aspect of this application are all applicable to the system for constructing a dynamic three-dimensional shape of the heart disclosed in the second aspect, so they will not be elaborated here.

[0210] For the detailed description of this embodiment, reference can be made to the corresponding descriptions in the foregoing embodiments, and they will not be elaborated here.

[0211] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above-disclosed specific details are only for the purpose of illustration and easy understanding, rather than limitations, and the above details do not limit the present disclosure to necessarily adopt the above specific details to implement.

[0212] In this disclosure, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any way. Words such as "including", "comprising", "having", etc. are open-ended words, meaning "including but not limited to", and can be used interchangeably with each other. The words "or" and "and" used herein refer to the word "and / or" and can be used interchangeably with it, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to" and can be used interchangeably with it.

[0213] In addition, as used herein, "or" in a listing of items beginning with "at least one" indicates a disjunctive listing, so that for example a listing of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Further, the term "exemplary" does not mean that the examples described are preferred or better than other examples.

[0214] It should also be noted that in the systems and methods of this disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of this disclosure.

[0215] equivalent solutions.

[0216] Various changes, substitutions, and alterations to the technologies described herein can be made without departing from the teachings defined by the appended claims. In addition, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and acts described above. Current or later-developed processes, machines, manufactures, compositions of events, means, methods, or acts that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Accordingly, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or acts within their scope.

[0217] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

[0218] The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the present disclosure to the form disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for constructing a dynamic three-dimensional shape of the heart, characterized in that Including: Segment the original cardiac image to obtain mask image information; Preprocess the mask image information to obtain a set of voxel coordinate points; Establish a target neural network model based on implicit neural representation; Construct a preset loss function based on the set of voxel coordinate points; Train the target neural network model based on the preset loss function, and denote the trained model as the first neural network model; Obtain the signed distance function value for each frame based on the first neural network model; Obtain the dynamic three-dimensional shape of the heart based on the signed distance function values of all frames; The preprocessing of the mask image information to obtain a set of voxel coordinate points includes: Process each frame of the three-dimensional original image based on a cropping strategy to obtain a three-dimensional cropped image for each frame; Merge the three-dimensional cropped images of all frames to obtain a reconstructed cropped image; Create a grid coordinate matrix based on the size of the reconstructed cropped image and a preset resolution; Unfold the coordinates of the grid coordinate matrix to obtain a first array; Obtain the set of target pixel points and the set of background pixel points in the reconstructed cropped image; Perform conversion processing on the set of target pixel points and the set of background pixel points based on the voxel dimension information to obtain a second array and a third array; Obtain the set of inner boundary points and the set of outer boundary points of the set of target pixel points; Perform conversion processing on the set of inner boundary points and the set of outer boundary points based on the voxel dimension information to obtain a fourth array and a fifth array; Normalize the three-dimensional coordinates and time of the first array, the second array, the third array, the fourth array, and the fifth array; Randomly sample the first array, the second array, the third array, the fourth array, and the fifth array after normalization processing to obtain a first sampled array, a second sampled array, a third sampled array, a fourth sampled array, and a fifth sampled array; The first sampled array, the second sampled array, the third sampled array, the fourth sampled array, and the fifth sampled array constitute the set of voxel coordinate points; The constructing of the preset loss function based on the set of voxel coordinate points includes: determining a first loss function based on the first sampled array; determining a second loss function based on the second sampled array, the third sampled array, the fourth sampled array, and the fifth sampled array; determining a third loss function based on the first sampled array; determining the preset loss function based on the first loss function, the second loss function, and the third loss function.

2. The method for constructing a dynamic three-dimensional shape of the heart according to claim 1, wherein The segmenting of the original cardiac image to obtain mask image information includes: segmenting the original cardiac image based on a deep learning segmentation algorithm to obtain a mask image; The mask image includes several frames of three-dimensional original images and voxel dimension information.

3. The method for constructing a dynamic three-dimensional shape of the heart according to claim 1, wherein, The processing of each frame of the three-dimensional original image based on a cropping strategy to obtain a three-dimensional cropped image for each frame includes: Determine the cardiac reconstruction sub-structure; Traverse the three-dimensional original images of all time frames to determine the minimum coordinate information and the maximum coordinate information of the cardiac reconstruction sub-structure; If the target bounding box formed by the minimum coordinate information and the maximum coordinate information is smaller than the preset image size, expand the target bounding box formed by the minimum coordinate information and the maximum coordinate information to obtain an expanded bounding box, and crop the three-dimensional original image based on the expanded bounding box to obtain the three-dimensional cropped image; If the target bounding box formed by the minimum coordinate information and the maximum coordinate information is not smaller than the preset image size, crop the three-dimensional original image based on the target bounding box, and fill the background pixels of the cropped image to obtain the three-dimensional cropped image.

4. The method for constructing a dynamic three-dimensional shape of the heart according to claim 1, characterized in that, The establishing of the target neural network model based on the implicit neural representation includes: Establishing an initial model, where the initial model includes an implicit neural representation with a periodic activation function; Initializing the weights of the initial model using a multi-frequency geometric initialization method; Replacing the linear layer in the initialized initial model with a quadratic layer to obtain the target neural network model.

5. The method for constructing a dynamic three-dimensional shape of the heart according to claim 1, characterized in that, The first loss function is : ; Among them, represents the function at the midpoint of the first sampling array the gradient at this point, represents the 2-norm of the gradient, is the number of sample points in the first sampling array, is the coordinate of the i-th point in the first sampling array; The second loss function is : ； wherein, is the predicted value of the signed distance function of the points in the second sampling array or the fourth sampling array, is the predicted value of the signed distance function of the points in the third sampling array or the fifth sampling array, is the number of points in the fourth sampling array or the fifth sampling array, is the number of points in the second sampling array or the third sampling array, and are the corresponding coefficients; The third loss function is : ; Among them, represents the function at the i -th point in the first sampling array, is the Frobenius norm; The preset loss function is : ; Among them, , and are respectively the loss term, the loss term, the weights of the loss terms.

6. The method for constructing a dynamic three-dimensional shape of the heart according to claim 1, wherein The obtaining of the signed distance function value for each frame based on the first neural network model includes: Creating a grid coordinate matrix based on a preset numerical range, a preset quantity of spatial dimensions, and a preset quantity of time dimensions; Wherein, the preset numerical range is a range from 0 to 1; the preset quantity of spatial dimensions ≥ 128; the preset quantity of time dimensions ≥ the number of time frames in the masked image; Unfolding the coordinates of the grid coordinate matrix to obtain a uniform point set; Inputting the uniform point set into the first neural network model to obtain the signed distance function value of each point.

7. The method for constructing a dynamic three-dimensional shape of the heart according to claim 6, characterized in that The obtaining of the dynamic three-dimensional shape of the heart based on the signed distance function values of all frames includes: Extracting the signed distance function values of the same frame according to the time frames of the uniform point set, denoted as the target time frame set; Extracting the zero isosurface from the target time frame set to obtain the dynamic three-dimensional shape of the heart.

8. The method for constructing a dynamic three-dimensional shape of the heart according to claim 7, wherein After the extraction of the zero isosurface, it further includes: Obtaining a triangular mesh for each frame based on the extracted zero isosurface; Rendering all the triangular meshes to obtain rendered sub-images; Merging all the rendered sub-images according to the time frames to generate a video of the dynamic three-dimensional shape of the heart.

Images