Heart dynamic three-dimensional shape construction method

Abstract

The invention discloses a heart dynamic three-dimensional shape construction method, which comprises the following steps: segmenting a heart original image to obtain mask image information; preprocessing the mask image information to obtain a voxel coordinate point cloud set; constructing a preset loss function based on the voxel coordinate point cloud set; training the established target neural network model based on a preset loss function, and recording the trained model as a first neural network model; obtaining a signed distance function value of each frame based on the first neural network model; and obtaining the dynamic three-dimensional shape of the heart based on the signed distance function values of all the frames. According to the method, the dynamic three-dimensional shape of the heart can be quickly and accurately reconstructed from a single heart image, the natural ventricle shape can be reconstructed from an out-of-plane low-resolution image without a data set, the calculation speed is high, the fitting accuracy is high, and the proposed method has good generalization and is suitable for large-scale popularization and application. The method can be suitable for heart movie magnetic resonance images of the whole time sequence, various pathologies and various resolutions.

Description

Technical Field

[0001] The present disclosure relates to the field of medical technology, and in particular to a method for constructing a dynamic three-dimensional shape of a heart. Background Art

[0002] Among the existing cardiac magnetic resonance imaging technologies, cine imaging is the most important imaging method, which can visualize the structure of the heart chambers and detect ventricular motion function. Compared with the traditional method of only referring to two-dimensional image information, the construction of a three-dimensional model of the heart can provide a more accurate and comprehensive understanding of the structure and function of the heart. However, the short-axis images obtained by fast-scanning cine imaging usually show anisotropic spatial resolution, resulting in low resolution between slices, making it challenging to accurately and efficiently reconstruct the shape of the heart.

[0003] So far, medical 3D reconstruction technology has achieved some research results, mainly including: 1) Methods based on statistical shape models, however, this method requires a large amount of grid data as input, depends on data distribution and cannot completely fit the contour corresponding to the image; 2) Deep learning methods based on template grid deformation are only applicable to isotropic data and specific populations; Methods based on high-resolution segmentation to obtain the shape of the heart do not utilize geometric prior knowledge and often require a large number of nuclear magnetic resonance images, and there is no research on the construction of the dynamic 3D shape of the heart. Summary of the invention

[0004] In view of this, an embodiment of the present disclosure provides a method for constructing a dynamic three-dimensional shape of a heart, which can solve the problems existing in the prior art such as the inability to accurately and efficiently reconstruct the shape of the heart.

[0006] In a first aspect, the present disclosure provides a method for constructing a dynamic three-dimensional shape of a heart, which specifically includes the following contents:

[0007] Segment the original heart image to obtain mask image information;

[0008] Preprocessing the mask image information to obtain a voxel coordinate point cloud set;

[0009] Establish a target neural network model;

[0010] Constructing a preset loss function based on the voxel coordinate point cloud set;

[0011] Training the target neural network model based on the preset loss function, and recording the trained model as a first neural network model;

[0012] Obtaining a signed distance function value for each frame based on the first neural network model;

[0013] Based on the signed distance function values ​​of all frames, the dynamic three-dimensional shape of the heart is obtained.

[0014] Optionally, segmenting the original heart image to obtain mask image information includes: segmenting the original heart image based on a deep learning segmentation algorithm to obtain a mask image;

[0015] The mask image includes several frames of three-dimensional original images and voxel dimension information.

[0016] Optionally, preprocessing the mask image information to obtain a voxel coordinate point cloud set includes:

[0017] Processing each frame of the three-dimensional original image based on the cropping strategy to obtain a three-dimensional cropped image of each frame;

[0018] Merging the three-dimensional cropped images of all frames to obtain a reconstructed cropped image;

[0019] Creating a grid coordinate matrix based on the size of the reconstructed cropped image and a preset resolution;

[0020] Expand the coordinates of the grid coordinate matrix to obtain a first array;

[0021] Obtaining a target pixel point set and a background pixel point set in the reconstructed cropped image;

[0022] Based on the voxel dimension information, the target pixel point set and the background pixel point set are converted to obtain a second array and a third array;

[0023] Obtaining an inner boundary point set and an outer boundary point set of the target pixel point set;

[0024] Based on the voxel dimension information, the inner boundary point set and the outer boundary point set are converted to obtain a fourth array and a fifth array;

[0025] Normalizing the three-dimensional coordinates and time of the first array, the second array, the third array, the fourth array, and the fifth array;

[0026] Randomly sampling the first array, the second array, the third array, the fourth array, and the fifth array after normalization to obtain a first sampling array, the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array;

[0027] The first sampling array, the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array constitute the voxel coordinate point cloud set.

[0028] Optionally, the processing of each frame of the 3D original image based on the cropping strategy to obtain each frame of the 3D cropped image includes:

[0029] Identify cardiac reconstructive substructures;

[0030] Traversing the three-dimensional original images of all time frames to determine minimum coordinate information and maximum coordinate information of the cardiac reconstruction substructure;

[0031] If the target bounding box formed by the minimum coordinate information and the maximum coordinate information is smaller than a preset image size, expanding the target bounding box formed by the minimum coordinate information and the maximum coordinate information to obtain an extended bounding box, and cropping the 3D original image based on the extended bounding box to obtain the 3D cropped image;

[0032] If the target bounding box formed by the minimum coordinate information and the maximum coordinate information is not smaller than the preset image size, the three-dimensional original image is cropped based on the target bounding box, and the cropped image is filled with background pixels to obtain the three-dimensional cropped image.

[0033] Optionally, establishing a target neural network model based on implicit neural representation includes:

[0034] establishing an initial model comprising an implicit neural representation with a periodic activation function;

[0035] Initializing the weights of the initial model using a multi-frequency geometric initialization method;

[0036] The linear layer in the initialized initial model is replaced with a quadratic layer to obtain a target neural network model.

[0037] Optionally, constructing a preset loss function based on the voxel coordinate point cloud set includes:

[0038] Determining a first loss function based on the first sampling array;

[0039] Determine a second loss function based on the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array;

[0040] Determining a third loss function based on the first sampling array;

[0041] The preset loss function is determined based on the first loss function, the second loss function, and the third loss function.

[0042] Optionally, the first loss function is L eik :

[0043]

[0044] in, represents the function f at the midpoint x of the first array i The gradient at , ||·||2 represents the 2-norm of the gradient, N grid is the number of sample points in the first array, x i is the coordinate of the i-th point in the first array;

[0045] The second loss function is L mask :

[0046]

[0047] Among them, f in (x i ) is the predicted value of the signed distance function of a point in the second sampling array or in the fourth sampling array, f out (x i ) is the predicted value of the signed distance function of a point in the third sampling array or the fifth sampling array, N bou is the number of points in the fourth sampling array or the fifth sampling array, N mask is the number of points in the second sampling array or the third sampling array;

[0048] The third loss function is L hes : Among them, H f (x i ) represents the Hessian matrix of the function f at the i-th point in the first sampling array, ||·|| F represents the Frobenius norm;

[0049] The preset loss function is L: L = f0L eik +λ1L mask +λ2L hes ; where λ0, λ1 and λ2 are L eik Loss term, L mask Loss term, L hes The weight of the loss term.

[0050] Optionally, obtaining a signed distance function value for each frame based on the first neural network model includes:

[0051] Based on a preset value range, a preset amount of a spatial dimension, and a preset amount of a temporal dimension, a grid coordinate matrix is ​​created; the preset value range is a range from 0 to 1; the preset amount of the spatial dimension is ≥ 128, and the preset amount of the temporal dimension is ≥ the number of time frames in the mask image;

[0052] Expanding the coordinates of the grid coordinate matrix to obtain a uniform point set;

[0053] The uniform point set is input into the first neural network model to obtain a signed distance function value of each point.

[0054] Optionally, obtaining the dynamic three-dimensional shape of the heart based on the signed distance function values ​​of all frames comprises:

[0056] According to the time frame of the uniform point set, the signed distance function value of the same frame is extracted and recorded as a target time frame set;

[0057] A zero isosurface is extracted from the target time frame set to obtain a dynamic three-dimensional shape of the heart.

[0058] Optionally, after the zero isosurface is extracted, the method further includes:

[0059] Based on the extracted zero isosurface, obtaining a triangular mesh of each frame;

[0060] Rendering the triangular meshes to obtain rendered sub-images;

[0061] All the rendered sub-images are merged according to time frames to generate a video of the dynamic three-dimensional shape of the heart.

[0062] In a second aspect, the present disclosure also provides a system for constructing a dynamic three-dimensional shape of a heart, which specifically includes the following contents:

[0063] A segmentation module is used to segment the original heart image and obtain mask image information;

[0064] A preprocessing module, used for preprocessing the mask image information to obtain a voxel coordinate point cloud set;

[0065] A model building module, used to build a target neural network model;

[0066] A training module, used for training the target neural network model based on a preset loss function, and recording the trained model as a first neural network model;

[0067] A signed distance function value acquisition module, used to obtain a signed distance function value of each frame based on the first neural network model;

[0068] A construction module is provided for obtaining a dynamic three-dimensional shape of the heart based on the signed distance function values ​​of all frames.

[0069] The method for constructing the dynamic three-dimensional shape of the heart disclosed in the present application can effectively avoid background noise by obtaining mask image information through image segmentation, so that subsequent processing can be more focused on the heart area; by preprocessing the mask image information, a voxel coordinate point cloud set is obtained, so that the subsequent neural network model can efficiently process three-dimensional data, and the flexibility and accuracy of the processing are enhanced; the target neural network model is established and trained, and the complex representation of the heart shape can be learned and optimized, and the signed distance function value of each frame image is calculated by using the trained first neural network model, which can provide an accurate surface representation and help to construct an accurate three-dimensional shape. By integrating the signed distance function values ​​of all frames, the dynamic three-dimensional shape of the heart can be obtained. This method can not only reflect the static structure of the heart, but also capture its movement and changes, and provide a more comprehensive and detailed analysis. The method for constructing the dynamic three-dimensional shape of the heart disclosed in the present application can quickly and accurately reconstruct the dynamic three-dimensional shape of the heart from a single cardiac movie magnetic resonance image, and can reconstruct the natural ventricular shape from an out-of-plane low-resolution image without a data set, with fast calculation speed and high fitting accuracy. The proposed method has good generalization and can adapt to the entire time series, various pathologies, and cardiac movie magnetic resonance images of various resolutions.

[0070] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0072] Figure 1 A schematic flow chart of a method for constructing a dynamic three-dimensional shape of a heart provided in an embodiment of the present disclosure.

[0073] Figure 2 A schematic flow chart of a method for acquiring a voxel coordinate point cloud set provided in an embodiment of the present disclosure.

[0074] Figure 3 A schematic flow chart of a method for acquiring a three-dimensional cropped image provided in an embodiment of the present disclosure.

[0075] Figure 4 A flowchart of a method for establishing a target neural network model provided in an embodiment of the present disclosure.

[0076] Figure 5 A flowchart of a method for constructing a preset loss function based on a voxel coordinate point cloud set provided in an embodiment of the present disclosure.

[0077] Figure 6 A schematic diagram of a flow chart of a method for obtaining a signed distance function value provided in an embodiment of the present disclosure.

[0078] Figure 7 A schematic flow chart of a method for obtaining a dynamic three-dimensional shape of a heart based on signed distance function values ​​of all frames provided in an embodiment of the present disclosure.

[0079] Figure 8 A flowchart of a video acquisition method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0080] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0081] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0082] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0083] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0084] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0085] Reference Figure 1 The present application discloses a method for constructing a dynamic three-dimensional shape of a heart, comprising:

[0086] S100, segmenting the original heart image to obtain mask image information;

[0087] The mask image information is preprocessed to obtain a voxel coordinate point cloud set.

[0088] Among them, segmenting the original heart image specifically includes: segmenting the original heart image based on a deep learning segmentation algorithm to obtain a mask image marked with labels; the mask image includes several frames of three-dimensional original images and voxel dimension information.

[0089] The original cardiac image includes any one of a CT image and a magnetic resonance image; further, the original cardiac image is preferably a cardiac movie magnetic resonance image.

[0090] Furthermore, the format of the 3D original image is NIFTI.

[0091] S200, establishing a target neural network model.

[0092] S300, constructing a preset loss function based on the voxel coordinate point cloud set.

[0093] S400, training the target neural network model based on a preset loss function, and recording the trained model as a first neural network model.

[0094] Through training, the model can adaptively learn the characteristics of the heart shape and improve reconstruction accuracy.

[0095] S500, obtaining a signed distance function value of each frame based on the first neural network model;

[0096] Based on the signed distance function values ​​of all frames, the dynamic three-dimensional shape of the heart is obtained.

[0097] The method for constructing the dynamic three-dimensional shape of the heart disclosed in the present application can effectively avoid background noise by obtaining mask image information through image segmentation, so that subsequent processing can be more focused on the heart area; by preprocessing the mask image information, a voxel coordinate point cloud set is obtained, so that the subsequent neural network model can efficiently process three-dimensional data, and the flexibility and accuracy of the processing are enhanced; the target neural network model is established and trained, and the complex representation of the heart shape can be learned and optimized, and the signed distance function value of each frame image is calculated by using the trained first neural network model, which can provide an accurate surface representation and help to construct an accurate three-dimensional shape. By integrating the signed distance function values ​​of all frames, the dynamic three-dimensional shape of the heart can be obtained. This method can not only reflect the static structure of the heart, but also capture its movement and changes, and provide a more comprehensive and detailed analysis. The method for constructing the dynamic three-dimensional shape of the heart disclosed in the present application can quickly and accurately reconstruct the dynamic three-dimensional shape of the heart from a single cardiac movie magnetic resonance image, and can reconstruct the natural ventricular shape from an out-of-plane low-resolution image without a data set, with fast calculation speed and high fitting accuracy. The proposed method has good generalization and can adapt to the entire time series, various pathologies, and cardiac movie magnetic resonance images of various resolutions.

[0098] The dynamic three-dimensional heart model generated by the heart dynamic three-dimensional shape construction method disclosed in this application can be used for clinical diagnosis and treatment planning, helping doctors to better understand the structure and function of the heart and improve the accuracy of diagnosis and treatment effects; the entire process has a high degree of automation, which reduces human intervention and errors and improves processing efficiency, which is particularly important for the processing of large amounts of medical imaging data and has broad application prospects and important clinical value.

[0099] Reference Figure 2 , the method for obtaining the voxel coordinate point cloud set specifically includes:

[0100] A100 processes each frame of the 3D original image based on the cropping strategy to obtain a 3D cropped image of each frame.

[0101] Specific reference Figure 3 , a method for obtaining a three-dimensional cropped image, specifically comprising:

[0102] A110, Identification of cardiac reconstruction substructures.

[0103] Specifically, the heart substructure to be reconstructed (ie, the heart reconstruction substructure) may be labeled as 1, and the remaining labels may be 0; determining the heart reconstruction substructure ensures that the cropping area is concentrated on the key part of the heart, thereby improving the accuracy of heart reconstruction.

[0104] A120, traverse the three-dimensional original images of all time frames to determine the minimum coordinate information and the maximum coordinate information of the heart reconstruction substructure.

[0106] Specifically, the global minimum and maximum x, y, and z coordinate ranges of the pixels with the label 1 are found, that is, the minimum coordinate information and the maximum coordinate information of the heart reconstruction substructure.

[0107] A130, if the target bounding box formed by the minimum coordinate information and the maximum coordinate information is smaller than the preset image size, the target bounding box formed by the minimum coordinate information and the maximum coordinate information is expanded to obtain an extended bounding box, and the three-dimensional original image is cropped based on the extended bounding box to obtain a three-dimensional cropped image.

[0108] Among them, expanding the target bounding box composed of the minimum coordinate information and the maximum coordinate information includes: adding a pixel point on the basis of the minimum coordinate information and the maximum coordinate information, that is, expanding the minimum coordinate information inward by one pixel point, and expanding the maximum coordinate information outward by one pixel point. This can avoid the boundary being too close to the heart reconstruction substructure during cropping, effectively prevent the loss of part of the structure's information, and ensure that the cropped image contains sufficient background information.

[0109] In this step, the target bounding box is judged and expanded to ensure that the cropped area can cover the cardiac substructure without losing important information due to the bounding box being too small. This adjustment makes the cropped image more stable and applicable.

[0110] A140, if the target bounding box formed by the minimum coordinate information and the maximum coordinate information is not smaller than the preset image size, the three-dimensional original image is cropped based on the target bounding box, and the cropped image is filled with background pixels to obtain a three-dimensional cropped image.

[0111] The filling of background pixels on the cropped image includes: filling a circle of pixels with a label of 0 around the cropped image.

[0112] For the case where the target bounding box is not smaller than the preset image size, the cropped image is padded with background pixels to ensure the consistency of the size and format of the cropped image, which is convenient for subsequent image processing and analysis.

[0113] Whether cropping by extending the bounding box or filling the background pixels, a three-dimensional cropped image of the preset size can be obtained in the end, ensuring the consistency and comparability of the output image; this strategy can adapt to cardiac substructures of different sizes and shapes, and can obtain suitable cropped images regardless of whether it is a large heart or a small heart, enhancing the versatility and applicability of the algorithm.

[0114] A200, merge the 3D cropped images of all frames to obtain the reconstructed cropped image.

[0115] Merging all frames can generate a continuous three-dimensional image, which is convenient for overall analysis and processing. By merging, more useful information can be retained and the loss of single-frame image information can be avoided.

[0116] A300, creates a grid coordinate matrix based on the size of the reconstructed cropped image and the preset resolution.

[0117] A400, expand the coordinates of the grid coordinate matrix to obtain a first array.

[0118] Among them, the first array is of the form N grid ×(t,x,y,z).

[0119] A500, obtaining a target pixel point set and a background pixel point set in the reconstructed cropped image;

[0120] The target pixel point set and the background pixel point set are converted based on the voxel dimension information to obtain the second array and the third array.

[0121] The target pixel point set is a point set consisting of pixel points corresponding to the heart reconstruction substructure, and the background pixel point set is a point set consisting of all background pixel points.

[0122] The second and third arrays are also in the form of N grid ×(t,x,y,z).

[0123] A600, obtaining the inner boundary point set and the outer boundary point set of the target pixel point set;

[0124] The inner boundary point set and the outer boundary point set are converted based on the voxel dimension information to obtain a fourth array and a fifth array.

[0125] Among them, the fourth array and the fifth array are also in the form of N grid ×(t,x,y,z) two-dimensional array; further, N grid It can be preferably 20000;

[0126] Specifically, the method for obtaining the inner boundary point set and the outer boundary point set includes: first defining a 3×3 2D convolution kernel to detect neighborhood changes, and first performing convolution on each two-dimensional image of the traversed cropped image. If the two-dimensional pixel is 0 but there is at least one pixel with a label of 1 in the neighborhood, then it is an outer boundary point; if the two-dimensional pixel is 1 but the sum of the label values ​​of the neighboring pixels is less than 8, it means that there is at least one pixel with a label of 0, then it is an inner boundary point.

[0127] The shape and structure of the target can be better understood through the inner and outer boundary point sets. The boundary point sets can be used as important feature information to improve the classification and segmentation capabilities of the model.

[0128] A700, normalize the three-dimensional coordinates and time of the first array, the second array, the third array, the fourth array, and the fifth array.

[0129] Specifically, the three-dimensional coordinates of the first array, the second array, the third array, the fourth array, and the fifth array are normalized, that is, the maximum values ​​in the three directions of x, y, and z are calculated and normalized, and then the time t of the time frame is normalized.

[0130] Normalization can eliminate the dimensional differences caused by different coordinates and time scales, making the data more consistent. Normalized data can improve the training speed and stability of the model and reduce the risk of overfitting.

[0131] A800, randomly sampling the normalized first array, second array, third array, fourth array, and fifth array to obtain a first sampling array, a second sampling array, a third sampling array, a fourth sampling array, and a fifth sampling array;

[0132] The first sampling array, the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array constitute a voxel coordinate point cloud set.

[0133] The specific method of random sampling includes: 1) first sampling is required in the time dimension: retain the key frames and randomly select some of the remaining frames; 2) then sampling is performed in the spatial dimension: specifically, for the data in the first array, the second array, and the third array, randomly sample points to a fixed number, and for the fourth array and the fifth array, no sampling is performed because there is little data in the frame.

[0134] Furthermore, in the selection of time frames, the ED frame and the adjacent frame and the ES frame and the adjacent frame are retained, and for the remaining frames, 7 frames are randomly selected in each iteration.

[0135] In addition, since the heart beat is a cyclic process, a frame is added at the end of step A400, step A500 and step A600, and the data is the same as the first frame.

[0136] Through the above preprocessing steps, the mask image information can be effectively converted into a voxel coordinate point cloud set. This process not only improves the efficiency and accuracy of data processing, but also enhances the multi-dimensional feature representation capability of the data, providing a solid foundation for subsequent three-dimensional analysis and machine learning tasks.

[0137] Reference Figure 4 , the method for establishing the target neural network model specifically includes:

[0138] S210, establishing an initial model, the initial model including an implicit neural representation with a periodic activation function.

[0139] The initial model includes five hidden layers with a dimension of 128, the feature dimension of the network input is 4, the output dimension is 1, and the spherical initialization parameters are [1.6, 1.0].

[0140] The implicit neural representation parameterizes the signal as a continuous function that maps the domain of the signal to the value of the attribute at that coordinate.

[0141] S220, initializing the weights of the initial model using a multi-frequency geometric initialization method.

[0142] The multi-frequency geometric initialization method can accelerate the convergence of the model and reduce the training time by initializing weights at different frequencies; appropriate weight initialization can avoid the gradient vanishing and gradient exploding problems and improve the training stability of the model.

[0143] S230, replacing the linear layer in the initialized initial model with a quadratic layer to obtain a target neural network model.

[0144] The secondary layer can introduce more nonlinear characteristics and enhance the nonlinear fitting ability of the model; by adding nonlinearity, the model can better fit complex data distribution and improve the prediction accuracy and generalization ability of the model.

[0145] In this embodiment, the periodic activation function and implicit neural representation enhance the model's ability to express periodic and complex geometric shapes; the multi-frequency geometric initialization method can accelerate the convergence speed of the model and reduce the training time; the appropriate weight initialization can improve the training stability of the model and avoid gradient problems; replacing the linear layer with the quadratic layer can increase the nonlinear characteristics of the model and improve the model's nonlinear fitting ability and prediction accuracy.

[0146] Reference Figure 5 , a method for constructing a preset loss function based on a voxel coordinate point cloud set, specifically including:

[0147] S310, determining a first loss function based on a first sampling array.

[0148] The first loss function is L eik :

[0149] in, Represents the function f at the midpoint x of the first array i The gradient at , ||·||2 represents the 2-norm of the gradient, N grid is the number of sample points in the first array, x i is the coordinate of the i-th point in the first array.

[0150] Furthermore, the loss function L eikIt is used to constrain the norm of the gradient so that its modulus is close to 1, ensuring that the gradient modulus of the generated field at each point is close to 1, thereby keeping the "flow" of the field smooth and avoiding mutations and discontinuities.

[0151] S320, determining a second loss function based on the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array.

[0152] The second loss function is L mask :

[0153]

[0154] Among them, f in (x i ) is the predicted value of the signed distance function of the point in the second sampling array or the fourth sampling array, f out (x i ) is the predicted value of the signed distance function of the point in the third sampling array or the fifth sampling array, N bou is the number of points in the fourth sampling array or the fifth sampling array, N mask is the number of points in the second sampling array or the third sampling array, λ bou , mask is the corresponding coefficient.

[0155] Preferably, the number of points in the second sampling array is equal to the number of points in the third sampling array, and the number of points in the fourth sampling array is equal to the number of points in the fifth sampling array.

[0156] Loss function L mask It is used to ensure the sign constraints of points and boundaries, ensure that the internal point values ​​are negative and the external point values ​​are positive, ensure the sign consistency of the generated field in the boundary and internal areas, and ensure the continuity of the field and the difference between regions.

[0157] S330, determining a third loss function based on the first sampling array.

[0158] The third loss function is L hes : Among them, H f (x i ) represents the Hessian matrix of the function f at the i-th point in the first sampling array, ||·|| F represents the Frobenius norm.

[0159] Loss function L hes The modulus (Frobenius norm) used to constrain the second-order derivative ensures that the second-order variation of the generated field is small. By constraining the second-order derivative, the generated field is ensured to be locally smooth and not vary drastically, which helps to generate more natural shapes.

[0160] S340, determining a preset loss function based on the first loss function, the second loss function, and the third loss function.

[0161] The default loss function is L: L = λ0L eik +λ1L mask +λ2L hes ; where λ0, λ1 and λ2 are L eik Loss term, L mask Loss term, L hes The weight of the loss term.

[0162] Furthermore, the training method for training the target neural network model based on the preset loss function specifically includes:

[0163] B100, at the initial stage of training, 100≤λ2≤200, 5000≤λ0≤10000, 500≤λ1≤10000.

[0164] During the training process, λ2 is the annealing factor, which is 100 at the beginning of training. It is used to promote the rapid and robust convergence of the network by adding a loss function with a smoothing effect.

[0165] B200, in the middle of training, λ2 is gradually reduced to 1 according to the first preset interval, and λ0 and λ1 remain unchanged;

[0166] The first preset interval is the quotient of the first difference and the number of iterations in the middle of training, and the first difference is the difference between λ2 and 1 in the initial stage of training.

[0167] B300, at the end of the training, λ2 is gradually reduced to 0 according to the second preset interval, and λ0 and λ1 remain unchanged;

[0168] The second preset interval is the quotient of 1 and the number of iterations at the end of training.

[0169] At the end of training, it gradually drops to 0 to avoid affecting the fitting effect of image data.

[0170] Among them, the number of iterations in the initial stage of training is 20% of the total number of iterations, the number of iterations in the middle stage of training is 40% of the total number of iterations, and the number of iterations in the final stage of training is 40% of the total number of iterations.

[0171] During the training process, Adam is the preferred optimizer and the learning rate is 5×10 -5 , preferably using the CosineAnnealingLR learning rate scheduler, with a total of 4000 epochs.

[0172] Reference Figure 6 , the method for obtaining the signed distance function value includes:

[0173] C100, creates a grid coordinate matrix based on a preset value range, a preset amount of spatial dimension, and a preset amount of time dimension.

[0174] The preset value range is from 0 to 1.

[0175] The preset amount of spatial dimension is ≥ 128; the preset amount of temporal dimension is ≥ the number of time frames in the mask image.

[0176] By creating a grid coordinate matrix, the spatial and temporal dimensions are standardized to ensure that all input data is processed under the same standard. The grid coordinate matrix facilitates efficient parallel computing and indexing, improving overall computing efficiency. The preset value range is 0 to 1 to ensure that all input data is within a consistent value range and avoid errors caused by inconsistent value ranges.

[0177] C200, expand the coordinates of the grid coordinate matrix to obtain a uniform point set.

[0178] Expanding the grid coordinate matrix can facilitate subsequent array operations and data processing and simplify computational complexity; the expanded uniform point set ensures that each point is evenly distributed, which helps improve the training effect and prediction accuracy of the model.

[0179] Among them, the uniform point set is in the form of N test ×(t,x,y,z), the resolution of time t and spatial coordinates (x,y,z) can be determined as arbitrary values ​​and are no longer restricted by the shape of the original image.

[0180] C300, input the uniform point set into the first neural network model to obtain the signed distance function value of each point.

[0181] The signed distance function value of each point is obtained through the first neural network model, which can accurately represent the distance and direction of the point to the object surface, and is suitable for three-dimensional reconstruction and shape representation tasks.

[0182] Among them, the method for obtaining the uniform point set specifically includes: creating a grid coordinate matrix in the range of 0 to 1, which means that all coordinate values ​​are within this range; generating a uniform point set according to the required resolution, and the resolution can be set arbitrarily and is no longer limited by the shape of the original image.

[0183] Assume that the spatial coordinates are (x, y, z), the time coordinate is t, and the resolution is N x ,N y ,N z ,N t .

[0184] Then the point set in each coordinate dimension can be expressed as:

[0185]

[0186] Expand these coordinates and combine them into a two-dimensional array in the form of (i, j, k, l).

[0187] Reference Figure 7 , based on the signed distance function values ​​of all frames, the dynamic three-dimensional shape of the heart is obtained, including:

[0188] D100, according to the time frame of the uniform point set, extract the signed distance function value of the same frame and record it as the target time frame set.

[0189] By extracting the signed distance function values ​​of the same time frame, all data are aligned in the time dimension to avoid errors caused by time misalignment; grouping the data of all frames according to time frames simplifies subsequent processing steps and improves calculation efficiency.

[0190] D200, extract the zero isosurface from the target time frame set to obtain the dynamic three-dimensional shape of the heart.

[0191] Specifically, the marching cubes algorithm is used to extract the zero isosurface from the target time frame set to obtain the dynamic three-dimensional shape of the heart, that is, to obtain the dynamic three-dimensional shape composed of triangular meshes corresponding to all frames.

[0192] The zero isosurface (i.e., the surface with a signed distance function value of zero) represents the precise position of the object surface and can accurately reconstruct the three-dimensional shape of the heart; by extracting the zero isosurface under different time frames, the dynamic three-dimensional shape of the heart at different time points can be obtained, showing the dynamic changes of the heart; the dynamic three-dimensional shape of the heart can be used for high-quality visualization, helping doctors and researchers to understand the structure and function of the heart more intuitively.

[0193] Reference Figure 8 , the video acquisition method also includes:

[0194] E100, based on the extracted zero isosurface, obtains the triangular mesh of each frame.

[0195] Among them, triangular mesh is an efficient geometric representation method, suitable for reconstruction and rendering of three-dimensional shapes; the triangular mesh extracted by zero isosurface can accurately capture the three-dimensional shape of the heart and improve the accuracy of shape representation.

[0196] E200, render all triangular meshes to obtain rendered sub-images.

[0197] Specifically, the triangulated network of each frame may be sent to the rendering software Blender for rendering to obtain a rendered sub-image.

[0198] The rendering process can generate high-quality images that show the three-dimensional shape and details of the heart, making it easier for doctors and researchers to perform visual analysis. Rendering can introduce visual effects such as lighting and shadows to enhance the realism and visual effects of the image.

[0199] E300, merges all rendered sub-images according to time frames to generate a video of the dynamic 3D shape of the heart.

[0200] By merging the rendered sub-images of all frames, a dynamic three-dimensional shape video of the heart at different time points can be generated, showing the dynamic changes and movement trajectory of the heart; dynamic video can more intuitively display the function and structure of the heart, helping doctors to make more accurate diagnosis and evaluation; dynamic video can be used for doctor-patient communication and medical education, improving patient understanding and doctor's teaching effectiveness.

[0201] In the scheme of the method for constructing the dynamic three-dimensional shape of the heart disclosed in the present application, implicit neural representation is a method for representing complex geometric shapes through a neural network. In this method, the implicit neural representation maps points in space to their signed distance function values ​​by learning a continuous function, which means that for any given point in space, the neural network can output the distance between the point and the surface of the heart, whether inside or outside; in the preprocessing stage, the original image of the heart is segmented and converted into a voxel coordinate point cloud set, which contains the geometric information of the heart, including shape, size, and movement over time. Through this information, the neural network learns the geometric features of the heart during the training process; during the training process, the target neural network model is obtained by The preset loss function is used for training. The loss function ensures that the signed distance function value output by the model can accurately reflect the geometric shape and dynamic changes of the heart. Therefore, the trained model has the ability to infer the dynamic three-dimensional shape of the heart from the input point set (uniform point set); the uniform point set is generated within a preset numerical range, which covers the spatial area that the heart may occupy. These point sets are input into the trained model, and the model outputs the signed distance function value for each point. These values ​​reflect the distance between the heart surface and the point, thereby implicitly defining the shape of the heart; finally, the dynamic three-dimensional shape of the heart can be obtained by extracting the zero isosurface (that is, the surface composed of points with zero signed distance function value), because in the implicit representation, the zero isosurface corresponds exactly to the geometric surface of the heart.

[0202] In this application, the uniform point set itself does not directly contain the image information of the heart, but through implicit neural representation and model training, it actually inherits the geometric features of the heart. Therefore, when the uniform point set is input into the trained model, the model can use its learned geometric features to accurately output the signed distance function value of each point, thereby restoring the dynamic three-dimensional shape of the heart.

[0203] In a second aspect, the present disclosure also provides a system for constructing a dynamic three-dimensional shape of a heart, which specifically includes the following contents:

[0204] A segmentation module is used to segment the original heart image and obtain mask image information;

[0205] A preprocessing module is used to preprocess the mask image information to obtain a voxel coordinate point cloud set;

[0206] A model building module, used to build a target neural network model;

[0207] A training module, used for training the target neural network model based on a preset loss function, and recording the trained model as the first neural network model;

[0208] A signed distance function value acquisition module, used to obtain a signed distance function value of each frame based on the first neural network model;

[0209] A building block is used to obtain the dynamic three-dimensional shape of the heart based on the signed distance function values ​​of all frames.

[0210] It should be noted that the solutions in the cardiac dynamic three-dimensional shape construction method disclosed in the first aspect of this application are all applicable to the cardiac dynamic three-dimensional shape construction system disclosed in the second aspect, so they will not be repeated here.

[0211] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0212] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0213] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0214] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0215] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0217] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0218] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0219] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for constructing a dynamic three-dimensional shape of a heart, characterized in that: include: Segment the original heart image to obtain mask image information; Preprocessing the mask image information to obtain a voxel coordinate point cloud set; Establish a target neural network model based on implicit neural representation; Constructing a preset loss function based on the voxel coordinate point cloud set; Training the target neural network model based on the preset loss function, and recording the trained model as a first neural network model; Obtaining a signed distance function value for each frame based on the first neural network model; Based on the signed distance function values ​​of all frames, the dynamic three-dimensional shape of the heart is obtained.

2. The method for constructing a dynamic three-dimensional heart shape according to claim 1, characterized in that: The segmenting of the original heart image to obtain mask image information includes: segmenting the original heart image based on a deep learning segmentation algorithm to obtain a mask image; The mask image includes several frames of three-dimensional original images and voxel dimension information.

3. The method for constructing a dynamic three-dimensional heart shape according to claim 2, characterized in that: Preprocessing the mask image information to obtain a voxel coordinate point cloud set includes: Processing each frame of the three-dimensional original image based on the cropping strategy to obtain a three-dimensional cropped image of each frame; Merging the three-dimensional cropped images of all frames to obtain a reconstructed cropped image; Creating a grid coordinate matrix based on the size of the reconstructed cropped image and a preset resolution; Expand the coordinates of the grid coordinate matrix to obtain a first array; Obtaining a target pixel point set and a background pixel point set in the reconstructed cropped image; Based on the voxel dimension information, the target pixel point set and the background pixel point set are converted to obtain a second array and a third array; Obtaining an inner boundary point set and an outer boundary point set of the target pixel point set; Based on the voxel dimension information, the inner boundary point set and the outer boundary point set are converted to obtain a fourth array and a fifth array; Normalizing the three-dimensional coordinates and time of the first array, the second array, the third array, the fourth array, and the fifth array; Randomly sampling the first array, the second array, the third array, the fourth array, and the fifth array after normalization to obtain a first sampling array, the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array; The first sampling array, the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array constitute the voxel coordinate point cloud set.

4. The method for constructing a dynamic three-dimensional heart shape according to claim 3, characterized in that: The method of processing each frame of the 3D original image based on the cropping strategy to obtain each frame of the 3D cropped image includes: Identify cardiac reconstructive substructures; Traversing the three-dimensional original images of all time frames to determine minimum coordinate information and maximum coordinate information of the cardiac reconstruction substructure; If the target bounding box formed by the minimum coordinate information and the maximum coordinate information is smaller than a preset image size, expanding the target bounding box formed by the minimum coordinate information and the maximum coordinate information to obtain an extended bounding box, and cropping the 3D original image based on the extended bounding box to obtain the 3D cropped image; If the target bounding box formed by the minimum coordinate information and the maximum coordinate information is not smaller than the preset image size, the three-dimensional original image is cropped based on the target bounding box, and the cropped image is filled with background pixels to obtain the three-dimensional cropped image.

5. The method for constructing a dynamic three-dimensional heart shape according to claim 1, characterized in that: The method of establishing a target neural network model based on implicit neural representation includes: establishing an initial model comprising an implicit neural representation with a periodic activation function; Initializing the weights of the initial model using a multi-frequency geometric initialization method; The linear layer in the initialized initial model is replaced with a quadratic layer to obtain a target neural network model.

6. The method for constructing a dynamic three-dimensional heart shape according to claim 3, characterized in that: The step of constructing a preset loss function based on the voxel coordinate point cloud set includes: Determining a first loss function based on the first sampling array; Determine a second loss function based on the second sampling array, the third sampling array, the fourth sampling array, and the fifth sampling array; Determining a third loss function based on the first sampling array; The preset loss function is determined based on the first loss function, the second loss function, and the third loss function.

7. The method for constructing a dynamic three-dimensional heart shape according to claim 6, characterized in that: The first loss function is L eik : in, represents the function f at the midpoint x of the first array i The gradient at , ||·||2 represents the 2-norm of the gradient, N grid is the number of sample points in the first array, x i is the coordinate of the i-th point in the first array; The second loss function is L mask : Among them, n (x i ) is the predicted value of the signed distance function of a point in the second sampling array or in the fourth sampling array, f out (x i ) is the predicted value of the signed distance function of a point in the third sampling array or the fifth sampling array, N bou is the number of points in the fourth sampling array or the fifth sampling array, N mask is the number of points in the second sampling array or the third sampling array, λ bou , mask is the corresponding coefficient; The third loss function is L hes : Among them, H f (x i ) represents the Hessian matrix of the function f at the i-th point in the first sampling array, ||·|| F is the Frobenius norm; The preset loss function is L: L = λ0L eik +λ1L mask +λ2L hes ; Among them, λ0, λ1 and λ2 are L eik Loss term, L mask Loss term, L hes The weight of the loss term.

8. The method for constructing a dynamic three-dimensional heart shape according to claim 1, characterized in that: The step of obtaining a signed distance function value for each frame based on the first neural network model includes: Create a grid coordinate matrix based on a preset value range, a preset amount of a spatial dimension, and a preset amount of a temporal dimension; The preset value range is from 0 to 1; the preset value of the spatial dimension is ≥ 128; the preset value of the time dimension is ≥ the number of time frames in the mask image; Expanding the coordinates of the grid coordinate matrix to obtain a uniform point set; The uniform point set is input into the first neural network model to obtain a signed distance function value of each point.

9. The method for constructing a dynamic three-dimensional heart shape according to claim 8, characterized in that: The step of obtaining the dynamic three-dimensional shape of the heart based on the signed distance function values ​​of all frames comprises: According to the time frame of the uniform point set, the signed distance function value of the same frame is extracted and recorded as a target time frame set; A zero isosurface is extracted from the target time frame set to obtain a dynamic three-dimensional shape of the heart.

10. The method for constructing a dynamic three-dimensional heart shape according to claim 9, characterized in that: After the zero isosurface is extracted, the method further includes: Based on the extracted zero isosurface, obtaining a triangular mesh of each frame; Rendering the triangular meshes to obtain rendered sub-images; All the rendered sub-images are merged according to time frames to generate a video of the dynamic three-dimensional shape of the heart.

Images