Method for constructing myocardial tissue bionic unit model based on digital twin

Abstract

The present disclosure relates to a method for constructing a bionic unit model of myocardial tissue based on digital twins, and relates to the field of digital twin technology; it uses medical imaging technologies such as MRI and CT to obtain detailed anatomical structure data of the heart, and combines it with electrophysiological data such as electrocardiogram to comprehensively reflect the physiological state of the heart. Computer vision technology is used to process multi-view MRI data of the heart to achieve ventricular segmentation and three-dimensional reconstruction, and generate a high-precision 3D model of the heart. Electrical activity simulation: an orthogonal anisotropic Eikonal model is used to simulate the propagation process of the heart's electrical activity and reproduce the electrophysiological characteristics of the myocardium. Electrocardiogram inversion: a dual-branch variational autoencoder architecture is designed to infer the characteristics of the myocardium from electrocardiogram data, and to achieve accurate evaluation of pathological conditions such as myocardial infarction.

Description

Technical Field

[0001] The present disclosure relates to the field of digital twin technology, and in particular, to a method for constructing a myocardial tissue bionic unit model based on digital twin. Background Art

[0002] Digital twins make full use of physical models, sensors, operation history and other data, integrate multi-disciplinary, multi-physical quantity, multi-scale, and multi-probability simulation processes, complete mapping in virtual space, and thus reflect the entire life cycle of the corresponding physical equipment.

[0003] With the development of digital twin technology, it is being applied to more and more scenarios. In medical scenarios, digital twin technology can be used to build bionic models, which can have a variety of applications.

[0004] For example, in cardiac medical scenarios, digital twin technology can be used to build a heart model. Based on the constructed heart model, the pathological state of the heart can be studied to assist in the research and diagnosis of various heart diseases.

[0005] In related technologies, the construction of heart models only stays at the basic information dimension of the heart, such as structural characteristics, tissue distribution characteristics, etc., and cannot achieve a comprehensive reflection of heart information. Summary of the Invention

[0006] The purpose of the present disclosure is to provide a method for constructing a myocardial tissue bionic unit model based on digital twins. The method for constructing a myocardial tissue bionic unit model based on digital twins can construct a myocardial tissue bionic unit model that comprehensively reflects the physiological state of the heart and improve the applicability of the model.

[0007] In order to achieve the above-mentioned objectives, the present disclosure provides a method for constructing a myocardial tissue bionic unit model based on digital twins, including: obtaining medical imaging data and electrocardiogram data corresponding to the heart to be modeled; constructing an original heart 3D model based on the medical imaging data; simulating the electrical activity propagation process of the original heart 3D model through an orthogonal anisotropic model to obtain the first myocardial physiological feature of the original heart 3D model; determining the second myocardial physiological feature of the original heart 3D model based on the electrocardiogram data and the medical imaging data through a pre-trained dual-branch variational autoencoder; and obtaining a myocardial tissue bionic unit model based on the first myocardial physiological feature, the second myocardial physiological feature and the original heart 3D model.

[0008] Optionally, the medical imaging data includes multiple types of medical images, different types of medical images correspond to different medical imaging devices, and each type of medical image includes medical images from multiple perspectives. The constructing of the original heart 3D model based on the medical imaging data includes: determining, based on the multiple types of medical images, multiple medical images corresponding to the multiple perspectives, the multiple medical images corresponding to each perspective include at least one type of medical image; screening the multiple medical images corresponding to the multiple perspectives to obtain multiple screened medical images corresponding to the multiple perspectives; converting the multiple screened medical images corresponding to the multiple perspectives into model images for 3D reconstruction; and performing 3D modeling based on the model images to obtain the original heart 3D model.

[0009] Optionally, the medical image data also includes diagnostic information corresponding to the multiple types of medical images, and the screening of the multiple medical images corresponding to the multiple perspectives to obtain multiple screened medical images corresponding to the multiple perspectives includes: determining a first medical image that has an association with the diagnostic information from the multiple medical images corresponding to the multiple perspectives based on the diagnostic information corresponding to the multiple types of medical images; if the perspective corresponding to the first medical image covers the multiple perspectives, determining multiple screened medical images corresponding to the multiple perspectives based on the first medical image; if the perspective corresponding to the first medical image does not cover the multiple perspectives, performing perspective expansion processing on the first medical image to obtain a second medical image; and determining multiple screened medical images corresponding to the multiple perspectives based on the first medical image and the second medical image.

[0010] Optionally, the electrical activity propagation process of the original heart 3D model is simulated by the orthogonal anisotropic model to obtain the first myocardial physiological characteristic of the original heart 3D model, including: determining a first anisotropy parameter according to the model parameters of the original heart 3D model; determining a second anisotropy parameter according to a pre-configured myocardial physiological characteristic acquisition requirement; determining an initial heart 3D model state corresponding to the electrical activity propagation process; and simulating the electrical activity propagation process of the original heart 3D model according to the first anisotropy parameter, the second anisotropy parameter and the initial heart 3D model state through the orthogonal anisotropic model to obtain the first myocardial physiological characteristic of the original heart 3D model.

[0011] Optionally, the pre-configured myocardial physiological characteristic acquisition requirements include: myocardial cell excitation characteristic acquisition requirements, myocardial cell conduction characteristic acquisition requirements and myocardial cell contraction characteristic acquisition requirements, and determining the second anisotropy parameter based on the pre-configured myocardial physiological characteristic acquisition requirements includes: determining the anisotropy parameter related to myocardial cell excitation based on the myocardial cell excitation characteristic acquisition requirements; determining the anisotropy parameter related to myocardial cell conduction based on the myocardial cell conduction characteristic acquisition requirements; determining the anisotropy parameter related to myocardial cell contraction based on the myocardial cell contraction characteristic acquisition requirements; determining the second anisotropy parameter based on the anisotropy parameter related to myocardial cell excitation, the anisotropy parameter related to myocardial cell conduction and the anisotropy parameter related to myocardial cell contraction.

[0012] Optionally, the dual-branch autoencoder includes an encoder and a decoder, the encoder includes a first branch and a second branch, and the pre-trained dual-branch variational autoencoder determines the second myocardial physiological characteristics of the original heart 3D model based on the electrocardiogram data and the medical imaging data, including: performing feature extraction based on the electrocardiogram data through the first branch to obtain electrocardiogram features; performing feature extraction based on the medical imaging data through the second branch to obtain medical imaging features; and determining the second myocardial physiological characteristics based on the electrocardiogram features and the medical imaging features through the decoder.

[0013] Optionally, the electrocardiogram features include a first myocardial cell excitation feature, a myocardial cell rhythm excitation feature, a myocardial cell conduction feature and a first myocardial cell contraction feature, and the medical image features include a second myocardial cell excitation feature and a second myocardial cell contraction feature, and determining the second myocardial physiological feature according to the electrocardiogram features and the medical image features by the decoder includes: fusing the first myocardial cell excitation feature and the second myocardial cell excitation feature by the decoder to obtain a target myocardial cell excitation feature; fusing the first myocardial cell contraction feature and the second myocardial cell contraction feature according to the target myocardial cell excitation feature to obtain a target myocardial contraction feature; determining the second myocardial physiological feature according to the target myocardial cell excitation feature, the target myocardial contraction feature, the myocardial cell rhythm excitation feature and the myocardial cell conduction feature by the decoder.

[0014] Optionally, the model building method further includes:

[0015] Acquire a first training data set and a second training data set, wherein the first training data set includes multiple first training samples, the second training data set includes multiple second training samples, each first training sample includes: a medical image sample and a medical image feature corresponding to the medical image sample, and each second training sample includes: an electrocardiogram sample and an electrocardiogram feature corresponding to the electrocardiogram sample; train the first branch according to the first training data set, and train the second branch according to the second training data set to obtain a pre-trained encoder; acquire a third training data set, wherein the third training data set includes multiple third training samples, and each third training sample includes: a medical image feature sample, an electrocardiogram feature sample, and a myocardial physiological feature label; train the decoder to be trained according to the third training data set to obtain a pre-trained decoder; obtain the pre-trained two-branch variational autoencoder according to the pre-trained encoder and the pre-trained decoder.

[0016] Optionally, the myocardial tissue bionic unit model is obtained based on the first myocardial physiological characteristics, the second myocardial physiological characteristics and the original heart 3D model, including: determining the target myocardial physiological characteristics based on the first myocardial physiological characteristics and the second myocardial physiological characteristics; generating a myocardial tissue bionic unit based on the target myocardial physiological characteristics and the original heart 3D model; determining the identification information corresponding to the myocardial tissue bionic unit based on the target myocardial physiological characteristics, the identification information being used to characterize the pathological state of the heart; and obtaining the myocardial tissue bionic unit model based on the identification information and the myocardial tissue bionic unit.

[0017] Optionally, the model construction method also includes: in response to detecting a model display request, determining a model display mode according to the model display request, the model display mode being a dynamic display mode or a static display mode; when the model display mode is a dynamic display mode, displaying the original heart 3D model, the preset model segmentation animation and the myocardial tissue bionic unit model in sequence; when the model display mode is a static display mode, displaying the original heart 3D model and the myocardial tissue bionic unit model respectively in a preset model display template.

[0018] Through the above technical solution, a high-precision 3D heart model is established using medical imaging data. The orthogonal anisotropic model is used to simulate the propagation process of the heart's electrical activity and reproduce the electrophysiological characteristics of the myocardium. Through the dual-branch variational autoencoder architecture, the physiological characteristics of the myocardium are determined based on electrocardiogram data and medical imaging data. Furthermore, by combining multi-dimensional electrophysiological characteristics with a high-precision 3D heart model, the construction of a myocardial tissue bionic unit is realized. This myocardial tissue bionic unit is constructed based on multi-dimensional electrophysiological characteristics and can comprehensively reflect the physiological state of the heart. It has strong applicability and can assist in the accurate assessment of the pathological state of the heart in the evaluation scenario of the heart's pathological state.

[0019] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0021] Figure 1 It is a schematic diagram showing an application scenario according to an exemplary embodiment.

[0022] Figure 2 This is a flowchart of a method for constructing a myocardial tissue bionic unit model based on digital twins according to an exemplary embodiment.

[0023] Figure 3 It is a schematic diagram showing an application of an orthotropic model according to an exemplary embodiment.

[0024] Figure 4 The figure is a block diagram of a dual-branch autoencoder according to an exemplary embodiment.

[0025] Figure 5 The figure is a schematic diagram showing a dynamic display effect of a model according to an exemplary embodiment.

[0026] Figure 6 It is a block diagram of a device for constructing a myocardial tissue bionic unit model based on digital twins according to an exemplary embodiment.

[0027] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0029] Digital twins make full use of physical models, sensors, operation history and other data, integrate multi-disciplinary, multi-physical quantity, multi-scale, and multi-probability simulation processes, complete mapping in virtual space, and thus reflect the entire life cycle of the corresponding physical equipment.

[0030] With the development of digital twin technology, it is being applied to more and more scenarios. In medical scenarios, digital twin technology can be used to build bionic models, which can have a variety of applications.

[0031] For example, in cardiac medical scenarios, digital twin technology can be used to build a heart model. Based on the constructed heart model, the pathological state of the heart can be studied to assist in the research and diagnosis of various heart diseases.

[0032] In related technologies, the construction of heart models only stays at the basic information dimension of the heart, such as structural characteristics, tissue distribution characteristics, etc., and cannot achieve a comprehensive reflection of heart information.

[0033] Based on this, the disclosed embodiments provide a technical solution for building a high-precision 3D heart model using medical imaging data. Using an orthogonal anisotropic model, the propagation of cardiac electrical activity is simulated, reproducing the electrophysiological characteristics of the myocardium. Using a dual-branch variational autoencoder architecture, the physiological characteristics of the myocardium are determined based on electrocardiogram (ECG) and medical imaging data.

[0034] Furthermore, by combining multi-dimensional electrophysiological characteristics with a high-precision 3D heart model, a bionic myocardial tissue unit was constructed. This bionic myocardial tissue unit, based on multi-dimensional electrophysiological characteristics, can comprehensively reflect the physiological state of the heart and has strong applicability. It can assist in the accurate assessment of cardiac pathological conditions in scenarios where this is necessary.

[0035] Figure 1 is a schematic diagram showing an application scenario according to an exemplary embodiment. Figure 1 As shown, in this application scenario, it includes: modeling platform, medical imaging equipment and electrocardiogram data acquisition equipment.

[0036] The modeling platform may be a host computer or other equipment with modeling capabilities, the medical imaging equipment may be a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, etc., and the ECG data acquisition device may be an ECG data acquisition instrument.

[0037] In some embodiments, a medical imaging device can collect medical imaging data and synchronize it to the modeling platform. An electrocardiogram (ECG) data acquisition device can collect electrocardiogram (ECG) data and synchronize it to the modeling platform. Thus, the modeling platform can construct a model based on the medical imaging data and the ECG data.

[0038] Figure 2 is a flowchart of a method for constructing a myocardial tissue bionic unit model based on digital twins according to an exemplary embodiment. Figure 2 As shown, this method can be applied to Figure 1 The modeling platform shown in FIG. 1 includes the following steps:

[0039] Step S21: Obtain medical image data and electrocardiogram data corresponding to the heart to be modeled.

[0040] Step S22: constructing an original heart 3D model based on the medical imaging data.

[0041] Step S23 , simulating the electrical activity propagation process of the original heart 3D model by using an orthotropic model to obtain the first myocardial physiological characteristics of the original heart 3D model.

[0042] In step S24 , a second myocardial physiological feature of the original heart 3D model is determined based on the electrocardiogram data and the medical imaging data using a pre-trained two-branch variational autoencoder.

[0043] Step S25 , obtaining a myocardial tissue bionic unit model according to the first myocardial physiological characteristics, the second myocardial physiological characteristics and the original heart 3D model.

[0044] In step S21, the heart to be modeled can be a virtual heart, a heart to be pathologically studied, or a real heart, a heart to be pathologically diagnosed. In different scenarios, a heart model can be constructed when there is a need for heart modeling.

[0045] In some embodiments, the medical imaging data is data collected by medical imaging equipment.

[0046] As an implementation method, the medical imaging data includes multiple types of medical images. Different types of medical images correspond to different medical imaging devices, and each type of medical image includes medical images from multiple perspectives.

[0047] In some embodiments, multiple viewing angles can be configured based on the structural anatomy (analysis) requirements of the heart. For example, if the structural anatomy of the heart is required to be viewed from the inside out, the multiple viewing angles can be multiple cross-sectional views of the heart. Therefore, in different scenarios, medical images can be collected from multiple viewing angles based on different requirements.

[0048] In some embodiments, medical images from multiple perspectives may be captured using multiple medical imaging devices to obtain different types of medical images.

[0049] Therefore, in some embodiments, the multiple viewing angles may also be viewing angles that match the acquisition method of the medical imaging device.

[0050] In some embodiments, the electrocardiogram data may reflect the potential changes of the heart under stimulation, and may be in the form of waveforms or intuitive potential data, which are not limited here.

[0051] In step S22, the original heart 3D model can be constructed using the medical image data. The 3D model construction technology can adopt mature technologies in the art.

[0052] In some embodiments, in order to realize the construction of a 3D model, it is necessary to obtain an image for constructing the 3D model, so that rendering, mapping, etc. can be performed according to the image to realize the construction of the 3D model.

[0053] Therefore, as an optional implementation, step S22 includes: determining, based on multiple types of medical images, multiple medical images corresponding to multiple perspectives, wherein the multiple medical images corresponding to each perspective include at least one type of medical image; screening the multiple medical images corresponding to the multiple perspectives to obtain multiple screened medical images corresponding to the multiple perspectives; converting the multiple screened medical images corresponding to the multiple perspectives into model images for 3D reconstruction; and performing 3D modeling based on the model images to obtain an original heart 3D model.

[0054] In this embodiment, multiple types of medical images can be first classified according to perspective to obtain multiple medical images at different perspectives. These multiple medical images may only involve one type, that is, they are collected by one medical imaging device, or they may involve multiple types, that is, they are collected by multiple medical imaging devices.

[0055] Furthermore, screening can be performed based on multiple medical images corresponding to multiple perspectives.

[0056] As an optional implementation, the medical imaging data also includes diagnostic information corresponding to various types of medical images. This diagnostic information can be provided by a doctor or by a medical imaging device, and can be used to indicate whether a heart problem exists. This diagnostic information can then be used to screen medical images.

[0057] Furthermore, as an optional implementation, the multiple medical images corresponding to the multiple perspectives are screened to obtain multiple screened medical images corresponding to the multiple perspectives, including: determining a first medical image that is associated with the diagnostic information from the multiple medical images corresponding to the multiple perspectives based on the diagnostic information corresponding to the multiple types of medical images; if the perspective corresponding to the first medical image covers multiple perspectives, determining multiple screened medical images corresponding to the multiple perspectives based on the first medical image; if the perspective corresponding to the first medical image does not cover multiple perspectives, performing perspective expansion processing on the first medical image to obtain a second medical image; and determining multiple screened medical images corresponding to the multiple perspectives based on the first medical image and the second medical image.

[0058] In some embodiments, different medical images may correspond to different cardiac structures, and the diagnostic information will indicate whether each structure has a problem, such as whether there is a problem with the ventricle, atrium, or cardiovascular system. Thus, associated medical images and diagnostic information indicating a problem may correspond to the same cardiac structure.

[0059] For example, if the diagnosis information includes: no abnormalities in the ventricles and atria, but abnormalities in the cardiovascular system, then the medical image with which the diagnosis is related based on the association relationship is a medical image related to the cardiovascular system.

[0060] Furthermore, it can be determined whether the perspective corresponding to the first medical image covers multiple perspectives. If so, it means that the first medical image has comprehensive perspectives, so the first medical image can be directly determined as multiple filtered medical images corresponding to multiple perspectives.

[0061] If not, it means that the first medical image does not have the comprehensiveness of the viewing angle. In this case, the viewing angle expansion process can be performed on the first medical image to obtain the second medical image to ensure the comprehensiveness of the viewing angle.

[0062] In some embodiments, a medical image taken at a perspective that does not overlap with the current perspective may be selected from medical images that do not have an associated relationship as the extended medical image, ie, the second medical image.

[0063] In some embodiments, the pre-configured medical image under multiple viewing angles may be used as an extended medical image, ie, a second medical image.

[0064] In some embodiments, the first medical image may be subjected to perspective-based image transformation processing to obtain a second medical image.

[0065] Furthermore, the first medical image and the second medical image may be used as final screened medical images.

[0066] Furthermore, based on the multiple screened medical images corresponding to the multiple perspectives, the model image is converted to obtain an image that can be used for 3D modeling.

[0067] Image conversion processing may include: image grayscale processing, image contrast enhancement processing, image denoising processing, etc. In different application scenarios, different conversion processing methods may be used according to the requirements for 3D modeling images, which are not limited here.

[0068] Furthermore, 3D modeling can be performed based on the model image to obtain the original 3D model of the heart. Regarding 3D modeling technology, reference can be made to mature digital twin technology in the field, which will not be described in detail here. For example, modeling can be achieved using mature modeling software.

[0069] In step S23, the electrical activity propagation process of the original heart 3D model is simulated by using an orthotropic model to obtain the first myocardial physiological characteristics of the original heart 3D model.

[0070] The orthotropic model may be an Eikonal model, which is a mathematical model used to describe wave propagation in an orthotropic medium.

[0071] In myocardial tissue research, the arrangement of myocardial fibers exhibits significant anisotropy. The orthotropic Eikonal model can be used to simulate the propagation of myocardial electrical activity. This model more accurately reflects the electrical conduction characteristics of myocardial cells in different directions, thus providing support for the study and diagnosis of diseases such as arrhythmias.

[0072] Therefore, this model can be used to analyze the anisotropy of myocardial tissue to obtain myocardial psychological characteristics.

[0073] In some embodiments, the application of the orthotropic Eikonal model involves: anisotropy parameters and initial state parameters. By configuring these two parameters, the model can simulate the propagation process to obtain corresponding characteristics.

[0074] Therefore, step S23 may include: determining a first anisotropy parameter based on the model parameters of the original heart 3D model; determining a second anisotropy parameter based on a pre-configured myocardial physiological characteristic acquisition requirement; determining an initial heart 3D model state corresponding to the electrical activity propagation process; simulating the electrical activity propagation process of the original heart 3D model through an orthogonal anisotropy model based on the first anisotropy parameter, the second anisotropy parameter and the initial heart 3D model state to obtain the first myocardial physiological characteristic of the original heart 3D model.

[0075] In some embodiments, the first anisotropic parameter may include: elastic modulus, Poisson's ratio, and shear modulus. The first anisotropic parameter may be determined based on model parameters of the original 3D cardiac model. For example, after the model is constructed, the parameters of the constructed model may be statistically analyzed. The model parameters may include structural parameters such as material and size. Based on these structural parameters, anisotropic parameters such as elastic modulus, Poisson's ratio, and shear modulus related to mechanics may be determined.

[0076] In some embodiments, the second anisotropy parameter may be a specific parameter associated with the propagation of electrical activity of the heart.

[0077] In some embodiments, the pre-configured myocardial physiological characteristic acquisition requirements may include: myocardial cell excitation characteristic acquisition requirements, myocardial cell conduction characteristic acquisition requirements, and myocardial cell contraction characteristic acquisition requirements.

[0078] Regarding myocardial physiological characteristics, they may include excitation characteristics, rhythmic excitation characteristics, conduction characteristics and contraction characteristics.

[0079] Excitability: Cardiac myocytes can become excited upon stimulation, manifesting as the generation of action potentials and mechanical contraction on the cell membrane. Excitability is characterized by periodic changes, including an effective refractory period, a relative refractory period, and a supernormal period.

[0080] Characteristics of Rhythmic Excitation: Cardiac myocardial cells can spontaneously generate rhythmic excitation without external stimulation. The sinoatrial node has the highest degree of automaticity and serves as the pacemaker for a normal heartbeat, followed by the atrioventricular junction and Purkinje fibers.

[0081] Conduction characteristics: Cardiac myocytes have the ability to conduct excitation, not only within the same cardiomyocyte but also between cells. Conduction velocity varies across different locations, for example, the atrioventricular junction is the slowest, while the ventricles are the fastest.

[0082] Contraction Characteristics: Myocardial cells mechanically contract upon stimulation through excitation-contraction coupling. Myocardial contraction exhibits the following characteristics: an "all-or-none" contraction, whereby myocardial cells contract synchronously; the absence of tetanic contractions due to the myocardium's exceptionally long effective refractory period; and a strong dependence on extracellular calcium concentration.

[0083] Therefore, when the requirements for obtaining the excitation characteristics of the myocardial cells, the conduction characteristics of the myocardial cells, and the contraction characteristics of the myocardial cells are configured, anisotropy parameters associated with different characteristics can be determined.

[0084] In some embodiments, the anisotropy parameter associated with cardiomyocyte excitation may be the variation period of the aforementioned first anisotropy parameter. The anisotropy parameter associated with cardiomyocyte conduction may be the variation rate of the aforementioned first anisotropy parameter. The anisotropy parameter associated with cardiomyocyte contraction may be the variation period of the aforementioned first anisotropy parameter.

[0085] Furthermore, the anisotropy parameter related to myocardial cell excitation, the anisotropy parameter related to myocardial cell conduction, and the anisotropy parameter related to myocardial cell contraction may be deduplicated to obtain a second anisotropy parameter.

[0086] Therefore, the second anisotropy parameter can also be understood as a description / definition parameter for the first anisotropy parameter.

[0087] In some embodiments, the initial heart 3D model state corresponding to the electrical activity propagation process may be the heart 3D model state in a quiet state, where various features are relatively inconspicuous and there is no corresponding stimulation.

[0088] Furthermore, by utilizing the simulation characteristics of the orthogonal anisotropic model, based on the first anisotropy parameter, the second anisotropy parameter and the initial heart 3D model state, the electrical activity propagation process of the original heart 3D model is simulated, and the first myocardial physiological characteristics of the original heart 3D model can be obtained.

[0089] Among them, regarding the specific simulation method of the orthotropic model, reference can be made to the mature technology in the field. The embodiment of the present disclosure mainly configures the input parameters required for its simulation.

[0090] In some embodiments, the first myocardial physiological characteristic may include at least one of an excitability characteristic, a rhythmic excitability characteristic, a conduction characteristic, and a contraction characteristic.

[0091] Figure 3 is a schematic diagram of an application of an orthotropic model according to an exemplary embodiment. Figure 3 As shown, by inputting the first anisotropy parameter, the second anisotropy parameter and the initial heart 3D model state into the orthotropic model, the orthotropic model can output corresponding myocardial physiological characteristics.

[0092] In step S24, the second myocardial physiological feature of the original heart 3D model is determined based on the electrocardiogram data and the medical imaging data through the pre-trained two-branch variational autoencoder.

[0093] The dual-branch variational autoencoder is an extended variational autoencoder architecture designed to process different types of data inputs through two independent encoder branches and perform feature fusion and output through a shared decoder. This architecture is particularly suitable for multimodal data processing. Therefore, this encoder can be used to combine electrogram data and medical imaging data to determine myocardial physiological characteristics.

[0094] Figure 4 is a block diagram of a dual-branch autoencoder according to an exemplary embodiment. Figure 4 As shown, the dual-branch autoencoder includes an encoder and a decoder, and the encoder includes a first branch and a second branch.

[0095] Combine Figure 4 Step S24 may include: performing feature extraction based on the electrocardiogram data through the first branch to obtain electrocardiogram features; performing feature extraction based on the medical image data through the second branch to obtain medical image features; and determining the second myocardial physiological features based on the electrocardiogram features and the medical image features through the decoder.

[0096] In this embodiment, the first branch is used to extract electrocardiogram features, the second branch is used to extract medical image features, and the decoder is used to perform feature fusion based on the two features to obtain a second myocardial physiological feature.

[0097] As an optional embodiment, the electrocardiogram characteristics include a first myocardial cell excitation characteristic, a myocardial cell rhythm excitation characteristic, a myocardial cell conduction characteristic and a first myocardial cell contraction characteristic, and the medical imaging characteristics include a second myocardial cell excitation characteristic and a second myocardial cell contraction characteristic.

[0098] Accordingly, the decoder determines the second myocardial physiological feature based on the electrocardiogram features and the medical imaging features, including: fusing the first myocardial cell excitation feature and the second myocardial cell excitation feature through the decoder to obtain the target myocardial cell excitation feature; fusing the first myocardial cell contraction feature and the second myocardial cell contraction feature based on the target myocardial cell excitation feature through the decoder to obtain the target myocardial contraction feature; determining the second myocardial physiological feature based on the target myocardial cell excitation feature, the target myocardial contraction feature, the myocardial cell rhythm excitation feature and the myocardial cell conduction feature through the decoder.

[0099] In this embodiment, the myocardial cell excitation features obtained through different data extractions can be fused, and the myocardial cell contraction features obtained through different data extractions can be fused.

[0100] Furthermore, the myocardial cell contraction characteristics are associated with the myocardial cell excitation characteristics. Therefore, when fusing the myocardial cell contraction characteristics, the myocardial cell excitation characteristics can be referred to.

[0101] Furthermore, the decoder can ultimately determine the final second myocardial physiological feature based on the four types of features.

[0102] It can be understood that the second myocardial physiological feature may also include at least one of the first myocardial cell excitation feature, the myocardial cell rhythm excitation feature, the myocardial cell conduction feature and the first myocardial cell contraction feature, but the feature here is a fusion feature, which may take into account the mutual influence between the features and has higher accuracy than the extraction of a single feature.

[0103] In some embodiments, the training of the dual-branch autoencoder may include: obtaining a first training data set and a second training data set, the first training data set including multiple first training samples, the second training data set including multiple second training samples, each first training sample including: a medical image sample and a medical image feature corresponding to the medical image sample, each second training sample including: an electrocardiogram sample and an electrocardiogram feature corresponding to the electrocardiogram sample; training the first branch according to the first training data set, and training the second branch according to the second training data set to obtain a pre-trained encoder; obtaining a third training data set, the third training data set including multiple third training samples, each third training sample including: a medical image feature sample, an electrocardiogram feature sample and a myocardial physiological feature label; training the decoder to be trained according to the third training data set to obtain a pre-trained decoder; obtaining a pre-trained dual-branch variational autoencoder based on the pre-trained encoder and the pre-trained decoder.

[0104] In this embodiment, the first branch and the second branch are respectively used to extract corresponding features, so feature extraction needs to be performed based on corresponding samples and feature labels.

[0105] Also, for the decoder, feature fusion is required. Therefore, the training samples of the decoder need to involve pre-fusion features (i.e., medical image feature samples and electrocardiogram feature samples) and post-fusion features (i.e., myocardial physiological feature labels).

[0106] In some embodiments, different types of medical image feature samples and electrocardiogram feature samples may be combined to obtain a variety of feature combinations to be fused, and then these combinations may be used for training respectively to improve the generalization performance of the decoder.

[0107] Furthermore, after the encoder and decoder are trained, a pre-trained two-branch variational autoencoder can be constructed based on the input-output relationship between the encoder and decoder.

[0108] Furthermore, in step S25, a myocardial tissue bionic unit model is obtained according to the first myocardial physiological characteristics, the second myocardial physiological characteristics and the original heart 3D model.

[0109] In some embodiments, the original 3D heart model does not have corresponding myocardial physiological characteristics. By reflecting the myocardial physiological characteristics on the model, a myocardial tissue bionic unit model can be obtained.

[0110] Therefore, as an optional embodiment, step S25 includes: determining the target myocardial physiological characteristics based on the first myocardial physiological characteristics and the second myocardial physiological characteristics; generating a myocardial tissue bionic unit based on the target myocardial physiological characteristics and the original heart 3D model; determining the identification information corresponding to the myocardial tissue bionic unit based on the target myocardial physiological characteristics, the identification information is used to characterize the pathological state of the heart; and obtaining a myocardial tissue bionic unit model based on the identification information and the myocardial tissue bionic unit.

[0111] In this embodiment, the first myocardial physiological feature and the second myocardial physiological feature can be integrated first. Integration methods include, but are not limited to, weighting and deduplication. Regarding deduplication, for identical features of the same feature type, one can be retained. Regarding weighting, for different features of the same feature type, weighting can be applied. The second myocardial physiological feature is weighted higher than the first myocardial physiological feature.

[0112] In some embodiments, based on the physiological characteristics of the target myocardium, the original heart 3D model can be segmented or modified to obtain a myocardial tissue bionic unit.

[0113] For example, if the target myocardial physiological characteristic involves contraction of a specific cardiac structural position, the myocardial tissue of that portion is presented in a contracted form, and the myocardial tissue of that portion can be cut out separately as an independent myocardial tissue bionic unit.

[0114] It can be understood that the number of myocardial tissue bionic units can be one or more. In the case of more than one, the multiple units can correspond to the same myocardial physiological characteristics or different myocardial physiological characteristics.

[0115] Furthermore, the pathological state of the heart can be evaluated based on the myocardial physiological characteristics and identification information can be generated.

[0116] Regarding how to assess the pathological state of the heart based on myocardial physiological characteristics, reference can be made to mature technologies in the field and will not be described in detail here. For example, if the contraction is abnormal, it may be myocarditis, etc.

[0117] Furthermore, identification information can be marked on the basis of the myocardial tissue bionic unit, and integrated to obtain a myocardial tissue bionic unit model.

[0118] Therefore, the myocardial tissue bionic unit model can reflect the overall state of the heart from a three-dimensional perspective, and can also reflect the pathological state, so that the model can be applied in a variety of scenarios.

[0119] In some embodiments, the constructed model can be displayed. Therefore, the model construction method may further include: in response to detecting a model display request, determining a model display mode according to the model display request, wherein the model display mode is a dynamic display mode or a static display mode; when the model display mode is the dynamic display mode, sequentially displaying the original heart 3D model, a preset model segmentation animation, and a myocardial tissue bionic unit model; when the model display mode is the static display mode, displaying the original heart 3D model and the myocardial tissue bionic unit model in a preset model display template.

[0120] In some embodiments, the model display request may be a request initiated by a user who has a need to view the model.

[0121] In some embodiments, the model display mode may be indicated in the model display request, and thus the display mode may be determined directly according to the request.

[0122] Furthermore, when the model display mode is a dynamic display mode, the original heart 3D model, the preset model segmentation animation and the myocardial tissue bionic unit model are displayed in sequence.

[0123] The preset model segmentation animation can be configured in different scenarios according to different needs and is not limited here.

[0124] Through this implementation, a dynamic display effect can be provided to users viewing the model, thereby improving user experience.

[0125] When the model display mode is a static display mode, the original heart 3D model and the myocardial tissue bionic unit model are respectively displayed in the preset model display template.

[0126] The preset model display template can be used to limit the display position and display form of the two models, for example, to display the original heart 3D model in an enlarged manner and to display the myocardial tissue bionic unit model locally.

[0127] Figure 5 is a schematic diagram showing a model dynamic display effect according to an exemplary embodiment. Figure 5As shown, the interactive interface first displays the original 3D heart model and related information. Next, a preset model segmentation animation plays. The specific effects of this animation are dynamic and difficult to illustrate, so they are not shown in the figure. Furthermore, after the preset model segmentation animation finishes playing, the myocardial tissue biomimetic unit model and identification information are presented.

[0128] In some embodiments, the model can also be saved and directly retrieved and applied when needed.

[0129] It can be seen that in the embodiments disclosed herein, medical imaging technologies such as MRI and CT are used to obtain detailed anatomical data of the heart, which is combined with electrophysiological data such as electrocardiograms to comprehensively reflect the physiological state of the heart. Computer vision technology is used to process multi-view MRI data of the heart to achieve ventricular segmentation and three-dimensional reconstruction, generating a high-precision 3D model of the heart. Electrical activity simulation: The orthogonal anisotropic Eikonal model is used to simulate the propagation process of the heart's electrical activity and reproduce the electrophysiological characteristics of the myocardium. Electrocardiogram inversion: A dual-branch variational autoencoder architecture is designed to infer the characteristics of the myocardium from electrocardiogram data, thereby achieving accurate assessment of pathological conditions such as myocardial infarction.

[0130] Figure 6 is a block diagram of a device for constructing a myocardial tissue bionic unit model based on digital twins according to an exemplary embodiment. Figure 6 As shown, the device includes:

[0131] The acquisition module 601 is used to acquire medical image data and electrocardiogram data corresponding to the heart to be modeled.

[0132] The construction module 602 is configured to construct an original heart 3D model based on the medical image data.

[0133] The feature extraction module 603 is configured to simulate the electrical activity propagation process of the original heart 3D model by using an orthotropic model to obtain a first myocardial physiological feature of the original heart 3D model.

[0134] The feature extraction module 603 is further configured to determine a second myocardial physiological feature of the original heart 3D model based on the electrocardiogram data and the medical imaging data using a pre-trained two-branch variational autoencoder.

[0135] The construction module 602 is further configured to obtain a myocardial tissue bionic unit model according to the first myocardial physiological characteristics, the second myocardial physiological characteristics, and the original heart 3D model.

[0136] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0137] Figure 7 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 may include: a processor 701 , a memory 702 , and may further include one or more of a multimedia component 703 , an input / output (I / O) interface 704 , and a communication component 705 .

[0138] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned method for constructing a myocardial tissue biomimetic unit model based on digital twins. The memory 702 is used to store various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, as well as application-related data, such as contact information, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 702 or transmitted via the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules. The above-mentioned other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more thereof, so the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module.

[0139] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned digital twin-based myocardial tissue bionic unit model construction method.

[0140] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the aforementioned method for constructing a myocardial tissue biomimetic unit model based on digital twins. For example, the computer-readable storage medium may be the aforementioned memory 702 including the program instructions. The program instructions may be executed by the processor 701 of the electronic device 700 to complete the aforementioned method for constructing a myocardial tissue biomimetic unit model based on digital twins.

[0141] In another exemplary embodiment, a computer program product is also provided, which includes a computer program that can be executed by a processor, and when the computer program is executed by the processor, the steps of the above-mentioned method for constructing a myocardial tissue bionic unit model based on digital twins are implemented.

[0142] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.

[0143] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0144] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for constructing a myocardial tissue bionic unit model based on digital twins, characterized in that: include: Obtain medical imaging data and electrocardiogram data corresponding to the heart to be modeled; constructing an original heart 3D model based on the medical imaging data; Simulating the electrical activity propagation process of the original heart 3D model using an orthotropic model to obtain first myocardial physiological characteristics of the original heart 3D model, wherein the first myocardial physiological characteristics include: excitability characteristics, rhythmic excitability characteristics, conduction characteristics, and contraction characteristics; Determining, by a pre-trained two-branch variational autoencoder, a second myocardial physiological feature of the original heart 3D model based on the electrocardiogram data and the medical imaging data; Obtaining a myocardial tissue bionic unit model according to the first myocardial physiological characteristics, the second myocardial physiological characteristics, and the original heart 3D model; The simulating the electrical activity propagation process of the original heart 3D model by using the orthogonal anisotropic model to obtain the first myocardial physiological characteristics of the original heart 3D model includes: Determining first anisotropy parameters according to the model parameters of the original heart 3D model, where the first anisotropy parameters include: elastic modulus, Poisson's ratio, and shear modulus; determining a second anisotropy parameter according to a pre-configured myocardial physiological characteristic acquisition requirement, where the second anisotropy parameter is a description or limiting parameter for the first anisotropy parameter; Determine the initial heart 3D model state corresponding to the electrical activity propagation process; Simulating the electrical activity propagation process of the original heart 3D model using the orthogonal anisotropic model according to the first anisotropy parameter, the second anisotropy parameter, and the state of the initial heart 3D model to obtain a first myocardial physiological characteristic of the original heart 3D model; The dual-branch variational autoencoder includes an encoder and a decoder, the encoder includes a first branch and a second branch, and the pre-trained dual-branch variational autoencoder determines the second myocardial physiological feature of the original heart 3D model based on the electrocardiogram data and the medical imaging data, including: Extracting features based on the electrocardiogram data by the first branch to obtain electrocardiogram features, the electrocardiogram features including a first myocardial cell excitation feature, a myocardial cell rhythm excitation feature, a myocardial cell conduction feature, and a first myocardial cell contraction feature; Extracting features based on the medical image data by the second branch to obtain medical image features, wherein the medical image features include a second myocardial cell excitation feature and a second myocardial cell contraction feature; The decoder determines the second myocardial physiological feature based on the electrocardiogram feature and the medical image feature.

2. The model building method according to claim 1, characterized in that The medical image data includes multiple types of medical images. Different types of medical images correspond to different medical imaging devices. Each type of medical image includes medical images from multiple perspectives. Constructing the original heart 3D model based on the medical image data includes: Determining, based on the multiple types of medical images, multiple medical images corresponding to the multiple perspectives, wherein the multiple medical images corresponding to each perspective include at least one type of medical image; screening the plurality of medical images corresponding to the plurality of perspectives to obtain a plurality of screened medical images corresponding to the plurality of perspectives; Converting the plurality of screened medical images corresponding to the plurality of perspectives into model images for 3D reconstruction; 3D modeling is performed based on the model image to obtain the original heart 3D model.

3. The model building method according to claim 2, characterized in that: The medical image data also includes diagnostic information corresponding to the multiple types of medical images, and the screening of the multiple medical images corresponding to the multiple perspectives to obtain the multiple screened medical images corresponding to the multiple perspectives includes: According to the diagnosis information respectively corresponding to the multiple types of medical images, determining a first medical image associated with the diagnosis information from the multiple medical images respectively corresponding to the multiple perspectives; If the viewing angle corresponding to the first medical image covers the multiple viewing angles, determining, based on the first medical image, a plurality of filtered medical images corresponding to the multiple viewing angles respectively; If the perspective corresponding to the first medical image does not cover the multiple perspectives, the first medical image is subjected to perspective expansion processing to obtain a second medical image; and based on the first medical image and the second medical image, a plurality of filtered medical images corresponding to the multiple perspectives are determined.

4. The model building method according to claim 1, characterized in that The pre-configured myocardial physiological characteristic acquisition requirements include: myocardial cell excitation characteristic acquisition requirements, myocardial cell conduction characteristic acquisition requirements, and myocardial cell contraction characteristic acquisition requirements. Determining the second anisotropy parameter based on the pre-configured myocardial physiological characteristic acquisition requirements includes: Determining anisotropy parameters related to myocardial cell excitation according to the requirement for obtaining the myocardial cell excitation characteristics; Determining anisotropy parameters related to myocardial cell conduction according to the requirement for obtaining the myocardial cell conduction characteristics; Determining anisotropy parameters related to myocardial cell contraction according to the requirement for obtaining the myocardial cell contraction characteristics; A second anisotropy parameter is determined according to the anisotropy parameter related to myocardial cell excitation, the anisotropy parameter related to myocardial cell conduction, and the anisotropy parameter related to myocardial cell contraction.

5. The model building method according to claim 1, characterized in that: The step of determining the second myocardial physiological characteristic according to the electrocardiogram characteristic and the medical image characteristic by the decoder includes: fusing the first myocardial cell excitation feature and the second myocardial cell excitation feature through the decoder to obtain a target myocardial cell excitation feature; fusing the first myocardial cell contraction feature and the second myocardial cell contraction feature according to the target myocardial cell excitation feature by the decoder to obtain a target myocardial cell contraction feature; The second myocardial physiological characteristic is determined by the decoder according to the target myocardial cell excitation characteristic, the target myocardial contraction characteristic, the myocardial cell rhythm excitation characteristic and the myocardial cell conduction characteristic.

6. The model building method according to claim 1, characterized in that: The model building method further comprises: Acquire a first training data set and a second training data set, wherein the first training data set includes a plurality of first training samples, and the second training data set includes a plurality of second training samples, each first training sample includes: a medical image sample and a medical image feature corresponding to the medical image sample, and each second training sample includes: an electrocardiogram sample and an electrocardiogram feature corresponding to the electrocardiogram sample; Training the first branch according to the first training data set, and training the second branch according to the second training data set to obtain a pre-trained encoder; Acquire a third training data set, wherein the third training data set includes a plurality of third training samples, each of which includes: a medical image feature sample, an electrocardiogram feature sample, and a myocardial physiological feature label; Training the decoder to be trained according to the third training data set to obtain a pre-trained decoder; According to the pre-trained encoder and the pre-trained decoder, the pre-trained two-branch variational autoencoder is obtained.

7. The model building method according to claim 1, characterized in that: The step of obtaining a myocardial tissue bionic unit model according to the first myocardial physiological characteristics, the second myocardial physiological characteristics, and the original heart 3D model includes: determining a target myocardial physiological characteristic according to the first myocardial physiological characteristic and the second myocardial physiological characteristic; generating a myocardial tissue biomimetic unit according to the target myocardial physiological characteristics and the original heart 3D model; determining identification information corresponding to the myocardial tissue bionic unit according to the target myocardial physiological characteristics, wherein the identification information is used to characterize a pathological state of the heart; A myocardial tissue bionic unit model is obtained according to the identification information and the myocardial tissue bionic unit.

8. The model building method according to claim 1, characterized in that: The model building method further includes: In response to detecting a model display request, determining a model display mode according to the model display request, the model display mode being a dynamic display mode or a static display mode; When the model display mode is a dynamic display mode, the original heart 3D model, the preset model segmentation animation and the myocardial tissue bionic unit model are displayed in sequence; When the model display mode is a static display mode, the original heart 3D model and the myocardial tissue bionic unit model are respectively displayed in a preset model display template.

Images