Myocardial tissue bionic unit model construction method based on digital twinning

Abstract

The invention relates to a cardiac muscle tissue bionic unit model construction method based on digital twinning, and relates to the technical field of digital twinning. Detailed anatomical structure data of the heart is obtained through medical image technologies such as MRI and CT, and the physiological state of the heart is comprehensively reflected in combination with electrophysiological data such as electrocardiogram. A computer vision technology is utilized to process multi-view MRI data of the heart, ventricle segmentation and three-dimensional reconstruction are realized, and a high-precision heart 3D model is generated. And electrical activity simulation: simulating an electrical activity transmission process of the heart by adopting an orthotropic Eikoral model, and reproducing electrophysiological characteristics of the cardiac muscle. And electrocardiogram backstepping: designing a double-branch variational auto-encoder architecture, and backstepping the characteristics of the cardiac muscle from electrocardiogram data to realize accurate evaluation of pathological states such as myocardial infarction.

Description

Technical Field

[0001] The present disclosure relates to the technical field of digital twins, and in particular, to a method for constructing a myocardial tissue bionic unit model based on digital twins. Background Art

[0002] Digital twins make full use of physical models, sensors, operation history and other data, integrate multi-disciplinary, multi-physical quantity, multi-scale, and multi-probability simulation processes, complete mapping in virtual space, and thus reflect the entire life cycle of the corresponding physical equipment.

[0003] With the development of digital twin technology, digital twin technology is being applied to more and more scenarios. In medical scenarios, bionic models can be established through digital twin technology, and the established bionic models can have multiple applications.

[0004] For example, in cardiac medical scenarios, digital twin technology can be used to build a heart model. Based on the constructed heart model, the pathological state of the heart can be studied to assist in the research and diagnosis of various heart diseases.

[0005] In the related art, the construction of the heart model only stays at the basic information dimension of the heart, such as structural characteristics, tissue distribution characteristics, etc., and cannot achieve a comprehensive reflection of the heart information. Summary of the invention

[0006] The purpose of the present disclosure is to provide a method for constructing a myocardial tissue bionic unit model based on digital twins. The method for constructing a myocardial tissue bionic unit model based on digital twins can construct a myocardial tissue bionic unit model that comprehensively reflects the physiological state of the heart and improve the applicability of the model.

[0007] In order to achieve the above-mentioned objectives, the present disclosure provides a method for constructing a myocardial tissue bionic unit model based on digital twins, including: obtaining medical imaging data and electrocardiogram data corresponding to the heart to be modeled; constructing an original heart 3D model based on the medical imaging data; simulating the electrical activity propagation process of the original heart 3D model through an orthogonal anisotropic model to obtain a first myocardial physiological feature of the original heart 3D model; determining a second myocardial physiological feature of the original heart 3D model based on the electrocardiogram data and the medical imaging data through a pre-trained dual-branch variational autoencoder; and obtaining a myocardial tissue bionic unit model based on the first myocardial physiological feature, the second myocardial physiological feature and the original heart 3D model.

[0008] Optionally, the medical image data includes multiple types of medical images, different types of medical images correspond to different medical imaging devices, and each type of medical image includes medical images from multiple perspectives. The constructing of the original heart 3D model based on the medical image data includes: determining, based on the multiple types of medical images, multiple medical images corresponding to the multiple perspectives, the multiple medical images corresponding to each perspective include at least one type of medical image; screening the multiple medical images corresponding to the multiple perspectives to obtain multiple screened medical images corresponding to the multiple perspectives; converting the multiple screened medical images corresponding to the multiple perspectives into model images for 3D reconstruction; and performing 3D modeling based on the model images to obtain the original heart 3D model.

[0009] Optionally, the medical image data also includes diagnostic information corresponding to the multiple types of medical images, and the screening of the multiple medical images corresponding to the multiple perspectives to obtain the multiple screened medical images corresponding to the multiple perspectives includes: determining, according to the diagnostic information corresponding to the multiple types of medical images, a first medical image having an association with the diagnostic information from the multiple medical images corresponding to the multiple perspectives; if the perspective corresponding to the first medical image covers the multiple perspectives, determining, according to the first medical image, the multiple screened medical images corresponding to the multiple perspectives; if the perspective corresponding to the first medical image does not cover the multiple perspectives, performing perspective expansion processing on the first medical image to obtain a second medical image; and determining, according to the first medical image and the second medical image, the multiple screened medical images corresponding to the multiple perspectives.

[0010] Optionally, the electrical activity propagation process of the original heart 3D model is simulated by the orthogonal anisotropic model to obtain the first myocardial physiological characteristic of the original heart 3D model, including: determining a first anisotropy parameter according to the model parameters of the original heart 3D model; determining a second anisotropy parameter according to a pre-configured myocardial physiological characteristic acquisition requirement; determining an initial heart 3D model state corresponding to the electrical activity propagation process; and simulating the electrical activity propagation process of the original heart 3D model according to the first anisotropy parameter, the second anisotropy parameter and the initial heart 3D model state through the orthogonal anisotropic model to obtain the first myocardial physiological characteristic of the original heart 3D model.

[0011] Optionally, the pre-configured myocardial physiological characteristic acquisition requirements include: myocardial cell excitation characteristic acquisition requirements, myocardial cell conduction characteristic acquisition requirements and myocardial cell contraction characteristic acquisition requirements, and determining the second anisotropy parameter according to the pre-configured myocardial physiological characteristic acquisition requirements includes: determining anisotropy parameters related to myocardial cell excitation according to the myocardial cell excitation characteristic acquisition requirements; determining anisotropy parameters related to myocardial cell conduction according to the myocardial cell conduction characteristic acquisition requirements; determining anisotropy parameters related to myocardial cell contraction according to the myocardial cell contraction characteristic acquisition requirements; determining the second anisotropy parameter according to the anisotropy parameters related to myocardial cell excitation, the anisotropy parameters related to myocardial cell conduction and the anisotropy parameters related to myocardial cell contraction.

[0012] Optionally, the dual-branch autoencoder includes an encoder and a decoder, the encoder includes a first branch and a second branch, and the pre-trained dual-branch variational autoencoder determines the second myocardial physiological characteristics of the original heart 3D model according to the electrocardiogram data and the medical image data, including: performing feature extraction according to the electrocardiogram data through the first branch to obtain electrocardiogram features; performing feature extraction according to the medical image data through the second branch to obtain medical image features; and determining the second myocardial physiological characteristics according to the electrocardiogram features and the medical image features through the decoder.

[0013] Optionally, the electrocardiogram features include a first myocardial cell excitation feature, a myocardial cell rhythm excitation feature, a myocardial cell conduction feature and a first myocardial cell contraction feature, the medical image features include a second myocardial cell excitation feature and a second myocardial cell contraction feature, and determining the second myocardial physiological feature according to the electrocardiogram features and the medical image features by the decoder includes: fusing the first myocardial cell excitation feature and the second myocardial cell excitation feature by the decoder to obtain a target myocardial cell excitation feature; fusing the first myocardial cell contraction feature and the second myocardial cell contraction feature by the decoder according to the target myocardial cell excitation feature to obtain a target myocardial contraction feature; determining the second myocardial physiological feature by the decoder according to the target myocardial cell excitation feature, the target myocardial contraction feature, the myocardial cell rhythm excitation feature and the myocardial cell conduction feature.

[0014] Optionally, the model building method further includes: A first training data set and a second training data set are obtained, wherein the first training data set includes a plurality of first training samples, the second training data set includes a plurality of second training samples, each first training sample includes: a medical image sample and a medical image feature corresponding to the medical image sample, and each second training sample includes: an electrocardiogram sample and an electrocardiogram feature corresponding to the electrocardiogram sample; the first branch is trained according to the first training data set, and the second branch is trained according to the second training data set to obtain a pre-trained encoder; a third training data set is obtained, wherein the third training data set includes a plurality of third training samples, and each third training sample includes: a medical image feature sample, an electrocardiogram feature sample, and a myocardial physiological feature label; a decoder to be trained is trained according to the third training data set to obtain a pre-trained decoder; the pre-trained two-branch variational autoencoder is obtained according to the pre-trained encoder and the pre-trained decoder.

[0015] Optionally, the myocardial tissue bionic unit model is obtained according to the first myocardial physiological characteristics, the second myocardial physiological characteristics and the original heart 3D model, including: determining target myocardial physiological characteristics according to the first myocardial physiological characteristics and the second myocardial physiological characteristics; generating a myocardial tissue bionic unit according to the target myocardial physiological characteristics and the original heart 3D model; determining identification information corresponding to the myocardial tissue bionic unit according to the target myocardial physiological characteristics, the identification information being used to characterize the pathological state of the heart; and obtaining a myocardial tissue bionic unit model according to the identification information and the myocardial tissue bionic unit.

[0016] Optionally, the model building method also includes: in response to detecting a model display request, determining a model display mode according to the model display request, the model display mode being a dynamic display mode or a static display mode; when the model display mode is the dynamic display mode, displaying the original heart 3D model, the preset model segmentation animation and the myocardial tissue bionic unit model in sequence; when the model display mode is the static display mode, displaying the original heart 3D model and the myocardial tissue bionic unit model respectively in a preset model display template.

[0017] Through the above technical solution, a high-precision 3D model of the heart is established through medical imaging data. The orthogonal anisotropic model is used to simulate the propagation process of the heart's electrical activity and reproduce the electrophysiological characteristics of the myocardium. Through the dual-branch variational autoencoder architecture, the physiological characteristics of the myocardium are determined based on electrocardiogram data and medical imaging data. Furthermore, the construction of a myocardial tissue bionic unit is realized by combining multi-dimensional electrophysiological characteristics and a high-precision 3D model of the heart. The myocardial tissue bionic unit is constructed based on multi-dimensional electrophysiological characteristics, which can comprehensively reflect the physiological state of the heart and has strong applicability. In the evaluation scenario of the pathological state of the heart, it can assist in the accurate evaluation of the pathological state of the heart.

[0018] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 It is a schematic diagram showing an application scenario according to an exemplary embodiment.

[0020] Figure 2 It is a flowchart of a method for constructing a myocardial tissue bionic unit model based on digital twins according to an exemplary embodiment.

[0021] Figure 3 It is a schematic diagram of an application of an orthotropic model according to an exemplary embodiment.

[0022] Figure 4 It is a block diagram of a dual-branch autoencoder according to an exemplary embodiment.

[0023] Figure 5 The figure is a schematic diagram of a model dynamic display effect according to an exemplary embodiment.

[0024] Figure 6 It is a block diagram of a device for constructing a myocardial tissue bionic unit model based on digital twins according to an exemplary embodiment.

[0025] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0026] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0027] Digital twins make full use of physical models, sensors, operation history and other data, integrate multi-disciplinary, multi-physical quantity, multi-scale, and multi-probability simulation processes, complete mapping in virtual space, and thus reflect the entire life cycle of the corresponding physical equipment.

[0028] With the development of digital twin technology, digital twin technology is being applied to more and more scenarios. In medical scenarios, bionic models can be established through digital twin technology, and the established bionic models can have multiple applications.

[0029] For example, in cardiac medical scenarios, digital twin technology can be used to build a heart model. Based on the constructed heart model, the pathological state of the heart can be studied to assist in the research and diagnosis of various heart diseases.

[0030] In the related art, the construction of the heart model only stays at the basic information dimension of the heart, such as structural characteristics, tissue distribution characteristics, etc., and cannot achieve a comprehensive reflection of the heart information.

[0031] Based on this, the embodiment of the present disclosure provides a technical solution to establish a high-precision 3D heart model through medical imaging data. The orthogonal anisotropic model is used to simulate the propagation process of the heart's electrical activity and reproduce the electrophysiological characteristics of the myocardium. The dual-branch variational autoencoder architecture is used to determine the physiological characteristics of the myocardium based on electrocardiogram data and medical imaging data.

[0032] Furthermore, the myocardial tissue bionic unit is constructed by combining multi-dimensional electrophysiological characteristics and a high-precision 3D heart model. The myocardial tissue bionic unit is constructed based on multi-dimensional electrophysiological characteristics, which can fully reflect the physiological state of the heart and has strong applicability. In the evaluation scenario of the pathological state of the heart, it can assist in the accurate evaluation of the pathological state of the heart.

[0033] Figure 1 is a schematic diagram showing an application scenario according to an exemplary embodiment. Figure 1 As shown, in this application scenario, it includes: a modeling platform, a medical imaging device and an ECG data acquisition device.

[0034] Among them, the modeling platform can be a host computer or other equipment with modeling capabilities, the medical imaging equipment can be a CT (Computed Tomography) device, MRI (Magnetic Resonance Imaging) device, etc., and the ECG data acquisition device can be an ECG data acquisition instrument.

[0035] In some embodiments, the medical imaging device can collect medical imaging data and synchronize it to the modeling platform. The ECG data acquisition device can collect ECG data and synchronize it to the modeling platform. Thus, the modeling platform can build a model based on the medical imaging data and the ECG data.

[0036] Figure 2 is a flowchart of a method for constructing a myocardial tissue bionic unit model based on digital twins according to an exemplary embodiment. Figure 2 As shown, this method can be applied to Figure 1 The modeling platform shown, the method comprises the following steps: Step S21, obtaining medical image data and electrocardiogram data corresponding to the heart to be modeled.

[0037] Step S22: constructing an original heart 3D model based on the medical imaging data.

[0038] Step S23, simulating the electrical activity propagation process of the original heart 3D model by using an orthogonal anisotropic model to obtain the first myocardial physiological characteristics of the original heart 3D model.

[0039] Step S24, determining the second myocardial physiological characteristics of the original heart 3D model according to the electrocardiogram data and the medical imaging data through the pre-trained dual-branch variational autoencoder.

[0040] Step S25, obtaining a myocardial tissue bionic unit model according to the first myocardial physiological characteristics, the second myocardial physiological characteristics and the original heart 3D model.

[0041] In step S21, the heart to be modeled can be a virtual heart, a heart to be pathologically studied, or a real heart, a heart to be pathologically diagnosed. In different scenarios, a heart model can be constructed when there is a need for heart modeling.

[0042] In some embodiments, the medical imaging data is data acquired by a medical imaging device.

[0043] As an implementation mode, the medical image data includes multiple types of medical images. Different types of medical images correspond to different medical imaging devices. Each type of medical image includes medical images from multiple perspectives.

[0044] In some embodiments, multiple viewing angles may be configured according to the structural anatomy (analysis) requirements of the heart. For example, if the structural anatomy requirements of the heart are from the inside to the outside, the multiple viewing angles may be multiple cross-sectional viewing angles of the heart. Therefore, in different scenarios, medical images of multiple viewing angles may be collected according to different requirements.

[0045] In some embodiments, medical images from multiple perspectives may be collected using multiple medical imaging devices to obtain different types of medical images.

[0046] Therefore, in some embodiments, the multiple viewing angles may also be viewing angles that match the acquisition method of the medical imaging device.

[0047] In some embodiments, the electrocardiogram data may reflect the potential changes of the heart under stimulation, and may be in the form of waveforms or intuitive potential data, which are not limited here.

[0048] In step S22, the original heart 3D model can be constructed using the medical image data. As for the construction technology of the 3D model, mature technologies in the art can be adopted.

[0049] In some embodiments, in order to realize the construction of a 3D model, it is necessary to obtain an image for constructing the 3D model, so that rendering, mapping, etc. can be performed according to the image to realize the construction of the 3D model.

[0050] Therefore, as an optional implementation, step S22 includes: determining, based on multiple types of medical images, multiple medical images corresponding to multiple perspectives, wherein the multiple medical images corresponding to each perspective include at least one type of medical image; screening the multiple medical images corresponding to the multiple perspectives to obtain multiple screened medical images corresponding to the multiple perspectives; converting the multiple screened medical images corresponding to the multiple perspectives into model images for 3D reconstruction; and performing 3D modeling based on the model images to obtain an original heart 3D model.

[0051] In this implementation, multiple types of medical images can be first classified according to viewing angles to obtain multiple medical images at different viewing angles. These multiple medical images may involve only one type, that is, they are collected by one medical imaging device, or they may involve multiple types, that is, they are collected by multiple medical imaging devices.

[0052] Furthermore, screening can be performed based on multiple medical images corresponding to multiple perspectives.

[0053] As an optional implementation, the medical image data also includes diagnostic information corresponding to various types of medical images. The diagnostic information may be diagnostic information given by a doctor or diagnostic information given by a medical image acquisition device, which may be used to characterize whether there is a corresponding problem with the heart. Thus, the medical images may be screened using the diagnostic information.

[0054] Furthermore, as an optional implementation, the multiple medical images corresponding to the multiple perspectives are screened to obtain multiple screened medical images corresponding to the multiple perspectives, including: according to the diagnostic information corresponding to the multiple types of medical images, determining a first medical image that has an association with the diagnostic information from the multiple medical images corresponding to the multiple perspectives; if the perspective corresponding to the first medical image covers multiple perspectives, determining multiple screened medical images corresponding to the multiple perspectives based on the first medical image; if the perspective corresponding to the first medical image does not cover multiple perspectives, performing perspective expansion processing on the first medical image to obtain a second medical image; and determining multiple screened medical images corresponding to the multiple perspectives based on the first medical image and the second medical image.

[0055] In some embodiments, different medical images may correspond to different cardiac structures, and the diagnostic information may indicate whether there are problems with each structure, for example, whether there are problems with the ventricle, atrium, or cardiovascular system. Thus, medical images with an associated relationship and diagnostic information indicating problems may correspond to the same cardiac structure.

[0056] For example, if the diagnosis information includes: there is no abnormality in the ventricle and atrium, but there is an abnormality in the cardiovascular system, then the medical image with a diagnosis relationship based on an association relationship is a medical image related to the cardiovascular system.

[0057] Furthermore, it can be determined whether the perspective corresponding to the first medical image covers multiple perspectives. If so, it means that the first medical image has comprehensive perspectives, so that the first medical image can be directly determined as multiple screened medical images corresponding to multiple perspectives.

[0058] If not, it means that the first medical image does not have the comprehensiveness of the viewing angle. At this time, the first medical image can be processed by viewing angle expansion to obtain the second medical image to ensure the comprehensiveness of the viewing angle.

[0059] In some embodiments, a medical image at a perspective that is not overlapping with the current perspective may be selected from medical images that are not associated with each other as an extended medical image, that is, a second medical image.

[0060] In some embodiments, the pre-configured medical image under multiple viewing angles may be used as an extended medical image, that is, a second medical image.

[0061] In some embodiments, the first medical image may be subjected to perspective-based image transformation processing to obtain a second medical image.

[0062] Furthermore, the first medical image and the second medical image may be used as final screened medical images.

[0063] Furthermore, based on a plurality of screened medical images corresponding to a plurality of perspectives, the model image is converted to obtain an image that can be used for 3D modeling.

[0064] The image conversion processing may include: image grayscale processing, image contrast enhancement processing, image denoising processing, etc. In different application scenarios, different conversion processing methods may be used according to the requirements for 3D modeling images, which are not limited here.

[0065] Further, based on the model image, 3D modeling can be performed to obtain the original heart 3D model. Regarding the 3D modeling technology, reference can be made to the mature digital twin technology in the field, which will not be introduced in detail here. For example, modeling can be achieved through mature modeling software.

[0066] In step S23, the electrical activity propagation process of the original heart 3D model is simulated by using an orthogonal anisotropic model to obtain the first myocardial physiological characteristics of the original heart 3D model.

[0067] The orthotropic model may be an Eikonal model. The orthotropic Eikonal model is a mathematical model for describing wave propagation in an orthotropic medium.

[0068] In the study of myocardial tissue, the arrangement of myocardial fibers has obvious anisotropy, and the orthogonal anisotropic Eikonal model can be used to simulate the propagation process of myocardial electrical activity. This model can more accurately reflect the electrical conduction characteristics of myocardial cells in different directions, thus providing support for the mechanism research and diagnosis of diseases such as arrhythmias.

[0069] Therefore, this model can be used to analyze the anisotropy of myocardial tissue to obtain myocardial psychological characteristics.

[0070] In some embodiments, the application of the orthotropic Eikonal model needs to involve: anisotropic parameters and initial state parameters. By configuring these two parameters, the model can simulate the propagation process to obtain corresponding characteristics.

[0071] Therefore, step S23 may include: determining a first anisotropy parameter based on the model parameters of the original heart 3D model; determining a second anisotropy parameter based on a pre-configured myocardial physiological characteristic acquisition requirement; determining an initial heart 3D model state corresponding to the electrical activity propagation process; and simulating the electrical activity propagation process of the original heart 3D model through an orthogonal anisotropic model based on the first anisotropy parameter, the second anisotropy parameter and the initial heart 3D model state to obtain the first myocardial physiological characteristic of the original heart 3D model.

[0072] In some embodiments, the first anisotropic parameter may include: elastic modulus, Poisson's ratio and shear modulus. The first anisotropic parameter may be determined according to the model parameters of the original heart 3D model. For example, after the model is constructed, the parameters of the constructed model may be statistically analyzed, and the model parameters may involve structural parameters such as materials and dimensions. Based on these structural parameters, anisotropic parameters such as elastic modulus, Poisson's ratio and shear modulus related to mechanics may be determined.

[0073] In some embodiments, the second anisotropy parameter may be a specific parameter associated with the propagation of electrical activity of the heart.

[0074] In some embodiments, the preconfigured myocardial physiological characteristics acquisition requirements may include: myocardial cell excitation characteristics acquisition requirements, myocardial cell conduction characteristics acquisition requirements, and myocardial cell contraction characteristics acquisition requirements.

[0075] Regarding the physiological characteristics of the myocardium, they may include excitation characteristics, rhythmic excitation characteristics, conduction characteristics and contraction characteristics.

[0076] Excitation characteristics: Myocardial cells can become excited after being stimulated, which is manifested by the generation of action potentials and mechanical contraction on the cell membrane. Excitation characteristics have periodic changes, including effective refractory period, relative refractory period and supernormal period.

[0077] Characteristics of rhythmic excitation: Myocardial cells can automatically generate rhythmic excitation without external stimulation. The sinoatrial node has the highest autonomy and is the pacemaker of normal heartbeat, followed by the atrioventricular junction and Purkinje fibers.

[0078] Conduction characteristics: Cardiac myocytes have the ability to conduct excitement, and excitement conduction occurs not only within the same cardiomyocyte, but also between cells. The conduction speed varies in different parts, for example, the conduction speed is the slowest at the atrioventricular junction and the fastest in the ventricle.

[0079] Contraction characteristics: After being stimulated, myocardial cells produce mechanical contraction through excitation-contraction coupling. Myocardial contraction has the following characteristics: "all or nothing" contraction, that is, myocardial cells contract synchronously; no tetanic contraction occurs because the effective refractory period of the myocardium is particularly long; and it is highly dependent on the extracellular calcium ion concentration.

[0080] Therefore, when the requirements for obtaining the characteristics of myocardial cell excitation, the requirements for obtaining the characteristics of myocardial cell conduction, and the requirements for obtaining the characteristics of myocardial cell contraction are configured, anisotropy parameters associated with different characteristics can be determined.

[0081] In some embodiments, the anisotropy parameter associated with myocardial cell excitation may be a change period for the aforementioned first anisotropy parameter. The anisotropy parameter associated with myocardial cell conduction may be a change speed for the aforementioned first anisotropy parameter. The anisotropy parameter associated with myocardial cell contraction may be a change period for the aforementioned first anisotropy parameter.

[0082] Furthermore, the anisotropy parameter related to the excitation of the myocardial cells, the anisotropy parameter related to the conduction of the myocardial cells, and the anisotropy parameter related to the contraction of the myocardial cells may be deduplicated to obtain a second anisotropy parameter.

[0083] Therefore, the second anisotropy parameter can also be understood as a description / definition parameter for the first anisotropy parameter.

[0084] In some embodiments, the initial heart 3D model state corresponding to the electrical activity propagation process may be the heart 3D model state in a quiet state, where various features are less obvious and there is no corresponding stimulation.

[0085] Furthermore, by utilizing the simulation characteristics of the orthogonal anisotropic model, based on the first anisotropy parameter, the second anisotropy parameter and the initial heart 3D model state, the electrical activity propagation process of the original heart 3D model is simulated, and the first myocardial physiological characteristics of the original heart 3D model can be obtained.

[0086] Among them, regarding the specific simulation method of the orthotropic model, reference can be made to the mature technology in the field, and the embodiment of the present disclosure mainly configures the input parameters required for its simulation.

[0087] In some embodiments, the first myocardial physiological characteristic may include at least one of an excitation characteristic, a rhythmic excitation characteristic, a conduction characteristic, and a contraction characteristic.

[0088] Figure 3 is a schematic diagram of an application of an orthotropic model according to an exemplary embodiment. Figure 3 As shown, by inputting the first anisotropy parameter, the second anisotropy parameter and the initial heart 3D model state into the orthogonal anisotropic model, the orthogonal anisotropic model can output corresponding myocardial physiological characteristics.

[0089] In step S24, the second myocardial physiological characteristics of the original heart 3D model are determined based on the electrocardiogram data and the medical imaging data through the pre-trained dual-branch variational autoencoder.

[0090] The dual-branch variational autoencoder is an extended variational autoencoder architecture that is designed to process different types of data inputs through two independent encoder branches and perform feature fusion and output through a shared decoder. This architecture is particularly suitable for multimodal data processing. Therefore, the encoder can be used to combine electrogram data and medical imaging data to determine myocardial physiological characteristics.

[0091] Figure 4 is a block diagram of a dual-branch autoencoder according to an exemplary embodiment, such as Figure 4 As shown, the dual-branch autoencoder includes an encoder and a decoder, and the encoder includes a first branch and a second branch.

[0092] Combination Figure 4 Step S24 may include: extracting features based on the electrocardiogram data through the first branch to obtain electrocardiogram features; extracting features based on the medical image data through the second branch to obtain medical image features; and determining the second myocardial physiological features based on the electrocardiogram features and the medical image features through the decoder.

[0093] In this embodiment, the first branch is used to extract electrocardiogram features, the second branch is used to extract medical image features, and the decoder is used to perform feature fusion based on the two features to obtain a second myocardial physiological feature.

[0094] As an optional embodiment, the electrocardiogram characteristics include a first myocardial cell excitation characteristic, a myocardial cell rhythm excitation characteristic, a myocardial cell conduction characteristic and a first myocardial cell contraction characteristic, and the medical imaging characteristics include a second myocardial cell excitation characteristic and a second myocardial cell contraction characteristic.

[0095] Correspondingly, the decoder determines the second myocardial physiological feature according to the electrocardiogram features and the medical image features, including: fusing the first myocardial cell excitation feature and the second myocardial cell excitation feature through the decoder to obtain the target myocardial cell excitation feature; fusing the first myocardial cell contraction feature and the second myocardial cell contraction feature according to the target myocardial cell excitation feature through the decoder to obtain the target myocardial contraction feature; determining the second myocardial physiological feature according to the target myocardial cell excitation feature, the target myocardial contraction feature, the myocardial cell rhythm excitation feature and the myocardial cell conduction feature through the decoder.

[0096] In this embodiment, the myocardial cell excitation features obtained by different data extractions can be fused. Also, the myocardial cell contraction features obtained by different data extractions can be fused.

[0097] Furthermore, the myocardial cell contraction characteristics are associated with the myocardial cell excitation characteristics. Therefore, when fusing the myocardial cell contraction characteristics, the myocardial cell excitation characteristics can be referred to.

[0098] Furthermore, the decoder can ultimately determine the final second myocardial physiological characteristics based on the four types of characteristics.

[0099] It can be understood that the second myocardial physiological characteristics may also include at least one of the first myocardial cell excitation characteristics, myocardial cell rhythm excitation characteristics, myocardial cell conduction characteristics and first myocardial cell contraction characteristics, but the characteristics here are fusion characteristics, which may take into account the mutual influence between the features, and have higher accuracy than the extraction of a single feature.

[0100] In some embodiments, the training of the dual-branch autoencoder may include: obtaining a first training data set and a second training data set, the first training data set including multiple first training samples, the second training data set including multiple second training samples, each first training sample including: a medical image sample and a medical image feature corresponding to the medical image sample, each second training sample including: an electrocardiogram sample and an electrocardiogram feature corresponding to the electrocardiogram sample; training the first branch according to the first training data set, and training the second branch according to the second training data set to obtain a pre-trained encoder; obtaining a third training data set, the third training data set including multiple third training samples, each third training sample including: a medical image feature sample, an electrocardiogram feature sample and a myocardial physiological feature label; training the decoder to be trained according to the third training data set to obtain a pre-trained decoder; obtaining a pre-trained dual-branch variational autoencoder according to the pre-trained encoder and the pre-trained decoder.

[0101] In this implementation, the first branch and the second branch are respectively used to extract corresponding features, so feature extraction needs to be performed based on corresponding samples and feature labels.

[0102] Also, for the decoder, feature fusion is required, so the training samples of the decoder need to involve pre-fusion features (i.e., medical image feature samples and electrocardiogram feature samples) and post-fusion features (i.e., myocardial physiological feature labels).

[0103] In some embodiments, different types of medical image feature samples and electrocardiogram feature samples may be combined to obtain a variety of feature combinations to be fused, and then these combinations may be used for training respectively to improve the generalization performance of the decoder.

[0104] Furthermore, after the encoder and decoder are trained, a pre-trained two-branch variational autoencoder can be constructed based on the input-output relationship between the encoder and the decoder.

[0105] Furthermore, in step S25, a myocardial tissue bionic unit model is obtained according to the first myocardial physiological characteristics, the second myocardial physiological characteristics and the original heart 3D model.

[0106] In some embodiments, the original heart 3D model does not have corresponding myocardial physiological characteristics. By reflecting the myocardial physiological characteristics on the model, a myocardial tissue bionic unit model can be obtained.

[0107] Therefore, as an optional implementation, step S25 includes: determining the target myocardial physiological characteristics based on the first myocardial physiological characteristics and the second myocardial physiological characteristics; generating a myocardial tissue bionic unit based on the target myocardial physiological characteristics and the original heart 3D model; determining the identification information corresponding to the myocardial tissue bionic unit based on the target myocardial physiological characteristics, the identification information is used to characterize the pathological state of the heart; and obtaining a myocardial tissue bionic unit model based on the identification information and the myocardial tissue bionic unit.

[0108] In this embodiment, the first myocardial physiological feature and the second myocardial physiological feature may be integrated first, and the integration method includes but is not limited to: weighting, deduplication, etc. Regarding deduplication, for features that are the same and of the same feature type, one may be retained. Regarding weighting, for features that are different and of the same feature type, weighting may be performed. Among them, the weight of the second myocardial physiological feature is higher than the weight of the first myocardial physiological feature.

[0109] In some embodiments, the original heart 3D model may be segmented or modified based on the physiological characteristics of the target myocardium to obtain a myocardial tissue bionic unit.

[0110] For example, if the target myocardial physiological characteristics involve contraction of a specific cardiac structural position, the myocardial tissue of that portion is presented in a contracted form, and the myocardial tissue of that portion can be cut out separately as an independent myocardial tissue bionic unit.

[0111] It can be understood that the number of myocardial tissue bionic units can be one or more. In the case of more than one, the multiple units can correspond to the same myocardial physiological characteristics or different myocardial physiological characteristics.

[0112] Furthermore, the pathological state of the heart can be evaluated based on the myocardial physiological characteristics and identification information can be generated.

[0113] Regarding how to evaluate the pathological state of the heart based on the physiological characteristics of the myocardium, reference can be made to the mature technology in the art, which will not be described in detail here. For example, if the contraction is abnormal, myocarditis may occur.

[0114] Furthermore, on the basis of the myocardial tissue bionic unit, identification information can be marked and integrated to obtain a myocardial tissue bionic unit model.

[0115] Therefore, the myocardial tissue bionic unit model can reflect the overall state of the heart from a three-dimensional perspective, and can also reflect the pathological state, so that the model can be applied in a variety of scenarios.

[0116] In some embodiments, the constructed model can be displayed. Therefore, the model construction method may also include: in response to detecting a model display request, determining a model display mode according to the model display request, the model display mode being a dynamic display mode or a static display mode; when the model display mode is a dynamic display mode, sequentially displaying the original heart 3D model, the preset model segmentation animation, and the myocardial tissue bionic unit model; when the model display mode is a static display mode, displaying the original heart 3D model and the myocardial tissue bionic unit model in a preset model display template.

[0117] In some embodiments, the model display request may be a request initiated by a user who has a need to view the model.

[0118] In some embodiments, the model display mode may be indicated in the model display request, and thus the display mode may be determined directly according to the request.

[0119] Furthermore, when the model display mode is a dynamic display mode, the original heart 3D model, the preset model segmentation animation and the myocardial tissue bionic unit model are displayed in sequence.

[0120] The preset model segmentation animation can be configured in different scenes according to different requirements and is not limited here.

[0121] Through this implementation, a dynamic display effect can be provided to users viewing the model, thereby improving the user experience.

[0122] When the model display mode is a static display mode, the original heart 3D model and the myocardial tissue bionic unit model are respectively displayed in the preset model display template.

[0123] The preset model display template can be used to limit the display position and display form of the two models, for example, to enlarge and display the original heart 3D model, to partially display the myocardial tissue bionic unit model, etc.

[0124] Figure 5 is a schematic diagram of a model dynamic display effect according to an exemplary embodiment. Figure 5As shown, on the interactive interface, the original heart 3D model and related information of the model are first displayed. Then, the preset model segmentation animation is played. The specific effect of the preset model segmentation animation is a dynamic effect, which is not convenient to show, so it is not shown in the figure. Further, after the preset model segmentation animation is played, the myocardial tissue bionic unit model and identification information are presented.

[0125] In some embodiments, the model can also be saved and directly retrieved and applied when needed.

[0126] It can be seen that in the disclosed embodiments, medical imaging technologies such as MRI and CT are used to obtain detailed anatomical data of the heart, combined with electrophysiological data such as electrocardiograms to comprehensively reflect the physiological state of the heart. Computer vision technology is used to process multi-view MRI data of the heart to achieve ventricular segmentation and three-dimensional reconstruction, and generate a high-precision 3D model of the heart. Electrical activity simulation: The orthogonal anisotropic Eikonal model is used to simulate the propagation process of the heart's electrical activity and reproduce the electrophysiological characteristics of the myocardium. Electrocardiogram inversion: A dual-branch variational autoencoder architecture is designed to infer the characteristics of the myocardium from electrocardiogram data, and to achieve accurate evaluation of pathological conditions such as myocardial infarction.

[0127] Figure 6 is a block diagram of a device for constructing a myocardial tissue bionic unit model based on digital twins according to an exemplary embodiment, such as Figure 6 As shown, the device comprises: The acquisition module 601 is used to acquire medical image data and electrocardiogram data corresponding to the heart to be modeled.

[0128] The construction module 602 is used to construct an original heart 3D model according to the medical image data.

[0129] The feature extraction module 603 is used to simulate the electrical activity propagation process of the original heart 3D model through an orthogonal anisotropic model to obtain the first myocardial physiological feature of the original heart 3D model.

[0130] The feature extraction module 603 is also used to determine the second myocardial physiological feature of the original heart 3D model according to the electrocardiogram data and the medical image data through a pre-trained dual-branch variational autoencoder.

[0131] The construction module 602 is further used to obtain a myocardial tissue bionic unit model according to the first myocardial physiological characteristics, the second myocardial physiological characteristics and the original heart 3D model.

[0132] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0133] Figure 7 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 may include: a processor 701 , a memory 702 . The electronic device 700 may also include one or more of a multimedia component 703 , an input / output (I / O) interface 704 , and a communication component 705 .

[0134] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned method for constructing a myocardial tissue bionic unit model based on digital twins. The memory 702 is used to store various types of data to support the operation of the electronic device 700, and these data may include, for example, instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the above-mentioned other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module.

[0135] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned digital twin-based myocardial tissue bionic unit model construction method.

[0136] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by the processor, the steps of the above-mentioned method for constructing a myocardial tissue bionic unit model based on digital twins are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions can be executed by the processor 701 of the electronic device 700 to complete the above-mentioned method for constructing a myocardial tissue bionic unit model based on digital twins.

[0137] In another exemplary embodiment, a computer program product is also provided, which includes a computer program that can be executed by a processor, and when the computer program is executed by the processor, the steps of the above-mentioned method for constructing a myocardial tissue bionic unit model based on digital twins are implemented.

[0138] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0139] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0140] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for constructing a myocardial tissue bionic unit model based on digital twins, characterized in that: include: Obtain medical imaging data and electrocardiogram data corresponding to the heart to be modeled; constructing an original heart 3D model according to the medical imaging data; Simulating the electrical activity propagation process of the original heart 3D model by using an orthogonal anisotropic model to obtain a first myocardial physiological characteristic of the original heart 3D model; Determining, by a pre-trained dual-branch variational autoencoder, a second myocardial physiological feature of the original heart 3D model according to the electrocardiogram data and the medical imaging data; A myocardial tissue bionic unit model is obtained according to the first myocardial physiological characteristics, the second myocardial physiological characteristics and the original heart 3D model.

2. The model building method according to claim 1, characterized in that: The medical image data includes multiple types of medical images. Different types of medical images correspond to different medical imaging devices. Each type of medical image includes medical images from multiple perspectives. The constructing of the original heart 3D model according to the medical image data includes: Determining, according to the multiple types of medical images, a plurality of medical images corresponding to the multiple perspectives, wherein the plurality of medical images corresponding to each perspective include at least one type of medical image; Screening the plurality of medical images respectively corresponding to the plurality of perspectives to obtain a plurality of screened medical images respectively corresponding to the plurality of perspectives; Converting the plurality of screened medical images corresponding to the plurality of viewing angles into model images for 3D reconstruction; 3D modeling is performed according to the model image to obtain the original heart 3D model.

3. The model building method according to claim 2, characterized in that: The medical image data also includes diagnostic information corresponding to the multiple types of medical images, and the screening of the multiple medical images corresponding to the multiple perspectives to obtain the multiple screened medical images corresponding to the multiple perspectives includes: According to the diagnosis information respectively corresponding to the multiple types of medical images, determining a first medical image associated with the diagnosis information from the multiple medical images respectively corresponding to the multiple perspectives; If the viewing angle corresponding to the first medical image covers the multiple viewing angles, determining, according to the first medical image, multiple screened medical images corresponding to the multiple viewing angles respectively; If the viewing angle corresponding to the first medical image does not cover the multiple viewing angles, the first medical image is subjected to viewing angle expansion processing to obtain a second medical image; and multiple screened medical images corresponding to the multiple viewing angles are determined based on the first medical image and the second medical image.

4. The model building method according to claim 1, characterized in that: The simulating the electrical activity propagation process of the original heart 3D model by using the orthogonal anisotropic model to obtain the first myocardial physiological characteristics of the original heart 3D model includes: Determining a first anisotropy parameter according to the model parameters of the original heart 3D model; determining a second anisotropy parameter according to a pre-configured myocardial physiological characteristic acquisition requirement; Determine the initial heart 3D model state corresponding to the electrical activity propagation process; The electrical activity propagation process of the original heart 3D model is simulated by the orthogonal anisotropic model according to the first anisotropy parameter, the second anisotropy parameter and the initial heart 3D model state to obtain the first myocardial physiological characteristics of the original heart 3D model.

5. The model building method according to claim 4, characterized in that: The pre-configured myocardial physiological characteristic acquisition requirements include: myocardial cell excitation characteristic acquisition requirements, myocardial cell conduction characteristic acquisition requirements and myocardial cell contraction characteristic acquisition requirements. The second anisotropy parameter is determined according to the pre-configured myocardial physiological characteristic acquisition requirements, including: According to the requirement of obtaining the excitation characteristics of the myocardial cells, determining anisotropy parameters related to the excitation of the myocardial cells; According to the requirement of obtaining the conduction characteristics of the myocardial cells, determining anisotropy parameters related to the conduction of the myocardial cells; Determining anisotropy parameters related to myocardial cell contraction according to the requirement for obtaining the myocardial cell contraction characteristics; A second anisotropy parameter is determined according to the anisotropy parameter associated with myocardial cell excitation, the anisotropy parameter associated with myocardial cell conduction, and the anisotropy parameter associated with myocardial cell contraction.

6. The model building method according to claim 1, characterized in that: The dual-branch autoencoder includes an encoder and a decoder, the encoder includes a first branch and a second branch, and the pre-trained dual-branch variational autoencoder determines the second myocardial physiological feature of the original heart 3D model according to the electrocardiogram data and the medical image data, including: Extracting features based on the electrocardiogram data through the first branch to obtain electrocardiogram features; Extracting features from the medical image data through the second branch to obtain medical image features; The decoder determines the second myocardial physiological characteristics according to the electrocardiogram characteristics and the medical image characteristics.

7. The model building method according to claim 6, characterized in that: The electrocardiogram feature includes a first myocardial cell excitation feature, a myocardial cell rhythm excitation feature, a myocardial cell conduction feature, and a first myocardial cell contraction feature, the medical image feature includes a second myocardial cell excitation feature and a second myocardial cell contraction feature, and determining the second myocardial physiological feature according to the electrocardiogram feature and the medical image feature by the decoder includes: fusing the first myocardial cell excitation feature and the second myocardial cell excitation feature through the decoder to obtain a target myocardial cell excitation feature; The decoder fuses the first myocardial cell contraction feature and the second myocardial cell contraction feature according to the target myocardial cell excitation feature to obtain a target myocardial cell contraction feature; The second myocardial physiological characteristic is determined by the decoder according to the target myocardial cell excitation characteristic, the target myocardial contraction characteristic, the myocardial cell rhythm excitation characteristic and the myocardial cell conduction characteristic.

8. The model building method according to claim 6, characterized in that: The model building method also includes: Acquire a first training data set and a second training data set, wherein the first training data set includes a plurality of first training samples, and the second training data set includes a plurality of second training samples, each of the first training samples includes: a medical image sample and a medical image feature corresponding to the medical image sample, and each of the second training samples includes: an electrocardiogram sample and an electrocardiogram feature corresponding to the electrocardiogram sample; Training the first branch according to the first training data set, and training the second branch according to the second training data set to obtain a pre-trained encoder; Acquire a third training data set, wherein the third training data set includes a plurality of third training samples, each of which includes: a medical image feature sample, an electrocardiogram feature sample, and a myocardial physiological feature label; Training the decoder to be trained according to the third training data set to obtain a pre-trained decoder; According to the pre-trained encoder and the pre-trained decoder, the pre-trained two-branch variational autoencoder is obtained.

9. The model building method according to claim 1, characterized in that: The step of obtaining a myocardial tissue bionic unit model according to the first myocardial physiological characteristics, the second myocardial physiological characteristics and the original heart 3D model includes: determining a target myocardial physiological characteristic according to the first myocardial physiological characteristic and the second myocardial physiological characteristic; generating a myocardial tissue bionic unit according to the target myocardial physiological characteristics and the original heart 3D model; Determining identification information corresponding to the myocardial tissue bionic unit according to the target myocardial physiological characteristics, wherein the identification information is used to characterize a pathological state of the heart; A myocardial tissue bionic unit model is obtained according to the identification information and the myocardial tissue bionic unit.

10. The model building method according to claim 1, characterized in that: The model building method also includes: In response to detecting a model display request, determining a model display mode according to the model display request, wherein the model display mode is a dynamic display mode or a static display mode; When the model display mode is a dynamic display mode, the original heart 3D model, the preset model segmentation animation and the myocardial tissue bionic unit model are displayed in sequence; When the model display mode is a static display mode, the original heart 3D model and the myocardial tissue bionic unit model are respectively displayed in a preset model display template.

Images