Multi-modal cardiac image registration method, registration system and readable storage medium

Abstract

The present invention provides a cardiac multimodal image registration method, a registration system and a readable storage medium. The cardiac multimodal image registration method includes: obtaining first scan data through optical scanning and establishing a magnetocardiography (MCG) device model; registering a design model of an MCG sensor with the MCG sensor model to obtain a first transformation matrix between the design model and the MCG sensor model; obtaining second scan data through optical scanning and establishing a first torso model; obtaining a medical image of a target object and establishing a cardiac model and a second torso model; registering the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; obtaining the MCG data of the target object through the MCG sensor of the MCG device, and fusing the MCG data with the cardiac model based on the aligned sensor coordinate system, MCG device coordinate system and cardiac model coordinate system.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and particularly to a cardiac multimodal image registration method, a registration system and a readable storage medium. Background Art

[0002] A magnetocardiography (MCG) source imaging device is an imaging device that uses an atomic magnetometer to detect the magnetic field signals of the heart, and can be used for the auxiliary diagnosis of heart diseases such as heart failure, ischemic heart disease, arrhythmia, and fetal congenital heart disease. This device uses an atomic magnetometer to collect the magnetic signals on the surface of the patient's heart, and through feature analysis and extraction of the magnetic signals and combined with radiomics, the precise location of abnormal MCG signals is achieved, providing a reference for subsequent further treatment. The precise location of abnormal MCG signals requires the construction of a cardiac electrophysiological model, and precise tracing is achieved through forward modeling and inverse solution. Among them, the forward problem is used to describe the mathematical relationship between the cardiac endogenous source and the MCG signals measured by the MCG sensors, and the inverse problem reconstructs the source activity through the forward model and the measured MCG signals.

[0003] The core of MCG source imaging includes three parts: a cardiac electrophysiological model, the position of the sensor chamber, and the MCG signals. By segmenting, reconstructing, and meshing the CT images of the heart and combining the chamber position and MCG signals, the endogenous activity of the heart is reconstructed. Since the CT images and the MCG sensors are not in the same spatial coordinate system, it is difficult to accurately fuse the MCG signals with the cardiac electrophysiological model, resulting in difficulty in ensuring the accuracy of subsequent MCG tracing. Summary of the Invention

[0004] The purpose of the present invention is to provide a cardiac multimodal image registration method, a registration system and a readable storage medium to solve the problem in the prior art that it is difficult to accurately fuse the MCG signals with the cardiac electrophysiological model.

[0005] To solve the above technical problems, the present invention provides a cardiac multimodal image registration method, which includes:

[0006] Through optical scanning, obtain the first scan data of the MCG device, and establish an MCG device model based on the first scan data; obtain the design model of the MCG sensor of the MCG device, register the design model with the MCG sensor model in the MCG device model to obtain the first transformation matrix between the design model and the MCG sensor device model; align the MCG sensor coordinate system with the MCG device coordinate system according to the first transformation matrix;

[0007] Obtain the second scan data of the target object through optical scanning, and establish a first torso model based on the second scan data; obtain the medical image of the target object, and establish a heart model and a second torso model of the target object based on the medical image; register the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; align the heart model coordinate system with the magnetocardiography device coordinate system through the second transformation matrix;

[0008] Obtain the magnetocardiography data of the target object through the magnetocardiography sensor of the magnetocardiography device, and fuse the magnetocardiography data with the heart model based on the aligned sensor coordinate system, magnetocardiography device coordinate system and heart model coordinate system.

[0009] Optionally, when obtaining the medical image of the target object, synchronously obtain an electrocardiogram signal; obtain first gating information based on the electrocardiogram signal, where the first gating information reflects the systolic or diastolic phase of the heart; the heart model is established based on the medical image sequence corresponding to the first gating information.

[0010] Optionally, when obtaining the medical image of the target object, synchronously obtain a respiration signal; obtain second gating information based on the respiration signal, where the second gating information reflects the respiration state of the target object; the second torso model is established based on the medical image sequence corresponding to the second gating information.

[0011] Optionally, when obtaining the second scan data of the target object through optical scanning, correspond to the respiration state reflected by the second gating information, so that the first torso model and the second torso model are based on the same respiration state.

[0012] Optionally, the steps of registering the design model of the magnetocardiography sensor with the magnetocardiography sensor model in the magnetocardiography device model to obtain a first transformation matrix between the design model and the magnetocardiography sensor model include:

[0013] Select multiple groups of matching feature points on the design model and the magnetocardiography sensor model, align the matching feature points on the design model and the magnetocardiography sensor model through the iterative closest point algorithm to obtain a first rough alignment transformation matrix; use the first rough alignment transformation matrix to roughly align the design model and the magnetocardiography sensor model to obtain a first roughly aligned model;

[0014] Select multiple sets of matching feature points on the first rough alignment model and the magnetocardiogram sensor model, and align the matching feature points on the first rough alignment model and the magnetocardiogram sensor model through the iterative closest point algorithm to obtain a first fine alignment transformation matrix; use the first fine alignment transformation matrix to perform fine alignment on the first rough alignment model and the magnetocardiogram sensor model;

[0015] The first transformation matrix = the first rough alignment transformation matrix × the first fine alignment transformation matrix.

[0016] Optionally, the step of registering the second torso model and the first torso model to obtain a second transformation matrix between the second torso model and the first torso model includes:

[0017] Select multiple sets of matching feature points on the second torso model and the first torso model, and align the matching feature points on the second torso model and the first torso model through the iterative closest point algorithm to obtain a second rough alignment transformation matrix; use the second rough alignment transformation matrix to perform rough alignment on the second torso model and the first torso model to obtain a second rough alignment model;

[0018] Select multiple sets of matching feature points on the second rough alignment model and the first torso model, and align the matching feature points on the second rough alignment model and the first torso model through the iterative closest point algorithm to obtain a second fine alignment transformation matrix; use the second fine alignment transformation matrix to perform fine alignment on the second rough alignment model and the first torso model;

[0019] The second transformation matrix = the second rough alignment transformation matrix × the second fine alignment transformation matrix.

[0020] Optionally, the step of establishing a heart model and a second torso model of the target object based on the medical image includes:

[0021] Segment the medical image to obtain heart images of multiple time axes; establish the heart model based on the heart images;

[0022] Process the medical image with the heart image as a mask to obtain a torso image; establish the second torso model based on the torso image.

[0023] To solve the above technical problems, the present invention also provides a readable storage medium, on which a program is stored, and when the program is executed, the steps of the above-mentioned heart multimodal image registration method are implemented.

[0024] To solve the above technical problems, the present invention also provides a heart multimodal image registration system, which includes: an optical scanner, a magnetocardiogram device, and a registration execution module;

[0025] The optical scanner is used to acquire first scan data of the magnetocardiogram device and second scan data of a target object; the magnetocardiogram device includes a magnetocardiogram sensor, and the magnetocardiogram device is used to acquire magnetocardiogram data of the target object.

[0026] The registration execution module is configured to establish a magnetocardiogram device model based on the first scan data; acquire a design model of the magnetocardiogram sensor of the magnetocardiogram device, register the design model with the magnetocardiogram sensor model in the magnetocardiogram device model to obtain a first transformation matrix between the design model and the magnetocardiogram sensor model; align the magnetocardiogram sensor coordinate system with the magnetocardiogram device coordinate system according to the first transformation matrix; establish a first torso model based on the second scan data; acquire a medical image of the target object, and establish a heart model and a second torso model of the target object based on the medical image; register the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; align the heart model coordinate system with the magnetocardiogram device coordinate system through the second transformation matrix; fuse the magnetocardiogram data with the heart model based on the aligned sensor coordinate system, magnetocardiogram device coordinate system, and heart model coordinate system.

[0027] Optionally, the cardiac multimodal image registration system further includes an electrocardiogram acquisition device; the electrocardiogram acquisition device is used to synchronously acquire an electrocardiogram signal of the target object when acquiring a medical image of the target object.

[0028] The registration execution module is further configured to obtain first gating information based on the electrocardiogram signal, where the first gating information reflects the systolic or diastolic phase of the heart; the heart model is established based on the medical image sequence corresponding to the first gating information.

[0029] In summary, in the cardiac multimodal image registration method, registration system, and readable storage medium provided by the present invention, the cardiac multimodal image registration method includes: obtaining first scan data of a magnetocardiogram (MCG) device through optical scanning, and establishing an MCG device model based on the first scan data; obtaining a design model of an MCG sensor of the MCG device, registering the design model with the MCG sensor model in the MCG device model to obtain a first transformation matrix between the design model and the MCG sensor model; aligning the MCG sensor coordinate system with the MCG device coordinate system according to the first transformation matrix; obtaining second scan data of a target object through optical scanning, and establishing a first torso model based on the second scan data; obtaining a medical image of the target object, and establishing a cardiac model and a second torso model of the target object based on the medical image; registering the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; aligning the cardiac model coordinate system with the MCG device coordinate system through the second transformation matrix; obtaining MCG data of the target object through the MCG sensor of the MCG device, and fusing the MCG data with the cardiac model based on the aligned sensor coordinate system, MCG device coordinate system, and cardiac model coordinate system.

[0030] With such a configuration, by registering the design model of the MCG sensor with the MCG sensor model established based on optical scanning, and registering the first torso model established based on optical scanning with the second torso model established based on medical images, the alignment and unification of the sensor coordinate system, MCG device coordinate system, and cardiac model coordinate system are achieved, enabling the accurate fusion of MCG data and the cardiac model, providing reliable input for subsequent MCG source tracing. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Those of ordinary skill in the art will understand that the provided drawings are used to better understand the present invention and do not constitute any limitation to the scope of the present invention.

[0032] Figure 1 is a schematic diagram of an MCG device according to an embodiment of the present invention.

[0033] Figure 2 is a schematic diagram of an MCG panel according to an embodiment of the present invention.

[0034] Figure 3 is a schematic flowchart of a cardiac multimodal image registration method according to an embodiment of the present invention.

[0035] In the drawings: 1 - MCG device; 10 - MCG panel; 11 - bed body; 12 - MCG sensor; 2 - target object. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] To make the objectives, advantages and features of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings are in very simplified forms and are not drawn to scale, and are only used to conveniently and clearly assist in explaining the objectives of the embodiments of the present invention. In addition, the structures shown in the accompanying drawings are often part of the actual structures. In particular, the accompanying drawings need to show different emphases and sometimes use different scales.

[0037] As used in the present invention, the singular forms "a", "an", "one" and "the" include plural objects, the term "or" is generally used in the sense of including "and / or", the term "several" is generally used in the sense of including "at least one", the term "at least two" is generally used in the sense of including "two or more", in addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second", "third" may explicitly or implicitly include one or at least two of such features. "One end" and "the other end" and "proximal end" and "distal end" generally refer to two corresponding parts, which not only include the endpoints. In addition, as used in the present invention, "mounted", "connected", "coupled", a component "disposed" on another component should be understood in a broad sense, and generally only means that there is a connection, coupling, cooperation or transmission relationship between the two components, and the two components can be directly or indirectly connected, coupled, cooperated or transmitted through an intermediate component, and cannot be construed as indicating or implying the spatial position relationship between the two components, that is, a component can be in any position such as inside, outside, above, below or on one side of another component, unless otherwise explicitly specified in the content. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, directional terms such as above, below, up, down, upward, downward, left, right, etc. are used relative to the exemplary embodiments as shown in the figures, and the upward or upward direction faces the top of the corresponding figure, and the downward or downward direction faces the bottom of the corresponding figure.

[0038] The objective of the present invention is to provide a cardiac multi-modal image registration method, a registration system and a readable storage medium to solve the problem in the prior art that it is difficult to accurately fuse the cardiac magnetic signals and the cardiac electrophysiological model. The following is a description with reference to the accompanying drawings.

[0039] Please refer to Figure 1 , which shows a magnetocardiogram device 1, and the magnetocardiogram device 1 includes a bed body 11 and a magnetocardiogram sensor 12. The bed body 11 is used for the target object 2 (such as a patient) to lie on for scanning, and the magnetocardiogram sensor 12 is used to acquire the magnetocardiogram signals of the target object 2 lying on the bed body 11. As Figure 2As shown, in a demonstration example, the number of magnetocardiogram sensors 12 can be multiple, and preferably they are arranged in an array on the magnetocardiogram panel 10. The specific structure and detection principle of the magnetocardiogram device 1 can refer to the prior art and will not be elaborated here. Of course, the number and arrangement form of the magnetocardiogram sensors 12 are not limited.

[0040] It can be understood that the magnetocardiogram signal is acquired based on the magnetocardiogram sensors 12, so the magnetocardiogram signal is expressed based on the sensor coordinate system. As described in the background art, the magnetocardiogram signal needs to be accurately reflected on the heart model to achieve subsequent magnetocardiogram tracing and locate abnormal magnetocardiogram signals. The heart model is often established based on medical images (such as CT images, MRI images, etc.), and the medical images are expressed based on the medical image coordinate system. Therefore, the magnetocardiogram signal and the heart model are expressed based on different coordinate systems, resulting in the problem of difficult accurate fusion as described in the background art.

[0041] To solve the fusion problem of the magnetocardiogram signal and the heart model, please refer to Figure 3 , an embodiment of the present invention provides a method for multi-modal cardiac image registration, which includes:

[0042] Step S1: Through optical scanning, obtain the first scan data of the magnetocardiogram device 1, and establish a magnetocardiogram device model based on the first scan data; obtain the design model of the magnetocardiogram sensors 12 of the magnetocardiogram device 1, register the design model with the magnetocardiogram sensor model in the magnetocardiogram device model to obtain the first transformation matrix between the design model and the magnetocardiogram sensor model; the magnetocardiogram sensor coordinate system is aligned with the magnetocardiogram device coordinate system according to the first transformation matrix;

[0043] Step S2: Through optical scanning, obtain the second scan data of the target object 2, and establish a first torso model based on the second scan data; obtain the medical image of the target object 2, and establish the heart model and the second torso model of the target object 2 based on the medical image; register the second torso model with the first torso model to obtain the second transformation matrix between the second torso model and the first torso model; the heart model coordinate system is aligned with the magnetocardiogram device coordinate system through the second transformation matrix;

[0044] Step S3: Obtain the magnetocardiogram data of the target object 2 through the magnetocardiogram sensors 12 of the magnetocardiogram device 1, and fuse the magnetocardiogram data with the heart model based on the aligned sensor coordinate system, magnetocardiogram device coordinate system, and heart model coordinate system.

[0045] With such a configuration, by registering the design model of the magnetocardiogram sensor 12 with the magnetocardiogram sensor model established based on optical scanning, and registering the first torso model established based on optical scanning with the second torso model established based on medical images, the alignment and unification of the sensor coordinate system, the magnetocardiogram device coordinate system, and the heart model coordinate system are achieved, enabling the accurate fusion of magnetocardiogram data and the heart model, and providing reliable input for subsequent magnetocardiogram tracing. The multi-modal cardiac image registration method provided in this embodiment is particularly applicable to application scenarios such as catheter radiofrequency ablation for the surgical treatment of atrial fibrillation. Due to the occasional occurrence of paroxysmal atrial fibrillation and the difficulty in accurately locating the position of atrial fibrillation, most ablation surgeries are performed by doctors based on electrophysiological indicators, radiomics, and clinical knowledge. Since there is no quantitative standard for the ablation degree, the surgical effects of different operators vary greatly. By using the multi-modal cardiac image registration method provided in this embodiment to register the magnetocardiogram data and the heart model, the source activity of the heart can be traced using the magnetocardiogram data, providing a relatively reliable basis for the radiofrequency ablation of atrial fibrillation.

[0046] The following further illustrates steps S1 to S3 in combination with a demonstration example.

[0047] In step S1, through optical scanning, the first scan data of the magnetocardiogram device 1 is obtained. For example, the magnetocardiogram device 1 can be scanned by an optical scanner, and the structural data of the bed 11 and the magnetocardiogram panel 10 of the magnetocardiogram device 1 is scanned into a 3D point cloud and saved in formats such as obj, stl, or ply. That is, the obtained first scan data contains the structural data of the outer contours of components such as the bed 11 and the magnetocardiogram panel 10.

[0048] Based on the first scan data, a magnetocardiogram device model is established. For example, the 3D point cloud can be imported through MESHLAB software, and the noise points in the 3D point cloud can be deleted using the built-in controls of MESHLAB software, and the 3D point cloud can be cut to obtain some features reflecting the solid outer contour of the magnetocardiogram panel 10, and this feature is saved as an stl format file. Here, the feature of the magnetocardiogram panel 10 is denoted as the magnetocardiogram sensor model, which reflects the outer contour structure form of the magnetocardiogram panel 10 obtained based on optical scanning.

[0049] Using MESHLAB software to import the design model of the magnetocardiogram sensor 12, for example, it can be the design model of the magnetocardiogram panel 10, which contains information such as the number and position of each magnetocardiogram sensor 12 on the magnetocardiogram panel 10. Using the built-in controls of MESHLAB software to cut the design model, a feature file similar to the magnetocardiogram sensor model can be obtained, and after cutting, it is saved in the stl file format.

[0050] Furthermore, the design model and the magnetocardiogram sensor model are simultaneously imported into the MESHLAB software, which can register and align the design model and the magnetocardiogram sensor model, and obtain the first transformation matrix between the design model and the magnetocardiogram sensor model. Since the design model contains information such as the number and position of the magnetocardiogram sensors 12, and the magnetocardiogram sensor model is cut from the magnetocardiogram device model, after the design model and the magnetocardiogram sensor model are registered and aligned, it is equivalent to reflecting the coordinates of the magnetocardiogram sensors 12 into the magnetocardiogram device coordinate system, and the coordinates of the magnetocardiogram signals acquired by the magnetocardiogram sensors 12 can also be reflected into the magnetocardiogram device coordinate system, that is, the magnetocardiogram sensor coordinate system is aligned with the magnetocardiogram device coordinate system according to the first transformation matrix. It should be noted that the MESHLAB software is only an example rather than a limitation of the 3D model processing software, and those skilled in the art can also select other software with similar functions, and this embodiment is not limited thereto.

[0051] In step S2, through optical scanning, the second scan data of the target object is obtained, and the first torso model is established based on the second scan data. The second scan data can be obtained, for example, by scanning the target object 2 with an optical scanner. In a specific example, the target object 2 takes off the upper body clothes and lies flat on the bed 11, and the laser marking points are used to locate the lying position of the target object 2 to ensure that the requirements for collecting magnetocardiogram signals are met, and then the second scan data of the target object 2 can be collected by using an optical scanner. Furthermore, after the scanning is completed, the second scan data is saved in the stl file format to obtain the first torso model, which reflects the torso outer contour of at least the upper body of the target object 2.

[0052] The medical image of the target object 2 can be obtained by a medical imaging device. Preferably, the medical image includes multiple time phases, that is, the medical image is 4D data, which is composed of information such as scanning time, number of layers, image width, and height.

[0053] Optionally, the steps of establishing the heart model and the second torso model of the target object based on the medical image include:

[0054] Step S21: Segment the medical image to obtain heart images of multiple time axes; establish the heart model based on the heart images; in an alternative exemplary example, the Unet series deep learning segmentation algorithm can be used to segment the medical image to obtain heart images. Three-dimensional reconstruction is performed on the heart images based on the three-dimensional reconstruction algorithm and saved in the stl file format to obtain the heart model.

[0055] Step S22: Process the medical image using the cardiac image as a mask to obtain a torso image; establish the second torso model based on the torso image. After the cardiac image is segmented in step S21, processing the medical image using the cardiac image as a mask can obtain the torso image after removing the heart. Based on the 3D reconstruction algorithm, the torso image is 3D reconstructed and saved in the stl file format, that is, the second torso model is obtained.

[0056] Furthermore, import the first torso model into the MESHLAB software and use its built-in controls to delete the redundant noise points. After completion, import the second torso model, which can realize registering and aligning the second torso model with the first torso model, and obtain the second transformation matrix between the second torso model and the first torso model.

[0057] The first torso model is obtained by scanning the state of the target object 2 lying flat on the bed 11 of the magnetocardiogram device 1. Therefore, the coordinates of the first torso model can be directly reflected in the magnetocardiogram device coordinate system. The second torso model and the cardiac model originate from the same medical image coordinate system. After registering and aligning the second torso model with the first torso model, it is equivalent to reflecting the coordinates of the cardiac model in the magnetocardiogram device coordinate system, that is, the cardiac model coordinate system is aligned with the magnetocardiogram device coordinate system through the second transformation matrix.

[0058] In step S3, the magnetocardiogram data of the target object 2 is acquired by the magnetocardiogram sensor 12 of the magnetocardiogram device 1. Since the magnetocardiogram sensor coordinate system is aligned with the magnetocardiogram device coordinate system in step S1, and the cardiac model coordinate system is aligned with the magnetocardiogram device coordinate system in step S2, the alignment between the cardiac model coordinate system and the magnetocardiogram sensor coordinate system is realized. Thus, the magnetocardiogram data acquired by the magnetocardiogram sensor 12 can be directly expressed in the cardiac model coordinate system, realizing the accurate fusion of the magnetocardiogram data and the cardiac model.

[0059] The inventor's further research found that the heart is not a rigid body structure. During the depolarization and repolarization processes of the heart (which can be simply understood as the heartbeat), its size and shape vary greatly, and the fusion accuracy of the magnetocardiogram data and the cardiac model is greatly affected by the cardiac depolarization and repolarization.

[0060] Optionally, in order to reduce or avoid the influence of cardiac depolarization and repolarization on the fusion accuracy, when acquiring the medical image of the target object 2 in step S2, an electrocardiogram signal is also acquired synchronously; a first gating information is obtained based on the electrocardiogram signal, and the first gating information reflects the systolic or diastolic phase of the heart; the cardiac model is established based on the medical image sequence corresponding to the first gating information.

[0061] Before acquiring the medical image of the target object 2 using a medical imaging device, an electrocardiogram acquisition device can be pasted on the chest of the target object 2. During the process of acquiring the medical image, the electrocardiogram acquisition device is used to synchronously acquire the electrocardiogram signal of the target object 2. It can be understood that based on the electrocardiogram signal, the systolic and diastolic phases of the heart can be distinguished. Therefore, the first gating information reflecting the systolic or diastolic phase of the heart can be obtained using the electrocardiogram signal. The medical image sequence corresponding to the first gating information is the continuous cardiac slice images of the heart in the systolic or diastolic phase, reducing or eliminating the source position positioning error caused by heart movement and effectively improving the fusion accuracy of the magnetocardiogram data and the heart model.

[0062] The inventors further found that since the registration method of this embodiment is implemented by means of the torso model of the target object 2, the breathing movement of the target object 2 will also have a great impact on the registration process. To reduce or avoid the influence of the breathing movement on the registration process, optionally, when acquiring the medical image of the target object 2 in step S2, the breathing signal is also acquired synchronously; the second gating information is obtained based on the breathing signal, and the second gating information reflects the breathing state of the target object 2; the second torso model is established based on the medical image sequence corresponding to the second gating information.

[0063] In a demonstration example, when using a medical imaging device to acquire the medical image of the target object 2, the target object 2 can hold its breath or exhale under the doctor's instruction, and the second gating information reflects the breathing state of the target object 2 (such as being in a state of holding breath or exhaling). In this way, the medical images acquired by the medical imaging device are continuous slice images corresponding to the target object 2 in a certain breathing state, reducing or eliminating the torso positioning error caused by breathing movement.

[0064] Further, when obtaining the second scan data of the target object 2 by optical scanning in step S2, it corresponds to the breathing state reflected by the second gating information, so that the first torso model and the second torso model are based on the same breathing state.

[0065] When scanning the target object 2 using an optical scanner, the target object 2 can hold its breath or exhale under the doctor's instruction, and this instruction should be the same as the instruction when acquiring the medical image, that is, corresponding to the breathing state reflected by the second gating information. Thus, the first torso model and the second torso model are based on the same breathing state, reducing or eliminating the registration error caused by different breathing states.

[0066] Optionally, in step S1, the registration of the design model and the magnetocardiogram sensor model can be realized in two steps, that is, first perform rough registration and then perform fine registration to improve the registration accuracy.

[0067] Specifically, the steps of registering the design model of the magnetocardiogram sensor with the magnetocardiogram sensor model in the magnetocardiogram device model to obtain the first transformation matrix between the design model and the magnetocardiogram sensor model include:

[0068] Select multiple sets of matching feature points on the design model and the magnetocardiogram sensor model, and align the matching feature points on the design model and the magnetocardiogram sensor model through the iterative closest point algorithm to obtain the first rough alignment transformation matrix; use the first rough alignment transformation matrix to roughly align the design model and the magnetocardiogram sensor model to obtain the first roughly aligned model;

[0069] Select multiple sets of matching feature points on the first roughly aligned model and the magnetocardiogram sensor model, and align the matching feature points on the first roughly aligned model and the magnetocardiogram sensor model through the iterative closest point algorithm to obtain the first fine alignment transformation matrix; use the first fine alignment transformation matrix to finely align the first roughly aligned model and the magnetocardiogram sensor model;

[0070] The first transformation matrix = the first rough alignment transformation matrix × the first fine alignment transformation matrix.

[0071] In an exemplary example, when importing the design model and the magnetocardiogram sensor model into the MESHLAB software, use the point selection control of the MESHLAB software to select 3 sets of matching feature points from the design model and the magnetocardiogram sensor model respectively, denoted as n1, n2, n3, m1, m2, m3. The above 6 points are all in three-dimensional space and are composed of three coordinate points x, y, and z. The alignment of the above 3 sets of feature points is realized through the iterative closest point algorithm (ICP, IterativeClosestPoint) and the first rough alignment transformation matrix T is obtained 1 , through the first rough alignment transformation matrix T 1 The rough alignment of the design model and the magnetocardiogram sensor model can be realized, and the first roughly aligned model is obtained after the rough alignment is completed. Furthermore, by using the iterative closest point algorithm again, the fine alignment of the first roughly aligned model and the magnetocardiogram sensor model can be realized, and the first fine alignment transformation matrix T is obtained 2 , where T 1 、T 2 are both 4×4 matrices. The specific registration and alignment principle can refer to the prior art and will not be elaborated here. Let A be the spatial position of the magnetocardiogram sensor 12. After the design model and the magnetocardiogram sensor model are registered and aligned, the position B of the magnetocardiogram sensor 12 in the magnetocardiogram device coordinate system = T 1 ×T 2×A. It can be understood that the iterative closest point algorithm and several groups of feature points are a demonstration rather than a limitation for registration. Those skilled in the art can select other registration algorithms according to the prior art, and this embodiment is not limited thereto.

[0072] Optionally, in step S2, the registration of the second torso model and the first torso model can also be achieved in two steps, that is, first perform rough registration and then perform fine registration to improve the registration accuracy.

[0073] Specifically, the steps of registering the second torso model and the first torso model to obtain the second transformation matrix between the second torso model and the first torso model include:

[0074] Select multiple groups of matching feature points on the second torso model and the first torso model, and align the matching feature points on the second torso model and the first torso model through the iterative closest point algorithm to obtain a second rough alignment transformation matrix; use the second rough alignment transformation matrix to roughly align the second torso model and the first torso model to obtain a second roughly aligned model;

[0075] Select multiple groups of matching feature points on the second roughly aligned model and the first torso model, and align the matching feature points on the second roughly aligned model and the first torso model through the iterative closest point algorithm to obtain a second fine alignment transformation matrix; use the second fine alignment transformation matrix to finely align the second roughly aligned model and the first torso model;

[0076] The second transformation matrix = the second rough alignment transformation matrix × the second fine alignment transformation matrix.

[0077] In an exemplary example, similar to the registration and alignment steps of the design model and the magnetocardiogram sensor model, after importing the second torso model and the first torso model into the MESHLAB software, the alignment of multiple groups of feature points is achieved through the iterative closest point algorithm, and the second rough alignment transformation matrix T is obtained 3 , and through the first rough alignment transformation matrix T 3 , the rough alignment of the second torso model and the first torso model can be realized. After the rough alignment is completed, a second roughly aligned model is obtained. Furthermore, by using the iterative closest point algorithm again, the fine alignment of the second roughly aligned model and the first torso model can be realized, and the second fine alignment transformation matrix T is obtained 4 . Let C be the spatial position of the heart model after three-dimensional reconstruction. After the registration and alignment of the second torso model and the first torso model, the position D of the heart model in the magnetocardiogram device coordinate system = T 3 ×T 4 ×C.

[0078] Based on this, the alignment and unification of the sensor coordinate system, the magnetocardiogram (MCG) device coordinate system, and the heart model coordinate system are realized, enabling the accurate fusion of MCG data and the heart model, and providing reliable input for subsequent MCG source tracing.

[0079] An embodiment of the present invention further provides a cardiac multimodal image registration system, which includes: an optical scanner, an MCG device 1, and a registration execution module;

[0080] The optical scanner is used to obtain the first scan data of the MCG device 1 and the second scan data of the target object 2; the MCG device 1 includes an MCG sensor 12, and the MCG device 1 is used to obtain the MCG data of the target object 2;

[0081] The registration execution module is configured to establish an MCG device model based on the first scan data; obtain the design model of the MCG sensor of the MCG device, register the design model with the MCG sensor model in the MCG device model to obtain the first transformation matrix between the design model and the MCG sensor model; align the MCG sensor coordinate system with the MCG device coordinate system according to the first transformation matrix; establish a first torso model based on the second scan data; obtain the medical image of the target object 2, and establish the heart model and the second torso model of the target object 2 based on the medical image; register the second torso model with the first torso model to obtain the second transformation matrix between the second torso model and the first torso model; align the heart model coordinate system with the MCG device coordinate system through the second transformation matrix; and fuse the MCG data and the heart model based on the aligned sensor coordinate system, MCG device coordinate system, and heart model coordinate system.

[0082] Optionally, the cardiac multimodal image registration system further includes an electrocardiogram (ECG) acquisition device (not shown); the ECG acquisition device is used to synchronously acquire the ECG signal of the target object 2 when obtaining the medical image of the target object 2; the registration execution module is further configured to obtain the first gating information based on the ECG signal, and the first gating information reflects the systolic or diastolic phase of the heart; and the heart model is established based on the medical image sequence corresponding to the first gating information.

[0083] For the structures and principles of other components of the cardiac multimodal image registration system, reference can be made to the prior art, and no further elaboration will be made in this embodiment.

[0084] An embodiment of the present invention further provides a readable storage medium, on which a program is stored, and when the program is executed, the steps of the above cardiac multimodal image registration method are implemented. The readable storage medium can be independently set or integrated into a certain component in the cardiac multimodal image registration system as described above, for example, integrated into the registration execution module.

[0085] In summary, in the cardiac multimodal image registration method, registration system and readable storage medium provided by the present invention, the cardiac multimodal image registration method includes: obtaining first scan data of a magnetocardiogram (MCG) device through optical scanning, and establishing an MCG device model based on the first scan data; obtaining a design model of an MCG sensor of the MCG device, registering the design model with an MCG sensor model in the MCG device model to obtain a first transformation matrix between the design model and the MCG sensor model; aligning the MCG sensor coordinate system with the MCG device coordinate system according to the first transformation matrix; obtaining second scan data of a target object through optical scanning, and establishing a first torso model based on the second scan data; obtaining a medical image of the target object, and establishing a cardiac model and a second torso model of the target object based on the medical image; registering the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; aligning the cardiac model coordinate system with the MCG device coordinate system through the second transformation matrix; obtaining MCG data of the target object through the MCG sensor of the MCG device, and fusing the MCG data with the cardiac model based on the aligned sensor coordinate system, MCG device coordinate system and cardiac model coordinate system. With such configuration, by registering the design model of the MCG sensor with the MCG sensor model established based on optical scanning, and registering the first torso model established based on optical scanning with the second torso model established based on medical images, alignment and unification of the sensor coordinate system, MCG device coordinate system and cardiac model coordinate system are achieved, enabling accurate fusion of the MCG data and the cardiac model, and providing reliable input for subsequent MCG source tracing.

[0086] It should be noted that the above-mentioned several embodiments can be combined with each other. The above description is only a description of the preferred embodiments of the present invention, and does not limit the scope of the present invention in any way. Any changes and modifications made by those of ordinary skill in the field of the present invention according to the above disclosure shall fall within the protection scope of the present invention.

Claims

1. A cardiac multimodal image registration method, characterized in that: include: Acquire first scanning data of a magnetocardiographic device through optical scanning, and establish a magnetocardiographic device model based on the first scanning data; acquire a design model of a magnetocardiographic sensor of the magnetocardiographic device, wherein the design model includes the number and position information of the magnetocardiographic sensor; align the design model with the magnetocardiographic sensor model in the magnetocardiographic device model to obtain a first transformation matrix between the design model and the magnetocardiographic sensor model; align the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system according to the first transformation matrix; Acquire second scan data of the target object through optical scanning, and establish a first torso model based on the second scan data, wherein the first torso model reflects the outer contour of at least the upper body of the target object, and the coordinates of the first torso model are reflected in the coordinate system of the magnetocardiometry device; Acquire a medical image of the target object, and establish a heart model and a second torso model of the target object based on the medical image; the second torso model and the heart model are derived from the same medical image coordinate system; register the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; align the heart model coordinate system with the magnetocardiographic device coordinate system through the second transformation matrix; The step of establishing the heart model and the second torso model of the target object based on the medical image includes segmenting the medical image to obtain heart images of multiple time axes; Establishing the heart model based on the heart image; processing the medical image using the heart image as a mask to obtain a torso image; establishing the second torso model based on the torso image; The magnetocardiographic data of the target object is acquired by a magnetocardiographic sensor of the magnetocardiographic device, and the magnetocardiographic data is fused with the heart model based on the aligned sensor coordinate system, the magnetocardiographic device coordinate system and the heart model coordinate system.

2. The cardiac multimodal image registration method according to claim 1, characterized in that: When acquiring medical images of a target object, an electrocardiogram signal is synchronously acquired; first gating information is obtained based on the electrocardiogram signal, and the first gating information reflects the systole or diastole of the heart; and the heart model is established based on a medical image sequence corresponding to the first gating information.

3. The cardiac multimodal image registration method according to claim 1, characterized in that: When acquiring a medical image of a target object, synchronously acquiring a respiratory signal; obtaining second gating information based on the respiratory signal, wherein the second gating information reflects a respiratory state of the target object; The second torso model is established based on a medical image sequence corresponding to the second gating information.

4. The cardiac multimodal image registration method according to claim 3, characterized in that: When acquiring the second scan data of the target object through optical scanning, the first torso model and the second torso model are based on the same respiratory state corresponding to the respiratory state reflected by the second gating information.

5. The cardiac multimodal image registration method according to claim 1, characterized in that: The step of aligning the design model of the magnetocardiogram sensor with the magnetocardiogram sensor model in the magnetocardiogram device model to obtain a first conversion matrix between the design model and the magnetocardiogram sensor model comprises: Selecting a plurality of groups of matching feature points on the design model and the magnetocardio sensor model, aligning the design model with the matching feature points on the magnetocardio sensor model by a nearest point iteration algorithm, and obtaining a first coarse alignment transformation matrix; using the first coarse alignment transformation matrix, coarsely aligning the design model with the magnetocardio sensor model, and obtaining a first coarse alignment model; Selecting a plurality of groups of matching feature points on the first coarse alignment model and the magnetocardio sensor model, aligning the first coarse alignment model with the matching feature points on the magnetocardio sensor model by a nearest point iteration algorithm to obtain a first fine alignment transformation matrix; and finely aligning the first coarse alignment model with the magnetocardio sensor model by using the first fine alignment transformation matrix; The first transformation matrix=the first coarse alignment transformation matrix×the first fine alignment transformation matrix.

6. The cardiac multimodal image registration method according to claim 1, characterized in that: The step of registering the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model comprises: Selecting a plurality of groups of matching feature points on the second torso model and the first torso model, aligning the matching feature points on the second torso model and the first torso model by a nearest point iteration algorithm to obtain a second coarse alignment transformation matrix; using the second coarse alignment transformation matrix, coarsely aligning the second torso model and the first torso model to obtain a second coarse alignment model; Selecting a plurality of groups of matching feature points on the second coarse alignment model and the first torso model, aligning the second coarse alignment model with the matching feature points on the first torso model by a nearest point iteration algorithm to obtain a second fine alignment transformation matrix; and finely aligning the second coarse alignment model with the first torso model by using the second fine alignment transformation matrix; The second transformation matrix=the second coarse alignment transformation matrix×the second fine alignment transformation matrix.

7. A readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the steps of the cardiac multimodal image registration method according to any one of claims 1 to 6 are implemented.

8. A cardiac multimodal image registration system, characterized in that: include: Optical scanner, magnetocardiographic device and registration execution module; The optical scanner is used to obtain first scanning data of the magnetocardiographic device and second scanning data of the target object; the magnetocardiographic device includes a magnetocardiographic sensor, and the magnetocardiographic device is used to obtain the magnetocardiographic data of the target object; The registration execution module is configured to establish a magnetocardiographic device model based on the first scanning data; obtain a design model of the magnetocardiographic sensor of the magnetocardiographic device, wherein the design model includes the number and position information of the magnetocardiographic sensor; register the design model with the magnetocardiographic sensor model in the magnetocardiographic device model to obtain a first transformation matrix between the design model and the magnetocardiographic sensor model; and align the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system according to the first transformation matrix; Establishing a first torso model based on the second scan data, wherein the first torso model reflects the outer contour of at least the upper body of the target object, and the coordinates of the first torso model are reflected in the magnetocardiometry device coordinate system; Acquire a medical image of the target object, and establish a heart model and a second torso model of the target object based on the medical image; the second torso model and the heart model are derived from the same medical image coordinate system; register the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; align the heart model coordinate system with the magnetocardiographic device coordinate system through the second transformation matrix; The step of establishing the heart model and the second torso model of the target object based on the medical image includes segmenting the medical image to obtain heart images of multiple time axes; Establishing the heart model based on the heart image; Processing the medical image using the heart image as a mask to obtain a torso image; establishing the second torso model based on the torso image; The magnetocardiographic data is fused with the heart model based on the aligned sensor coordinate system, the magnetocardiographic device coordinate system and the heart model coordinate system.

9. The cardiac multimodal image registration system according to claim 8, characterized in that: It also includes an ECG acquisition device; the ECG acquisition device is used to synchronously acquire the ECG signal of the target object when acquiring the medical image of the target object; The registration execution module is further configured to obtain first gating information based on the electrocardiogram signal, where the first gating information reflects the systole or diastole of the heart; and the heart model is established based on a medical image sequence corresponding to the first gating information.

Images