Cardiac multimodal image registration method, registration system and readable storage medium

Abstract

The present invention provides a cardiac multimodal image registration method, a registration system and a readable storage medium. The cardiac multimodal image registration method comprises: obtaining first scanning data and establishing a magnetocardiographic device model by optical scanning; registering a design model of a magnetocardiographic sensor with a magnetocardiographic sensor model to obtain a first conversion matrix between the design model and the magnetocardiographic sensor model; obtaining second scanning data and establishing a first trunk model by optical scanning; obtaining a medical image of a target object and establishing a heart model and a second trunk model; registering the second trunk model with the first trunk model to obtain a second conversion matrix between the second trunk model and the first trunk model; obtaining magnetocardiographic data of the target object by using the magnetocardiographic sensor of the magnetocardiographic device, and fusing the magnetocardiographic data with the heart model based on an aligned sensor coordinate system, a magnetocardiographic device coordinate system and a heart model coordinate system.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to a cardiac multimodal image registration method, a registration system and a readable storage medium. Background Art

[0002] The magnetic heart source imaging device is an imaging device that uses an atomic magnetometer to detect cardiac magnetic field signals. It can be used for auxiliary diagnosis of heart diseases such as heart failure, ischemic heart disease, arrhythmia, and fetal congenital heart disease. The device uses an atomic magnetometer to collect magnetic signals from the surface of the patient's heart, and accurately locates abnormal magnetic heart signals by analyzing and extracting the characteristics of the magnetic signals and combining them with imaging omics, providing a reference for further treatment. The accurate positioning of abnormal magnetic heart signals requires the construction of a cardiac electrophysiological model, and accurate tracing of the source through forward modeling and inverse solving. The forward problem is used to describe the mathematical relationship between the magnetic heart signals measured by the endogenous cardiac source and the magnetic heart sensor, and the inverse problem reconstructs the source activity through the forward model and the measured magnetic heart signals.

[0003] The core of magnetic heart source imaging includes three parts: cardiac electrophysiological model, sensor air chamber position, and magnetic heart signal. The heart's intrinsic activity is reconstructed by segmenting, reconstructing, and gridding the CT image of the heart and combining the air chamber position and magnetic heart signal. Since the CT image and the magnetic heart sensor are not in the same spatial coordinate system, it is difficult to accurately merge the magnetic heart signal with the cardiac electrophysiological model, making it difficult to guarantee the accuracy of subsequent magnetic heart tracing. Summary of the invention

[0004] The purpose of the present invention is to provide a cardiac multimodal image registration method, a registration system and a readable storage medium to solve the problem in the prior art that it is difficult to accurately fuse central magnetic signals with cardiac electrophysiological models.

[0005] In order to solve the above technical problems, the present invention provides a cardiac multimodal image registration method, which comprises: Acquire first scanning data of a magnetocardiographic device through optical scanning, and establish a magnetocardiographic device model based on the first scanning data; acquire a design model of a magnetocardiographic sensor of the magnetocardiographic device, align the design model with a magnetocardiographic sensor model in the magnetocardiographic device model, and obtain a first transformation matrix between the design model and the magnetocardiographic sensor model; align a magnetocardiographic sensor coordinate system with a magnetocardiographic device coordinate system according to the first transformation matrix; Acquire second scan data of the target object through optical scanning, and establish a first torso model based on the second scan data; acquire a medical image of the target object, and establish a heart model and a second torso model of the target object based on the medical image; align the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; align the heart model coordinate system with the magnetocardiographic device coordinate system through the second transformation matrix; The magnetocardiographic data of the target object is acquired through the magnetocardiographic sensor of the magnetocardiographic device, and the magnetocardiographic data is fused with the heart model based on the aligned sensor coordinate system, the magnetocardiographic device coordinate system and the heart model coordinate system.

[0006] Optionally, when acquiring medical images of the target object, an electrocardiogram signal is synchronously acquired; first gating information is obtained based on the electrocardiogram signal, and the first gating information reflects the systole or diastole of the heart; and the heart model is established based on the medical image sequence corresponding to the first gating information.

[0007] Optionally, when acquiring the medical image of the target object, a respiratory signal is acquired synchronously; second gating information is obtained based on the respiratory signal, and the second gating information reflects the respiratory state of the target object; and the second torso model is established based on the medical image sequence corresponding to the second gating information.

[0008] Optionally, when acquiring the second scanning data of the target object through optical scanning, the first torso model and the second torso model are based on the same breathing state corresponding to the breathing state reflected by the second gating information.

[0009] Optionally, the step of registering the design model of the magnetocardiographic sensor with the magnetocardiographic sensor model in the magnetocardiographic device model to obtain a first conversion matrix between the design model and the magnetocardiographic sensor model includes: Selecting a plurality of groups of matching feature points on the design model and the magnetocardio sensor model, aligning the design model with the matching feature points on the magnetocardio sensor model by a nearest point iteration algorithm, and obtaining a first coarse alignment transformation matrix; using the first coarse alignment transformation matrix, coarsely aligning the design model with the magnetocardio sensor model, and obtaining a first coarse alignment model; Selecting a plurality of groups of matching feature points on the first coarse alignment model and the magnetocardio sensor model, aligning the first coarse alignment model with the matching feature points on the magnetocardio sensor model by a nearest point iteration algorithm to obtain a first fine alignment transformation matrix; and finely aligning the first coarse alignment model with the magnetocardio sensor model by using the first fine alignment transformation matrix; The first transformation matrix=the first coarse alignment transformation matrix×the first fine alignment transformation matrix.

[0010] Optionally, the step of registering the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model includes: Selecting a plurality of groups of matching feature points on the second torso model and the first torso model, aligning the matching feature points on the second torso model and the first torso model by a nearest point iteration algorithm to obtain a second coarse alignment transformation matrix; using the second coarse alignment transformation matrix, coarsely aligning the second torso model and the first torso model to obtain a second coarse alignment model; Selecting a plurality of groups of matching feature points on the second coarse alignment model and the first torso model, aligning the second coarse alignment model with the matching feature points on the first torso model by a nearest point iteration algorithm to obtain a second fine alignment transformation matrix; and finely aligning the second coarse alignment model with the first torso model by using the second fine alignment transformation matrix; The second transformation matrix=the second coarse alignment transformation matrix×the second fine alignment transformation matrix.

[0011] Optionally, the step of establishing the heart model and the second torso model of the target object based on the medical image includes: Segmenting the medical image to obtain cardiac images of multiple time axes; establishing the cardiac model based on the cardiac images; The medical image is processed using the heart image as a mask to obtain a torso image; and the second torso model is established based on the torso image.

[0012] In order to solve the above technical problem, the present invention also provides a readable storage medium, on which a program is stored, and when the program is executed, the steps of the cardiac multimodal image registration method as described above are implemented.

[0013] In order to solve the above technical problems, the present invention also provides a cardiac multimodal image registration system, which includes: an optical scanner, a magnetic heart device and a registration execution module; The optical scanner is used to obtain first scanning data of the magnetocardiographic device and second scanning data of the target object; the magnetocardiographic device includes a magnetocardiographic sensor, and the magnetocardiographic device is used to obtain the magnetocardiographic data of the target object; The registration execution module is configured to establish a magnetocardiographic device model based on the first scanning data; obtain a design model of the magnetocardiographic sensor of the magnetocardiographic device, align the design model with the magnetocardiographic sensor model in the magnetocardiographic device model, and obtain a first transformation matrix between the design model and the magnetocardiographic sensor model; align the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system according to the first transformation matrix; establish a first torso model based on the second scanning data; obtain a medical image of the target object, and establish a heart model and a second torso model of the target object based on the medical image; align the second torso model with the first torso model, and obtain a second transformation matrix between the second torso model and the first torso model; align the heart model coordinate system with the magnetocardiographic device coordinate system through the second transformation matrix; and fuse the magnetocardiographic data with the heart model based on the aligned sensor coordinate system, the magnetocardiographic device coordinate system and the heart model coordinate system.

[0014] Optionally, the cardiac multimodal image registration system further includes an electrocardiogram acquisition device; the electrocardiogram acquisition device is used to synchronously acquire the electrocardiogram signal of the target object when acquiring the medical image of the target object; The registration execution module is further configured to obtain first gating information based on the electrocardiogram signal, where the first gating information reflects the systole or diastole of the heart; and the heart model is established based on a medical image sequence corresponding to the first gating information.

[0015] In summary, in the cardiac multimodal image registration method, registration system and readable storage medium provided by the present invention, the cardiac multimodal image registration method includes: obtaining first scanning data of a magnetocardiographic device through optical scanning, and establishing a magnetocardiographic device model based on the first scanning data; obtaining a design model of a magnetocardiographic sensor of the magnetocardiographic device, aligning the design model with the magnetocardiographic sensor model in the magnetocardiographic device model, and obtaining a first transformation matrix between the design model and the magnetocardiographic sensor model; aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system according to the first transformation matrix; obtaining second scanning data of the target object through optical scanning, and aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system based on the first transformation matrix; obtaining second scanning data of the target object through optical scanning, and aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system based on the first transformation matrix; obtaining second scanning data of the target object through optical scanning, and aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system based on the first transformation matrix; obtaining second scanning data of the target object through optical scanning, and aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system based on the first transformation matrix; obtaining first ... The second scanning data establishes a first torso model; a medical image of the target object is acquired, and based on the medical image, a heart model and a second torso model of the target object are established; the second torso model is registered with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; the heart model coordinate system is aligned with the magnetocardio device coordinate system through the second transformation matrix; the magnetocardio data of the target object is acquired through the magnetocardio sensor of the magnetocardio device, and the magnetocardio data is fused with the heart model based on the aligned sensor coordinate system, the magnetocardio device coordinate system and the heart model coordinate system.

[0016] With this configuration, by aligning the design model of the magnetocardio sensor with the magnetocardio sensor model established based on optical scanning, and aligning the first torso model established based on optical scanning with the second torso model established based on medical imaging, the sensor coordinate system, the magnetocardio device coordinate system and the heart model coordinate system are aligned and unified, so that the magnetocardio data and the heart model can be accurately integrated, providing reliable input for subsequent magnetocardio tracing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Those skilled in the art will appreciate that the drawings are provided for a better understanding of the present invention, but do not constitute any limitation on the scope of the present invention.

[0018] Figure 1 Schematic diagram of a magnetocardiographic device according to an embodiment of the present invention.

[0019] Figure 2 Schematic diagram of a magnetic heart panel according to an embodiment of the present invention.

[0020] Figure 3 It is a flowchart of a cardiac multimodal image registration method according to an embodiment of the present invention.

[0021] In the attached drawings: 1-magnetic heart device; 10-magnetic heart panel; 11-bed; 12-magnetic heart sensor; 2-target object. DETAILED DESCRIPTION

[0022] In order to make the purpose, advantages and features of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and are not drawn to scale, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the drawings is often a part of the actual structure. In particular, the emphasis of each drawing is different, and sometimes different scales are used.

[0023] As used in the present invention, the singular forms "a", "an", "one" and "the" include plural objects, the term "or" is usually used to include the meaning of "and / or", the term "several" is usually used to include the meaning of "at least one", and the term "at least two" is usually used to include the meaning of "two or more". In addition, the terms "first", "second" and "third" are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined as "first", "second" and "third" may explicitly or implicitly include one or at least two of the features, "one end" and "the other end" as well as "proximal end" and "distal end" generally refer to two corresponding parts, which include not only endpoints. In addition, as used in the present invention, "installed", "connected", "connected", and one element "set" on another element should be understood in a broad sense, usually only indicating that there is a connection, coupling, cooperation or transmission relationship between the two elements, and the connection, coupling, cooperation or transmission between the two elements can be direct or indirect through an intermediate element, and cannot be understood as indicating or implying the spatial position relationship between the two elements, that is, one element can be in any orientation such as inside, outside, above, below or one side of another element, unless the content clearly indicates otherwise. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, directional terms such as above, below, up, down, upward, downward, left, right, etc. are used relative to the exemplary embodiments as shown in the figures, with the upward or upper direction toward the top of the corresponding figure, and the downward or lower direction toward the bottom of the corresponding figure.

[0024] The purpose of the present invention is to provide a cardiac multimodal image registration method, registration system and readable storage medium to solve the problem that the central magnetic signal and the cardiac electrophysiological model are difficult to accurately merge in the prior art.

[0025] Please refer to Figure 1 , which shows a magnetocardiographic device 1, which includes a bed 11 and a magnetocardiographic sensor 12. The bed 11 is used for a target object 2 (such as a patient) to be scanned to lie on, and the magnetocardiographic sensor 12 is used to obtain a magnetocardiographic signal of the target object 2 lying on the bed 11. Figure 2 As shown, in an exemplary embodiment, the number of the magnetocardiographic sensors 12 can be multiple, and they are preferably arranged in an array on the magnetocardiographic panel 10. The specific structure and detection principle of the magnetocardiographic device 1 can refer to the prior art, and will not be described in detail here. Of course, the number and arrangement of the magnetocardiographic sensors 12 are not limited.

[0026] It is understandable that the magnetic heart signal is collected based on the magnetic heart sensor 12, so the magnetic heart signal is expressed based on the sensor coordinate system. As described in the background technology, the magnetic heart signal needs to be accurately reflected on the heart model in order to achieve subsequent magnetic heart tracing and locate abnormal magnetic heart signals. The heart model is often established based on medical images (such as CT images, MRI images, etc.), and the medical images are expressed based on the medical image coordinate system. Therefore, the magnetic heart signal and the heart model are expressed based on different coordinate systems, resulting in the problem of difficulty in accurate fusion as described in the background technology.

[0027] To solve the problem of integrating the magnetic heart signal with the heart model, please refer to Figure 3 The embodiment of the present invention provides a cardiac multimodal image registration method, which includes: Step S1: acquiring first scanning data of the magnetocardiographic device 1 by optical scanning, and establishing a magnetocardiographic device model based on the first scanning data; acquiring a design model of the magnetocardiographic sensor 12 of the magnetocardiographic device 1, aligning the design model with the magnetocardiographic sensor model in the magnetocardiographic device model, and obtaining a first conversion matrix between the design model and the magnetocardiographic sensor model; aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system according to the first conversion matrix; Step S2: acquiring second scan data of the target object 2 by optical scanning, and establishing a first torso model based on the second scan data; acquiring a medical image of the target object 2, and establishing a heart model and a second torso model of the target object 2 based on the medical image; registering the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; aligning the heart model coordinate system with the magnetocardiographic device coordinate system through the second transformation matrix; Step S3: acquiring the magnetocardiographic data of the target object 2 through the magnetocardiographic sensor 12 of the magnetocardiographic device 1, and fusing the magnetocardiographic data with the heart model based on the aligned sensor coordinate system, the magnetocardiographic device coordinate system and the heart model coordinate system.

[0028] With such configuration, by aligning the design model of the magnetic heart sensor 12 with the magnetic heart sensor model established based on optical scanning, and aligning the first torso model established based on optical scanning with the second torso model established based on medical imaging, the alignment and unification of the sensor coordinate system, the magnetic heart device coordinate system and the heart model coordinate system are achieved, so that the magnetic heart data and the heart model can be accurately integrated, providing reliable input for subsequent magnetic heart tracing. The multimodal cardiac image registration method provided in this embodiment is particularly suitable for application scenarios such as catheter radiofrequency ablation for surgical treatment of atrial fibrillation. Due to the sporadic nature of paroxysmal atrial fibrillation and the difficulty in accurately locating the location of atrial fibrillation, most ablation surgeries are performed by doctors through electrophysiological indicators, imaging genomics combined with clinical knowledge. Due to the lack of quantitative standards for the degree of ablation, the surgical effects of different operators vary greatly. Using the multimodal cardiac image registration method provided in this embodiment, after aligning the magnetic heart data with the heart model, the magnetic heart data can be used to trace the source activity of the heart, providing a relatively reliable basis for radiofrequency ablation of atrial fibrillation.

[0029] The following further describes steps S1 to S3 with reference to an example.

[0030] In step S1, the first scanning data of the magnetocardiographic device 1 is obtained by optical scanning. For example, the magnetocardiographic device 1 can be scanned by an optical scanner, and the structural data of the bed 11 and the magnetocardiographic panel 10 of the magnetocardiographic device 1 are scanned into a 3D point cloud and saved in a format such as obj, stl or ply. That is, the first scanning data obtained includes the structural data of the outer contours of components such as the bed 11 and the magnetocardiographic panel 10.

[0031] A magnetocardio device model is established based on the first scanning data; for example, a 3D point cloud can be imported through MESHLAB software, noise points in the 3D point cloud can be deleted using the built-in controls of the MESHLAB software, and the 3D point cloud can be cut to obtain partial features reflecting the physical outer contour of the magnetocardio panel 10, and the features are saved as an stl format file. Here, the features of the magnetocardio panel 10 are recorded as a magnetocardio sensor model, which reflects the outer contour structural morphology of the magnetocardio panel 10 obtained based on optical scanning.

[0032] The design model of the magnetocardiographic sensor 12 is imported using the MESHLAB software, for example, the design model of the magnetocardiographic panel 10, which includes information such as the number and position of each magnetocardiographic sensor 12 on the magnetocardiographic panel 10. The design model is cut using the built-in control of the MESHLAB software to obtain a feature file similar to the magnetocardiographic sensor model, which is saved in the stl file format after cutting.

[0033] Then, the design model and the magnetocardio sensor model are simultaneously imported into the MESHLAB software, so that the design model and the magnetocardio sensor model can be registered and aligned, and the first conversion matrix between the design model and the magnetocardio sensor model can be obtained. Since the design model contains the number, position and other information of the magnetocardio sensor 12, and the magnetocardio sensor model is obtained by cutting from the magnetocardio device model, after the design model and the magnetocardio sensor model are registered and aligned, it is equivalent to reflecting the coordinates of the magnetocardio sensor 12 in the magnetocardio device coordinate system, and the coordinates of the magnetocardio signal obtained by the magnetocardio sensor 12 can also be reflected in the magnetocardio device coordinate system, that is, the magnetocardio sensor coordinate system is aligned with the magnetocardio device coordinate system according to the first conversion matrix. It should be noted that the MESHLAB software is only a demonstration of the three-dimensional model processing software and not a limitation. Those skilled in the art can also choose other software with similar functions, which is not limited in this embodiment.

[0034] In step S2, the second scan data of the target object is obtained by optical scanning, and a first torso model is established based on the second scan data. The second scan data can be obtained, for example, by scanning the target object 2 with an optical scanner. In a specific example, the target object 2 takes off the clothes on the upper body and lies flat on the bed 11. The lying position of the target object 2 is located by using a laser marking point to ensure that the acquisition requirements of the magnetic cardio signal are met, and then the second scan data of the target object 2 can be collected by using an optical scanner. After the scanning is completed, the second scan data is saved in an stl file format to obtain a first torso model, which reflects the outer contour of the torso of at least the upper body of the target object 2.

[0035] The medical image of the target object 2 can be acquired by a medical imaging device. Preferably, the medical image includes multiple time phases, that is, the medical image is 4D data, which is composed of information such as scanning time, number of layers, image width, height, etc.

[0036] Optionally, the step of establishing the heart model and the second torso model of the target object based on the medical image includes: Step S21: segment the medical image to obtain cardiac images of multiple time axes; establish the cardiac model based on the cardiac image; in an alternative example, the Unet series deep learning segmentation algorithm can be used to segment the medical image to obtain the cardiac image. Perform three-dimensional reconstruction on the cardiac image based on the three-dimensional reconstruction algorithm and save it in the stl file format to obtain the cardiac model.

[0037] Step S22: Process the medical image with the heart image as a mask to obtain a torso image; establish the second torso model based on the torso image. After the heart image is segmented in step S21, the medical image is processed with the heart image as a mask to obtain a torso image after the heart is removed, and the torso image is 3D reconstructed based on a 3D reconstruction algorithm and saved in an stl file format, thereby obtaining a second torso model.

[0038] Then, the first torso model is imported into MESHLAB software, and redundant noise points are deleted using its built-in controls. After completion, the second torso model is imported to register and align the second torso model with the first torso model, and a second transformation matrix between the second torso model and the first torso model is obtained.

[0039] The first torso model is obtained by scanning the target object 2 lying flat on the bed 11 of the magnetic heart device 1, so the coordinates of the first torso model can be directly reflected in the magnetic heart device coordinate system. The second torso model and the heart model are derived from the same medical imaging coordinate system. After the second torso model is aligned with the first torso model, it is equivalent to reflecting the coordinates of the heart model in the magnetic heart device coordinate system, that is, the heart model coordinate system is aligned with the magnetic heart device coordinate system through the second transformation matrix.

[0040] Step S3 acquires the magnetic heart data of the target object 2 through the magnetic heart sensor 12 of the magnetic heart device 1. Since the magnetic heart sensor coordinate system has been aligned with the magnetic heart device coordinate system in step S1, and the heart model coordinate system has been aligned with the magnetic heart device coordinate system in step S2, the heart model coordinate system and the magnetic heart sensor coordinate system are aligned. Therefore, the magnetic heart data acquired by the magnetic heart sensor 12 can be directly expressed in the heart model coordinate system, realizing the accurate fusion of the magnetic heart data and the heart model.

[0041] The inventors further discovered that the heart is not a rigid structure. Its size and shape vary greatly during the process of depolarization and repolarization (which can be simply understood as heartbeat). The accuracy of the fusion of cardiac magnetic data and the heart model is greatly affected by cardiac depolarization and repolarization.

[0042] Optionally, in order to reduce or avoid the impact of cardiac depolarization on fusion accuracy, when acquiring the medical image of the target object 2 in step S2, the electrocardiogram signal is also synchronously acquired; based on the electrocardiogram signal, the first gating information is obtained, and the first gating information reflects the systole or diastole of the heart; the heart model is established based on the medical image sequence corresponding to the first gating information.

[0043] Before using the medical imaging device to collect the medical image of the target object 2, an ECG acquisition device can be attached to the chest of the target object 2. During the process of collecting the medical image, the ECG acquisition device is used to synchronously collect the ECG signal of the target object 2. It can be understood that based on the ECG signal, the systolic period and diastolic period of the heart can be distinguished, so the ECG signal can be used to obtain the first gating information reflecting the systolic period or diastolic period of the heart. The medical image sequence corresponding to the first gating information, that is, the continuous heart slice image of the heart in the systolic period or diastolic period, reduces or eliminates the source position positioning error caused by the heart movement, and effectively improves the fusion accuracy of the magnetic heart data and the heart model.

[0044] The inventors further discovered that since the registration method of this embodiment is implemented with the help of the torso model of the target object 2, the respiratory movement of the target object 2 will also have a significant impact on the registration process. In order to reduce or avoid the impact of respiratory movement on the registration process, optionally, when acquiring the medical image of the target object 2 in step S2, a respiratory signal is also acquired synchronously; second gating information is obtained based on the respiratory signal, and the second gating information reflects the respiratory state of the target object 2; the second torso model is established based on the medical image sequence corresponding to the second gating information.

[0045] In an exemplary embodiment, when a medical imaging device is used to acquire a medical image of the target object 2, the target object 2 may hold its breath or exhale under the doctor's instruction, and the second gating information reflects the breathing state (holding breath or exhaling state, etc.) of the target object 2. In this way, the medical image acquired by the medical imaging device corresponds to a continuous slice image of the target object 2 in a certain breathing state, which reduces or eliminates the trunk positioning error caused by respiratory movement.

[0046] Furthermore, when the second scan data of the target object 2 is acquired by optical scanning in step S2, the first torso model and the second torso model are based on the same respiratory state corresponding to the respiratory state reflected by the second gating information.

[0047] When the target object 2 is scanned by the optical scanner, the target object 2 can hold its breath or exhale under the doctor's instruction, and the instruction should be the same as the instruction when the medical image is collected, that is, corresponding to the respiratory state reflected by the second gating information. Therefore, the first torso model and the second torso model are based on the same respiratory state, reducing or eliminating the registration error caused by different respiratory states.

[0048] Optionally, in step S1, the registration of the design model and the magnetocardiographic sensor model may be implemented in two steps, namely, first performing a coarse registration and then performing a fine registration, so as to improve the registration accuracy.

[0049] Specifically, the step of registering the design model of the magnetocardiographic sensor with the magnetocardiographic sensor model in the magnetocardiographic device model to obtain a first conversion matrix of the design model and the magnetocardiographic sensor model includes: Selecting a plurality of groups of matching feature points on the design model and the magnetocardio sensor model, aligning the design model with the matching feature points on the magnetocardio sensor model by a nearest point iteration algorithm, and obtaining a first coarse alignment transformation matrix; using the first coarse alignment transformation matrix, coarsely aligning the design model with the magnetocardio sensor model, and obtaining a first coarse alignment model; Selecting a plurality of groups of matching feature points on the first coarse alignment model and the magnetocardio sensor model, aligning the first coarse alignment model with the matching feature points on the magnetocardio sensor model by a nearest point iteration algorithm to obtain a first fine alignment transformation matrix; and finely aligning the first coarse alignment model with the magnetocardio sensor model by using the first fine alignment transformation matrix; The first transformation matrix=the first coarse alignment transformation matrix×the first fine alignment transformation matrix.

[0050] In an example, after importing the design model and the magnetocardiographic sensor model into the MESHLAB software, the point selection control of the MESHLAB software is used to select three sets of matching feature points from the design model and the magnetocardiographic sensor model, respectively, and they are recorded as n1, n2, n3, m1, m2, and m3. The above six points are all in three-dimensional space and consist of three coordinate points of x, y, and z. The alignment of the above three sets of feature points is achieved through the iterative closest point algorithm (ICP, IterativeClosestPoint) and the first rough alignment transformation matrix T is obtained. 1 , through the first coarse alignment transformation matrix T 1 The design model and the magnetocardiographic sensor model can be roughly aligned, and the first roughly aligned model is obtained after the rough alignment is completed. Then, the nearest point iteration algorithm is used again to achieve the fine alignment of the first roughly aligned model and the magnetocardiographic sensor model, and the first fine alignment transformation matrix T is obtained. 2 , where T 1 , T 2 The specific registration and alignment principle can be referred to the prior art and will not be elaborated here. Let A be the spatial position of the magnetocardiographic sensor 12. After the design model is registered and aligned with the magnetocardiographic sensor model, the position of the magnetocardiographic sensor 12 in the magnetocardiographic device coordinate system is B = T 1 ×T 2 ×A. It can be understood that the closest point iteration algorithm and several feature point groups are an example of registration rather than a limitation, and those skilled in the art can select other registration algorithms according to the prior art, and this embodiment is not limited thereto.

[0051] Optionally, in step S2, the registration of the second torso model with the first torso model may also be implemented in two steps, namely, first performing a coarse registration and then performing a fine registration, so as to improve the registration accuracy.

[0052] Specifically, the step of registering the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model includes: Selecting a plurality of groups of matching feature points on the second torso model and the first torso model, aligning the matching feature points on the second torso model and the first torso model by a nearest point iteration algorithm to obtain a second coarse alignment transformation matrix; using the second coarse alignment transformation matrix, coarsely aligning the second torso model and the first torso model to obtain a second coarse alignment model; Selecting a plurality of groups of matching feature points on the second coarse alignment model and the first torso model, aligning the second coarse alignment model with the matching feature points on the first torso model by a nearest point iteration algorithm to obtain a second fine alignment transformation matrix; and finely aligning the second coarse alignment model with the first torso model by using the second fine alignment transformation matrix; The second transformation matrix=the second coarse alignment transformation matrix×the second fine alignment transformation matrix.

[0053] In an example, similar to the registration and alignment steps of the design model and the magnetocardiographic sensor model, after the second torso model and the first torso model are imported into the MESHLAB software, the closest point iteration algorithm is used to align multiple sets of feature points and obtain the second rough alignment transformation matrix T 3 , through the first coarse alignment transformation matrix T 3 The second torso model can be roughly aligned with the first torso model, and the second rough alignment model can be obtained after the rough alignment is completed. Then, the closest point iteration algorithm can be used again to achieve the fine alignment of the second rough alignment model with the first torso model, and the second fine alignment transformation matrix T can be obtained. 4 Let C be the spatial position of the heart model after 3D reconstruction. After the second torso model is aligned with the first torso model, the position of the heart model in the coordinate system of the magnetic heart device is D = T 3 ×T 4 ×C.

[0054] Based on this, the alignment and unification of the sensor coordinate system, the magnetic heart device coordinate system and the heart model coordinate system were achieved, so that the magnetic heart data and the heart model can be accurately integrated, providing reliable input for subsequent magnetic heart tracing.

[0055] The embodiment of the present invention also provides a cardiac multimodal image registration system, which includes: an optical scanner, a magnetocardiographic device 1 and a registration execution module; The optical scanner is used to obtain first scanning data of the magnetocardiographic device 1 and second scanning data of the target object 2; the magnetocardiographic device 1 includes a magnetocardiographic sensor 12, and the magnetocardiographic device 1 is used to obtain the magnetocardiographic data of the target object 2; The registration execution module is configured to establish a magnetocardiographic device model based on the first scanning data; obtain a design model of the magnetocardiographic sensor of the magnetocardiographic device, align the design model with the magnetocardiographic sensor model in the magnetocardiographic device model, and obtain a first transformation matrix between the design model and the magnetocardiographic sensor model; align the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system according to the first transformation matrix; establish a first torso model based on the second scanning data; obtain a medical image of the target object 2, and establish a heart model and a second torso model of the target object 2 based on the medical image; align the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; align the heart model coordinate system with the magnetocardiographic device coordinate system through the second transformation matrix; and fuse the magnetocardiographic data with the heart model based on the aligned sensor coordinate system, the magnetocardiographic device coordinate system and the heart model coordinate system.

[0056] Optionally, the cardiac multimodal image registration system also includes an ECG acquisition device (not shown); the ECG acquisition device is used to synchronously acquire the ECG signal of the target object 2 when acquiring the medical image of the target object 2; the registration execution module is also configured to obtain first gating information based on the ECG signal, the first gating information reflecting the systole or diastole of the heart; the heart model is established based on the medical image sequence corresponding to the first gating information.

[0057] The structures and principles of other components of the cardiac multimodal image registration system may refer to the prior art and will not be described in detail in this embodiment.

[0058] The embodiment of the present invention further provides a readable storage medium having a program stored thereon, and when the program is executed, the steps of the above cardiac multimodal image registration method are implemented. The readable storage medium can be independently provided, or can be integrated in a component of the cardiac multimodal image registration system as described above, for example, integrated in the registration execution module.

[0059] In summary, in the cardiac multimodal image registration method, registration system and readable storage medium provided by the present invention, the cardiac multimodal image registration method includes: obtaining first scanning data of a magnetocardiographic device through optical scanning, and establishing a magnetocardiographic device model based on the first scanning data; obtaining a design model of a magnetocardiographic sensor of the magnetocardiographic device, aligning the design model with the magnetocardiographic sensor model in the magnetocardiographic device model, and obtaining a first transformation matrix between the design model and the magnetocardiographic sensor model; aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system according to the first transformation matrix; obtaining second scanning data of the target object through optical scanning, and aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system based on the first transformation matrix; obtaining second scanning data of the target object through optical scanning, and aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system based on the first transformation matrix; obtaining second scanning data of the target object through optical scanning, and aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system based on the first transformation matrix; obtaining second scanning data of the target object through optical scanning, and aligning the magnetocardiographic sensor coordinate system with the magnetocardiographic device coordinate system based on the first transformation matrix; obtaining first ... The second scanning data establishes a first torso model; obtains a medical image of the target object, and establishes a heart model and a second torso model of the target object based on the medical image; aligns the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; the heart model coordinate system is aligned with the heart magneto device coordinate system through the second transformation matrix; the heart magneto data of the target object is acquired through the heart magneto sensor of the heart magneto device, and the heart magnet data is fused with the heart model based on the aligned sensor coordinate system, the heart magneto device coordinate system and the heart model coordinate system. With such configuration, by aligning the design model of the heart magneto sensor with the heart magneto sensor model established based on optical scanning, and aligning the first torso model established based on optical scanning with the second torso model established based on medical images, the alignment and unification of the sensor coordinate system, the heart magneto device coordinate system and the heart model coordinate system are achieved, so that the heart magneto data and the heart model can be accurately fused, providing a reliable input for subsequent heart magneto tracing.

[0060] It should be noted that the above embodiments can be combined with each other. The above description is only a description of the preferred embodiments of the present invention, and does not limit the scope of the present invention. Any changes and modifications made by ordinary technicians in the field of the present invention based on the above disclosure are within the protection scope of the present invention.

Claims

1. A cardiac multimodal image registration method, characterized in that: include: Acquire first scanning data of a magnetocardiographic device through optical scanning, and establish a magnetocardiographic device model based on the first scanning data; acquire a design model of a magnetocardiographic sensor of the magnetocardiographic device, align the design model with a magnetocardiographic sensor model in the magnetocardiographic device model, and obtain a first transformation matrix between the design model and the magnetocardiographic sensor model; align a magnetocardiographic sensor coordinate system with a magnetocardiographic device coordinate system according to the first transformation matrix; Acquire second scan data of the target object through optical scanning, and establish a first torso model based on the second scan data; acquire a medical image of the target object, and establish a heart model and a second torso model of the target object based on the medical image; align the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; align the heart model coordinate system with the magnetocardiographic device coordinate system through the second transformation matrix; The magnetocardiographic data of the target object is acquired through the magnetocardiographic sensor of the magnetocardiographic device, and the magnetocardiographic data is fused with the heart model based on the aligned sensor coordinate system, the magnetocardiographic device coordinate system and the heart model coordinate system.

2. The cardiac multimodal image registration method according to claim 1, characterized in that: When acquiring medical images of a target object, an electrocardiogram signal is synchronously acquired; first gating information is obtained based on the electrocardiogram signal, and the first gating information reflects the systole or diastole of the heart; and the heart model is established based on a medical image sequence corresponding to the first gating information.

3. The cardiac multimodal image registration method according to claim 1, characterized in that: When acquiring a medical image of a target object, synchronously acquiring a respiratory signal; obtaining second gating information based on the respiratory signal, wherein the second gating information reflects a respiratory state of the target object; The second torso model is established based on a medical image sequence corresponding to the second gating information.

4. The cardiac multimodal image registration method according to claim 3, characterized in that: When acquiring the second scan data of the target object through optical scanning, the first torso model and the second torso model are based on the same respiratory state corresponding to the respiratory state reflected by the second gating information.

5. The cardiac multimodal image registration method according to claim 1, characterized in that: The step of registering the design model of the magnetocardiogram sensor with the magnetocardiogram sensor model in the magnetocardiogram device model to obtain a first conversion matrix between the design model and the magnetocardiogram sensor model comprises: Selecting a plurality of groups of matching feature points on the design model and the magnetocardio sensor model, aligning the design model with the matching feature points on the magnetocardio sensor model by a nearest point iteration algorithm, and obtaining a first coarse alignment transformation matrix; using the first coarse alignment transformation matrix, coarsely aligning the design model with the magnetocardio sensor model, and obtaining a first coarse alignment model; Selecting a plurality of groups of matching feature points on the first coarse alignment model and the magnetocardio sensor model, aligning the first coarse alignment model with the matching feature points on the magnetocardio sensor model by a nearest point iteration algorithm to obtain a first fine alignment transformation matrix; and finely aligning the first coarse alignment model with the magnetocardio sensor model by using the first fine alignment transformation matrix; The first transformation matrix=the first coarse alignment transformation matrix×the first fine alignment transformation matrix.

6. The cardiac multimodal image registration method according to claim 1, characterized in that: The step of registering the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model comprises: Selecting a plurality of groups of matching feature points on the second torso model and the first torso model, aligning the matching feature points on the second torso model and the first torso model by a nearest point iteration algorithm to obtain a second coarse alignment transformation matrix; using the second coarse alignment transformation matrix, coarsely aligning the second torso model and the first torso model to obtain a second coarse alignment model; Selecting a plurality of groups of matching feature points on the second coarse alignment model and the first torso model, aligning the second coarse alignment model with the matching feature points on the first torso model by a nearest point iteration algorithm to obtain a second fine alignment transformation matrix; and finely aligning the second coarse alignment model with the first torso model by using the second fine alignment transformation matrix; The second transformation matrix=the second coarse alignment transformation matrix×the second fine alignment transformation matrix.

7. The cardiac multimodal image registration method according to claim 1, characterized in that: The step of establishing a heart model and a second torso model of the target object based on the medical image comprises: Segmenting the medical image to obtain cardiac images of multiple time axes; establishing the cardiac model based on the cardiac images; The medical image is processed using the heart image as a mask to obtain a torso image; and the second torso model is established based on the torso image.

8. A readable storage medium, characterized in that: A program is stored thereon, and when the program is executed, the steps of the cardiac multimodal image registration method according to any one of claims 1 to 7 are implemented.

9. A cardiac multimodal image registration system, characterized in that: include: Optical scanner, magnetocardiographic device and registration execution module; The optical scanner is used to obtain first scanning data of the magnetocardiographic device and second scanning data of the target object; the magnetocardiographic device includes a magnetocardiographic sensor, and the magnetocardiographic device is used to obtain the magnetocardiographic data of the target object; The registration execution module is configured to establish a magnetocardiographic device model based on the first scanning data; obtain a design model of a magnetocardiographic sensor of the magnetocardiographic device, register the design model with a magnetocardiographic sensor model in the magnetocardiographic device model, and obtain a first transformation matrix between the design model and the magnetocardiographic sensor model; and align a magnetocardiographic sensor coordinate system with a magnetocardiographic device coordinate system according to the first transformation matrix; Establishing a first torso model based on the second scan data; acquiring a medical image of the target object, and establishing a heart model and a second torso model of the target object based on the medical image; registering the second torso model with the first torso model to obtain a second transformation matrix between the second torso model and the first torso model; aligning the heart model coordinate system with the magnetocardiographic device coordinate system through the second transformation matrix; The magnetocardiographic data is fused with the heart model based on the aligned sensor coordinate system, the magnetocardiographic device coordinate system and the heart model coordinate system.

10. The cardiac multimodal image registration system according to claim 9, characterized in that: It also includes an ECG acquisition device; the ECG acquisition device is used to synchronously acquire the ECG signal of the target object when acquiring the medical image of the target object; The registration execution module is further configured to obtain first gating information based on the electrocardiogram signal, where the first gating information reflects the systole or diastole of the heart; and the heart model is established based on a medical image sequence corresponding to the first gating information.

Images