Heart body surface multi-point joint acquisition and multi-modal fusion system, method and equipment

Abstract

The invention provides a heart body surface multi-point joint acquisition and multi-modal fusion system, method and device, and belongs to the technical field of heart state monitoring, the system comprises: a cardiac seismic signal sensor used for acquiring a plurality of initial cardiac seismic signals at a plurality of acquisition positions of a measured chest, each acquisition position corresponding to one initial cardiac seismic signal; the first signal processing module is used for preprocessing the plurality of initial cardiac seismic signals to obtain a plurality of target cardiac seismic signals; the radar signal sensor is used for transmitting a radar signal and collecting an initial radar echo signal returned by the tested chest; the second signal processing module is used for preprocessing the initial radar echo signal to obtain an area array radar echo signal; and the data processing module is used for determining target physiological state information for reflecting the measured chest according to the plurality of target cardiac shock signals and the area array radar echo signals. More accurate heart activity information can be provided, so that the diagnosis accuracy is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of cardiac status monitoring, and in particular to a cardiac surface multi-point joint acquisition and multi-modal fusion system, method and device. Background Art

[0002] With the rapid development of the medical and health field, heart health monitoring technology has received more and more attention. As one of the leading causes of death worldwide, early diagnosis and real-time monitoring of heart disease are crucial to improving treatment outcomes and patients’ quality of life.

[0003] Traditional heart monitoring methods, such as electrocardiogram (ECG), play an important role in clinical diagnosis, but ECG often only collects electrocardiograms. However, the structure of the heart is complex. In addition to ECG signals, the mechanical vibration of cardiac activity is also important for the assessment of heart health. However, the mechanical vibration of cardiac activity is greatly affected by the external environment, the signal is weak, and it is not easy to analyze. Summary of the invention

[0004] The present disclosure provides a system, method and device for joint acquisition of multi-points on the heart surface and multi-modal fusion; the system can reduce the impact caused by a weak single signal or noise interference, thereby making the key features obtained by the final analysis more accurate and richer in content, thereby improving the accuracy of diagnosis.

[0005] The technical solution of the present disclosure is achieved as follows: In a first aspect, the present disclosure provides a cardiac surface multi-point joint acquisition and multi-modal fusion system, which includes: a cardiac tremor signal sensor, a first signal processing module connected to the cardiac tremor signal sensor, a radar signal sensor, a second signal processing module connected to the radar signal sensor, and a data processing module connected to the first signal processing module and the second signal processing module; the cardiac tremor signal sensor is configured to acquire multiple initial cardiac tremor signals at multiple acquisition positions of a measured chest cavity, each acquisition position corresponds to an initial cardiac tremor signal; the first signal processing module is configured to pre-process the multiple initial cardiac tremor signals to obtain multiple target cardiac tremor signals; the radar signal sensor is configured to transmit a radar signal and acquire an initial radar echo signal returned by the measured chest cavity, and the second signal processing module is configured to pre-process the initial radar echo signal to obtain an array radar echo signal; the data processing module is configured to determine target physiological state information reflecting the measured chest cavity according to the multiple target cardiac tremor signals and the array radar echo signal.

[0006] In a second aspect, the present disclosure provides a method for multi-point joint acquisition and multi-modal fusion of cardiac surface, the method comprising: acquiring multiple initial cardiac shock signals at multiple acquisition positions of the measured chest cavity, each acquisition position corresponding to an initial cardiac shock signal; preprocessing the multiple initial cardiac shock signals to obtain multiple target cardiac shock signals; transmitting a radar signal and acquiring an initial radar echo signal returned by the measured chest cavity; preprocessing the initial radar echo signal to obtain an array radar echo signal; and determining target physiological state information reflecting the measured chest cavity based on the multiple target cardiac shock signals and the array radar echo signal.

[0007] In a third aspect, the present disclosure provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method for joint acquisition of multiple points on the cardiac surface and multimodal fusion as described in the second aspect.

[0008] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing a program or instruction, which, when executed by a processor, implements the steps of the method for joint acquisition of multiple points on the cardiac surface and multi-modal fusion as described in the second aspect.

[0009] In a fifth aspect, the present disclosure provides a computer program product, wherein the computer program product includes a computer program or instructions. When the computer program product runs on a processor, the processor executes the computer program or instructions to implement the steps of the method for joint acquisition of multiple points on the cardiac surface and multi-modal fusion as described in the second aspect.

[0010] In a sixth aspect, the present disclosure provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method for joint acquisition of multiple points on the cardiac surface and multi-modal fusion as described in the second aspect.

[0011] The present disclosure provides a cardiac surface multi-point joint acquisition and multi-modal fusion system, which includes: a cardiac tremor signal sensor, a first signal processing module connected to the cardiac tremor signal sensor, a radar signal sensor, a second signal processing module connected to the radar signal sensor, and a data processing module connected to the first signal processing module and the second signal processing module; the cardiac tremor signal sensor is configured to acquire multiple initial cardiac tremor signals at multiple acquisition positions of a measured chest cavity, each acquisition position corresponds to an initial cardiac tremor signal; the first signal processing module is configured to pre-process the multiple initial cardiac tremor signals to obtain multiple target cardiac tremor signals; the radar signal sensor is configured to transmit a radar signal and acquire an initial radar echo signal returned by the measured chest cavity, and the second signal processing module is configured to pre-process the initial radar echo signal to obtain a planar array radar echo signal; the data processing module is configured to determine target physiological state information reflecting the measured chest cavity according to the multiple target cardiac tremor signals and the planar array radar echo signal. The physiological status information that can reflect cardiac activity included in both the target cardiotocracycline signal and the area array radar echo signal is fused. The fusion can enhance the signal strength and reduce the impact caused by a weak single signal or noise interference, so that the final target physiological status information can more accurately indicate cardiac activity and improve the accuracy of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is one of the structural schematic diagrams of a cardiac surface multi-point joint acquisition and multi-modal fusion system provided by the present disclosure; Figure 2 A schematic diagram of the collection location provided for the present disclosure; Figure 3 A schematic diagram of the echo detection principle based on a multi-input multi-output millimeter-wave radar provided in the present disclosure; Figure 4 The second structural schematic diagram of a cardiac surface multi-point joint acquisition and multi-modal fusion system provided by the present invention; Figure 5 A schematic diagram of characteristic points provided by the present disclosure; Figure 6 A schematic diagram of a process of a method for joint acquisition of multi-points on the cardiac surface and multi-modal fusion provided by the present invention; Figure 7 A schematic diagram of the hardware structure of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the present disclosure to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of the present application.

[0014] The terms "first", "second", etc. in the specification of this application are used to distinguish similar objects, rather than to describe a specific order or precedence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the present disclosure can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more.

[0015] Figure 1 The present invention discloses a cardiac surface multi-point joint acquisition and multi-modal fusion system. Figure 1 As shown, the cardiac body surface multi-point joint acquisition and multi-modal fusion system 10 includes: a cardiac shock signal sensor 101, a first signal processing module 102 connected to the cardiac shock signal sensor 101, a radar signal sensor 103, a second signal processing module 104 connected to the radar signal sensor 103, and a data processing module 105 connected to the first signal processing module 102 and the second signal processing module 104; The cardiac tremor signal sensor 101 is configured to collect multiple initial cardiac tremor signals at multiple collection positions of the measured chest cavity; the first signal processing module 102 is configured to pre-process the multiple initial cardiac tremor signals to obtain multiple target cardiac tremor signals; the radar signal sensor 103 is configured to transmit radar signals and collect initial radar echo signals returned by the measured chest cavity; the second signal processing module 104 is configured to pre-process the initial radar echo signals to obtain array radar echo signals; the data processing module 105 is configured to determine the target physiological state information reflecting the measured chest cavity based on the multiple target cardiac tremor signals and the array radar echo signals.

[0016] The cardiac tremor signal sensor 101 is configured to collect a plurality of initial cardiac tremor signals at a plurality of collection positions of the chest cavity to be measured.

[0017] Cardiac shock signals generally refer to tiny vibrations that are synchronized with the heartbeat and are caused by the contraction of the heart and the impact of blood on large blood vessels. Based on this, in the present disclosure, the cardiac shock signal sensor 101 can be implemented as a vibration sensor for measuring information related to object vibration (such as amplitude, frequency, acceleration, etc.), for example, a piezoelectric crystal acceleration sensor, a piezoelectric crystal displacement sensor, a micro-electromechanical system acceleration sensor, a micro-electromechanical system displacement sensor, etc. Taking a piezoelectric crystal acceleration sensor or a micro-electromechanical system acceleration sensor as an example, it can capture the linear acceleration component of the heart vibration. These signals reflect the tiny body vibrations caused by the physical movement of the heart. By analyzing these signals, information related to heart activity can be extracted, such as peak value (corresponding to the closing and opening of each heart valve), signal amplitude, energy, area, and time interval, etc., and a piezoelectric crystal displacement sensor or a micro-electromechanical system displacement sensor can capture the amplitude of the heart vibration. Since the chest cavity under test is divided into multiple collection positions, the number of cardiac shock signal sensors 101 is consistent with the collection positions, and during the implementation process, each shock sensor is attached to the corresponding collection position of the chest cavity under test, so as to collect the initial cardiac shock signal at each collection position.

[0018] Since the vibration signals at different positions of the tested chest cavity can reflect different cardiac physiological activities, in order to make the collected initial cardiac vibration signal reflect the cardiac activity more comprehensively, multiple collection positions can be set at certain intervals to cover the entire tested chest cavity.

[0019] However, if there are too many acquisition positions, it is necessary to screen out the cardiac tremor signals that can represent the physiological activities of the heart from multiple initial cardiac tremor signals, which increases the processing flow. Therefore, in order to ensure that the acquired cardiac tremor signals can reflect more cardiac activities, the number of acquisition positions is reduced to save hardware resources and reduce user discomfort.

[0020] In some embodiments, since the mechanical vibrations generated by the physiological activities of the human heart mainly come from the opening and closing movements of the four valves in the heart, Figure 2 As shown, it is a schematic diagram of the set acquisition positions, and the acquisition positions include: the mitral valve area M at the strongest point of the apex beat located at the 5th intercostal space, 0.5-1.0 cm inside the left clavicle midline, the pulmonary valve area P located at the 2nd intercostal space on the left edge of the sternum, the aortic valve area A located at the 2nd intercostal space on the right edge of the sternum, the aortic valve second area E located at the 3rd intercostal space on the left edge of the sternum, and the tricuspid valve area T located at the left edge of the lower end of the sternum, that is, between the 4th and 5th intercostals on the left edge of the sternum.

[0021] The sampling position can be set according to the height and weight of the subject to be tested; or the initial radar echo signal can be first collected for a certain period of time, and the shape and size of the heart to be tested in the chest cavity to be tested can be determined according to the initial radar echo signal, and then a more reasonable sampling position can be set according to the shape and size.

[0022] Since the acquired initial cardiac arrest signals usually include noise, the first signal processing module 102 is configured to pre-process the multiple initial cardiac arrest signals to obtain multiple target cardiac arrest signals.

[0023] In some embodiments, the preprocessing includes filtering and removing baseline drift. The first signal processing module 102 is specifically configured to filter and remove baseline drift on a plurality of initial cardiac arrest signals to obtain a plurality of target cardiac arrest signals.

[0024] Most of the noise in the original signal comes from the power frequency interference generated by the power supply when the system is working, and some noise comes from the bioelectric signals of the human body, such as electromyographic signals, etc. Since the vibration frequency generated by human heart activity is mainly in the range of 0.05Hz to 2Hz, a Butterworth bandpass filter can be used to filter out signals with frequencies not in the range of 0.05Hz to 2Hz to remove noise signals unrelated to heart activity.

[0025] Baseline drift refers to the slowly changing non-periodic trends that appear in the signal. These trends are not directly caused by cardiac activity, but are signal changes caused by other factors. The essence of baseline drift is the superposition of DC components and low-frequency components on the original signal, which may affect the accuracy of the signal and subsequent data processing results.

[0026] The causes of baseline drift usually include: the interaction between cardiac activity and respiratory pattern (i.e., cardiopulmonary coupling) introduces low-frequency fluctuations related to breathing into the initial cardiotonic signal, causing baseline drift; random movements caused by changes in body posture or muscle activity will cause additional fluctuations in the initial cardiotonic signal, causing baseline drift.

[0027] The purpose of removing baseline drift is to remove non-physiological, non-interesting low-frequency trend components in the signal. These components are not directly generated by cardiac activity, but are generated by other factors such as cardiopulmonary coupling and random body movement. In order to eliminate the influence of these baseline drifts, a variety of methods can be used, such as: baseline drift can be removed by detrend function, wavelet transform, zero point calibration, empirical mode decomposition, etc. However, because the detrend function is a simple and effective method to remove trend items in the signal, it can subtract a best-fitting straight line (linear trend) or a higher-order polynomial trend from the signal, so that the mean of the signal is close to zero. In practice, this method is often used and is particularly suitable for removing linear or piecewise linear trend items.

[0028] The radar signal sensor 103 is configured to transmit a radar signal and collect an initial radar echo signal returned by the chest cavity under test.

[0029] The radar signal sensor 103 transmits a frequency modulated continuous wave to the chest cavity under test and receives the reflected initial radar echo signal, thereby capturing the tiny vibration of the chest cavity under test caused by the heartbeat. The radar signal sensor 103 receives the initial radar echo signal through multiple virtual transceiver channels composed of multiple transmitting antennas and multiple receiving antennas. The radar signal is transmitted in a time division multiple access (TDMA) mode, thereby ensuring that only one pair of transceiver antennas are working each time the radar signal is transmitted and received, and the radiation area of ​​the radar signal sensor 103 covers the entire chest cavity under test. The initial radar signal includes information such as the speed, acceleration, phase, and direction of vibration.

[0030] Similar to the above-mentioned heart shock signal, the collected initial radar echo signal also includes noise. Therefore, the second signal processing module 104 is configured to pre-process the initial radar echo signal to obtain the array radar echo signal.

[0031] In some embodiments, preprocessing includes filtering, removing static background and sidelobe interference; the second signal processing module 104 is specifically configured to filter, remove static background and sidelobe interference on the initial radar echo signal to obtain an array radar echo signal.

[0032] The filtering process may refer to the above-mentioned filtering process on the initial cardiac shock signal, which will not be described in detail here.

[0033] Static background mainly comes from the radar system itself and fixed objects in the surrounding environment. In order to remove these interferences, hybrid analog and digital compensation techniques can be used to suppress leakage and static clutter. For example, a portion of the static background can be removed by a sliding average filter algorithm; a frequency domain filter can also be used to remove the static background. For example, by designing a band-stop filter, signals within a specific frequency range can be suppressed, thereby removing the interference of the static background.

[0034] Sidelobe interference refers to the scattering of radar signals in non-target directions. These scattered signals may interfere with the detection of target signals. In order to remove sidelobe interference, sidelobe suppression technology can be used, such as using a window function to reduce the sidelobe level; using the characteristics of Radon transform to distinguish the sidelobe from the target main axis. When the main axis and sidelobe have different directions, the peak points are in different positions in the Radon transform domain, and they can be distinguished in the Radon transform domain; integrating along the azimuth and range directions to find sidelobes, and using local mean filtering to eliminate their influence, etc.

[0035] The data processing module 105 determines target physiological state information reflecting the measured chest cavity according to the multiple target cardiac shock signals and the area array radar echo signals.

[0036] Specifically, since both the target cardiotocracyptus signal and the area array radar echo signal can collect the vibration signal of the chest cavity to be tested, the features included in both the target cardiotocracyptus signal and the area array radar echo signal are fused. The fused signal can enhance the signal strength and reduce the impact caused by a weak single signal or noise interference.

[0037] Since the radiation surface of the radar signal sensor 103 is the entire chest cavity to be tested, the usual analysis is to fuse the radar echo signal returned by the entire chest cavity to be tested, but the accuracy of this analysis method is not enough, and there may be two signals reflecting vibrations in different areas, and there are large differences between the signals themselves. The result of direct fusion may not accurately reflect the actual heart activity. Therefore, the heart shock signal sensor 101 of the present disclosure has a fixed collection position on the chest cavity to be tested, so that the radar echo signal and the target heart shock signal are heart activity signals collected at the same position.

[0038] In some embodiments, the radar signal sensor 103 includes: a radar antenna array, a data processing module 105, which is specifically configured to: determine a reference point mapped from the center of the radar antenna array to the chest cavity under test; determine a multi-point micro-motion radar echo signal at each collection position from the array radar echo signal based on the distance between the reference point and each collection position and the radar transmission distance, the radar transmission distance being the distance between the radar signal sensor 103 and the chest cavity under test; determine a target physiological state reflecting the chest cavity under test based on the target heart shock signal at each collection position and the corresponding multi-point micro-motion radar echo signal.

[0039] A radar antenna array is a system composed of multiple antenna units (also called array elements) that are arranged in a certain geometric pattern and work together to achieve a specific radar function. The individual antenna units in the radar antenna array can be arranged as needed, such as a linear array arranged linearly, a planar array arranged on a plane (such as a rectangular array, a circular array, etc.), and a three-dimensional array distributed in a certain space (such as a spherical array). The radar antenna array in the present disclosure is a planar array. Radar antenna arrays with different arrangements have different radiation areas, and the center of the radiation area is the center of the radar antenna array.

[0040] Specifically, Figure 3 The schematic diagram of echo detection based on multi-input multi-output millimeter wave radar is shown. The coordinates of the five acquisition positions in the plane diagram of the chest cavity to be measured corresponding to the three-dimensional chest cavity to be measured are recorded as If the specific coordinate value is , , , , , , determine the reference point where the center of the radar antenna is mapped to the chest cavity being measured, that is, the position of the reference point is where the azimuth and elevation angles of the radar are both 0. If the acquisition position A is taken as the reference point, it is expressed as The transmission distance from the reference point, i.e. the acquisition position A, to the radar is recorded as , and calculate the azimuth and elevation angles of the acquisition position P, acquisition position T, acquisition position E, and acquisition position M relative to the radar, which are expressed as , , , ,in: ,and , According to the azimuth and elevation angle of each acquisition position, the multi-point micro-motion radar echo signals at each acquisition position are identified from the radar array radar echo signals.

[0041] In this embodiment, each multi-point micro-motion radar echo signal and the corresponding target cardiac activity signal reflect the cardiac activity at the same acquisition position. The two signals have a higher matching degree and a higher fusion accuracy, and the target physiological state information obtained to reflect the cardiac activity is more accurate.

[0042] In some embodiments, Figure 4 As shown, the cardiac surface multi-point joint acquisition and multi-modal fusion system 10 also includes: an ECG signal sensor 106, and a third signal processing module 107 connected to the ECG signal sensor 106; the ECG signal sensor 106 is configured to acquire the initial ECG signal of the measured chest cavity; the third signal processing module 107 is configured to pre-process the initial ECG signal to obtain a target ECG signal; and the data processing module 105 is specifically configured to determine target physiological state information reflecting the measured chest cavity according to the target ECG signal, multiple target cardiac tremor signals and multiple multi-point micro-motion radar echo signals.

[0043] The beating of the heart is controlled by the electrical signals generated by the cardiac pacemaker cells. These electrical signals are generated as different parts of the heart are excited, forming a series of electrical signals. The ECG signal sensor 106 is used to collect the electrical signals of the heart activity.

[0044] Since the collected initial ECG signal is weak and contains noise, it is necessary to preprocess the initial ECG signal to obtain a target ECG signal with reduced noise.

[0045] In some embodiments, preprocessing includes: amplification, filtering and baseline drift removal; the third signal processing module 107 is specifically configured to amplify, filter and baseline drift removal the initial ECG signal to obtain a target ECG signal.

[0046] After the initial ECG signal is amplified by the signal amplifier, it is filtered and baseline drift is removed. For details of the filtering and baseline drift removal, reference may be made to the above description of the initial cardiac shock signal processing process, which will not be repeated here.

[0047] Because it combines the features of the target ECG signal, multiple target cardiac tremor signals and multi-point micro-motion radar echo signals, including mechanical activity features and electrical activity features, it contains richer cardiac activity information.

[0048] The acquired signals include multiple features, but because the opening and closing of the aortic valve marks the beginning and end of ventricular contraction and relaxation, and the opening and closing of the mitral valve reflects the hemodynamics between the atria and ventricles, the four feature points of aortic valve opening, aortic valve closing, mitral valve opening, and mitral valve closing can best reflect the physiological activities of the heart. By monitoring these feature points, the mechanical function and hemodynamic state of the heart can be effectively evaluated.

[0049] Therefore, in some embodiments, the data processing module 105 is configured to determine the aortic valve opening and closing characteristic points and the mitral valve opening and closing characteristic points at each acquisition position based on the target electrocardiogram signal, multiple target cardiac tremor signals and multiple multi-point micro-motion radar echo signals.

[0050] Specifically, in some embodiments, the data processing module 105 is specifically configured to: determine the unit ECG signal of a cardiac cycle, the unit radar echo signal and the unit cardiac shock signal at each acquisition position according to the target ECG signal, multiple target cardiac shock signals and multiple multi-point micro-motion radar echo signals; divide the unit radar echo signal and the unit cardiac shock signal at each acquisition position into a first sub-wave and a second sub-wave according to the time points corresponding to the R peak and the T peak in the unit ECG signal; in the first sub-wave, determine the maximum value within the first preset time length before the first reference origin as the mitral valve closure feature point, and determine the maximum value within the second preset time length after the first reference origin as the aortic valve opening feature point, and the first reference origin is the minimum value point within the first experience time length after the R peak; in the second sub-wave, determine the maximum value within the third preset time length before the second reference origin as the aortic valve closure feature point, and determine the minimum value within the fourth preset time length after the second reference origin as the mitral valve opening feature point, and the second reference origin is the maximum value point within the second experience time length after the T peak.

[0051] Each of the target ECG signal, target heartbeat signal and multi-point micro-motion radar echo signal includes signals of multiple cardiac cycles (the entire time period from the beginning of one heartbeat to the beginning of the next heartbeat). For each signal, each cardiac cycle is basically the same. Therefore, extracting target physiological state information from the target ECG signal, target heartbeat signal and multi-point micro-motion radar echo signal can be simplified to extracting target physiological state information from the signal of one cardiac cycle of each signal.

[0052] In the target ECG signal, the R peak is the mark of ventricular depolarization, that is, the mark of ventricular contraction. A cardiac cycle can be determined based on the R peak. One cardiac cycle is about 1 second, so 0.5 seconds forward and backward can be taken from the R peak as the origin as a cardiac cycle. Since the target ECG signal, the multi-point micro-motion radar echo signal, and the target cardiac tremor signal are collected synchronously, the multi-point micro-motion radar echo signal and the target cardiac tremor signal are divided into multiple cardiac cycles according to the starting time point and the ending time point of each cardiac cycle of the target ECG signal.

[0053] Since the signal of one cardiac cycle may not effectively represent the actual state of cardiac activity due to noise or system reasons, the signal of one cardiac cycle can be fused based on the signals of multiple cardiac cycles. The specific fusion can be weighted average or direct superposition. For example, the ECG signals of 10 consecutive cardiac cycles in the target ECG signal are weighted summed to obtain a unit ECG signal; the cardiac tremor signals of 10 consecutive cardiac cycles in the target cardiac tremor signal are weighted summed to obtain a unit cardiac tremor signal; the radar echo signals of 10 consecutive cardiac cycles in the multi-point micro-motion radar echo signal are weighted summed to obtain a unit radar echo signal. In this way, it is ensured that the unit ECG signal, unit radar echo signal and unit cardiac tremor signal to be analyzed represent the actual cardiac activity.

[0054] The unit radar echo signal and the unit heart shock signal can be divided into a first sub-wave and a second sub-wave according to the contraction of the heart. The first sub-wave represents the contraction of the heart, and the second sub-wave represents the diastole of the heart. In the target electrocardiogram signal, the T peak is a marker of ventricular repolarization, that is, a marker of ventricular diastole. Therefore, the first sub-wave and the second sub-wave are divided according to the R peak and the T peak. Since the time interval between the T peak and the R peak is fixed, the T peak can also be determined according to the R peak.

[0055] To divide the first sub-wave and the second sub-wave, it is necessary to first determine the R peak. The maximum value in the unit ECG signal corresponds to the R peak. The extreme point within the preset time length after the R peak is the T peak. The first sub-wave and the second sub-wave are divided according to the time points corresponding to the R peak and the T peak. Specifically, according to the experimental value, the wave corresponding to the first time length range after the first time length with the time point corresponding to the R peak as the origin is the first sub-wave, and the wave corresponding to the second time length range after the second time length with the time point corresponding to the R peak as the origin is the second sub-wave, or the wave corresponding to the second time length range after the third time length with the T peak as the origin is the second sub-wave, and the third time length is greater than the second time length.

[0056] After dividing the first sub-wave and the second sub-wave, it is necessary to determine the characteristic points. The first sub-wave includes the mitral valve closure characteristic point and the active valve opening characteristic point, and the second sub-wave includes the mitral valve opening characteristic point and the aortic valve closure characteristic point. Specifically, the first preset time length and the second preset time length are determined according to the experiment, and the maximum value within the first preset time length before the first reference far point is the mitral valve closure characteristic point, and the maximum value within the second preset time length after the first reference origin is the aortic valve opening characteristic point. The third preset time length and the fourth preset time length are determined according to the experiment, and the maximum value within the third preset time length before the second reference origin is the aortic valve closure characteristic point, and the minimum value within the fourth preset time length after the second reference origin is the mitral valve opening characteristic point.

[0057] For example, Figure 5 The figure shows an exemplary schematic diagram of determining characteristic points. The R peak and T peak are determined from the target electrocardiogram, S1 represents the first sub-wave, S2 represents the second sub-wave, the first empirical duration range is 125ms to 155ms after the R peak, the second empirical duration range is 135ms to 175ms after the T peak, the first reference origin is ICP, the second reference origin is IRP, according to the first reference origin, the maximum value is determined to be the aortic valve opening characteristic point AO and the mitral valve closing characteristic point MC, according to the second reference origin, the maximum value is determined to be the aortic valve closing characteristic point AC, and the minimum value is the mitral valve opening characteristic point MO.

[0058] It should be noted that the first sub-wave and the second sub-wave described above represent: the first sub-wave and the second sub-wave in the unit cardiac tremor signal, and the first sub-wave and the second sub-wave in the unit radar echo signal. In addition, the above description is based on a unit radar echo signal, a unit cardiac tremor signal and a unit electrocardiogram signal. In an actual acquisition process, multiple unit radar echo signals, unit cardiac tremor signals and unit electrocardiogram signals can be obtained, and the processing process of each unit radar echo signal, unit cardiac tremor signal and unit electrocardiogram signal is the same.

[0059] The aortic valve opening and closing feature points and the mitral valve opening and closing feature points at each acquisition position are fused to obtain the target physiological state information. Specifically, at each acquisition position, the aortic valve opening and closing feature points and the mitral valve opening and closing feature points of the unit heart shock signal, as well as the aortic valve opening and closing feature points and the mitral valve opening and closing feature points of the unit radar echo signal are obtained, and the corresponding feature points are fused (such as weighted summation), such as: the aortic valve opening feature points of the unit heart shock signal are fused with the aortic valve opening feature points of the unit radar echo signal, the mitral valve opening feature points of the unit heart shock signal are fused with the mitral valve opening feature points of the unit radar echo signal, etc., and each fused feature point after fusion is the target physiological state information.

[0060] It should be noted that the waveforms of the same characteristic points of the unit radar echo signal and the corresponding unit heart shock signal may be opposite, and the waveforms need to be flipped to make the two waveforms consistent before fusion.

[0061] In this way, the four feature points that best reflect heart activity are extracted from the complex radar echo signals, heart shock signals and electrocardiogram signals. Because multiple collection positions are set, four feature points can be obtained at each collection position. The feature points at each position are compared and analyzed to obtain more accurate heart status assessment results.

[0062] In some embodiments, the data processing module 105 is specifically configured to determine the first physiological state information according to the multiple target cardiac shock signals and determine the second physiological state information according to the array radar echo signal, wherein the first physiological state information and the second physiological state information are the cardiac state information other than the target physiological state information.

[0063] Since the radar echo signal monitors the chest cavity to be tested in a non-contact manner, the vibration caused by breathing and human body shaking is much greater than the vibration caused by heart activity, so some subtle vibrations in the heart cannot be detected by the radar signal sensor 103. However, the heart shock signal is collected in a contact manner, and the chest cavity to be tested is usually static. Therefore, the heart shock sensor 101 can monitor some subtle vibrations of the heart, such as: the tiny vibrations of the human body caused by the heart pumping blood. Therefore, the first physiological state information included in some tiny vibrations that can reflect heart activity is analyzed from the target heart shock signal.

[0064] In addition, the radar echo signal may also include some cardiac activity information that is not included in the cardiac tremor signal. For example, the radar signal sensor 103 can extract a four-dimensional cardiac mechanical activity signal, which includes not only a time dimension but also a space dimension. Therefore, the second physiological state information that is not included in the target cardiac tremor signal is analyzed from the array radar echo signal.

[0065] In this way, the unique features of the target cardiotocracycline signal and the area array radar echo signal can also be extracted, which can provide richer cardiac activity information than a single sensor, making the key features obtained in the final analysis more accurate and richer in content, thereby improving the accuracy of diagnosis.

[0066] Since the radar signal sensor 103 is non-contact, the monitored vibration signal may be inaccurate, while the cardiac shock signal sensor 101 is contact-type, with less noise and more accurate vibration signals. However, the radar signal sensor 103 has the advantage of being non-contact. In order to satisfy both non-contact and accuracy. In some embodiments, according to the difference between the vibration signal collected by the cardiac shock signal sensor 101 and the vibration signal collected by the radar signal sensor 103, the working parameters of the radar signal sensor 103 are adjusted, so as to finally adjust the difference between the vibration signals collected by the cardiac shock signal sensor 101 and the radar signal sensor 103 at the same collection position to be less than the difference threshold. Thereby improving the accuracy of the vibration signal collected by the radar signal sensor 103.

[0067] As for the hardware setting of the multi-point joint acquisition and multi-modal fusion system of the heart surface provided by the present disclosure, since the radar signal is collected in a non-contact manner, and both the electrocardiogram signal and the cardiac tremor signal require sensors that are in direct contact with the human body, the cardiac tremor signal sensor 101, the first signal processing module 102, the electrocardiogram signal sensor 106, and the third signal processing module 107 can be integrated together in the hardware setting and made using flexible printed circuit board technology. This technology is a printed circuit with high reliability and excellent flexibility made of polyester film or polyimide as a substrate. By embedding the circuit design on a bendable and thin plastic sheet, a large number of precision components are stacked in a narrow and limited space, thereby forming a bendable flexible circuit. This circuit can be bent and folded at will (to facilitate fitting with the irregular chest surface), is light in weight, small in size, has good heat dissipation, and is easy to install. The physical dimensions of the integrated circuit are actually 3 cm long, 3 cm wide, and 1.5 cm high. This size can more conveniently and easily place the sensor on the chest surface of the subject while ensuring the realization of all functions of the system, and can effectively reduce the additional vibration caused by the sensor's own shaking, thereby improving the authenticity and accuracy of the measurement results.

[0068] The radar sensor 103 and the second signal processing module 104 can be set as an independent hardware module. The first signal processing module 102, the second signal processing module 104 and the third signal processing module 107 can be connected to the data processing module 105 by wire or wireless, and the data processing module 105 can be a terminal (such as a computer).

[0069] For example, the sampling frequency of the radar signal sensor 103 is set to 5 MHz, the sampling depth is 16 bits, the radar start frequency is 77 GHz, the end frequency is 81 GHz, and the effective bandwidth is 4 GHz. Actual experimental data show that under this parameter, the mechanical movement generated by the physiological activity of the heart can be recorded more accurately.

[0070] The mechanical vibration generated by the physiological activity of the human heart is about 20-40 mg, and the minimum time difference between two adjacent feature points is about 25 ms. Therefore, the sampling rate of the cardiac signal sensor 101 needs to be set to 250 Hz and the sampling depth to 20 bits to ensure that the feature point information is recorded with high enough accuracy.

[0071] The frequency of normal human heart mechanical vibration is about 1Hz to 2Hz, and the QRS wave group duration is 0.06 to 0.10s, and the longest is no more than 0.11s. Therefore, the sampling rate of the ECG signal sensor 106 is set to 125Hz to accurately record the QRS wave group information while reducing the amount of data transmission.

[0072] The power consumption of the system mainly comes from each module, so the total power consumption of the whole system can be estimated by counting the current of each module under normal working conditions. For example, when the system works at a 3.7V power supply voltage, the total power consumption is about 0.74W. When using a 700mAh battery, the total battery life of the system is about 3.5h when continuously collecting data.

[0073] Based on the same design concept as the above-mentioned cardiac surface multi-point joint acquisition and multi-modal fusion system, the present disclosure also provides a cardiac surface multi-point joint acquisition and multi-modal fusion method, such as Figure 6 As shown, the method for joint acquisition of multiple points on the heart surface and multi-modal fusion includes the following steps S601 to S605.

[0074] In step S601, multiple initial cardiac tremor signals are acquired at multiple acquisition positions of the chest cavity to be tested, wherein each acquisition position corresponds to an initial cardiac tremor signal.

[0075] In step S602, a plurality of initial cardiac arrest signals are preprocessed to obtain a plurality of target cardiac arrest signals.

[0076] In step S603, a radar signal is transmitted and an initial radar echo signal returned by the chest cavity under test is collected.

[0077] In step S604, the initial radar echo signal is preprocessed to obtain an array radar echo signal.

[0078] In step S605, target physiological state information reflecting the measured chest cavity is determined according to a plurality of target cardiac arrest signals and area array radar echo signals.

[0079] In some embodiments, the above step S605 determines the target physiological state information reflecting the measured chest cavity based on multiple target cardiac tremor signals and array radar echo signals, including: determining a reference point where the center of the radar antenna array is mapped to the measured chest cavity; determining the multi-point micro-motion radar echo signal at each collection position from the array radar echo signal based on the distance between the reference point and each collection position and the radar transmission distance, the radar transmission distance being the distance between the radar signal sensor 103 and the measured chest cavity; determining the target physiological state information reflecting the measured chest cavity based on the target cardiac tremor signal and the multi-point micro-motion radar echo signal at each collection position.

[0080] In some embodiments, the method for joint acquisition of multiple points on the heart surface and multimodal fusion also includes: acquiring the initial ECG signal of the chest cavity under test; preprocessing the initial ECG signal to obtain a target ECG signal; the above-mentioned step S605 determines the target physiological state information reflecting the chest cavity under test based on multiple target cardiac tremor signals and array radar echo signals, including: determining the target physiological state information reflecting the chest cavity under test based on the target ECG signal, multiple target cardiac tremor signals and array radar echo signals.

[0081] In some embodiments, the above-mentioned step S605 determines the target physiological state information reflecting the measured chest cavity based on multiple target cardiac tremor signals and array radar echo signals, including: determining the aortic valve opening and closing feature points and the mitral valve opening and closing feature points at each acquisition position based on the target electrocardiogram signal, multiple target cardiac tremor signals and multiple multi-point micro-motion radar echo signals; fusing the aortic valve opening and closing feature points and the mitral valve opening and closing feature points at each acquisition position to obtain the target physiological state information.

[0082] In some embodiments, the above step S605 determines the target physiological state information reflecting the measured chest cavity according to the multiple target cardiac tremor signals and the array radar echo signals, including: determining the unit ECG signal of a cardiac cycle, the unit radar echo signal and the unit cardiac tremor signal at each acquisition position according to the target ECG signal, the multiple target cardiac tremor signals and the multiple multi-point micro-motion radar echo signals; dividing the unit radar echo signal and the unit cardiac tremor signal at each acquisition position into a first sub-wave and a second sub-wave according to the time points corresponding to the R peak and the T peak in the unit ECG signal; and in the first sub-wave In the second sub-wave, the maximum value within the first preset time length before the first reference origin is determined as the mitral valve closing characteristic point, and the maximum value within the second preset time length after the first reference origin is determined as the aortic valve opening characteristic point, and the first reference origin is the minimum point in the first empirical time length after the R peak; in the second sub-wave, the maximum value within the third preset time length before the second reference origin is determined as the aortic valve closing characteristic point, and the minimum value within the fourth preset time length after the second reference origin is determined as the mitral valve opening characteristic point, and the second reference origin is the maximum point in the second empirical time length after the T peak.

[0083] In some embodiments, the multiple acquisition locations are: mitral valve area, pulmonary valve area, aortic valve area, aortic valve second area, and tricuspid valve area.

[0084] In some embodiments, the above-mentioned step S605 determines the target physiological state information reflecting the measured chest cavity based on multiple target cardiac tremor signals and array radar echo signals, including: determining the first physiological state information based on multiple target cardiac tremor signals; determining the second physiological state information based on the array radar echo signal, the first physiological state information and the second physiological state information being heart state information other than the target physiological state information.

[0085] In the embodiments of the present application, the technical effects of the above-mentioned cardiac surface multi-point joint acquisition and multi-modal fusion method embodiment can refer to the above-mentioned cardiac surface multi-point joint acquisition and multi-modal fusion system, and will not be repeated here to avoid repetition.

[0086] Please refer to Figure 7 , which shows a schematic diagram of the hardware structure of an electronic device provided by an exemplary embodiment of the present disclosure. In some examples, the electronic device may be at least one of a smart phone, a smart watch, a desktop computer, a laptop, a virtual reality terminal, an augmented reality terminal, a wireless terminal, and a laptop computer. The electronic device has a communication function and can access a wired network or a wireless network. The electronic device may generally refer to one of a plurality of terminals, and those skilled in the art may know that the number of the above terminals may be more or less. It can be understood that the electronic device undertakes the calculation and processing work of the technical solution of the present disclosure, and the present disclosure does not limit this.

[0087] like Figure 7 As shown, the electronic device in the present disclosure may include one or more of the following components: a processor 710 and a memory 720 .

[0088] Optionally, the processor 710 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 720, and calling data stored in the memory 720. Optionally, the processor 710 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 710 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a baseband chip. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content that needs to be displayed on the touch display; the NPU is used to implement artificial intelligence (AI) functions; and the baseband chip is used to process wireless communications. It is understandable that the above baseband chip may not be integrated into the processor 710, but may be implemented by a separate chip.

[0089] The memory 720 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 720 includes a non-transitory computer-readable storage medium. The memory 720 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above various method embodiments, etc.; the data storage area may store data created according to the use of the electronic device, etc.

[0090] In addition, those skilled in the art can understand that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the electronic device also includes a display screen, a camera assembly, a microphone, a speaker, a radio frequency circuit, an input unit, a sensor (such as an acceleration sensor, an angular velocity sensor, a light sensor, etc.), an audio circuit, a WiFi module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0091] The present disclosure also provides a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the method for joint acquisition of multiple points on the cardiac surface and multi-modal fusion as described in the above embodiments.

[0092] The present disclosure also provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes to implement the method for joint acquisition of multiple points on the heart surface and multi-modal fusion as described in the above-mentioned embodiments.

[0093] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the method for joint acquisition of multi-points on the cardiac surface and multimodal fusion, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0094] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0095] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices, servers and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0096] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0097] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0099] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present disclosure can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by a general or special-purpose computer.

[0100] It should be noted that the technical solutions described in the present disclosure can be combined arbitrarily without conflict.

[0101] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A cardiac surface multi-point joint acquisition and multi-modal fusion system, characterized in that: The cardiac surface multi-point joint acquisition and multi-modal fusion system comprises: a cardiac shock signal sensor, a first signal processing module connected to the cardiac shock signal sensor, a radar signal sensor, a second signal processing module connected to the radar signal sensor, and a data processing module connected to the first signal processing module and the second signal processing module; The cardiac tremor signal sensor is configured to collect multiple initial cardiac tremor signals at multiple collection positions of the chest cavity being tested, each collection position corresponding to an initial cardiac tremor signal; The first signal processing module is configured to pre-process the multiple initial cardiac shock signals to obtain multiple target cardiac shock signals; The radar signal sensor is configured to transmit a radar signal and collect an initial radar echo signal returned by the chest cavity under test; The second signal processing module is configured to pre-process the initial radar echo signal to obtain an array radar echo signal; The data processing module is configured to determine target physiological state information reflecting the measured chest cavity according to the multiple target cardiac shock signals and area array radar echo signals.

2. The cardiac surface multi-point joint acquisition and multi-modal fusion system according to claim 1, characterized in that: The radar signal sensor comprises: a radar antenna array, and the data processing module is specifically configured as follows: Determine a reference point where the center of the radar antenna array is mapped to the chest cavity being measured; Determine the multi-point micro-motion radar echo signal at each collection position from the array radar echo signal according to the distance between the reference point and each collection position and the radar transmission distance, wherein the radar transmission distance is the distance between the radar signal sensor and the chest cavity to be measured; According to the target cardiac tremor signal and the multi-point micro-motion radar echo signal at each acquisition position, target physiological state information reflecting the measured chest cavity is determined.

3. The cardiac surface multi-point joint acquisition and multi-modal fusion system according to claim 2, characterized in that: The cardiac surface multi-point joint acquisition and multi-modal fusion system further includes: an electrocardiogram signal sensor, and a third signal processing module connected to the electrocardiogram signal sensor; The electrocardiogram signal sensor is configured to collect an initial electrocardiogram signal of the chest cavity being tested; The third signal processing module is configured to pre-process the initial ECG signal to obtain a target ECG signal; The data processing module is specifically configured to determine target physiological state information reflecting the measured chest cavity based on the target electrocardiogram signal, the multiple target cardiac tremor signals and multiple multi-point micro-motion radar echo signals.

4. The cardiac surface multi-point joint acquisition and multi-modal fusion system according to claim 3, characterized in that: The data processing module is specifically configured to determine the aortic valve opening and closing feature points and the mitral valve opening and closing feature points at each acquisition position according to the target electrocardiogram signal, the multiple target cardiac tremor signals and the multiple multi-point micro-motion radar echo signals; The aortic valve opening and closing feature points and the mitral valve opening and closing feature points at each acquisition position are fused to obtain target physiological state information.

5. The system according to claim 4, characterized in that The data processing module is specifically configured as follows: Determine a unit ECG signal of a cardiac cycle, a unit radar echo signal at each acquisition position, and a unit cardiac tremor signal according to the target ECG signal, the multiple target cardiac tremor signals, and the multiple multi-point micro-motion radar echo signals; According to the time points corresponding to the R peak and the T peak in the unit electrocardiogram signal, the unit radar echo signal and the unit heart shock signal at each acquisition position are divided into a first sub-wave and a second sub-wave; In the first sub-wave, the maximum value within the first preset time length before the first reference origin is determined as the mitral valve closing feature point, and the maximum value within the second preset time length after the first reference origin is determined as the aortic valve opening feature point, and the first reference origin is the minimum value point within the first experience time length after the R peak; In the second sub-wave, the maximum value within the third preset time length before the second reference origin is determined as the aortic valve closing characteristic point, and the minimum value within the fourth preset time length after the second reference origin is determined as the mitral valve opening characteristic point. The second reference origin is the maximum value point within the second empirical time length range after the T peak.

6. The system according to any one of claims 1 to 5, characterized in that: The multiple acquisition positions are: mitral valve area, pulmonary valve area, aortic valve area, aortic valve second area, and tricuspid valve area.

7. The system according to any one of claims 1 to 5, characterized in that: The data processing module is specifically configured to determine first physiological state information according to the multiple target cardiac shock signals; The second physiological state information is determined according to the area array radar echo signal, wherein the first physiological state information and the second physiological state information are heart state information other than the target physiological state information.

8. A method for joint acquisition of multi-points on the cardiac surface and multi-modal fusion, characterized in that: The method comprises: A plurality of initial cardiac shock signals are acquired at a plurality of acquisition positions of the chest cavity to be tested, each acquisition position corresponding to an initial cardiac shock signal; Preprocessing the multiple initial cardiac arrest signals to obtain multiple target cardiac arrest signals; Transmitting a radar signal and collecting an initial radar echo signal returned by the chest cavity under test; Preprocessing the initial radar echo signal to obtain an array radar echo signal; According to the multiple target cardiac arrest signals and the area array radar echo signals, target physiological state information reflecting the measured chest cavity is determined.

9. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of the method for joint acquisition of multi-points on the cardiac surface and multi-modal fusion as described in claim 8 are implemented.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the method for joint acquisition of multiple points on the heart surface and multi-modal fusion as described in claim 8 are implemented.

Images