Triggering method and device of magnetic resonance imaging equipment based on multi-modal signal fusion, terminal equipment and storage medium

Abstract

The application relates to the technical field of biomedical engineering. The application discloses a triggering method and device of a magnetic resonance imaging equipment based on multi-modal signal fusion, a terminal equipment and a storage medium, which can improve the accuracy of triggering the magnetic resonance imaging equipment to collect images. The method comprises the following steps: acquiring a first morphological feature of a heart vibration signal of a target object, a second morphological feature of a jugular vein pulsation signal and a third morphological feature of a photoplethysmogram signal; constructing a heterogeneous graph based on the first morphological feature, the second morphological feature and the third morphological feature; performing global graph feature vector extraction processing on the heterogeneous graph by using a heterogeneous graph attention network to obtain a target global graph feature vector; performing time prediction processing on the target global graph feature vector by using a triggering control model to obtain a target time, and triggering the magnetic resonance imaging equipment to collect images when the target object is in a diastolic middle and late stage based on the target time.

Description

Technical Field

[0001] This application relates to the field of biomedical engineering technology. More specifically, this application relates to a triggering method, apparatus, terminal device, and storage medium for a magnetic resonance imaging device based on multimodal signal fusion. Background Technology

[0002] Traditional triggering methods for magnetic resonance imaging (MRI) devices typically involve acquiring the IPPG signal of the target object; determining the respiratory and pulse cycles based on the IPPG signal and calculating the ratio of the respiratory period to the pulse cycle; determining the respiratory time interval corresponding to each pulse cycle based on the ratio of the respiratory cycle to the pulse cycle; combining the target object's electrocardiogram (ECG) signal with a pre-trained regression model to determine the delay time of the IPPG signal peak relative to the ECG R-wave peak; calculating the composite gating time interval for MRI in each pulse cycle based on the respiratory time interval and the delay time; and generating a composite gating signal to trigger the MRI device for MRI based on this composite gating time interval. However, in this method, the ECG signal itself reflects an electroexcitation event, which has a physiological electromechanical delay with the mechanical contraction of the heart. This electromechanical delay directly affects the accuracy of the delay time of the IPPG signal peak relative to the ECG R-wave peak, and thus affects the accuracy of triggering MRI image acquisition. Therefore, existing technologies need improvement. Summary of the Invention

[0003] The purpose of this application is to provide a triggering method, apparatus, terminal device, and storage medium for a magnetic resonance imaging (MRI) device based on multimodal signal fusion, which can improve the accuracy of triggering the MRI device to acquire images. This application is mainly achieved through the following technical solutions:

[0004] A first aspect of this application provides a triggering method for a magnetic resonance imaging device based on multimodal signal fusion, comprising:

[0005] Acquire the first morphological features of the cardiac vibration signal, the second morphological features of the jugular venous pulsation signal, and the third morphological features of the photoplethysmography pulse wave signal of the target object;

[0006] A heterogeneous graph is constructed based on the first morphological feature, the second morphological feature, and the third morphological feature;

[0007] A heterogeneous graph attention network is used to extract global graph feature vectors from the heterogeneous graph to obtain the target global graph feature vectors.

[0008] A trigger control model is used to perform time prediction processing on the feature vector of the target global map to obtain the target time, and the magnetic resonance imaging device is triggered to acquire the image of the target object when the target object is in the mid-to-late diastolic state of the heart based on the target time.

[0009] According to one embodiment of this application, the steps of obtaining the first morphological features of the cardiac vibration signal, the second morphological features of the jugular venous pulsation signal, and the third morphological features of the photoplethysmography (PPG) signal of the target object include:

[0010] Acquire cardiac vibration signals, jugular venous pulsation signals, and photoplethysmography (PPG) signals of the target object;

[0011] The cardiac vibration signal is processed to extract the aortic valve opening and closing points to obtain multiple target opening points and multiple target closing points, and all target opening points and all target closing points are used as the first morphological feature;

[0012] The peak points of the a wave, c wave and v wave of the jugular vein pulsation signal are extracted to obtain multiple first target peak points corresponding to the a wave, multiple second target peak points corresponding to the c wave and multiple third target peak points corresponding to the v wave, and all first target peak points, all second target peak points and all third target peak points are used as the second morphological feature;

[0013] The photoplethysmography (PPG) signal is processed to extract the pulse wave origin, peak point, and diabetic notch point to obtain multiple target pulse wave origins, multiple fourth target peak points, and multiple target diabetic notches. All target pulse wave origins, all fourth target peak points, and all target diabetic notches are used as the third morphological feature.

[0014] According to one embodiment of this application, the steps for obtaining the cardiac vibration signal, jugular venous pulsation signal, and photoplethysmography (PPG) wave signal of the target object include:

[0015] Acquire the raw cardiac vibration signal, raw jugular vein pulsation signal, and raw photoplethysmography pulse wave signal of the target object;

[0016] The original cardiac vibration signal is filtered to obtain a first filtered signal; the original jugular vein pulsation signal is filtered to obtain a second filtered signal; and the original photoplethysmography (PPG) pulse wave signal is filtered to obtain a third filtered signal.

[0017] The first filtered signal is detrended to obtain a first detrended signal; the second filtered signal is detrended to obtain a second detrended signal; and the third filtered signal is detrended to obtain a third detrended signal.

[0018] The first detrended signal is normalized to obtain the cardiac vibration signal of the target object; the second detrended signal is normalized to obtain the jugular vein pulsation signal of the target object; and the third detrended signal is normalized to obtain the photoplethysmography (PPG) signal of the target object.

[0019] According to one embodiment of this application, the step of constructing a heterogeneous graph based on the first morphological feature, the second morphological feature, and the third morphological feature includes:

[0020] The feature points that have a direct physiological causal relationship among the first morphological feature, the second morphological feature, and the third morphological feature are connected to obtain the first image to be processed;

[0021] Based on the first image to be processed, all feature points in the first morphological feature are connected in chronological order, all feature points in the second morphological feature are connected in chronological order, and all feature points in the third morphological feature are connected in chronological order to obtain the second image to be processed.

[0022] Based on the second image to be processed, all similar feature points in the first morphological feature are connected, all similar feature points in the second morphological feature are connected, and all similar feature points in the third morphological feature are connected to obtain the heterogeneous image.

[0023] According to one embodiment of this application, each feature point of the first morphological feature, each feature point of the second morphological feature, and each feature point of the third morphological feature contain temporal features, morphological features, and modal labels.

[0024] According to one embodiment of this application, the step of using a heterogeneous graph attention network to extract global graph feature vectors from the heterogeneous graph to obtain target global graph feature vectors includes:

[0025] The encoding network of the heterogeneous graph attention network is used to map each point in the heterogeneous graph to the latent space to obtain the first fusion feature at the node level;

[0026] The attention mechanism of the heterogeneous graph attention network is used to transfer information along different types of edges in the heterogeneous graph to obtain a second fusion feature at the relation level;

[0027] The weighted pooling module of the heterogeneous graph attention network is used to perform global graph feature vector extraction processing on the first fused feature and the second fused feature to obtain the target global graph feature vector.

[0028] According to one embodiment of this application, when the trigger control model is a fully connected network, the step of using the trigger control model to perform time prediction processing on the target global graph feature vector to obtain the target time includes:

[0029] The first fully connected layer of the fully connected network is used to perform a linear transformation on the feature vector of the target global graph to obtain the first vector to be processed.

[0030] The first vector to be processed is subjected to a nonlinear transformation using the first activation function of the fully connected network to obtain the second vector to be processed.

[0031] The second fully connected layer of the fully connected network is used to perform a linear transformation on the second vector to be processed to obtain the third vector to be processed.

[0032] The third vector to be processed is subjected to a nonlinear transformation using the second activation function of the fully connected network to obtain the fourth vector to be processed.

[0033] The time prediction algorithm of the fully connected network is used to perform time prediction processing on the fourth vector to be processed to obtain the target time.

[0034] A second aspect of this application provides a triggering device for a magnetic resonance imaging (MRI) device based on multimodal signal fusion, comprising:

[0035] The morphological feature acquisition module is used to acquire the first morphological features of the cardiac vibration signal, the second morphological features of the jugular venous pulsation signal, and the third morphological features of the photoplethysmography pulse wave signal of the target object.

[0036] A heterogeneous graph construction module is used to construct a heterogeneous graph based on the first morphological feature, the second morphological feature, and the third morphological feature;

[0037] The extraction module is used to perform global graph feature vector extraction processing on the heterogeneous graph using a heterogeneous graph attention network to obtain the target global graph feature vector;

[0038] The prediction module is used to perform time prediction processing on the feature vector of the target global map using a trigger control model to obtain the target time, and trigger the magnetic resonance imaging device to acquire the image of the target object when the target object is in the mid-to-late diastolic state of the heart based on the target time.

[0039] A third aspect of this application provides a terminal device, including a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to execute the steps of the triggering method for a magnetic resonance imaging device based on multimodal signal fusion provided in the first aspect of this application.

[0040] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that causes a computer to execute the steps of the triggering method for a magnetic resonance imaging device based on multimodal signal fusion provided in the first aspect of this application.

[0041] The beneficial effects of the embodiments of this application include:

[0042] This application embodiment acquires the first morphological features of the target object's cardiac vibration signal, the second morphological features of the jugular venous pulsation signal, and the third morphological features of the photoplethysmography (PPG) pulse wave signal. Based on these three morphological features, a heterogeneous graph is constructed. A heterogeneous graph attention network is used to extract global graph feature vectors from the heterogeneous graph to obtain the target global graph feature vector. A trigger control model is then used to perform time prediction processing on the target global graph feature vector to obtain the target time. Based on this target time, the magnetic resonance imaging (MRI) device is triggered to acquire images of the target object when it is in the mid-to-late diastolic phase of the heart. Compared with existing technologies, this application embodiment uses cardiac vibration signals and jugular venous pulsation signals instead of traditional electrocardiogram (ECG) signals, thereby avoiding the physiological electromechanical delay between the ECG signal and cardiac mechanical contraction. This improves the accuracy of triggering the MRI device to acquire images. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 The flowcharts for some embodiments of the triggering method of the magnetic resonance imaging device based on multimodal signal fusion of this application are shown below;

[0045] Figure 2 This is a block diagram illustrating the principle of the triggering device for a magnetic resonance imaging device based on multimodal signal fusion according to this application in some embodiments;

[0046] Figure 3 This is a schematic block diagram of the terminal device of this application in some embodiments. Detailed Implementation

[0047] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0048] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0049] The terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0050] The terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or that are inherent to such process, method, product, or apparatus.

[0051] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items.

[0052] The specific embodiments of this application will be further described below with reference to the accompanying drawings.

[0053] refer to Figure 1 The diagram shown is a flowchart of a triggering method for a magnetic resonance imaging device based on multimodal signal fusion, provided in the first aspect of an embodiment of this application. Figure 1 The triggering method for the magnetic resonance imaging device based on multimodal signal fusion includes:

[0054] S1. Acquire the first morphological features of the cardiac vibration signal, the second morphological features of the jugular venous pulsation signal, and the third morphological features of the photoplethysmography (PPG) pulse wave signal of the target object.

[0055] Further, step S1 includes: acquiring the cardiac vibration signal, jugular venous pulsation signal, and photoplethysmography (PPG) wave signal of the target object; extracting the aortic valve opening and closing points from the cardiac vibration signal to obtain multiple target opening points and multiple target closing points, and using all target opening points and all target closing points as the first morphological feature; extracting the peak points of the a, c, and v waves from the jugular venous pulsation signal to obtain multiple first target peak points corresponding to the a wave, multiple second target peak points corresponding to the c wave, and multiple third target peak points corresponding to the v wave, and using all first target peak points, all second target peak points, and all third target peak points as the second morphological feature; and extracting the pulse wave origin, peak point, and dicrotic notch from the PPG wave signal to obtain multiple target pulse wave origins, multiple fourth target peak points, and multiple target dicrotic notches, and using all target pulse wave origins, all fourth target peak points, and all target dicrotic notches as the third morphological feature.

[0056] Further, the steps of acquiring the cardiac vibration signal, jugular venous pulsation signal, and photoplethysmography (PPG) wave signal of the target object include: acquiring the original cardiac vibration signal, jugular venous pulsation signal, and PPG wave signal of the target object; filtering the original cardiac vibration signal to obtain a first filtered signal, filtering the original jugular venous pulsation signal to obtain a second filtered signal, and filtering the original PPG wave signal to obtain a third filtered signal; detrending the first filtered signal to obtain a first detrended signal, detrending the second filtered signal to obtain a second detrended signal, and detrending the third filtered signal to obtain a third detrended signal; normalizing the first detrended signal to obtain the cardiac vibration signal of the target object, normalizing the second detrended signal to obtain the jugular venous pulsation signal of the target object, and normalizing the third detrended signal to obtain the PPG wave signal of the target object.

[0057] Furthermore, the steps of acquiring the original cardiac vibration signal, original jugular vein pulsation signal, and original photoplethysmography (PPG) signal of the target object can be achieved using existing technologies.

[0058] In some embodiments, the acquisition of the raw cardiac vibration signal is based on the principle of optical interference. Heartbeats induce minute movements in the chest cavity, which in turn cause changes in the interference field of the reflected laser light on the chest surface. This change manifests as speckle shift on the camera's imaging plane. In this embodiment, the raw cardiac vibration signal can be extracted by capturing these speckle shift changes using a first defocus camera. The extraction process first requires precise adjustment of the configuration of the first defocus camera and the laser system. The first defocus camera is precisely positioned at the lower left fourth rib of the target object's chest cavity, and this area is illuminated by the laser. To improve the detection accuracy of the raw cardiac vibration signal, in this embodiment, the distance L1 between the focal plane of the first defocus camera and the chest cavity plane of the target object is greater than the distance L2 between the lens of the first defocus camera and the focal plane. This defocus configuration significantly amplifies the laser speckle image caused by cardiac vibration, allowing minute cardiac movements to be clearly presented. The ratio of L1 to L2 can be used to represent the degree of defocus of the first defocus camera; a higher degree of defocus corresponds to a more significant signal amplification effect. This configuration enables the first defocus camera to continuously acquire speckle images at a frame rate of at least 200 fps (Frames Per Second) and a resolution of 400×300, providing high-quality data for subsequent signal processing. After image acquisition, optical flow is used to perform in-depth analysis of the captured speckle images, accurately calculating the motion amplitude of each speckle in the X and Y axes. Then, the motion amplitude sequence of all speckles is rationally divided using a sliding window method, and the main motion angle θ of each segment is calculated. This motion angle represents the direction of cardiac vibration and can be further used to synthesize the final original cardiac vibration signal. This original cardiac vibration signal accurately reflects the periodic motion of the heart, becoming crucial for triggering synchronous imaging by the magnetic resonance imaging device.

[0059] The acquisition of the raw jugular pulsation signal is also based on the principle of optical interference, but the raw jugular pulsation signal is targeted at the neck region of the target object. Jugular pulsation causes minute movements of the neck skin, which are amplified and detected by defocus speckle imaging. In this embodiment, by precisely adjusting the configuration of the second defocus camera and the laser system, the jugular pulsation waveform reflecting right ventricular function can be captured, including its characteristic a, c, and v waves and the descending x and y branches. Extraction of the raw jugular pulsation signal requires precisely aligning the second defocus camera and the laser system with the neck region, typically the area along the right internal jugular vein. Similar to the raw cardiac vibration signal, the distance L3 between the focal plane of the second defocus camera and the neck region of the target object is greater than the distance L4 between the lens of the second defocus camera and the focal plane. This embodiment uses a defocus setting of L3>L4 to amplify the minute skin movements caused by jugular pulsation. When the laser irradiates the neck skin, the resulting speckle pattern shifts with the pulsation of the jugular vein. This application captures these speckle movements using a second defocused camera (frame rate ≥ 200 fps) and calculates the displacement (dx, dy) of the speckle in a two-dimensional plane using the optical flow method, where dx is the coordinate of the speckle on the x-axis and dy is the coordinate of the speckle on the y-axis. Since the skin movement caused by jugular venous pulsation has a specific directionality (related to the course of the vein and the direction of pulsation propagation), it is necessary to further separate the radial component (dr) through polar coordinate transformation. This component mainly reflects the expansion and contraction of the vessel wall, i.e., the pure, original jugular venous pulsation signal.

[0060] The acquisition of the raw photoplethysmography (PPG) signal is based on the principle of optical reflection. A light source illuminates the facial area of ​​the target object, and changes in the reflected light are detected to extract the raw PPG signal. When the light source illuminates the facial area, some light is absorbed by the skin tissue, while some is reflected back to the third defocus camera. Changes in light absorption caused by blood flow are detected through changes in the intensity of the reflected light. By extracting the alternating current (AC) signal of the reflected light, the characteristics of the raw PPG signal can be obtained. The extraction of the raw PPG signal first requires precise adjustment of the third defocus camera's position to clearly capture the facial area of ​​the target object. Therefore, the third defocus camera must be precisely positioned to ensure it covers key facial areas, such as the forehead, cheeks, or chin, as these areas effectively reflect changes in blood flow. The frame rate for acquiring the facial area is generally set to no less than 200fps to ensure efficient and accurate image acquisition and avoid signal loss or distortion caused by low frame rates. After acquiring facial region images, the first step is to accurately locate the skin region, extracting only the skin pixels from the image. The accuracy of this step directly affects the quality of subsequent signal extraction, therefore efficient image processing techniques must be employed. After successfully extracting the skin region, the G-channel image sequence, exhibiting stronger pulsatility, is analyzed to extract the final raw photoplethysmography (PPG) signal. This signal reflects changes in blood flow and is closely related to cardiac activity.

[0061] Further, the step of filtering the original cardiac vibration signal to obtain a first filtered signal includes: performing a Fourier transform on the original cardiac vibration signal to convert it to the frequency domain to obtain a first frequency domain signal; multiplying the first frequency domain signal with the frequency response of a bandpass filter and filtering out components other than the first target frequency to obtain a first signal to be processed; and performing an inverse Fourier transform on the first signal to be processed to obtain the first filtered signal.

[0062] Further, the step of filtering the original jugular vein pulsation signal to obtain a second filtered signal includes: performing a Fourier transform on the original jugular vein pulsation signal to convert it to the frequency domain to obtain a second frequency domain signal; multiplying the second frequency domain signal with the frequency response of the bandpass filter and filtering out components other than the second target frequency to obtain a second signal to be processed; and performing an inverse Fourier transform on the second signal to be processed to obtain the second filtered signal.

[0063] Further, the step of filtering the original photoplethysmography (PPG) signal to obtain the third filtered signal includes: performing a Fourier transform on the original PPG signal to convert it to the frequency domain to obtain a third frequency domain signal; multiplying the third frequency domain signal with the frequency response of the bandpass filter and filtering out components other than the third target frequency to obtain a third signal to be processed; and performing an inverse Fourier transform on the third signal to be processed to obtain the third filtered signal.

[0064] The first target frequency, the second target frequency, and the third target frequency may be the same or different, and can be set by those skilled in the art according to actual needs.

[0065] In other embodiments, the bandpass frequency ranges of the bandpass filters used for the original cardiac vibration signal, the original jugular vein pulsation signal, and the original photoplethysmography pulse wave signal may be different, and can be specifically set by those skilled in the art according to actual needs.

[0066] Furthermore, the detrending processing method described in this application embodiment involves fitting a linear or nonlinear function to represent the trend components and subtracting these trend components from the original signals (i.e., the original cardiac vibration signal, the original jugular venous pulsation signal, and the original photoplethysmography pulse wave signal). The purpose of detrending is to remove irrelevant components and retain only the parts relevant to the target signal.

[0067] Furthermore, normalization helps eliminate amplitude differences between different signals.

[0068] Furthermore, the cardiac vibration signal comprises multiple cycles, each cycle having a corresponding target open point and a target closed point.

[0069] Furthermore, the A wave refers to the first positive wave in one cycle of the jugular venous pulsation signal, caused by right atrial contraction in the target subject. The C wave refers to the second positive wave in one cycle of the jugular venous pulsation signal, caused by the compression of the right atrium during ventricular systole (or early ventricular systole with tricuspid valve bulging), and occurs after the A wave. The V wave refers to the third positive wave in one cycle of the jugular venous pulsation signal, caused by increased pressure due to peripheral venous blood return filling the right atrium at the end of ventricular systole and early diastole in the target subject.

[0070] Furthermore, the photoplethysmography signal contains multiple cycles, each cycle having a corresponding target pulse wave initiation point, a fourth target peak point, and a target dicrotic notch point.

[0071] The target pulse wave origin, the fourth target peak point, and the target dicrotic notch are all used to reflect the compliance of peripheral blood vessels and the blood flow wave reflection.

[0072] S2. Construct a heterogeneous graph based on the first morphological feature, the second morphological feature, and the third morphological feature.

[0073] Further, step S2 includes: connecting feature points with direct physiological causal relationships among the first morphological feature, the second morphological feature, and the third morphological feature to obtain a first image to be processed; based on the first image to be processed, connecting all feature points in the first morphological feature in chronological order, connecting all feature points in the second morphological feature in chronological order, and connecting all feature points in the third morphological feature in chronological order to obtain a second image to be processed; based on the second image to be processed, connecting all similar feature points in the first morphological feature, connecting all similar feature points in the second morphological feature, and connecting all similar feature points in the third morphological feature to obtain the heterogeneous image.

[0074] Furthermore, the feature points with a direct physiological causal relationship can be the target open point and the second target peak point in the same time period, as well as the target open point and the target pulse wave origin in the same time period.

[0075] Further, the step of connecting all feature points in the first morphological feature in a temporal sequence includes: connecting the target open point and the target closed point belonging to the first cycle of the cardiac vibration signal; then connecting the target open point belonging to the second cycle of the cardiac vibration signal and the target closed point of the first cycle of the cardiac vibration signal; then connecting the target open point and the target closed point of the second cycle of the cardiac vibration signal; then connecting the target open point belonging to the third cycle of the cardiac vibration signal and the target closed point of the second cycle of the cardiac vibration signal, and so on, until the target closed point of the last cycle of the cardiac vibration signal is connected, forming an event chain corresponding to the cardiac vibration signal.

[0076] Further, the step of sequentially connecting all feature points in the second morphological feature includes: connecting the first target peak point, the second target peak point, and the third target peak point belonging to the first cycle of the jugular pulsation signal; then connecting the first target peak point belonging to the second cycle of the jugular pulsation signal with the third target peak point belonging to the first cycle of the jugular pulsation signal; then connecting the first target peak point, the second target peak point, and the third target peak point belonging to the second cycle of the jugular pulsation signal; then connecting the first target peak point belonging to the third cycle of the jugular pulsation signal with the third target peak point belonging to the second cycle of the jugular pulsation signal, and so on, until the third target peak point of the last cycle of the jugular pulsation signal is connected, forming an event chain corresponding to the jugular pulsation signal.

[0077] Furthermore, the process of connecting all feature points in the third morphological feature in chronological order can refer to the process of connecting all feature points in the second morphological feature in chronological order.

[0078] Furthermore, the step of connecting all similar feature points in the first morphological feature includes: connecting the target open points belonging to the first cycle of the cardiac vibration signal with the target open points belonging to the second cycle of the cardiac vibration signal, connecting the target open points belonging to the second cycle of the cardiac vibration signal with the target open points belonging to the third cycle of the cardiac vibration signal, connecting the target open points belonging to the third cycle of the cardiac vibration signal with the target open points belonging to the fourth cycle of the cardiac vibration signal, and so on, until all target open points are connected; then, connecting the target closed points belonging to the first cycle of the cardiac vibration signal with the target closed points belonging to the second cycle of the cardiac vibration signal, connecting the target closed points belonging to the second cycle of the cardiac vibration signal with the target closed points belonging to the third cycle of the cardiac vibration signal, connecting the target closed points belonging to the third cycle of the cardiac vibration signal with the target closed points belonging to the fourth cycle of the cardiac vibration signal, and so on, until all target closed points are connected. These connection steps can characterize rhythm continuity.

[0079] The implementation methods for connecting all similar feature points in the second morphological feature and for connecting all similar feature points in the third morphological feature can refer to the implementation methods for connecting all similar feature points in the first morphological feature.

[0080] Furthermore, each feature point of the first morphological feature, each feature point of the second morphological feature, and each feature point of the third morphological feature contains temporal features, morphological features, and modal labels.

[0081] The temporal features include the relative time position of the current feature point within the period and the time interval between the current feature point and the previous similar event.

[0082] The morphological features include the amplitude, slope, and curvature of the current feature point.

[0083] The modal label includes the signal identifier to which the current feature point belongs. For example, if the current feature point is one of the feature points of the first morphological feature, then the modal label corresponding to the current feature point is SCG; if the current feature point is one of the feature points of the second morphological feature, then the modal label corresponding to the current feature point is JVP; if the current feature point is one of the feature points of the third morphological feature, then the modal label corresponding to the current feature point is PPG.

[0084] S3. Use a heterogeneous graph attention network to extract global graph feature vectors from the heterogeneous graph to obtain the target global graph feature vector.

[0085] Step S3 includes: using the encoding network of the heterogeneous graph attention network to map each point in the heterogeneous graph to the latent space to obtain a first fusion feature at the node level; using the attention mechanism of the heterogeneous graph attention network to pass information along different types of edges in the heterogeneous graph to obtain a second fusion feature at the relation level; and using the weighted pooling module of the heterogeneous graph attention network to perform global graph feature vector extraction processing on the first fusion feature and the second fusion feature to obtain the target global graph feature vector.

[0086] Furthermore, the latent space is a high-dimensional abstract representation space of graph nodes after model learning. Its essence is to automatically mine the potential features of graph nodes and relationships through neural networks, compressing the complex heterogeneous graph structure into a low-dimensional vector representation, thereby supporting efficient downstream task processing.

[0087] Furthermore, information transmission enables the embodiments of this application to adaptively learn the importance weights of different relationships. For example, in the same time period, the "edge formed by connecting the target open point and the target pulse wave origin" may be more informative than the "edge formed by connecting the first target peak point and the target pulse wave origin" at a specific heart rate.

[0088] Furthermore, the target global graph feature vector is used to represent the overall compressed cardiac cycle state and the coordination relationship between events.

[0089] Furthermore, the step of using the weighted pooling module of the heterogeneous graph attention network to perform global graph feature vector extraction processing on the first fused feature and the second fused feature to obtain the target global graph feature vector can be understood as using the weighted pooling module of the heterogeneous graph attention network to perform weighted pooling operation and aggregation processing on the first fused feature and the second fused feature to obtain the target global graph feature vector.

[0090] S4. The target global map feature vector is processed by a trigger control model to obtain the target time, and the magnetic resonance imaging device is triggered to acquire the image of the target object when the target object is in the mid-to-late diastolic state of the heart based on the target time.

[0091] Furthermore, the trigger control model is a fully connected network. In other embodiments, the trigger control model can also be other networks, which can be specifically set by those skilled in the art according to actual needs.

[0092] When the trigger control model is a fully connected network, the step of using the trigger control model to perform time prediction processing on the target global graph feature vector to obtain the target time includes: using the first fully connected layer of the fully connected network to perform linear transformation processing on the target global graph feature vector to obtain a first vector to be processed; using the first activation function of the fully connected network to perform nonlinear transformation processing on the first vector to be processed to obtain a second vector to be processed; using the second fully connected layer of the fully connected network to perform linear transformation processing on the second vector to be processed to obtain a third vector to be processed; using the second activation function of the fully connected network to perform nonlinear transformation processing on the third vector to be processed to obtain a fourth vector to be processed; and using the time prediction algorithm of the fully connected network to perform time prediction processing on the fourth vector to be processed to obtain the target time.

[0093] In other embodiments, the number of fully connected layers and the number of activation functions in the fully connected network can be set by those skilled in the art according to actual needs.

[0094] Both the first activation function and the second activation function can be ReLU (Rectified Linear Unit) activation functions. In other embodiments, the first activation function and the second activation function can be set by those skilled in the art according to actual needs.

[0095] Furthermore, the step of using the time prediction algorithm of the fully connected network to perform time prediction processing on the fourth vector to be processed and obtaining the target time includes: performing mode and median calculation processing on the fourth vector to be processed to obtain the target mode and target median; performing average calculation processing on the target mode and the target median to obtain the target average value, and using the target average value as the target time.

[0096] In other embodiments, the step of using the time prediction algorithm of the fully connected network to perform time prediction processing on the fourth vector to be processed and obtaining the target time can also be implemented in other ways, which can be set by those skilled in the art according to actual needs.

[0097] The target time is the optimal moment for the magnetic resonance imaging (MRI) device to acquire images.

[0098] Through the above technical solution, the embodiments of this application use cardiac vibration signals and jugular vein pulsation signals to replace traditional electrocardiogram signals, thereby avoiding the phenomenon of physiological electromechanical delay between electrocardiogram signals and cardiac mechanical contraction, which enables the embodiments of this application to improve the accuracy of triggering magnetic resonance imaging equipment to acquire images.

[0099] Furthermore, the heterogeneous graph in this embodiment possesses inherent adaptive robustness: if any graph node in the heterogeneous graph experiences a missed detection or false detection due to poor signal quality, the attention mechanism can automatically weaken the weight of that graph node in information propagation, making the system decision more reliant on high-confidence nodes and reliable relationships, thus ensuring trigger stability; at the same time, the system can dynamically adjust the graph structure of the heterogeneous graph in real time based on the detection confidence (e.g., removing low-confidence nodes and related edges) to achieve online adaptive optimization, thereby maintaining high-precision and robust trigger performance under different signal quality conditions.

[0100] In some embodiments, after step S1, the triggering method of the magnetic resonance imaging device based on multimodal signal fusion further includes: performing time-domain feature extraction processing on the cardiac vibration signal to obtain a first time-domain feature; performing time-domain feature extraction processing on the jugular vein pulsation signal to obtain a second time-domain feature; performing time-domain feature extraction processing on the photoplethysmography (PPG) wave signal to obtain a third time-domain feature; performing frequency-domain feature extraction processing on the cardiac vibration signal to obtain a first frequency-domain feature; performing frequency-domain feature extraction processing on the jugular vein pulsation signal to obtain a second frequency-domain feature; performing frequency-domain feature extraction processing on the PPG wave signal to obtain a third frequency-domain feature; performing phase synchronization feature extraction processing on the cardiac vibration signal, the jugular vein pulsation signal, and the PPG wave signal to obtain a target phase synchronization feature; and constructing the heterogeneous graph by combining the first time-domain feature, the second time-domain feature, the third time-domain feature, the first frequency-domain feature, the second frequency-domain feature, the third frequency-domain feature, and the target phase synchronization feature with the first morphological feature, the second morphological feature, and the third morphological feature.

[0101] Furthermore, the first time-domain feature includes the peak amplitude of each cycle in the cardiac vibration signal, the peak interval of each cycle in the cardiac vibration signal (e.g., the RR interval, i.e., the time interval between two adjacent R waves in the cardiac vibration signal), the waveform area of ​​each cycle in the cardiac vibration signal, and the rising slope and / or falling slope of each cycle in the cardiac vibration signal. The first time-domain feature directly reflects the amplitude, time interval, and morphological changes of the cardiac vibration signal, and is closely related to physiological parameters such as cardiac contractility, vascular tension, and blood flow velocity. Each peak amplitude, each peak interval, each waveform area, and each rising slope and / or falling slope in the first time-domain feature is a feature point of the first time-domain feature. Similarly, each element in the second time-domain feature, the third time-domain feature, the first frequency domain feature, the second frequency domain feature, the third frequency domain feature, and the target phase synchronization feature is also a feature point of itself.

[0102] Each feature point can be understood as an event point, and each event point is a key physiological event within each cardiac cycle.

[0103] The second time-domain features include the peak amplitude of each cycle of the jugular venous pulsation signal, the peak interval of each cycle of the jugular venous pulsation signal, the waveform area of ​​each cycle of the jugular venous pulsation signal, and the rising slope and / or falling slope of each cycle of the jugular venous pulsation signal.

[0104] The third time-domain feature includes the peak amplitude of each cycle of the photoplethysmography (PPG) signal, the peak interval of each cycle of the PPG signal, the waveform area of ​​each cycle of the PPG signal, and the rising slope and / or falling slope of each cycle of the PPG signal.

[0105] The first frequency domain feature includes the dominant frequency of each cycle in the cardiac vibration signal, the high-frequency component of each cycle in the cardiac vibration signal, the low-frequency component of each cycle in the cardiac vibration signal, and the energy ratio of each cycle in the cardiac vibration signal. The energy ratio of each cycle in the cardiac vibration signal is obtained by dividing the low-frequency component of each cycle in the cardiac vibration signal by the high-frequency component of each cycle in the cardiac vibration signal.

[0106] The second frequency domain feature includes the dominant frequency of each cycle of the jugular pulsation signal, the high-frequency component of each cycle of the jugular pulsation signal, the low-frequency component of each cycle of the jugular pulsation signal, and the energy ratio of each cycle of the jugular pulsation signal. The energy ratio of each cycle of the jugular pulsation signal is obtained by dividing the low-frequency component of each cycle of the jugular pulsation signal by the high-frequency component of each cycle of the jugular pulsation signal.

[0107] The third frequency domain feature includes the dominant frequency of each cycle of the photoplethysmography (PPG) signal, the high-frequency component of each cycle of the PPG signal, the low-frequency component of each cycle of the PPG signal, and the energy ratio of each cycle of the PPG signal. The energy ratio of each cycle of the PPG signal is obtained by dividing the low-frequency component of each cycle of the PPG signal by the high-frequency component of each cycle of the PPG signal.

[0108] The first frequency domain feature, the second frequency domain feature, and the third frequency domain feature reveal the rhythmicity and periodicity of cardiovascular system regulatory activities, such as the balance between the sympathetic and parasympathetic nervous systems.

[0109] The target phase synchronization feature is the time difference between key feature points of the cardiac vibration signal, the jugular venous pulsation signal, and the photoplethysmography (PPG) wave signal. For example, within the same time period, it is the time difference between the target opening point of the cardiac vibration signal, the second target peak point corresponding to the C-wave of the jugular venous pulsation signal, and the target pulse wave initiation point of the PPG wave signal. This time difference can quantify the phase relationship between cardiac electrical activity, mechanical contraction, and peripheral blood flow propagation. The target phase synchronization feature is a core physiological indicator for assessing cardiac function, vascular status, and overall cardiovascular system coordination, and its stability is far superior to absolute time points. The application of the target phase synchronization feature has better robustness to motion artifacts and individual differences.

[0110] Further, the step of constructing the heterogeneous graph by combining the first time-domain feature, the second time-domain feature, the third time-domain feature, the first frequency-domain feature, the second frequency-domain feature, the third frequency-domain feature, and the target phase synchronization feature with the first morphological feature, the second morphological feature, and the third morphological feature includes:

[0111] The feature points with direct physiological causal relationships among the first time-domain feature, the second time-domain feature, the third time-domain feature, the first frequency-domain feature, the second frequency-domain feature, the third frequency-domain feature, the target phase synchronization feature, the first morphological feature, the second morphological feature, and the third morphological feature are connected to obtain a third image to be processed. Based on the third image to be processed, all feature points in the first time-domain feature are connected in chronological order, as are all feature points in the second time-domain feature, the third time-domain feature, the first frequency-domain feature, the second frequency-domain feature, the third frequency-domain feature, the target phase synchronization feature, and the first morphological feature. The process involves performing connection processing on all feature points in the second morphological feature and the third morphological feature in a time-sequential manner to obtain a fourth image to be processed. Based on this fourth image, the process then proceeds by connecting all similar feature points in the first time-domain feature, the second time-domain feature, the third time-domain feature, the first frequency-domain feature, the second frequency-domain feature, the third frequency-domain feature, and the target phase synchronization feature, again connecting all similar feature points in the first, second, and third morphological features to obtain the heterogeneous image.

[0112] refer to Figure 2 The diagram shown is a schematic block diagram of a triggering device for a magnetic resonance imaging (MRI) device based on multimodal signal fusion, provided in the second aspect of an embodiment of this application. Figure 2 In the magnetic resonance imaging device based on multimodal signal fusion, the triggering device 100 includes:

[0113] The morphological feature acquisition module 101 is used to acquire the first morphological features of the cardiac vibration signal, the second morphological features of the jugular vein pulsation signal, and the third morphological features of the photoplethysmography pulse wave signal of the target object.

[0114] Heterogeneous graph construction module 102 is used to construct a heterogeneous graph based on the first morphological feature, the second morphological feature and the third morphological feature;

[0115] Extraction module 103 is used to perform global graph feature vector extraction processing on the heterogeneous graph using a heterogeneous graph attention network to obtain the target global graph feature vector;

[0116] The prediction module 104 is used to perform time prediction processing on the feature vector of the target global map using a trigger control model to obtain the target time, and trigger the magnetic resonance imaging device to acquire the image of the target object when the target object is in the mid-to-late diastolic state of the heart based on the target time.

[0117] A third aspect of this application provides a terminal device, the schematic diagram of which is as follows: Figure 3 As shown. The terminal device includes a processor, memory, network interface, display screen, and temperature sensor connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the terminal device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a triggering method for a magnetic resonance imaging device based on multimodal signal fusion. The display screen can be a liquid crystal display screen or an e-ink display screen, and the temperature sensor is pre-installed inside the terminal device to detect the operating temperature of the internal components.

[0118] Those skilled in the art will understand that Figure 3 The schematic diagram shown is only a partial structural diagram related to the present invention and does not constitute a limitation on the terminal device to which the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0119] In some embodiments, this application provides a terminal device, which includes a processor and a memory for storing computer programs. The processor is used to call and run the computer programs stored in the memory to execute the steps of the triggering method for a magnetic resonance imaging device based on multimodal signal fusion provided in the first aspect of this application.

[0120] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that causes a computer to execute the steps of the triggering method for a magnetic resonance imaging device based on multimodal signal fusion provided in the first aspect of this application.

[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0122] The technical features of the above embodiments can be combined without changing the basic principles of this application. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the patent protection scope of this application should be determined by the appended claims.

Claims

1. A triggering method for a magnetic resonance imaging device based on multimodal signal fusion, characterized in that, include: Acquire the first morphological features of the cardiac vibration signal, the second morphological features of the jugular venous pulsation signal, and the third morphological features of the photoplethysmography pulse wave signal of the target object; A heterogeneous graph is constructed based on the first morphological feature, the second morphological feature, and the third morphological feature; A heterogeneous graph attention network is used to extract global graph feature vectors from the heterogeneous graph to obtain the target global graph feature vectors. A trigger control model is used to perform time prediction processing on the feature vector of the target global map to obtain the target time, and the magnetic resonance imaging device is triggered to acquire the image of the target object when the target object is in the mid-to-late diastolic state of the heart based on the target time; The steps of using a heterogeneous graph attention network to extract global graph feature vectors from the heterogeneous graph and obtaining target global graph feature vectors include: using the encoding network of the heterogeneous graph attention network to map each point in the heterogeneous graph to the latent space to obtain the first fusion feature at the node level; The attention mechanism of the heterogeneous graph attention network is used to transmit information along different types of edges in the heterogeneous graph to obtain a second fusion feature at the relation level; the weighted pooling module of the heterogeneous graph attention network is used to perform global graph feature vector extraction processing on the first fusion feature and the second fusion feature to obtain the target global graph feature vector.

2. The triggering method for a magnetic resonance imaging device based on multimodal signal fusion according to claim 1, characterized in that, The steps for acquiring the first morphological features of the cardiac vibration signal, the second morphological features of the jugular venous pulsation signal, and the third morphological features of the photoplethysmography (PPG) pulse wave signal of the target object include: Acquire cardiac vibration signals, jugular venous pulsation signals, and photoplethysmography (PPG) signals of the target object; The cardiac vibration signal is processed to extract the aortic valve opening and closing points to obtain multiple target opening points and multiple target closing points, and all target opening points and all target closing points are used as the first morphological feature; The peak points of the a wave, c wave and v wave of the jugular vein pulsation signal are extracted to obtain multiple first target peak points corresponding to the a wave, multiple second target peak points corresponding to the c wave and multiple third target peak points corresponding to the v wave, and all first target peak points, all second target peak points and all third target peak points are used as the second morphological feature; The photoplethysmography (PPG) signal is processed to extract the pulse wave origin, peak point, and diabetic notch point to obtain multiple target pulse wave origins, multiple fourth target peak points, and multiple target diabetic notches. All target pulse wave origins, all fourth target peak points, and all target diabetic notches are used as the third morphological feature.

3. The triggering method for a magnetic resonance imaging device based on multimodal signal fusion according to claim 2, characterized in that, The steps for acquiring the cardiac vibration signal, jugular venous pulsation signal, and photoplethysmography (PPG) signal of the target object include: Acquire the raw cardiac vibration signal, raw jugular vein pulsation signal, and raw photoplethysmography pulse wave signal of the target object; The original cardiac vibration signal is filtered to obtain a first filtered signal; the original jugular vein pulsation signal is filtered to obtain a second filtered signal; and the original photoplethysmography (PPG) pulse wave signal is filtered to obtain a third filtered signal. The first filtered signal is detrended to obtain a first detrended signal; the second filtered signal is detrended to obtain a second detrended signal; and the third filtered signal is detrended to obtain a third detrended signal. The first detrended signal is normalized to obtain the cardiac vibration signal of the target object; the second detrended signal is normalized to obtain the jugular vein pulsation signal of the target object; and the third detrended signal is normalized to obtain the photoplethysmography (PPG) signal of the target object.

4. The triggering method for a magnetic resonance imaging device based on multimodal signal fusion according to claim 1, characterized in that, The steps for constructing a heterogeneous graph based on the first morphological feature, the second morphological feature, and the third morphological feature include: The feature points that have a direct physiological causal relationship among the first morphological feature, the second morphological feature, and the third morphological feature are connected to obtain the first image to be processed; Based on the first image to be processed, all feature points in the first morphological feature are connected in chronological order, all feature points in the second morphological feature are connected in chronological order, and all feature points in the third morphological feature are connected in chronological order to obtain the second image to be processed. Based on the second image to be processed, all similar feature points in the first morphological feature are connected, all similar feature points in the second morphological feature are connected, and all similar feature points in the third morphological feature are connected to obtain the heterogeneous image.

5. The triggering method for a magnetic resonance imaging device based on multimodal signal fusion according to claim 1, characterized in that, Each feature point of the first morphological feature, each feature point of the second morphological feature, and each feature point of the third morphological feature contains temporal features, morphological features, and modal labels.

6. The triggering method for a magnetic resonance imaging device based on multimodal signal fusion according to claim 1, characterized in that, When the trigger control model is a fully connected network, the steps of using the trigger control model to perform time prediction processing on the target global graph feature vector to obtain the target time include: The first fully connected layer of the fully connected network is used to perform a linear transformation on the feature vector of the target global graph to obtain the first vector to be processed. The first vector to be processed is subjected to a nonlinear transformation using the first activation function of the fully connected network to obtain the second vector to be processed. The second fully connected layer of the fully connected network is used to perform a linear transformation on the second vector to be processed to obtain the third vector to be processed. The third vector to be processed is subjected to a nonlinear transformation using the second activation function of the fully connected network to obtain the fourth vector to be processed. The time prediction algorithm of the fully connected network is used to perform time prediction processing on the fourth vector to be processed to obtain the target time.

7. A triggering device for a magnetic resonance imaging apparatus based on multimodal signal fusion, characterized in that, include: The morphological feature acquisition module is used to acquire the first morphological features of the cardiac vibration signal, the second morphological features of the jugular venous pulsation signal, and the third morphological features of the photoplethysmography pulse wave signal of the target object. A heterogeneous graph construction module is used to construct a heterogeneous graph based on the first morphological feature, the second morphological feature, and the third morphological feature; The extraction module is used to perform global graph feature vector extraction processing on the heterogeneous graph using a heterogeneous graph attention network to obtain the target global graph feature vector; The prediction module is used to perform time prediction processing on the feature vector of the target global map using a trigger control model to obtain the target time, and trigger the magnetic resonance imaging device to acquire the image of the target object when the target object is in the mid-to-late diastolic state of the heart based on the target time; The extraction module is further configured to map each point in the heterogeneous graph to the latent space using the encoding network of the heterogeneous graph attention network to obtain a first fusion feature at the node level; to use the attention mechanism of the heterogeneous graph attention network to transmit information along different types of edges in the heterogeneous graph to obtain a second fusion feature at the relation level; and to use the weighted pooling module of the heterogeneous graph attention network to perform global graph feature vector extraction processing on the first fusion feature and the second fusion feature to obtain the target global graph feature vector.

8. A terminal device, characterized in that, include: A processor and a memory, the memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform the steps of the triggering method of the magnetic resonance imaging device based on multimodal signal fusion as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the steps of the triggering method of the magnetic resonance imaging device based on multimodal signal fusion as described in any one of claims 1 to 6.

Images