Motion estimation guided magnetic resonance four-dimensional dynamic imaging method and device

Abstract

The invention relates to the technical field of magnetic resonance imaging, and discloses a motion estimation guided magnetic resonance four-dimensional dynamic imaging method and device, and the method comprises the steps: obtaining a three-dimensional single-frame static image of magnetic resonance and an under-acquired low-resolution four-dimensional dynamic image sequence, the low-resolution four-dimensional dynamic image sequence is a multi-frame low-resolution four-dimensional dynamic image obtained by carrying out undersampling on K space acquisition data; utilizing a motion encoder in the image sequence prediction model to extract motion estimation information in the low-resolution four-dimensional dynamic image sequence; inputting the three-dimensional single-frame static image into a structure encoder in the image sequence prediction model to obtain static anatomical structure information; and fusing and decoding the motion estimation information and the anatomical structure information through a decoder to obtain a high-resolution four-dimensional dynamic image sequence. The problems that an existing magnetic resonance four-dimensional imaging technology is long in scanning time, complex organ movement is difficult to capture, and time-space consistency is poor are solved.

Description

Technical Field

[0001] The invention relates to the technical field of magnetic resonance imaging, and in particular to a method and device for magnetic resonance four-dimensional dynamic imaging guided by motion estimation. Background Art

[0002] As an important progress in the field of medical imaging, four-dimensional magnetic resonance imaging technology can capture and analyze the dynamic changes of three-dimensional images over time, and is particularly suitable for observing and analyzing organ movement. However, traditional four-dimensional magnetic resonance imaging technology is limited by multiple data acquisitions, resulting in a lengthy scanning process, poor patient experience, and easy image distortion and artifacts due to movement. In order to balance the spatiotemporal resolution, the temporal resolution is sacrificed, which affects the capture of fast-moving structures. In recent years, deep learning has shown potential in medical image processing, which can reduce scanning time and improve image quality. However, the existing technology still has deficiencies in motion estimation and deformation field interpolation, resulting in the generated motion process being not realistic enough, affecting the accuracy and continuity of imaging. Especially when capturing fast or irregular motion, although the existing technology can generate intermediate frames, it cannot guarantee the consistency of the topological structure, making the motion process unrealistic. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a motion estimation-guided MRI four-dimensional dynamic imaging method and apparatus to solve the problems of the existing MRI four-dimensional imaging technology, such as long scanning time, difficulty in capturing complex organ motion, and poor temporal and spatial consistency.

[0004] In a first aspect, an embodiment of the present invention provides a method for motion estimation-guided magnetic resonance four-dimensional dynamic imaging, the method comprising:

[0005] Acquire a three-dimensional single-frame static image of magnetic resonance and a low-resolution four-dimensional dynamic image sequence with undersampling, wherein the low-resolution four-dimensional dynamic image sequence is a multi-frame low-resolution four-dimensional dynamic image obtained by undersampling K-space acquisition data;

[0006] Extracting motion estimation information from the low-resolution four-dimensional dynamic image sequence using a motion encoder in an image sequence prediction model;

[0007] Inputting the three-dimensional single-frame static image into a structure encoder in the image sequence prediction model to obtain static anatomical structure information;

[0008] The motion estimation information and the anatomical structure information are fused and decoded by a decoder to obtain a high-resolution four-dimensional dynamic image sequence.

[0009] Furthermore, before using the motion encoder in the image sequence prediction model to extract the motion estimation information in the low-resolution four-dimensional dynamic image sequence, the training method of the image sequence prediction model includes:

[0010] Obtain high-resolution four-dimensional dynamic image sequence samples of full magnetic resonance sampling;

[0011] Under-sampling the high-resolution four-dimensional dynamic image sequence samples to obtain low-resolution four-dimensional dynamic image sequence samples;

[0012] Acquire a three-dimensional single-frame static image sample of full magnetic resonance sampling, and input the four-dimensional dynamic image sequence sample and the three-dimensional single-frame static image sample into the original learning model for learning to obtain a predicted high-resolution four-dimensional dynamic image sequence sample;

[0013] The target loss value between the predicted high-resolution four-dimensional dynamic image sequence samples and the high-resolution four-dimensional dynamic image sequence samples obtained by full sampling is calculated, and the model parameters of the original learning model are adjusted using the target loss value until the target loss value meets the preset training conditions to obtain the image sequence prediction model.

[0014] Further, the target loss value is used to adjust the model parameters of the original learning model until the target loss value meets the preset training conditions to obtain the image sequence prediction model, including:

[0015] Obtain the number of training times of the current original learning model;

[0016] When the number of training times reaches a specified number, a target loss value of the original learning model is calculated using a preset validation set;

[0017] The downward trend of the target loss value is analyzed. If the downward trend stops decreasing, it is determined that the target loss value meets the preset training condition, and the original learning model corresponding to the lowest point of the target loss value is used as the image sequence prediction model.

[0018] Further, using a motion encoder in an image sequence prediction model to extract motion estimation information in the low-resolution four-dimensional dynamic image sequence includes:

[0019] Acquire low-resolution four-dimensional dynamic images of multiple time frames in the low-resolution four-dimensional dynamic image sequence;

[0020] By using the motion encoder in the image sequence prediction model, low-resolution four-dimensional dynamic images of multiple time frames are analyzed to obtain dynamic change characteristics of the low-resolution four-dimensional dynamic images between adjacent time frames and the dependency relationship of the low-resolution four-dimensional dynamic images between each time frame;

[0021] According to the dynamic change characteristics and the dependency relationship, motion estimation information corresponding to the low-resolution four-dimensional dynamic image sequence is extracted.

[0022] Furthermore, the motion estimation information and the anatomical structure information are fused and decoded by a decoder to obtain a high-resolution four-dimensional dynamic image sequence, including:

[0023] Obtaining a fusion mode pre-stored in the decoder;

[0024] The anatomical structure information and the motion estimation information are fused according to the fusion method to obtain a high-resolution four-dimensional dynamic feature sequence;

[0025] The high-resolution four-dimensional dynamic feature sequence is decoded into the high-resolution four-dimensional dynamic image sequence.

[0026] Furthermore, after the motion estimation information and the anatomical structure information are fused and decoded by a decoder to obtain a high-resolution four-dimensional dynamic image sequence, the evaluation method of the high-resolution four-dimensional dynamic image sequence includes:

[0027] Obtaining preset image evaluation indicators, wherein the image evaluation indicators include quantitative indicators and performance indicators;

[0028] According to the quantitative index, quantitative analysis is performed on the generated high-resolution four-dimensional dynamic image sequence to obtain a first analysis result;

[0029] According to the performance index, the generated high-resolution four-dimensional dynamic image sequence is compared with the standard dynamic image to obtain a comparison result, and the interference factors in the four-dimensional dynamic image are analyzed according to the comparison result to obtain a second analysis result;

[0030] Based on the first analysis result and the second analysis result, it is determined whether the generated high-resolution four-dimensional dynamic image sequence is qualified to obtain an evaluation result.

[0031] In a second aspect, an embodiment of the present invention provides a motion estimation-guided magnetic resonance four-dimensional dynamic imaging device, the device comprising:

[0032] An acquisition module, used for acquiring a three-dimensional single-frame static image of magnetic resonance and a low-resolution four-dimensional dynamic image sequence with undersampling, wherein the low-resolution four-dimensional dynamic image sequence is a multi-frame low-resolution four-dimensional dynamic image obtained by undersampling K-space acquisition data;

[0033] An extraction module, used to extract motion estimation information from the low-resolution four-dimensional dynamic image sequence using a motion encoder in an image sequence prediction model;

[0034] An input module, used for inputting the three-dimensional single-frame static image into a structure encoder in the image sequence prediction model to obtain static anatomical structure information;

[0035] The fusion module is used to fuse and decode the motion estimation information and the anatomical structure information through a decoder to obtain a high-resolution four-dimensional dynamic image sequence.

[0036] In a third aspect, an embodiment of the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0037] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0038] The method provided in the embodiment of the present application has the following beneficial effects:

[0039] The method provided in the embodiment of the present application uses rapid undersampling technology to acquire a low-resolution four-dimensional dynamic image sequence. This strategy significantly shortens the scanning time and improves imaging efficiency. Even under the condition of rapid undersampling, the method can still retain key information of moving parts, laying a solid foundation for subsequent analysis.

[0040] The method provided in the embodiment of the present application provides detailed anatomical structure information of the moving part through the acquired three-dimensional single-frame static image. This information is combined with the motion information provided by the low-resolution four-dimensional dynamic image sequence, so that the understanding of the moving part is more comprehensive. Further, the motion encoder can accurately extract motion estimation information from the low-resolution four-dimensional dynamic image sequence, which is conducive to capturing the motion state of complex organs and provides a key basis for the subsequent generation of high-resolution four-dimensional dynamic images. Through the extraction of motion estimation information, the dynamic changes of the moving part at different time points can be deeply understood, thereby enhancing the spatiotemporal consistency of the image. In addition, the motion encoder has a high degree of adaptability and can cope with different motion modes and speeds, which greatly improves the adaptability and generalization ability of the image sequence prediction model.

[0041] The method provided in the embodiment of the present application combines the anatomical structure information of a three-dimensional single-frame static image with the motion estimation information through a structural encoder to generate a multi-frame high-resolution four-dimensional dynamic image. This fusion strategy makes full use of the advantages of both, so that the detailed structure of the moving part can be clearly displayed, further improving the quality and accuracy of the image. Through the rapid processing of the image sequence prediction model, the target four-dimensional dynamic image sequence can be obtained in a short time, meeting the demand for high temporal and spatial resolution rapid imaging of the moving part. In order to ensure the image quality, quantitative indicators are used for quantitative analysis, and performance comparison is performed with standard dynamic images, so as to comprehensively evaluate the image quality and discover and solve potential problems. The target dynamic image finally obtained has high temporal and spatial resolution, which provides more accurate and detailed information support for the accurate diagnosis of the moving part. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 is a schematic flow chart of a method for four-dimensional dynamic magnetic resonance imaging guided by motion estimation according to an embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of a structural framework of an image sequence prediction model according to an embodiment of the present invention;

[0045] Figure 3 is a schematic flow chart of another motion estimation-guided magnetic resonance four-dimensional dynamic imaging method according to an embodiment of the present invention;

[0046] Figure 4 is a structural block diagram of a magnetic resonance four-dimensional dynamic imaging device guided by motion estimation according to an embodiment of the present invention;

[0047] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0049] According to an embodiment of the present invention, a motion estimation-guided magnetic resonance four-dimensional dynamic imaging method and apparatus are provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0050] In this embodiment, a motion estimation-guided magnetic resonance four-dimensional dynamic imaging method is provided. Figure 1 is a flow chart of a method for four-dimensional dynamic magnetic resonance imaging guided by motion estimation according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0051] Step S11, obtaining a three-dimensional single-frame static image of magnetic resonance and a low-resolution four-dimensional dynamic image sequence with undersampling, wherein the low-resolution four-dimensional dynamic image sequence is a multi-frame low-resolution four-dimensional dynamic image obtained by undersampling K-space acquisition data.

[0052] In the embodiment of the present application, a three-dimensional single-frame static image can provide detailed three-dimensional structural information at a certain static moment, while a low-resolution four-dimensional dynamic image sequence can provide real motion estimation information. The combination of the two helps to more comprehensively reflect the physiological and pathological state of the organism. For different application scenarios, such as the diagnosis of cardiovascular disease (which focuses more on the dynamic changes in the internal structure of the organism) and the localization and morphological assessment of tumors (which requires more accurate understanding of the internal structure of the organism at a certain moment), these data can meet different diagnostic needs.

[0053] Step S12, using the motion encoder in the image sequence prediction model to extract motion estimation information from the low-resolution four-dimensional dynamic image sequence.

[0054] It should be noted that if Figure 2 As shown, the structural framework corresponding to the model designed by the present invention includes: structural encoder Enc S , Motion encoder Enc m And decoder Dec. Among them, the structure encoder Enc S The 3D convolution structure is used to extract the anatomical structure information of a three-dimensional single-frame static image. m It is designed as a pseudo 4D convolutional encoder, which simulates a 3*3*3*3 4D convolution by combining a 1*3*3*3 3D spatial convolution with a 3*1*1*1 1D temporal convolution. mThe spatiotemporal self-attention module TSA is designed in this paper, which can focus on the spatial dimension and temporal dimension information in the low-resolution original four-dimensional dynamic image sequence respectively to obtain motion estimation information. The decoder Dec is used to fuse and decode the anatomical structure information and motion estimation information extracted by the encoder, and finally generate a target four-dimensional dynamic image sequence with high spatiotemporal resolution.

[0055] In the embodiment of the present application, step S12 includes the following steps A1-A3:

[0056] Step A1, obtaining low-resolution four-dimensional dynamic images of multiple time frames in a low-resolution four-dimensional dynamic image sequence.

[0057] Specifically, the low-resolution four-dimensional dynamic image sequence contains multiple frames of images in the time dimension. By obtaining these time frames, the motion encoder in the subsequent steps can capture dynamic change characteristics and dependencies from these time frames to provide a data basis. Only by obtaining multiple consecutive time frames can we further analyze the changes between adjacent frames and between different time points, thereby extracting accurate motion estimation information.

[0058] Step A2, through the motion encoder in the image sequence prediction model, analyze the low-resolution four-dimensional dynamic images of multiple time frames to obtain the dynamic change characteristics of the low-resolution four-dimensional dynamic images between adjacent time frames, and the dependency relationship of the low-resolution four-dimensional dynamic images between each time frame.

[0059] Specifically, the role of the motion encoder is to extract motion estimation information from the low-resolution four-dimensional dynamic image sequence. In this step, the motion encoder analyzes multiple time frames to capture the dynamic change characteristics between adjacent time frames, such as the displacement and deformation of voxels. These dynamic change characteristics are used to understand the changes in the internal structure of the organism over time. At the same time, capturing the dependencies between each time frame can ensure the continuity of the motion. This dependency reflects the change trend and speed of voxels at different time points, which helps to generate a smooth and realistic 4D dynamic sequence in the time dimension. The extracted dynamic change features and dependencies will be used to construct the motion estimation information corresponding to the low-resolution four-dimensional dynamic image.

[0060] Step A3: extracting motion estimation information corresponding to the low-resolution four-dimensional dynamic image sequence according to the dynamic change characteristics and the dependency relationship.

[0061] Specifically, the motion encoder captures the dynamic change characteristics between adjacent time frames and the dependencies between each time frame from these time frames. These characteristics and relationships are integrated to construct the motion estimation information corresponding to the low-resolution four-dimensional dynamic image sequence. This motion estimation information can accurately reflect the movement of the internal structure of the organism over time, and provide key motion feature basis for the subsequent generation of high-resolution magnetic resonance 4D dynamic sequences.

[0062] In the embodiment of the present application, the following steps S21-S24 are included before step S12, that is, the training method of the image sequence prediction model includes:

[0063] Step S21, obtaining high-resolution four-dimensional dynamic image sequence samples of full magnetic resonance sampling.

[0064] In the embodiment of the present application, in the preparation stage of deep learning model training, it is first necessary to obtain high-resolution four-dimensional dynamic image sequence samples with full sampling of magnetic resonance. Fully sampled high-resolution four-dimensional dynamic image sequence samples can provide the most complete and accurate dynamic change information of the internal structure of the organism, which serves as the basic data for subsequent processing. This step ensures that there is high-quality reference data available for model learning and optimization.

[0065] Step S22, under-sampling the high-resolution four-dimensional dynamic image sequence samples to obtain low-resolution four-dimensional dynamic image sequence samples.

[0066] In the embodiment of the present application, a highly under-sampled operation is performed on the fully sampled high-resolution four-dimensional dynamic image sequence samples in the K-space domain, in order to obtain low-resolution magnetic resonance 4D dynamic sequence samples. This can simulate the rapid acquisition situation in actual applications, and also provide real motion estimation information for the model. Through under-sampling, the amount of data and computational complexity can be reduced, and the model can also be prompted to learn how to recover high-quality images from limited data.

[0067] Step S23, obtaining a three-dimensional single-frame static image sample of full magnetic resonance sampling, and inputting the four-dimensional dynamic image sequence sample and the three-dimensional single-frame static image sample into the original learning model for learning to obtain a predicted high-resolution four-dimensional dynamic image sequence sample.

[0068] In an embodiment of the present application, in addition to obtaining low-resolution four-dimensional dynamic image sequence samples, it is also necessary to obtain three-dimensional single-frame static image samples of full magnetic resonance sampling. Then the low-resolution four-dimensional dynamic image sequence samples and the three-dimensional single-frame static image samples are input into the original learning model together. The original learning model will make predictions based on the input data and output predicted high-resolution four-dimensional dynamic image sequence samples. By inputting different types of data, the model learns how to fuse anatomical structure information and motion estimation information to generate high-quality 4D dynamic image sequences.

[0069] Step S24, calculate the target loss value between the predicted high-resolution four-dimensional dynamic image sequence samples and the high-resolution four-dimensional dynamic image sequence samples obtained by full sampling, and use the target loss value to adjust the model parameters of the original learning model until the target loss value meets the preset training conditions to obtain the image sequence prediction model.

[0070] In an embodiment of the present application, the difference between the predicted high-resolution four-dimensional dynamic image sequence samples and the high-resolution four-dimensional dynamic image sequence samples obtained by full sampling is calculated to obtain a target loss value. The loss value reflects the gap between the model prediction results and the real data. By using the target loss value and using optimization algorithms such as gradient descent to adjust the parameters of the original learning model, the model is continuously optimized and the prediction accuracy is gradually improved. This process is repeated until the target loss value meets the preset training conditions, and the model obtained at this time is the image sequence prediction model.

[0071] In the embodiment of the present application, the model parameters of the original learning model are adjusted using the target loss value until the target loss value meets the preset training conditions, thereby obtaining an image sequence prediction model, including the following steps B1-B3:

[0072] Step B1, obtaining the number of training times of the current original learning model.

[0073] Specifically, during the model training process, it is necessary to constantly check how many times the current original learning model has been trained. By obtaining the number of training times, it is possible to determine whether the model has reached one of the preset training conditions, that is, whether it has been trained a specified number of times. At the same time, the number of training times also provides a reference in the time dimension for determining the downward trend of the target loss value in subsequent steps.

[0074] Step B2, when the number of training times reaches the specified number, the target loss value of the original learning model is calculated using the preset verification set.

[0075] Specifically, when the model is trained for a specified number of times, the preset validation set is used to calculate the target loss value of the original learning model. The validation set is a part of data independent of the training data and is used to evaluate the performance of the model on different data. By calculating the target loss value, the gap between the model's prediction results and the real data can be quantified, thereby judging the training effect of the model.

[0076] It should be noted that in this model, the target loss value is composed of the generation loss value and the motion consistency loss value. On the one hand, the generation loss value mainly focuses on the difference between the model output and the target, and promotes the model to learn how to generate data more accurately. The generation loss value selects the root mean square error, which is defined as follows:

[0077]

[0078] Among them, G refers to the original learning model, χ 3D Refers to the input 3D single-frame static image sample, It refers to the input highly undersampled low-resolution four-dimensional dynamic image sequence samples, and γ refers to the high-resolution four-dimensional dynamic image sequence samples with full magnetic resonance sampling. The generated loss value can quantify the deviation between the predicted value and the actual value, and guide the update of model parameters through optimization algorithms such as gradient descent.

[0079] On the other hand, in order to ensure that the model can maintain the consistency of motion information when generating dynamic sequences, a motion consistency loss value is introduced to constrain the motion features of the model output, ensuring that the generated predicted four-dimensional dynamic image sequence shows a smooth and realistic motion pattern in the time dimension. The definition of the motion consistency loss value is as follows:

[0080]

[0081] Among them, E m It refers to the motion estimation information extraction encoder, which constrains the motion estimation of the input and target 4D dynamic image sequence samples by the mean absolute error. Combining these two loss forms, the model can not only learn to generate accurate data, but also ensure that the motion state of the generated predicted 4D dynamic image sequence is highly consistent with the real data.

[0082] Step B3, analyzing the downward trend of the target loss value. If the downward trend stops, it is determined that the target loss value meets the preset training conditions, and the original learning model corresponding to the lowest point of the target loss value is used as the image sequence prediction model.

[0083] Specifically, during the model training process, the downward trend of the target loss value is continuously analyzed. If the downward trend of the target loss value stops decreasing, it means that the model has converged to a relatively stable state, and it can be determined that the target loss value meets the preset training conditions. Then, the original learning model corresponding to the lowest point of the target loss value is used as the image sequence prediction model, because this model shows the best performance during the training process.

[0084] Step S13, inputting the three-dimensional single-frame static image into the structure encoder in the image sequence prediction model to obtain static anatomical structure information.

[0085] In an embodiment of the present application, a structural encoder in an image sequence prediction model after training is used to specifically process a three-dimensional single-frame static image. The function of the structural encoder is to extract anatomical structure information from static 3D data of magnetic resonance. These anatomical structure information provide detailed three-dimensional structural features of an organism at a certain static moment, which is used to understand the basic morphology and structure of the organism. For example, in the localization and morphological assessment of a tumor, anatomical structure information can clearly show the location, size and shape of the tumor, providing an important basis for doctors to formulate treatment plans. In the analysis of bone structure, anatomical structure information can accurately present the morphology, density and presence of abnormalities such as fractures of bones. In addition, for the study of the nervous system, anatomical structure information can help doctors observe the structure of important organs such as the brain and spinal cord, and provide support for the diagnosis of nervous system diseases.

[0086] Step S14: The motion estimation information and the anatomical structure information are fused and decoded by a decoder to obtain a high-resolution four-dimensional dynamic image sequence.

[0087] It should be noted that the decoder is used to fuse and decode the static anatomical structure information extracted by the encoder with the motion estimation information, with the aim of generating a 4D dynamic sequence of magnetic resonance images with high temporal and spatial resolution. Different fusion methods will significantly affect the quality of the generated images, so clarifying the specific fusion method is the key to accurate subsequent information fusion.

[0088] In the embodiment of the present application, step S14 includes the following steps C1-C3, namely the model testing stage:

[0089] Step C1, obtaining the fusion mode of the decoder in the trained model.

[0090] Specifically, the fusion methods include but are not limited to the following three types. On the one hand, a splicing fusion method at the feature level is adopted. First, the features of the anatomical structure information and the motion estimation information are extracted respectively. The former can obtain a set of structural feature vectors of a specific dimension, and the latter can obtain a corresponding motion feature vector. Then, the two feature vectors are spliced ​​in a specific dimension to form a comprehensive feature vector. This comprehensive feature vector has both static anatomical structure features and dynamic motion features, laying the foundation for generating high-quality magnetic resonance 4D dynamic sequences. On the one hand, a weighted fusion method can be adopted. According to a certain weight allocation strategy, different weights are assigned to anatomical structure information and motion estimation information. For example, in application scenarios where structural accuracy needs to be emphasized, a higher weight can be assigned to anatomical structure information; while in scenarios that are more sensitive to motion changes, the weight of motion estimation information is appropriately increased. Finally, the weighted two types of information are fused to achieve more targeted information integration.

[0091] The present invention adopts a new type of fusion method based on cross-attention. By introducing the attention mechanism, a more flexible association is established between anatomical structure information and motion estimation information. Specifically, the anatomical structure features and motion features are first extracted respectively, and then the correlation between the two is modeled through the cross-attention mechanism. This mechanism realizes two-way information interaction by calculating the attention weights of anatomical structure features to motion features, and the attention weights of motion features to anatomical structure features. This process can dynamically focus on important areas in the anatomical structure related to motion information, and vice versa. Finally, by fusing information between the attention-weighted features, not only can the static characteristics of the anatomical structure be fully retained, but also the details in the motion changes can be effectively captured, making the generated 4D MRI dynamic sequence more accurate and smooth.

[0092] Step C2, fusing the anatomical structure information and the motion estimation information according to a fusion method to obtain a high-resolution four-dimensional dynamic feature sequence.

[0093] Specifically, the anatomical structure information is fused with the motion estimation information according to the acquired fusion mode. By fusing these two different types of information, the static structure information and the dynamic motion information can be fully utilized to generate a high-resolution four-dimensional dynamic feature sequence. The predicted high-resolution four-dimensional dynamic feature sequence has a clear anatomical structure and can accurately reflect the dynamic changes of the internal structure of the organism over time.

[0094] Step C3, decoding the high-resolution four-dimensional dynamic feature sequence into a high-resolution four-dimensional dynamic image sequence.

[0095] Specifically, the decoder can convert a high-resolution four-dimensional dynamic feature sequence into a visual high-resolution four-dimensional dynamic image sequence through deconvolution operations and nonlinear transformations. It extracts key feature information containing a comprehensive description of static anatomical structure and dynamic motion from the feature sequence, uses deconvolution to expand the feature map size to restore spatial resolution, and applies nonlinear activation functions to enhance the expression ability. The time dimension information is considered in the decoding process, and the time smoothing constraint is introduced to ensure the temporal coherence and authenticity of the image sequence to avoid mutations. The final output image sequence has clear anatomical details and can accurately reflect the dynamic changes of the internal structure of the organism over time, providing doctors with more comprehensive and accurate diagnostic information, which is conducive to understanding the disease development process and treatment effect.

[0096] In the embodiment of the present application, after the motion estimation information and the anatomical structure information are fused and decoded by the decoder to obtain a high-resolution four-dimensional dynamic image sequence, the evaluation method of the high-resolution four-dimensional dynamic image sequence includes the following steps D1-D4:

[0097] Step D1, obtaining preset image evaluation indicators, wherein the image evaluation indicators include quantitative indicators and performance indicators.

[0098] Specifically, before evaluating the generated high-resolution four-dimensional dynamic image sequence, it is necessary to first obtain the preset image evaluation indicators. These indicators are divided into two categories: quantitative indicators and performance indicators. Quantitative indicators are mainly used to objectively measure the quality and detail fidelity of the generated image from a numerical perspective, while performance indicators focus on evaluating the performance of the image in practical applications, such as comparison with standard dynamic images, analysis of interference factors, etc. By clarifying the evaluation indicators, specific standards and directions can be provided for the subsequent evaluation process.

[0099] Step D2, performing quantitative analysis on the generated high-resolution four-dimensional dynamic image sequence according to the quantitative index to obtain a first analysis result.

[0100] Specifically, based on the quantitative indicators, the generated high-resolution four-dimensional dynamic image sequence is quantitatively analyzed. The quantitative analysis may include but is not limited to the following indicators: calculating the Peak Signal-to-Noise Ratio (PSNR), the Mean Squared Error (MSE) and the Structural Similarity Index (SSIM). These indicators can reflect the quality of the generated image sequence from different angles. For example, PSNR focuses on the ratio between the peak intensity and noise in the image sequence, MSE measures the difference between the predicted image sequence and the traditional sequence at the pixel level, and SSIM evaluates the performance of the image sequence in retaining spatial structural features. Through quantitative analysis, a specific numerical result, namely the first analysis result, can be obtained, which provides an objective basis for evaluating the quality of the generated image sequence.

[0101] Step D3, according to the performance index, the generated high-resolution four-dimensional dynamic image sequence is compared with the standard dynamic image sequence to obtain a comparison result, and the interference factors in the four-dimensional dynamic image are analyzed according to the comparison result to obtain a second analysis result.

[0102] Specifically, difference analysis is performed at multiple levels. At the pixel level, the differences in brightness, color, etc. of each corresponding pixel are carefully calculated to capture subtle changes. At the regional level, the differences in different tissue regions, such as tumor regions and vascular regions, are analyzed, focusing on the imaging quality of specific regions. At the same time, at the level of the overall image sequence, the differences in overall indicators such as peak signal-to-noise ratio and structural similarity are calculated.

[0103] Then, the interference factors are determined based on the difference analysis results. If the pixel-level differences are large, there is a high probability that there is a problem of detail loss or noise interference. When the regional level differences are obvious, it can be determined that the imaging of a specific tissue area is inaccurate. For example, unclear tumor boundaries usually mean loss of detail, while blurred blood vessel areas may be caused by motion blur. If the overall indicators differ greatly, further in-depth analysis is needed to determine which specific aspects cause this difference.

[0104] Finally, according to the characteristics of the four-dimensional dynamic image sequence, the changes in the time series are analyzed. Observe whether the changes of the predicted image sequence and the standard image sequence at different time points are consistent, and check whether there are time delays or abnormal changes. If there is inconsistency in the time series, it may indicate that the motion estimation is inaccurate or there is a problem with the processing of the time dimension during the sequence generation process.

[0105] Step D4, determining whether the generated high-resolution four-dimensional dynamic image sequence is qualified based on the first analysis result and the second analysis result, and obtaining an evaluation result.

[0106] Specifically, by comprehensively considering the analysis results of quantitative indicators and performance indicators, the quality and practicality of the generated image sequence can be more comprehensively evaluated as a whole, and then the evaluation results can be obtained. Only when the evaluation results show that the entire four-dimensional dynamic image sequence performs well in both quantitative indicators and performance indicators can it be determined as a high-resolution four-dimensional dynamic image sequence that has passed the evaluation. Such an image sequence can be used for high-temporal and spatial resolution rapid imaging of moving parts to meet the needs of clinical diagnosis and research.

[0107] In the embodiment of the present application, the method may further include the following steps E1-E2:

[0108] Step E1, obtaining current disease diagnosis requirements.

[0109] Specifically, in practical applications, different disease diagnosis needs will have different requirements for magnetic resonance imaging. This step is aimed at obtaining the current specific disease diagnosis needs. For example, for the diagnosis of cardiovascular diseases, more attention is paid to the dynamic changes of the internal structure of the organism, and dynamic images with high temporal and spatial resolution are required to observe the contraction and relaxation of the heart and other movements; while for the positioning and morphological evaluation of tumors, it is more necessary to accurately understand the internal structure of the organism at a certain moment, and higher requirements are placed on the details and clarity of the image. By clarifying the current disease diagnosis needs, a basis can be provided for the subsequent adjustment of the model to better meet the requirements of specific disease diagnosis.

[0110] Step E2, adjusting the relevant parameters of the encoder in the image sequence prediction model and the fusion mode of the decoder according to the disease diagnosis requirements to obtain an adjusted image sequence prediction model.

[0111] Specifically, according to the needs of disease diagnosis, the encoder-related parameters in the image sequence prediction model and the fusion method of the decoder are adjusted. If the disease diagnosis needs focus more on the observation of dynamic changes, the encoder needs to be adjusted to enhance the ability to capture motion features, such as adjusting the parameters of the pseudo-4D convolution and spatiotemporal autonomous attention modules to more sensitively capture dynamic change features such as displacement and deformation between adjacent frames. At the same time, the fusion method of the decoder can also be adjusted to make it more inclined to fuse motion estimation information to generate smoother and more realistic dynamic sequences. If the diagnostic needs focus more on the accurate display of static structures, the encoder can be adjusted to better extract anatomical structure information, and the fusion method of the decoder can be adjusted to increase the weight of anatomical structure information in the fusion. Through such adjustments, an image sequence prediction model that is more suitable for the current disease diagnosis needs can be obtained.

[0112] Figure 3 FIG. 1 is a flow chart of another motion estimation-guided magnetic resonance four-dimensional dynamic imaging method according to this embodiment. Figure 3As shown in the figure, the process is mainly divided into four stages: data acquisition and preprocessing stage, deep learning model training stage, magnetic resonance 4D dynamic sequence (i.e., target four-dimensional dynamic image sequence) generation stage, and dynamic sequence generation evaluation stage.

[0113] Specifically, the process is as follows: First, a fully sampled 4D magnetic resonance dynamic sequence is obtained, which is the target four-dimensional dynamic image sequence sample of the fully sampled magnetic resonance. Then, the fully sampled sequence is highly undersampled in K space to obtain a low-quality 4D magnetic resonance dynamic sequence, that is, the original four-dimensional dynamic image sequence sample, and a single-frame static 3D magnetic resonance data frame is obtained at the same time, that is, a three-dimensional single-frame static image sample of the fully sampled magnetic resonance. Then, the deep learning model (original learning model) is trained based on the static frame (three-dimensional single-frame static image sample) and motion prior (motion estimation information). For the actually collected static frame (three-dimensional single-frame static image of magnetic resonance) and the fast undersampled low-resolution 4D magnetic resonance dynamic sequence (original four-dimensional dynamic image sequence), the trained deep learning model (image sequence prediction model) can be used to obtain a high spatiotemporal resolution 4D magnetic resonance dynamic sequence, that is, the target four-dimensional dynamic image sequence. Finally, the generated 4D magnetic resonance dynamic sequence is evaluated.

[0114] In this embodiment, a motion estimation guided magnetic resonance four-dimensional dynamic imaging device is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0115] This embodiment provides a motion estimation guided 4D dynamic magnetic resonance imaging device. Figure 4 As shown, including:

[0116] An acquisition module 41 is used to acquire a three-dimensional single-frame static image of magnetic resonance and a low-resolution four-dimensional dynamic image sequence with undersampling, wherein the low-resolution four-dimensional dynamic image sequence is a multi-frame low-resolution four-dimensional dynamic image obtained by undersampling K-space acquisition data;

[0117] An extraction module 42, used to extract motion estimation information from a low-resolution four-dimensional dynamic image sequence using a motion encoder in an image sequence prediction model;

[0118] An input module 43 is used to input a three-dimensional single-frame static image into a structure encoder in an image sequence prediction model to obtain static anatomical structure information;

[0119] The fusion module 44 is used to fuse and decode the motion estimation information and the anatomical structure information through a decoder to obtain a high-resolution four-dimensional dynamic image sequence.

[0120] In an optional embodiment of the present application, the extraction module 42 is used to obtain low-resolution four-dimensional dynamic images of multiple time frames in a low-resolution four-dimensional dynamic image sequence; analyze the low-resolution four-dimensional dynamic images of multiple time frames through a motion encoder in an image sequence prediction model to obtain dynamic change characteristics of the low-resolution four-dimensional dynamic images between adjacent time frames, and the dependency relationship of the low-resolution four-dimensional dynamic images between each time frame; and extract motion estimation information corresponding to the low-resolution four-dimensional dynamic image sequence based on the dynamic change characteristics and the dependency relationship.

[0121] In an optional embodiment of the present application, the fusion module 44 is used to obtain a fusion method pre-stored in the decoder; fuse the anatomical structure information and the motion estimation information according to the fusion method to obtain a high-resolution four-dimensional dynamic feature sequence; and decode the high-resolution four-dimensional dynamic feature sequence into a high-resolution four-dimensional dynamic image sequence.

[0122] In an optional embodiment of the present application, the device also includes: an evaluation module, used to obtain preset image evaluation indicators, wherein the image evaluation indicators include quantitative indicators and performance indicators; according to the quantitative indicators, quantitatively analyze the generated high-resolution four-dimensional dynamic image sequence to obtain a first analysis result; according to the performance indicators, compare the generated high-resolution four-dimensional dynamic image sequence with the standard dynamic image to obtain a comparison result, and analyze the interference factors in the four-dimensional dynamic image based on the comparison result to obtain a second analysis result; determine whether the generated high-resolution four-dimensional dynamic image sequence is qualified based on the first analysis result and the second analysis result to obtain an evaluation result.

[0123] In an optional embodiment of the present application, the device also includes: a training module for obtaining high-resolution four-dimensional dynamic image sequence samples with full magnetic resonance sampling; under-sampling the high-resolution four-dimensional dynamic image sequence samples to obtain low-resolution four-dimensional dynamic image sequence samples; obtaining three-dimensional single-frame static image samples with full magnetic resonance sampling, and inputting the four-dimensional dynamic image sequence samples and the three-dimensional single-frame static image samples into the original learning model for learning to obtain predicted high-resolution four-dimensional dynamic image sequence samples; calculating the target loss value between the predicted high-resolution four-dimensional dynamic image sequence samples and the high-resolution four-dimensional dynamic image sequence samples obtained by full sampling, and using the target loss value to adjust the model parameters of the original learning model until the target loss value meets the preset training conditions to obtain an image sequence prediction model.

[0124] In an optional embodiment of the present application, a training module is used to obtain the number of training times of the current original learning model; when the number of training times reaches a specified number, a target loss value of the original learning model is calculated using a preset verification set; a downward trend of the target loss value is analyzed, and if the downward trend stops decreasing, it is determined that the target loss value meets the preset training conditions, and the original learning model corresponding to the lowest point of the target loss value is used as the image sequence prediction model.

[0125] In an optional embodiment of the present application, the device also includes: an adjustment module for obtaining current disease diagnosis requirements; adjusting relevant parameters of the encoder in the sequence prediction model and the fusion method of the decoder according to the disease diagnosis requirements to obtain an adjusted image sequence prediction model.

[0126] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0127] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0128] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0129] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0130] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0131] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0132] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0133] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A motion estimation guided magnetic resonance four-dimensional dynamic imaging method, characterized in that: The method comprises: Acquire a three-dimensional single-frame static image of magnetic resonance and a low-resolution four-dimensional dynamic image sequence with undersampling, wherein the low-resolution four-dimensional dynamic image sequence is a multi-frame low-resolution four-dimensional dynamic image obtained by undersampling K-space acquisition data; Extracting motion estimation information from the low-resolution four-dimensional dynamic image sequence using a motion encoder in an image sequence prediction model; Inputting the three-dimensional single-frame static image into a structure encoder in the image sequence prediction model to obtain static anatomical structure information; The motion estimation information and the anatomical structure information are fused and decoded by a decoder to obtain a high-resolution four-dimensional dynamic image sequence.

2. The method according to claim 1, characterized in that: Before extracting motion estimation information from the low-resolution four-dimensional dynamic image sequence using a motion encoder in the image sequence prediction model, the training method of the image sequence prediction model includes: Obtain high-resolution four-dimensional dynamic image sequence samples of full magnetic resonance sampling; Under-sampling the high-resolution four-dimensional dynamic image sequence samples to obtain low-resolution four-dimensional dynamic image sequence samples; Acquire a three-dimensional single-frame static image sample of full magnetic resonance sampling, and input the four-dimensional dynamic image sequence sample and the three-dimensional single-frame static image sample into the original learning model for learning to obtain a predicted high-resolution four-dimensional dynamic image sequence sample; The target loss value between the predicted high-resolution four-dimensional dynamic image sequence samples and the high-resolution four-dimensional dynamic image sequence samples obtained by full sampling is calculated, and the model parameters of the original learning model are adjusted using the target loss value until the target loss value meets the preset training conditions to obtain the image sequence prediction model.

3. The method according to claim 2, characterized in that The target loss value is used to adjust the model parameters of the original learning model until the target loss value meets the preset training conditions to obtain the image sequence prediction model, including: Obtain the number of training times of the current original learning model; When the number of training times reaches a specified number, a target loss value of the original learning model is calculated using a preset validation set; Analyze the downward trend of the target loss value. If the downward trend stops decreasing, determine that the target loss value meets the preset training condition, and use the original learning model corresponding to the lowest point of the target loss value as the image sequence prediction model.

4. The method according to claim 1, characterized in that Extracting motion estimation information from the low-resolution four-dimensional dynamic image sequence using a motion encoder in an image sequence prediction model includes: Acquire low-resolution four-dimensional dynamic images of multiple time frames in the low-resolution four-dimensional dynamic image sequence; By using the motion encoder in the image sequence prediction model, low-resolution four-dimensional dynamic images of multiple time frames are analyzed to obtain dynamic change characteristics of the low-resolution four-dimensional dynamic images between adjacent time frames and the dependency relationship of the low-resolution four-dimensional dynamic images between each time frame; According to the dynamic change characteristics and the dependency relationship, motion estimation information corresponding to the low-resolution four-dimensional dynamic image sequence is extracted.

5. The method according to claim 1, characterized in that: The motion estimation information and the anatomical structure information are fused and decoded by a decoder to obtain a high-resolution four-dimensional dynamic image sequence, including: Obtaining a fusion mode pre-stored in the decoder; The anatomical structure information and the motion estimation information are fused according to the fusion method to obtain a high-resolution four-dimensional dynamic feature sequence; The high-resolution four-dimensional dynamic feature sequence is decoded into the high-resolution four-dimensional dynamic image sequence.

6. The method according to claim 1, characterized in that After the motion estimation information and the anatomical structure information are fused and decoded by a decoder to obtain a high-resolution four-dimensional dynamic image sequence, the evaluation method of the high-resolution four-dimensional dynamic image sequence includes: Obtaining preset image evaluation indicators, wherein the image evaluation indicators include quantitative indicators and performance indicators; According to the quantitative index, quantitative analysis is performed on the generated high-resolution four-dimensional dynamic image sequence to obtain a first analysis result; According to the performance index, the generated high-resolution four-dimensional dynamic image sequence is compared with the standard dynamic image to obtain a comparison result, and the interference factors in the four-dimensional dynamic image are analyzed according to the comparison result to obtain a second analysis result; Based on the first analysis result and the second analysis result, it is determined whether the generated high-resolution four-dimensional dynamic image sequence is qualified to obtain an evaluation result.

7. A motion estimation guided 4D dynamic magnetic resonance imaging device, characterized in that: The device comprises: An acquisition module, used for acquiring a three-dimensional single-frame static image of magnetic resonance and a low-resolution four-dimensional dynamic image sequence with undersampling, wherein the low-resolution four-dimensional dynamic image sequence is a multi-frame low-resolution four-dimensional dynamic image obtained by undersampling K-space acquisition data; An extraction module, used to extract motion estimation information from the low-resolution four-dimensional dynamic image sequence using a motion encoder in an image sequence prediction model; An input module, used for inputting the three-dimensional single-frame static image into a structure encoder in the image sequence prediction model to obtain static anatomical structure information; The fusion module is used to fuse and decode the motion estimation information and the anatomical structure information through a decoder to obtain a high-resolution four-dimensional dynamic image sequence.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 6.

Images