An intelligent magnetic resonance holographic imaging method and system

Abstract

This invention discloses an intelligent magnetic resonance holographic imaging method and system. The system includes a data acquisition device, an image reconstruction device, and an image display device. The data acquisition device acquires undersampled magnetic resonance data in real time through undersampled scanning and sends it to the image reconstruction device. The image reconstruction device inputs the undersampled magnetic resonance data into a deep learning model and outputs a high-dimensional stereoscopic reconstructed image. The deep learning model takes the undersampled magnetic resonance data of the sample as input and the corresponding fully sampled magnetic resonance data as output, and is obtained through self-supervised training. The image display device renders the stereoscopic reconstructed image in stereo and displays it in stereoscopic form through holographic projection. This invention enables rapid magnetic resonance imaging and real-time stereoscopic magnetic resonance image display, thereby better assisting doctors in surgical interventions and treatments.

Description

Technical Field

[0001] This invention relates to the field of medical imaging technology, and more specifically, to an intelligent magnetic resonance holographic imaging method and system. Background Technology

[0002] Magnetic resonance imaging (MRI) offers excellent soft tissue contrast, is free of ionizing radiation, and allows for multi-angle, large-field imaging, making it a promising imaging modality for clinical surgical interventions. However, compared to CT and ultrasound imaging, MRI is slower, limiting its application in scenarios requiring real-time imaging. Furthermore, while MRI can acquire three-dimensional image data, current image display methods are limited to two-dimensional planar representations, making it difficult to accurately depict rich spatial and temporal information.

[0003] Numerous studies have addressed the slow speed of magnetic resonance imaging (MRI). For example, methods based on compressed sensing and deep learning have been developed to reduce scanning time while maintaining image quality. These methods increase scanning speed by reducing the amount of K-space data acquired, and then perform high-quality image reconstruction based on different priors. However, compressed sensing technology has a long iterative reconstruction time, which is not conducive to real-time display of imaging results. Deep learning technology requires fully sampled labeled data to supervise model training, and its clinical application is not yet mature. Regarding the problem of displaying two-dimensional planar images, there is currently no good solution. Doctors typically make diagnoses by simultaneously observing multiple two-dimensional images or two-dimensional planar projections of stereo images, which is clearly detrimental to the application of MRI in surgical interventions.

[0004] Deep learning-based rapid magnetic resonance imaging (MRI) methods have enormous potential for clinical applications. However, current technologies focus on improving imaging speed and restoring image quality. Even if high-dimensional images can be reconstructed well, the final results are limited to two-dimensional planar display. Users cannot fully and intuitively observe three-dimensional MRI images in real time, which limits the use of MRI technology in surgical intervention and treatment. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an intelligent magnetic resonance holographic imaging method and system.

[0006] According to a first aspect of the present invention, an intelligent magnetic resonance holographic imaging system is provided. The system includes: a data acquisition device, an image reconstruction device, and an image display device, wherein:

[0007] The data acquisition equipment is used to acquire undersampled magnetic resonance data in real time through undersampled scanning and send it to the image reconstruction equipment;

[0008] The image reconstruction device is used to input the undersampled magnetic resonance data into a deep learning model and output a stereo reconstructed image, wherein the stereo reconstructed image is not less than 3-dimensional;

[0009] The image display device is used to perform stereoscopic rendering of the stereoscopic reconstructed image and to display it stereoscopically via holographic projection;

[0010] The deep learning model is obtained through self-supervised training, using undersampled magnetic resonance data of the sample as input and corresponding fully sampled magnetic resonance data as output.

[0011] According to a second aspect of the present invention, a smart magnetic resonance holographic imaging method is provided. The method includes the following steps:

[0012] Undersampled magnetic resonance data is acquired in real time through undersampled scanning and input into a deep learning model to obtain a stereo reconstructed image, wherein the stereo reconstructed image is not less than 3-dimensional.

[0013] The reconstructed stereoscopic image is rendered in stereoscopic form and then displayed in stereoscopic form via holographic projection.

[0014] The deep learning model is obtained through self-supervised training, using undersampled magnetic resonance data of the sample as input and corresponding fully sampled magnetic resonance data as output.

[0015] In one embodiment, the deep learning model is trained according to the following steps:

[0016] The designed undersampled scanning trajectory is used to perform magnetic resonance scanning on the target to obtain the original undersampled magnetic resonance data;

[0017] The original undersampled magnetic resonance data is subjected to secondary undersampling, and the secondary undersampled magnetic resonance data and the original undersampled magnetic resonance data are combined to form a data pair, which is used for training the deep learning model.

[0018] After completing one round of training, the raw undersampled magnetic resonance data is input into the trained deep learning model to generate pseudo-full sampled data labels, and these pseudo-full sampled data labels are introduced into the next round of training of the deep learning model until the set model optimization criteria are met.

[0019] Compared with existing technologies, the advantages of this invention are that it proposes a new intelligent magnetic resonance holographic imaging scheme. First, it achieves rapid magnetic resonance scanning and reconstruction based on deep learning. Then, it performs stereoscopic rendering of the stereoscopic magnetic resonance image and realizes real-time stereoscopic magnetic resonance image display based on holographic projection. This allows doctors to comprehensively and intuitively observe the tissue of the imaging site, assisting doctors in surgical intervention and treatment.

[0020] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0022] Figure 1 This is a flowchart of an intelligent magnetic resonance holographic imaging method according to an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of an intelligent magnetic resonance holographic imaging system according to an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram illustrating the application process of an intelligent magnetic resonance holographic imaging system according to an embodiment of the present invention. Detailed Implementation

[0025] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0026] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0027] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0028] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0029] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0030] The intelligent magnetic resonance imaging scheme provided by this invention mainly includes: designing undersampled scanning trajectories and related sequences to perform magnetic resonance scanning and obtain undersampled data (or undersampled magnetic resonance data); training a self-supervised deep learning model as a magnetic resonance image reconstruction model; obtaining high-quality stereoscopic reconstructed images based on the trained model; performing stereoscopic rendering on the stereoscopic reconstructed images and displaying them in stereoscopic form through holographic projection. Ultimately, this achieves rapid magnetic resonance imaging and real-time stereoscopic image display.

[0031] Specifically, see Figure 1 As shown, the provided intelligent magnetic resonance holographic imaging method includes the following steps.

[0032] Step S110: Perform magnetic resonance scanning using the designed undersampled scanning trajectory to obtain undersampled data.

[0033] First, during magnetic resonance imaging (MRI) scanning, undersampled scanning sequences are designed and optimized. Appropriate scanning trajectories are selected based on the specific object being scanned to maximize scanning speed and thus obtain undersampled data. Examples include Cartesian space scanning, radial scanning, and helical scanning.

[0034] Step S120: Self-supervised training of a deep learning model based on the collected undersampled data.

[0035] Deep learning models can employ various network structures, such as convolutional neural networks or recurrent neural networks. Deep learning models are used to perform fast image reconstruction from undersampled magnetic resonance imaging (MRI) data, obtaining higher-quality reconstructed images. Therefore, in this paper, deep learning models are also referred to as MRI image reconstruction models.

[0036] In one embodiment, a deep learning network model is trained using a self-supervised approach based on collected data. The training data does not require fully sampled data labels; instead, undersampled data is subjected to secondary undersampling, forming a data pair with the original undersampled data, which is then used for network training. After one round of training, the original undersampled data is input into the network to generate pseudo-fully sampled data labels, which are then used in the next round of network training. This process is repeated until a predetermined loss criterion is met or the network performance no longer improves. This design expands the training dataset without requiring fully sampled data and when the original undersampled data sample size is small, improving not only the model reconstruction performance but also the efficiency of the model in subsequent practical applications.

[0037] Step S130: For the undersampled data acquired in real time, a stereo reconstructed image is obtained using a trained deep learning model.

[0038] After the model is trained, it can be used for actual stereo image reconstruction. That is, for the target, undersampled data is collected in real time, and the undersampled data is input into the trained deep learning model to obtain the corresponding fully sampled data, thereby obtaining a stereo reconstructed image. This stereo reconstructed image has higher clarity or higher quality and is at least a three-dimensional stereo image, such as three-dimensional or four-dimensional.

[0039] Step S140: Perform stereoscopic rendering on the stereoscopic reconstructed image and display it stereoscopically through holographic projection.

[0040] To achieve stereoscopic display of magnetic resonance images, the stereoscopic reconstructed images are further stereoscopically rendered and then displayed stereoscopically through holographic projection, thereby reproducing the real stereoscopic images more intuitively and clearly.

[0041] Accordingly, the present invention also provides an intelligent magnetic resonance holographic imaging system for implementing one or more aspects of the above-described methods. For example, see [link to documentation]. Figure 2 As shown, the system includes a data acquisition device 10, an image reconstruction device 20, and an image display device 30. The data acquisition device 10 acquires undersampled magnetic resonance data in real time through undersampled scanning and sends it to the image reconstruction device 20. The image reconstruction device 20 inputs the undersampled magnetic resonance data into a trained deep learning model and outputs a stereoscopic reconstructed image. The image display device 30 performs stereoscopic rendering on the stereoscopic reconstructed image and displays it in stereoscopic form through holographic projection. Each device involved can be implemented using general-purpose or dedicated hardware.

[0042] For example, see Figure 3 As shown, in practical applications, the signal acquisition computer transmits relevant settings such as the scan trajectory and sequence to the magnetic resonance imaging (MRI) instrument. The signal acquisition computer also collects raw undersampled K-space data from the MRI instrument and transmits it to a remote high-performance server via TCP / IP. The server then uses a GPU to quickly reconstruct high-quality stereo MRI images based on a trained deep learning model.

[0043] Next, holographic projection is used to achieve real-time stereoscopic display of magnetic resonance images. For example, after completing high-quality image reconstruction of undersampled magnetic resonance data, a high-performance server transmits the reconstructed magnetic resonance image data to the holographic projection device via TCP / IP protocol. The holographic projection device then renders and displays the transmitted stereoscopic image.

[0044] Taking a holographic projection device as an example of an image display device, during application, a doctor wears the holographic projection device in advance. Denote the signal acquisition time as t, and denote the time difference between the magnetic resonance signal being transmitted from the signal acquisition computer and the high-performance server to the holographic projection device and being stereoscopically displayed as t'. On the premise that t meets the requirement of real-time capturing of tissue changes (such as heart beating), as long as t' < t, the stereoscopic image can be projected and displayed before the second signal acquisition is completed. Through this method, real-time and stereoscopic magnetic resonance image display can be achieved, which can be used to assist doctors in surgical intervention.

[0045] It should be noted that the model training process involved in the present invention can be carried out offline on a server or in the cloud. Embedding the trained model into an electronic device can achieve real-time magnetic resonance holographic imaging. The electronic device can be a terminal device or a server. The terminal device includes any terminal device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sale (POS), an in-vehicle computer, a smart wearable device, etc. The server includes but is not limited to an application server or a web server, and can be an independent server, a cluster server or a cloud server, etc.

[0046] In summary, magnetic resonance imaging is an important tool for clinical imaging and scientific research. However, traditional imaging generally only displays in a two-dimensional plane, which is not direct enough for spatial precise positioning and guidance of surgical intervention. The intelligent magnetic resonance holographic imaging method proposed in the present invention can achieve fast magnetic resonance imaging and real-time stereoscopic magnetic resonance image display, and can better assist doctors in surgical intervention treatment in clinical practice. On the one hand, the proposed self-supervised magnetic resonance image reconstruction deep learning model training method based on secondary under-sampling and pseudo-label making can avoid the need for fully sampled label data in the training of conventional deep learning models and reduce the cost of data production. On the other hand, by using holographic projection technology for real-time display of magnetic resonance images, the limitation of traditional two-dimensional plane display is broken, and the signal loss caused by the currently commonly used two-dimensional plane image display method is effectively reduced. Doctors can observe the tissues in the imaging area more comprehensively and intuitively, so as to better assist doctors in surgical planning and implementation. In short, the present invention proposes a new intelligent magnetic resonance holographic imaging technology, which realizes fast magnetic resonance scanning and image reconstruction based on artificial intelligence algorithms, and at the same time combines advanced holographic projection technology to project and display the imaging results in real-time and stereoscopically. This can not only provide doctors with richer high-dimensional image information, but also provide a platform for more important human-computer interactive imaging.

[0047] The present invention can be a system, a method and / or a computer program product. The computer program product can include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0048] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0049] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0050] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0051] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0052] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0053] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0054] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0055] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. An intelligent magnetic resonance holographic imaging system, comprising: Data acquisition equipment, image reconstruction equipment, and image display equipment, among which: The data acquisition equipment is used to acquire undersampled magnetic resonance data in real time through undersampled scanning and send it to the image reconstruction equipment; The image reconstruction device is used to input the undersampled magnetic resonance data into a deep learning model and output a stereo reconstructed image, wherein the stereo reconstructed image is not less than 3-dimensional; The image display device is used to perform stereoscopic rendering of the stereoscopic reconstructed image and to display it stereoscopically via holographic projection; The deep learning model is obtained through self-supervised training by taking the undersampled magnetic resonance data of the sample as input and the corresponding fully sampled magnetic resonance data as output. The deep learning model is trained according to the following steps: The designed undersampled scanning trajectory is used to perform magnetic resonance scanning on the target to obtain the original undersampled magnetic resonance data; The original undersampled magnetic resonance data is subjected to secondary undersampling, and the secondary undersampled magnetic resonance data and the original undersampled magnetic resonance data are combined to form a data pair, which is used for training the deep learning model. After completing one round of training, the raw undersampled magnetic resonance data is input into the trained deep learning model to generate pseudo-full sampled data labels, and these pseudo-full sampled data labels are introduced into the next round of training of the deep learning model until the set model optimization criteria are met.

2. The system according to claim 1, characterized in that, The data acquisition device is a signal acquisition computer, the image reconstruction device is a server, and the image display device is a holographic projection device.

3. The system according to claim 1, characterized in that, The time required for the data acquisition device to acquire data once is less than the time difference between acquiring the data and displaying it in 3D.

4. The system according to claim 1, characterized in that, The undersampled scan trajectory includes Cartesian space scan, radial scan, or spiral scan.

5. The system according to claim 1, characterized in that, The deep learning model mentioned is a convolutional neural network model.

6. A smart magnetic resonance holographic imaging method, comprising the following steps: Undersampled magnetic resonance data is acquired in real time through undersampled scanning and input into a deep learning model to obtain a stereo reconstructed image, wherein the stereo reconstructed image is not less than 3-dimensional. The reconstructed stereoscopic image is rendered in stereoscopic form and then displayed in stereoscopic form via holographic projection. The deep learning model is obtained through self-supervised training by taking the undersampled magnetic resonance data of the sample as input and the corresponding fully sampled magnetic resonance data as output. The deep learning model is trained according to the following steps: The designed undersampled scanning trajectory is used to perform magnetic resonance scanning on the target to obtain the original undersampled magnetic resonance data; The original undersampled magnetic resonance data is subjected to secondary undersampling, and the secondary undersampled magnetic resonance data and the original undersampled magnetic resonance data are used to form a data pair for training the deep learning model. After completing one round of training, the raw undersampled magnetic resonance data is input into the trained deep learning model to generate pseudo-full sampled data labels, and these pseudo-full sampled data labels are introduced into the next round of training of the deep learning model until the set model optimization criteria are met.

7. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the steps of the method according to claim 6.

8. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 6.

Images