Image reconstruction method and system

Abstract

The embodiment of the invention provides an image reconstruction method and system. The method comprises the steps that a first medical image of a target object is acquired; generating N model inputs based on the first medical image, N being a positive integer greater than 1, each model input comprising at least one input image, each input image being the first medical image or a part of the first medical image; and processing the plurality of model inputs using an image processing model to generate a second medical image, the second medical image having higher image quality than the first medical image, the image processing model being a trained machine learning model.

Description

Technical Field

[0001] This specification relates to the field of medical imaging technology, and in particular to an image reconstruction method and system. Background Art

[0002] Magnetic resonance imaging (MRI) is widely used in medical diagnosis and treatment. To improve the efficiency of MRI, undersampling can be used to collect undersampled data and generate MRI images based on the undersampled data. However, the MRI images obtained in this way often suffer from low quality and require a long time to reconstruct.

[0003] Therefore, it is necessary to provide an image reconstruction method and system that can generate high-quality MRI images based on MRI under-sampled data. Summary of the Invention

[0004] One or more embodiments of the present specification provide an image reconstruction method, comprising: acquiring a first medical image of a target object; generating N model inputs based on the first medical image, where N is a positive integer greater than 1, and each model input includes at least one input image, and each of the input images is the first medical image or a portion of the first medical image; processing the multiple model inputs using an image processing model to generate a second medical image, where the second medical image has higher image quality than the first medical image, and the image processing model is a trained machine learning model.

[0005] In some embodiments, the image processing model includes N processing layers connected in sequence, the N processing layers correspond one-to-one to the N model inputs, and each of the processing layers is used to process its corresponding model input.

[0006] In some embodiments, among the N processing layers, the number of input images corresponding to the i-th processing layer is greater than the number of input images corresponding to the (i+1)-th processing layer, and i is a positive integer greater than 0 and less than N.

[0007] In some embodiments, an input to the Nth processing layer of the N processing layers includes at least the first medical image.

[0008] In some embodiments, each processing layer in the 1st to (N-1)th processing layers includes: a first attention mechanism module; an encoder-decoder module connected to the first attention mechanism module; and a second attention mechanism module connected to the encoder-decoder module.

[0009] In some embodiments, among the N processing layers, the output of the second attention mechanism module of the i-th processing layer is input into the encoder-decoder module of the (i+1)-th processing layer, where i is a positive integer greater than 0 and less than N.

[0010] In some embodiments, among the N processing layers, the output of the encoder-decoder module of the i-th processing layer is input to the encoder-decoder module of the (i+1)-th processing layer.

[0011] In some embodiments, the first medical image is a magnetic resonance under-acquisition image, and the second medical image is a simulated magnetic resonance full-acquisition image.

[0012] In some embodiments, the use of the image processing model to process the multiple model inputs to generate a second medical image includes: using the image processing model to process the multiple model inputs to obtain multiple output images; determining weights of the multiple output images based on target image quality parameters; and generating the second medical image based on the multiple output images and their weights.

[0013] One or more embodiments of the present specification provide an image reconstruction system, comprising: an acquisition module configured to acquire a first medical image of a target object; a first generation module configured to generate N model inputs based on the first medical image, where N is a positive integer greater than 1, and each model input includes at least one input image, and each of the input images is the first medical image or a portion of the first medical image; a second generation module configured to process the multiple model inputs using an image processing model to generate a second medical image, where the second medical image has higher image quality than the first medical image, and the image processing model is a trained machine learning model.

[0014] According to the above scheme, by combining a multi-scale, multi-stage neural network with a full-resolution path, both macroscopic and microscopic details of the image are fully captured, resulting in high-quality medical images and shortening image reconstruction time. This method can be applied to the field of MRI to generate high-quality fully acquired images from under-acquired images, thereby improving the quality of MRI image reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] This specification will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numbers represent like structures, wherein:

[0016] Figure 1 is a schematic diagram of an application scenario of an image reconstruction system according to some embodiments of this specification;

[0017] Figure 2 is an exemplary module diagram of an image reconstruction system according to some embodiments of this specification;

[0018] Figure 3 is an exemplary flow chart of an image reconstruction method according to some embodiments of this specification;

[0019] Figure 4 is an exemplary schematic diagram of an image processing model according to some embodiments of this specification;

[0020] Figure 5 is an exemplary schematic diagram of generating a second medical image using an image processing model according to some embodiments of this specification;

[0021] Figure 6 is another exemplary schematic diagram of the image processing model shown in some embodiments of this specification. DETAILED DESCRIPTION

[0022] To more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly describes the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of this specification. Those skilled in the art can apply this specification to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0023] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0024] As used in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not refer to the singular but also include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0025] Flowcharts are used throughout this specification to illustrate the operations performed by systems according to embodiments of this specification. It should be understood that preceding or following operations do not necessarily need to be performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0026] Figure 1 Schematic diagram of an application scenario of an image reconstruction system according to some embodiments of this specification. Figure 1 As shown, the application scenario 100 of the image reconstruction system includes an imaging device 110 , a processor 120 , a network 130 , a storage device 140 and a terminal 150 .

[0027] The imaging device 110 can be used to scan the target object and generate a first medical image. For example, the imaging device 110 may include a magnetic resonance imaging (MRI) scanner, a computed tomography (CT) scanner, a positron emission tomography (PET) scanner, etc. The target object may include a biological object (e.g., a human body, an animal, etc.), a non-biological object (e.g., a phantom), etc. In other embodiments, the target object may also include a specific part, organ, and / or tissue of a patient. For example, the target object may include the head, chest, legs, etc., or any combination thereof, without limitation. In some embodiments, the target object may include a specific part, organ, and / or tissue of a patient and other organs and / or tissues within a certain range around it. The target object may also be referred to as a patient hereinafter.

[0028] The processor 120 may be configured to process data and / or information obtained from the imaging device 110, the storage device 140, and / or the terminal 150. For example, the processor 120 may obtain a first medical image of a target object from the imaging device 110. In another example, the processor 120 may generate N model inputs based on the first medical image, where N is a positive integer greater than 1. In another example, the processor 120 may process the multiple model inputs using an image processing model to generate a second medical image.

[0029] In some embodiments, the processor 120 may send the processing result to the terminal 150. For example, the processor 120 may send the second medical image to the terminal 150 and display it on one or more display devices in the terminal 150.

[0030] In some embodiments, the processor 120 can be a single server or a server group. The server group can be centralized or distributed. In some embodiments, the processor 120 can be local or remote. In some embodiments, the processor 120 can be connected to the imaging device 110, the storage device 140 and / or the terminal 150 via the network 130 or directly to access the information and / or data stored thereon. In some embodiments, the processor 120 can be integrated into the imaging device 110. In some embodiments, the processor 120 can be implemented on a cloud platform. By way of example only, the cloud platform can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, or the like, or any combination thereof.

[0031] The network 130 may include any suitable network that can facilitate information and / or data exchange within the image reconstruction system. In some embodiments, one or more components of the image reconstruction system (e.g., the imaging device 110, the processor 120, the storage device 140, or the terminal 150) may be connected and / or communicate with other components of the image reconstruction system via the network 130. For example, the processor 120 may acquire a first medical image of a target object from the imaging device 110 via the network 130.

[0032] In some embodiments, the network 130 can be any one or more of a wired network or a wireless network. For example, the network 130 can include a cable network, a fiber optic network, a telecommunications network, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, near field communication (NFC), an intra-device bus, an intra-device line, a cable connection, or any combination thereof. The network connection between the various components can adopt one of the above methods or multiple methods. In some embodiments, the network 130 can be a point-to-point, shared, centralized, or other topological structure or a combination of multiple topological structures. In some embodiments, the network 130 can include one or more network access points. For example, the network 130 can include wired and / or wireless network access points, such as base stations and / or Internet access points, through which one or more components of the image reconstruction system can connect to the network 130 to exchange data and / or information.

[0033] The storage device 140 may store data, instructions, and / or any other information. In some embodiments, the storage device 140 may store data obtained from the imaging device 110, the processor 120, and / or the terminal 150. In some embodiments, the storage device 140 may store data and / or instructions used by the processor 120 to execute or use the exemplary methods described herein.

[0034] In some embodiments, the storage device 140 may include a mass storage device, a removable storage device, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. In some embodiments, the storage device 140 may be implemented on a cloud platform.

[0035] In some embodiments, the storage device 140 may be connected to the network 130 to communicate with one or more components of the image reconstruction system (e.g., the imaging device 110, the processor 120, the terminal 150, etc.). One or more components of the image reconstruction system may access data or instructions stored in the storage device 140 via the network 130. In some embodiments, the storage device 140 may be part of the processor 120.

[0036] The terminal 150 can communicate and / or connect with the imaging device 110, the processor 120, and / or the storage device 140. In some embodiments, the terminal 150 may include one or any combination of a mobile device 150-1, a tablet computer 150-2, a laptop computer 150-3, or other devices with input and / or output functions. In some embodiments, the terminal 150 can remotely operate the imaging device 110. In some embodiments, the terminal 150 can operate the imaging device 110 via a wireless connection. In some embodiments, the terminal 150 can receive data and / or information from the processor 120. For example, the terminal 150 can receive a second medical image from the processor 120. In some embodiments, the terminal 150 can be part of the processor 120. In some embodiments, the terminal 150 can be integrated with the processor 120 to serve as an operating console for the imaging device 110. In some embodiments, the terminal 150 can be omitted.

[0037] It should be noted that the application scenario 100 of the image reconstruction system is provided for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art will readily appreciate that various modifications and variations can be made based on the description of this specification. For example, the application scenario 100 of the image reconstruction system can be implemented on other devices to achieve similar or different functions. However, such variations and modifications do not deviate from the scope of this specification.

[0038] Figure 2 is an exemplary module diagram of an image reconstruction system according to some embodiments of this specification. In some embodiments, the image reconstruction system 200 includes an acquisition module 210, a first generation module 220 and a second generation module 230. In some embodiments, the image reconstruction system 200 may be composed of Figure 1 The processor 120 shown implements .

[0039] The acquisition module 210 is configured to acquire a first medical image of a target object.

[0040] The first generation module 220 is configured to generate N model inputs based on the first medical image, where N is a positive integer greater than 1, and each model input includes at least one input image, and each input image is the first medical image or a portion of the first medical image.

[0041] The second generation module 230 is configured to process the plurality of model inputs using an image processing model to generate a second medical image, wherein the second medical image has a higher image quality than the first medical image, and the image processing model is a trained machine learning model.

[0042] In some embodiments, the second generation module 230 is further configured to process multiple model inputs using an image processing model to obtain multiple output images; determine the weights of the multiple output images based on the target image quality parameters; and generate a second medical image based on the multiple output images and their weights.

[0043] For more information about the acquisition module and the first generation module, see Figure 3 For more information about the second generation module, see Figure 3 and Figure 5 and its related descriptions.

[0044] It should be noted that the above description of the image reconstruction system 200 and its modules is for convenience only and does not limit this specification to the scope of the embodiments. It is understood that after understanding the principles of the system, those skilled in the art may arbitrarily combine the modules or form subsystems connected to other modules without departing from the principles. In some embodiments, Figure 2 The acquisition module 210, first generation module 220, and second generation module 230 disclosed in the disclosure may be different modules in a system, or a single module may implement the functions of two or more of the aforementioned modules. For example, the modules may share a storage module, or each module may have its own storage module. In some embodiments, the image reconstruction system 200 further includes a training module for training the image processing model and / or the weight determination model. Such variations are within the scope of protection of this specification.

[0045] Figure 3is an exemplary flow chart of an image reconstruction method according to some embodiments of the present specification. In some embodiments, process 300 can be executed by processor 120 or image reconstruction system 200. For example, process 300 can be stored in the form of a program or instructions in storage device 140, and when processor 120 or image reconstruction system 200 executes the instructions, process 300 can be implemented. The operational diagram of process 300 presented below is illustrative. In some embodiments, the process can be completed using one or more additional operations not described and / or one or more operations not discussed. In addition, Figure 3 The order in which the operations of flow 300 are illustrated and described below is not intended to be limiting.

[0046] Step 310: Acquire a first medical image of the target object.

[0047] The target object is the object to be scanned and imaged. For example, the target object may include a patient or a specific part, organ, and / or tissue of the patient. For more information about target objects, see Figure 1 and its related descriptions.

[0048] The first medical image refers to a medical image that needs to be processed. The first medical image may include various types of medical images. For example, the first medical image may include an MRI scan image, a CT scan image, a PET scan image, etc.

[0049] In some embodiments, the processor may scan the target object using an imaging device and reconstruct based on the scan data to obtain a first medical image of the target object. As an example only, the first medical image may be an undersampled magnetic resonance image obtained by undersampling the target object using an MRI device.

[0050] Step 320: Generate N model inputs based on the first medical image, where N is a positive integer greater than 1.

[0051] Model input refers to data that needs to be input into the image processing model for processing. Each model input includes at least one input image. An input image refers to image data that needs to be input into the image processing model for processing. In some embodiments, each input image is a first medical image or a portion of a first medical image. For example, the processor may directly determine the first medical image as an input image. For another example, the processor may segment or block the first medical image and determine each resulting image as an input image.

[0052] In some embodiments, the processor determines the first medical image as an input image and uses the input image as a model input, that is, the model input includes a first medical image. In some embodiments, the processor can perform multiple segmentation or block processing on the first medical image, and determine the multiple input images obtained after each segmentation or block processing as a model input. For example, the processor divides the first medical image into two blocks to obtain two input images, and uses these two input images as a model input. For another example, the processor divides the first medical image into four blocks to obtain four input images, and uses these four input images as a model input. It should be understood that the processor can divide the first medical image into any number of input images and use these input images as a model input.

[0053] In some embodiments, the N model inputs contain different numbers of input images. In some embodiments, the N model inputs need to be simultaneously input into the image processing model for processing. Each model input is separately input into a different layer of the image processing model for processing. In some embodiments, some model inputs may be input into the same layer of the image processing model for processing.

[0054] Step 330 : Process the plurality of model inputs using the image processing model to generate a second medical image.

[0055] Image processing models are used to process input images to generate images with higher quality. Image processing models are trained machine learning models.

[0056] The second medical image refers to a simulated medical image obtained after being processed by the image processing model. The second medical image has higher image quality than the first medical image. Image quality can be measured using various image quality parameters. In some embodiments, image quality can be determined based on at least one of image resolution, signal-to-noise ratio, and sampling rate. For example, a higher image resolution of a medical image corresponds to higher image quality. For another example, a higher signal-to-noise ratio of a medical image corresponds to higher image quality. For another example, a higher sampling rate of a medical image corresponds to higher image quality.

[0057] As just an example, the first medical image may be an undersampled MRI image, while the second medical image may be a simulated fully sampled MRI image, where the second medical image has a higher sampling rate. For another example, the first medical image may be a low-resolution image, while the second medical image may be a simulated high-resolution image. For another example, the first medical image may be an X-ray image (e.g., a CT image) acquired with a low dose and short scan time, while the second medical image may be a simulated high-dose and long scan time X-ray image.

[0058] In some embodiments, the image processing model includes N processing layers connected in sequence, where the N processing layers correspond to N model inputs one by one, and each processing layer is used to process its corresponding model input. Figure 4 As shown, the first processing layer processes the first model input 410-1, the second processing layer processes the second model input 410-2, ..., and the Nth processing layer processes the Nth model input 410-N. It will be appreciated that the number of each processing layer is determined based on the order in which the data is processed by that processing layer. The earlier the processing layer processes data, the earlier the number corresponding to that processing layer. For example, the first processing layer processes data first, and the Nth processing layer processes data last.

[0059] In some embodiments, among the N processing layers, the number of input images corresponding to the i-th processing layer is greater than the number of input images corresponding to the (i+1)-th processing layer, where i is a positive integer greater than 0 and less than N. For example only, Figure 4 As shown, the number of input images corresponding to the first processing layer is greater than the number of input images corresponding to the second processing layer.

[0060] In some embodiments, as Figure 6 As shown, the image processing model can include three processing layers. The number of input images corresponding to the first, second, and third processing layers decreases in sequence. For example, the number of input images corresponding to the first processing layer is 4, the number of input images corresponding to the second processing layer is 2, and the number of input images corresponding to the third processing layer is 1.

[0061] In some embodiments, the Nth processing layer includes an original resolution subnet, and the original resolution subnet includes a plurality of original resolution blocks. Figure 4 As shown, the Nth processing layer includes an original resolution sub-network 450. The original resolution sub-network 450 includes a plurality of original resolution blocks 450-1, 450-2, ..., 450-n.

[0062] In some embodiments, each original resolution block includes multiple channel attention mechanism modules (CAB) and convolutional layers. The original resolution block is used to extract and optimize features. Since the CAB in the original resolution block does not use pooling, its output is a feature map of the original resolution, which can avoid affecting the resolution of the image and retain most of the details in the image. For multiple original resolution blocks in the original resolution subnet, the subsequent original resolution block will improve and add deeper features to the original resolution feature map output by the previous original resolution block, and enhance the features in the feature map.

[0063] In some embodiments, the input of the Nth processing layer in the N processing layers includes at least the first medical image. Figure 4 As shown, the Nth model input 410-N input to the Nth processing layer is a first medical image.

[0064] In some embodiments of the present specification, the Nth processing layer of the image processing model uses the original resolution subnet to perform convolution operations of the same scale on the first medical image, thereby avoiding data loss caused by image compression, retaining more details in the reconstructed second medical image, and obtaining high-quality images that can capture subtle features, providing data support for accurate medical diagnosis.

[0065] In some embodiments, each of the 1st to (N-1)th processing layers includes: a first attention mechanism module; an encoder-decoder module connected to the first attention mechanism module; and a second attention mechanism module connected to the encoder-decoder module. For example only, Figure 4 As shown, the first processing layer includes a first attention mechanism module 420-1, an encoder-decoder module 430-1 and a second attention mechanism module 440-1 connected in sequence.

[0066] In some embodiments, each of the first to Nth processing layers further includes a feature extraction module (not shown in the figure), which can be used to extract features from each input image corresponding to the processing layer. The feature extraction module is connected to the first attention mechanism module, and the output of the feature extraction module is input into the first attention mechanism module.

[0067] The first attention mechanism module is used to further process the output of the feature extraction module.

[0068] In some embodiments, the first attention mechanism module is a channel attention block (CAB), which includes a convolutional layer and a channel attention layer, where the convolutional layer is used for feature conversion and the channel attention layer is used for adaptively recalibrating channel feature responses.

[0069] In some embodiments, the first attention mechanism module can be used to perform the following operations: feature conversion, converting and refining the input features through the convolution layer; global average pooling, performing global average pooling on the feature map through the channel attention layer, reducing the spatial dimension to 1x1, and highlighting the global distribution of inter-channel features by generating a channel descriptor; channel attention, the channel descriptor generates a set of channel attention values ​​through a multilayer perceptron (MLP); activation, applying an activation function (e.g., a sigmoid activation function) to the channel attention value to obtain a set of channel attention weights between 0 and 1; feature recalibration, recalibrating the original feature map using the channel attention weights, effectively allowing the network to emphasize more informative features and suppress less useful features.

[0070] The first attention mechanism module enables the network to adaptively adjust the weight of each channel according to the input, making the network more focused and efficient in feature representation.

[0071] The encoder-decoder module is used to further process the output of the first attention module. For example, the encoder-decoder module may include a U-Net model, a transformer model, etc.

[0072] The encoder-decoder module includes an encoder and a decoder. For example, Figure 4 As shown, the black portion on the left side of the encoder-decoder module 430-1 is the encoder (also known as the downsampling channel). The encoder is used to compress the input into feature maps, reducing the input resolution during the downsampling process. The number of feature maps is related to the number of input images at each layer. For example, if the first processing layer corresponds to b input images, the encoder also outputs b feature maps. The white portion on the right side of the encoder-decoder module 430-1 is the decoder (also known as the upsampling channel). The decoder is used to merge the feature maps to obtain the output.

[0073] The second attention mechanism module is used to process the output of the encoder-decoder module and the original input image.

[0074] In some embodiments, the second attention mechanism module is a supervised attention mechanism module (SAM). The second attention mechanism module includes a series of convolutional layers and includes a mechanism to fuse the features of the current stage with the features of the input image.

[0075] In some embodiments, the input to the second attention mechanism module includes a feature map and an input image. The feature map is the feature map output by the encoder-decoder module. The output of the second attention mechanism module includes a refined feature map and a restored image. The refined feature map is the result of element-wise multiplication of the refined feature map and the attention map, and its image size is the same as the image size of the input feature map. The restored image is an image obtained by adding the convolutional layer output to the input image, and its image size is the same as the image size of the first medical image.

[0076] In some embodiments, the second attention mechanism module can be used to perform the following operations: feature refinement, performing preliminary refinement of the input feature map through a convolution layer; image restoration, generating a restored image through a convolution layer to enhance image details and texture; attention generation, generating an attention map based on an activation function (e.g., a sigmoid activation function) through another convolution layer, whose value is between 0 and 1; feature fusion, multiplying the refined feature map and the attention map element-by-element to obtain a refined feature map, so that the network can focus on more relevant areas in the feature space; output, outputting the refined feature map and the restored image.

[0077] The second attention mechanism module effectively guides the network to focus on more important areas in the image, enhancing the quality of restoration. The restored images generated at each stage also provide a form of intermediate supervision, contributing to better gradient flow and training stability.

[0078] In some embodiments of the present specification, the feature representation capability is enhanced by setting a first attention mechanism module to emphasize information-rich channels, and a second attention mechanism module is set to refine features in a supervised manner and generate intermediate restored images, thereby guiding the network to achieve better performance in image restoration tasks.

[0079] In some embodiments, among the N processing layers, the output of the second attention mechanism module of the i-th processing layer is input to the encoder-decoder module of the (i+1)-th processing layer, where i is a positive integer greater than 0 and less than N. For example only, Figure 4 As shown, the output of the second attention mechanism module 440-1 of the first processing layer (e.g., the refined feature map) is input into the encoder-decoder module 430-2 of the second processing layer.

[0080] In some embodiments, the output of the second attention mechanism module of the i-th processing layer needs to be processed before being input into the encoder-decoder module of the (i+1)-th processing layer. For example, the output of the second attention mechanism module of the i-th processing layer needs to be scaled to adapt to the encoder-decoder module of the (i+1)-th processing layer. For another example, the output of the second attention mechanism module of the i-th processing layer and the output of the first attention mechanism module of the (i+1)-th processing layer are spliced ​​(for example, the number of channels is multiplied by 2) by a splicing module (not shown in the figure) and then input into the encoder-decoder module of the (i+1)-th processing layer. Just as an example, Figure 4 As shown, the output of the second attention mechanism module 440-1 of the first processing layer and the output of the first attention mechanism module 420-2 of the second processing layer are spliced ​​and input into the encoder-decoder module 430-2 of the second processing layer.

[0081] In some embodiments, among the N processing layers, the output of the encoder-decoder module of the i-th processing layer is input to the encoder-decoder module of the (i+1)-th processing layer. For example only. Figure 4 As shown, the output of the encoder-decoder module 430-1 of the first processing layer (including the output of the encoder and / or the output of the decoder) is input to the encoder-decoder module 430-2 of the second processing layer.

[0082] In some embodiments of the present specification, by using the intermediate output data of the next processing layer as the intermediate input of the previous processing layer, different features of the aliased image can be learned at different levels, ensuring that macroscopic and microscopic details are captured, thereby performing comprehensive and detailed image reconstruction.

[0083] In some embodiments, as Figure 6As shown, the image processing model includes three processing layers. The first processing layer corresponds to four input images. The format of each input image is (batch size, 1, height / 2, width / 2). The output refined feature map and restored image format are (batch size, 1, height / 2, width). After the input image is processed by the feature extraction module (not shown in the figure), the output image format is (batch size, 1, height / 2, width / 2). After inputting it into the first attention mechanism module 420-1, the output image format is (batch size, number of feature maps, height / 2, width / 2). After being processed by the encoder-decoder module 430-1, the feature map format is (batch size, number of feature maps, height / 2, width / 2). Finally, it is input together with the input image into the second attention mechanism module 440-1 for processing to obtain the refined feature map and restored image. The second processing layer corresponds to two input images, each of which has the format of (batch size, 1, height / 2, width). The output of the refined feature map and restored image has the format of (batch size, 3, height, width). The output of the encoder-decoder module 430-1 of the first processing layer and the output of the second attention mechanism module 440-1 of the first processing layer are processed to obtain an image with the format of (batch size, number of feature maps, height / 2, width). This image is input into the encoder-decoder module 430-2 of the second processing layer, along with the output of the first attention mechanism module 420-2 of the second processing layer, to obtain a feature map with the format of (batch size, number of feature maps, height, width). This feature map is then input into the second attention mechanism module 440-2 along with the input image to obtain the refined feature map and restored image. The third processing layer corresponds to one input image with the format of (batch size, 1, height, width). The output of the refined feature map and restored image has the format of (batch size, 3, height, width). The output of the second attention mechanism module 440-2 of the second processing layer and the output of the encoder-decoder module 430-2 of the second processing layer are processed to obtain an image in the format of (batch size, number of feature maps, and number of feature maps of the original resolution subnet, height, width). This image is input together with the output of the first attention mechanism module 420-3 of the third processing layer into the original resolution subnet 610 of the third processing layer, and processed to obtain a refined feature map and a restored image. The processor can determine the refined feature map and restored image output by the third processing layer as the second medical image.

[0084] In some embodiments, the image processing model can be obtained by jointly training N initial processing layers. In some embodiments, the training samples of the image processing model include a first sample image and a second sample image, and the image quality of the second sample image is higher than that of the first sample image. The first sample image is the training input, and the second sample image is the training gold standard. For example, the second sample image is acquired under higher imaging conditions (such as high dose, long scan time, full acquisition scan mode), and the first sample image is acquired under lower imaging conditions (such as low dose, short scan time, under acquisition scan mode). Alternatively, the first sample image is obtained by processing the second sample image or its corresponding imaging data.

[0085] As an example, the training input for the image processing model includes sample undersampled images, and the training labels include sample fully sampled images. The image processing model is obtained by: obtaining fully sampled MRI data of a sample subject; generating undersampled MRI data based on the fully sampled data; using the sample fully sampled images corresponding to the fully sampled MRI data as the training gold standard, and the sample undersampled images corresponding to the undersampled MRI data as the training input, and then training the initial model to determine the image processing model.

[0086] In some embodiments, a full-acquisition magnetic resonance scan of a sample object is performed using an imaging device to obtain full-acquisition magnetic resonance data of the sample object. Based on the full-acquisition data, magnetic resonance under-acquisition data is generated using a preset sliding window. For example, if the full-acquisition magnetic resonance data includes 100 lines of full-acquisition data, the processor can select 25 consecutive lines of full-acquisition data using a preset sliding window of size 25 to obtain four times the magnetic resonance under-acquisition data. By moving the preset sliding window, multiple sets of magnetic resonance under-acquisition data can be obtained. Furthermore, the processor can perform image reconstruction based on the full-acquisition magnetic resonance data to obtain a sample full-acquisition image, which is determined as a training gold standard; and perform image reconstruction based on the under-acquisition magnetic resonance data to obtain a sample under-acquisition image, which is determined as a training input.

[0087] In some embodiments, the image processing model can be trained and generated by other processing devices (such as processing devices of model suppliers), and the processor can directly obtain the image processing model from other processing devices or storage devices.

[0088] In some embodiments, the processor may determine at least one image output by the image processing model as the second medical image. For example, the processor may determine the image output by the Nth processing layer of the image processing model as the second medical image. For another example, the processor may determine the image output by the 2nd processing layer of the image processing model as the second medical image.

[0089] In some embodiments, the processor may select one of the output images of the N processed layers as the second medical image based on the target image quality parameter. For example, the processor may select the output image whose image quality parameter is closest to the target image quality parameter as the second medical image. For more information about the target image quality parameter, see Figure 5 Related description.

[0090] In some embodiments, the processor may process multiple model inputs using an image processing model to obtain multiple output images, determine weights of the multiple output images, and generate a second medical image based on the multiple output images and their weights. For more information about the aforementioned embodiments, please refer to Figure 5 and its related descriptions.

[0091] In some embodiments, the first medical image is a sub-acquired MRI image. After processing the first medical image using process 300 of this specification, the generated second medical image is a simulated fully-acquired MRI image, that is, a simulated MRI image reconstructed from fully-acquired data. In some embodiments of this specification, processing the sub-acquired image using an image processing model to generate a simulated fully-acquired image can improve the efficiency of MRI scanning and produce high-quality MRI images.

[0092] In some embodiments, the first medical image is a low-resolution medical image, and after processing the first medical image through process 300 of this specification, the second medical image generated is a high-resolution medical image. In other embodiments, the first medical image is a low-dose and / or short-scan-time X-ray image, and after processing the first medical image through process 300 of this specification, the second medical image generated is a high-dose and / or long-scan-time X-ray image. In some embodiments of this specification, by subjecting a target object to low-dose and / or short-scan-time radiation, and then processing the image processing model to obtain a high-dose and / or long-scan-time X-ray image, radiation damage to the target object can be effectively reduced while improving imaging quality.

[0093] In some embodiments of this specification, a multi-scale, multi-stage neural network and a full-resolution image processing model are combined to ensure that both macroscopic and microscopic details of an image are fully captured, thereby reconstructing more image details and improving the image quality of the reconstructed medical images. Furthermore, the entire image reconstruction process does not involve iterative algorithms, shortening the time required for image reconstruction.

[0094] It should be noted that the above description of process 300 is for illustration and purpose only and does not limit the scope of application of this specification. Those skilled in the art may make various modifications and alterations to process 300 under the guidance of this specification. However, such modifications and alterations are still within the scope of this specification.

[0095] Figure 5 This is an exemplary schematic diagram of generating a second medical image using an image processing model according to some embodiments of this specification.

[0096] In some embodiments, as Figure 5 As shown, the processor can use the image processing model 520 to process multiple model inputs 510 to obtain multiple output images 530; determine the weights 550 of the multiple output images based on the target image quality parameter 540; and generate a second medical image 560 based on the multiple output images 530 and their weights 550.

[0097] Target image quality parameters refer to the desired image quality parameters of medical images. Image quality parameters are parameters used to reflect image quality. For example, image quality parameters include image resolution, signal-to-noise ratio, sampling rate, etc. In some embodiments, target image quality parameters can be default values, preset values, user-entered values, etc.

[0098] The weight is used to reflect the importance of each output image.

[0099] In some embodiments, for each output image, the processor may determine an image quality parameter corresponding to the output image, calculate the difference between the image quality parameter and the target image quality parameter, and determine a weight corresponding to the output image based on the difference. The smaller the difference between the image quality parameter corresponding to the output image and the target image quality parameter, the greater the weight corresponding to the output image.

[0100] In some embodiments, the processor may process the plurality of output images and the target image quality parameter using a weight determination model to determine weights of the plurality of output images.

[0101] The weight determination model may be a machine learning model for determining the weight corresponding to the output image. For example, the weight determination model may be a deep neural network (DNN) model.

[0102] In some embodiments, the input of the weight determination model includes an output image and a target image quality parameter, and the output includes a weight corresponding to the output image.

[0103] In some embodiments, the weight determination model can be trained using multiple image training samples with weight labels. For example, multiple image training samples can be input into an initial weight determination model, and the value of a loss function can be determined using the weight labels and the output of the initial weight determination model. The parameters of the initial weight determination model are then iteratively updated based on the loss function. When pre-set iteration conditions are met, model training is complete, resulting in a trained weight determination model. The pre-set iteration conditions may include convergence of the loss function, a threshold number of iterations, and the like.

[0104] In some embodiments, the image training samples may include multiple sample output images and sample target image quality parameters of the sample object, and the weight labels may include gold standard weights corresponding to the sample output images. In some embodiments, the image training samples may be obtained by: obtaining multiple sample images of the sample object under different imaging conditions; determining one of the sample images as the sample target image, and determining the image quality parameters corresponding to the sample target image as the sample target image quality parameters; and determining the other sample images as the sample output images. The weight labels may be obtained by: determining multiple weight sets by a brute force search method or other method, each weight set including weights of multiple sample output images; performing weighted summation on the sample output images based on the weights in each weight set to obtain a predicted image. If the predicted image corresponding to a certain sample weight set is closest to the sample target image, the sample weight set is used as the weight label.

[0105] In some embodiments of the present specification, the output image and target image quality parameters are processed through a weight determination model to determine the weight corresponding to the output image. This can simultaneously consider the influence of multiple factors, making the determination of the weight corresponding to the output image more efficient and accurate, and avoiding errors caused by manual determination.

[0106] In some embodiments, the processor may perform weighted summation on each pixel in the image based on the multiple output images and their weights to generate a second medical image.

[0107] In some embodiments of the present specification, post-processing of multiple output images in combination with target image quality parameters can obtain a second medical image that better meets the desired requirements.

[0108] While the basic concepts have been described above, it will be apparent to those skilled in the art that the detailed disclosure is merely illustrative and does not limit this specification. Although not explicitly stated herein, various modifications, improvements, and revisions to this specification may be made by those skilled in the art. Such modifications, improvements, and revisions are suggested in this specification and remain within the spirit and scope of the exemplary embodiments of this specification.

[0109] This specification also uses specific terms to describe the embodiments of this specification. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "one embodiment," "an embodiment," or "an alternative embodiment" two or more times in different locations in this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of this specification may be appropriately combined.

[0110] In addition, unless expressly stated in the claims, the order of the processing elements and sequences, the use of alphanumeric characters, or the use of other names described in this specification are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some of the invention embodiments currently considered useful through various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the spirit and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0111] Similarly, it should be noted that, in order to simplify the presentation of this specification and thus facilitate understanding of one or more embodiments of the invention, the foregoing descriptions of the embodiments of this specification sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not imply that the subject matter of this specification requires more features than those recited in the claims. In fact, an embodiment may have fewer features than all of the features of a single disclosed embodiment.

[0112] In some embodiments, numbers are used to describe the quantity of components and attributes. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise stated, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the description and claims are approximate values, which may change according to the required characteristics of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the settings of such numerical values ​​are as accurate as possible within the feasible range.

[0113] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, and documents, cited in this specification is hereby incorporated by reference in its entirety. This includes application history documents that are inconsistent with or conflict with the content of this specification, as well as documents (currently or subsequently attached to this specification) that limit the broadest scope of the claims of this specification. It should be noted that if the descriptions, definitions, and / or terminology used in the accompanying materials are inconsistent or conflicting with the content of this specification, the descriptions, definitions, and / or terminology used in this specification will control.

[0114] Finally, it should be understood that the embodiments described in this specification are intended only to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly described and illustrated in this specification.

Claims

1. An image reconstruction method, characterized in that: The method comprises: acquiring a first medical image of a target object; Based on the first medical image, generating N model inputs, where N is a positive integer greater than 1, each model input includes at least one input image, and each input image is the first medical image or a portion of the first medical image; The multiple model inputs are processed using an image processing model to generate a second medical image, where the second medical image has higher image quality than the first medical image, and the image processing model is a trained machine learning model.

2. The method according to claim 1, characterized in that The image processing model includes N processing layers connected in sequence, the N processing layers correspond one-to-one to the N model inputs, and each processing layer is used to process its corresponding model input.

3. The method according to claim 2, characterized in that In the N processing layers, the number of input images corresponding to the i-th processing layer is greater than the number of input images corresponding to the (i+1)-th processing layer, and i is a positive integer greater than 0 and less than N.

4. The method according to claim 3, characterized in that An input of the Nth processing layer among the N processing layers includes at least the first medical image.

5. The method according to claim 2, characterized in that Each of the 1st to (N-1)th processing layers includes: First attention mechanism module; an encoder-decoder module connected to the first attention mechanism module; A second attention mechanism module connected to the encoder-decoder module.

6. The method according to claim 5, characterized in that Among the N processing layers, the output of the second attention mechanism module of the i-th processing layer is input into the encoder-decoder module of the (i+1)-th processing layer, where i is a positive integer greater than 0 and less than N.

7. The method according to claim 6, characterized in that Among the N processing layers, the output of the encoder-decoder module of the i-th processing layer is input to the encoder-decoder module of the (i+1)-th processing layer.

8. The method according to claim 1, characterized in that The first medical image is a magnetic resonance under-acquisition image, and the second medical image is a simulated magnetic resonance full-acquisition image.

9. The method according to claim 1, characterized in that The processing of the plurality of model inputs by using an image processing model to generate a second medical image comprises: Processing the plurality of model inputs using the image processing model to obtain a plurality of output images; determining weights of the plurality of output images based on a target image quality parameter; The second medical image is generated based on the multiple output images and their weights.

10. An image reconstruction system, characterized in that: The system comprises: an acquisition module configured to acquire a first medical image of a target object; a first generating module configured to generate N model inputs based on the first medical image, where N is a positive integer greater than 1, each model input including at least one input image, and each input image being the first medical image or a portion of the first medical image; The second generation module is configured to use an image processing model to process the multiple model inputs to generate a second medical image, wherein the second medical image has higher image quality than the first medical image, and the image processing model is a trained machine learning model.

Images