Image processing method and system

Abstract

The embodiment of the invention discloses an image processing method and system. The image processing method comprises the following steps: acquiring a plurality of channel images corresponding to a plurality of channel coils of the magnetic resonance system; inputting the plurality of channel images into an image processing model for processing to obtain a target image; wherein the processing process comprises the following steps: carrying out filtering merging processing based on a plurality of channel images, and obtaining a plurality of filtered channel images corresponding to a plurality of channel coils; performing channel merging processing based on the plurality of channel images to obtain a channel merged image; fusing the plurality of filtered channel images and the channel combined image to generate a multi-channel fused image; and obtaining the target image by using the multi-channel fusion image.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method and system. Background Art

[0002] Medical imaging refers to the use of a medium to interact with the human body in a non-invasive manner to obtain images of the target internal tissues and organs to assist doctors in diagnosing and treating diseases. The accuracy of diagnosis is closely related to the image quality of medical imaging. High-quality images can better assist doctors in diagnosing and treating diseases.

[0003] Therefore, it is hoped to propose an image processing method to obtain high-quality medical images. Summary of the invention

[0004] One aspect of the embodiments of the present specification provides an image processing method. The image processing method includes: obtaining multiple channel images corresponding to multiple channel coils of a magnetic resonance system; inputting the multiple channel images into an image processing model for processing to obtain a target image; wherein the processing process includes: performing filtering and merging processing based on the multiple channel images to obtain multiple filtered channel images corresponding to the multiple channel coils; performing channel merging processing based on the multiple channel images to obtain a channel merged image; fusing the multiple filtered channel images and the channel merged image to generate a multi-channel fused image; and obtaining the target image using the multi-channel fused image.

[0005] Another aspect of the embodiments of the present specification provides an image processing system. The system includes: a channel image acquisition module, which is used to acquire multiple channel images corresponding to multiple channel coils of a magnetic resonance system; a target image acquisition module, which is used to input multiple channel images into an image processing model for processing to acquire a target image; wherein the target image acquisition module is further used to: perform filtering and merging processing based on multiple channel images to acquire multiple filtered channel images corresponding to multiple channel coils; perform channel merging processing based on multiple channel images to acquire a channel merged image; fuse multiple filtered channel images and channel merged images to generate a multi-channel fused image; and obtain the target image using the multi-channel fused image.

[0006] Another aspect of an embodiment of the present specification provides an image processing model, which includes: a first neural network, used to perform filtering and merging processing on multiple channel images corresponding to multiple channel coils of a magnetic resonance system to obtain multiple filtered channel images corresponding to the channel coils; an image merging module, used to perform channel merging processing based on the multiple channel images to obtain a channel merged image; an image fusion module, used to fuse the multiple filtered channel images and the channel merged image to generate a multi-channel fused image; and a second neural network, used to obtain the target image using the multi-channel fused image.

[0007] Another aspect of the embodiments of the present specification provides an image processing device including at least one storage medium and at least one processor, wherein the at least one storage medium is used to store computer instructions; and the at least one processor is used to execute the computer instructions to implement the above-mentioned image processing method.

[0008] Another aspect of the embodiments of this specification provides a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the above-mentioned image processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0010] Figure 1 is a schematic diagram of an exemplary application scenario of an image processing system according to some embodiments of this specification;

[0011] Figure 2 is an exemplary flow chart of an image processing method according to some embodiments of this specification;

[0012] Figure 3 An exemplary process for obtaining a filtered channel image according to some embodiments of this specification;

[0013] Figure 4 is an exemplary structural diagram of an image processing model system shown in some embodiments of the basic specification;

[0014] Figure 5 is an exemplary module diagram of an image processing system according to some embodiments of this specification;

[0015] Figures 6 to 8 It is a graph of experimental comparison results shown in some embodiments of this specification;

[0016] Fig. 9 It is an exemplary schematic diagram of obtaining a high-resolution image based on a low-resolution image according to some embodiments of the present specification. DETAILED DESCRIPTION

[0017] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0018] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0019] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0020] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0021] In the field of magnetic resonance imaging (MRI), the accuracy of the diagnosis of patients based on MRI images is closely related to the quality of MRI images. In the field of MRI images, in addition to the diagnostic images output by the system, the imaging process of the MRI system is accompanied by a series of image processing, including images generated by the data collected by each channel in the multi-channel coil. These images contain information unique to each channel. For example, the parameters of the multi-channel coil placement, distribution, acquisition, noise, etc. are different and vary greatly. Therefore, the information of each channel during imaging will be different, and this different information is the unique information of each channel.

[0022] The use of multi-channel information is very important for improving the quality of MRI images, for example, improving image resolution and reducing image noise. However, how to make use of this multi-channel information is an urgent problem to be solved. Based on multi-channel image input to the neural network, the improvement of MRI image quality requires a lot of calculations, which affects the network speed and cannot achieve universal effect. In addition, simply inputting multi-channel data into the neural network is extremely unstable. The reasons include: (1) The distribution of coil channels varies greatly, but the feature map of the neural network requires a fixed pattern, such as RGB. If the coil channels are placed in the feature map dimension for processing, the network will be overfitted to a fixed pattern and cannot be universal. For example, the number of feature maps of the network is fixed, but the number of coil channels of the magnetic resonance system is variable, and the two cannot be universal; (2) Placing multi-channel data directly in the training dimension will greatly affect the network speed and fail to utilize channel information. Placing it in the convolution dimension will greatly affect the performance and convergence speed of the model, resulting in a waste of resources. There is a method in the related art that uses a single magnetic resonance image obtained through system processing through a convolutional neural network (CNN) framework to improve the quality of MRI images. However, in this type of method, the input information of the network model is single, and it is difficult to achieve the expected effect of improving image quality.

[0023] Some embodiments of the present specification propose an image processing method for magnetic resonance images based on multi-channel information. The method can make full use of multi-channel and single-channel information of MRI images and can effectively improve the quality of MRI images.

[0024] Figure 1 It is a schematic diagram of an exemplary application scenario of the image processing system shown in some embodiments of this specification.

[0025] like Figure 1 As shown, in some embodiments, the application scenario 100 of the image processing system may include an imaging device 110 , a processing device 120 , a terminal 130 , a storage device 140 and a network 150 .

[0026] The imaging device 110 may be configured to scan the target object to collect scan data related to the target object. The scan data may be used to generate one or more images of the target object. In some embodiments, the imaging device 110 may be single-modal or multi-modal. In some embodiments, the single-modal device may be a magnetic resonance imaging device, and the multi-modal device may include a positron emission tomography-magnetic resonance imaging (PET-CT) scanner, a computed tomography-magnetic resonance imaging (CT-MRI) scanner, etc. The target object may be biological or non-biological. As an example only, the target object may include a patient, an artificial object (e.g., an artificial phantom), etc. In another example, the target object may also include a specific part, organ, and / or tissue of a patient. In some embodiments, the MRI device may include a magnet assembly and a coil assembly, etc. The coil assembly may include a gradient coil assembly and a radio frequency (RF) coil assembly, etc. In some embodiments, the coil assembly may include a plurality of coil channels, each of which may be used to obtain corresponding scan data and used for imaging.

[0027] The processing device 120 can process data and / or information obtained from the imaging device 110, the terminal 130 and / or the storage device 140. For example, the processing device 120 can process multiple channel images corresponding to multiple channel coils of the magnetic resonance system acquired by the imaging device 110, for example, input the multiple channel images into the image processing model for processing to obtain a target image. For another example, the processing device 120 can perform filtering and merging processing based on multiple channel images to obtain multiple filtered channel images corresponding to multiple channel coils; the processing device 120 can perform channel merging processing based on multiple channel images to obtain a channel merged image; the processing device 120 can fuse multiple filtered channel images and channel merged images to generate a multi-channel fused image; the processing device 120 can use the multi-channel fused image to obtain the target image. In some embodiments, the processing device 120 can be a single server or a server group. The server group can be centralized or distributed. In some embodiments, the processing device 120 can be local or remote. For example, the processing device 120 can access information and / or data from the imaging device 110, the terminal 130 and / or the storage device 140 through the network 150. For another example, the processing device 120 can be directly connected to the imaging device 110, the terminal 130 and / or the storage device 140 to access the information and / or data. In some embodiments, the processing device 120 can be implemented on a cloud platform. For example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud cloud, a multi-cloud, etc. or any combination thereof. In some embodiments, the processing device 120 can be a part of the imaging device 110. For example, the processing device 120 can be integrated in the imaging device 110.

[0028] The terminal 130 may include a mobile device 131, a tablet computer 132, a laptop computer 133, etc. or any combination thereof. In some embodiments, the terminal 130 may interact with other components in the scene 100 through the network 150. For example, the terminal 130 may send one or more control instructions to the imaging device 110 through the network 150 to control the imaging device 110 to scan the target object according to the instructions. In some embodiments, the mobile device 131 may include a smart home device, a wearable device, a mobile device, a virtual reality device, an augmented reality device, etc. or any combination thereof.

[0029] In some embodiments, the terminal 130 may be a part of the processing device 120. In some embodiments, the terminal 130 may be integrated with the processing device 120 as an operation console of the imaging device 110. For example, a user / operator (e.g., a doctor or a nurse) of the scene 100 may control the operation of the imaging device 110 through the operation console, for example, scanning a target object.

[0030] The storage device 140 may store data (e.g., multiple channel images corresponding to multiple channel coils, multiple filtered channel images, channel merged images, etc.), instructions, and / or any other information. In some embodiments, the storage device 140 may store data obtained from the imaging device 110, the processing device 120, and / or the terminal 130. For example, the storage device 140 may store multiple channel images corresponding to multiple channel coils obtained from the imaging device 110, multiple filtered channel images, channel merged images, etc. In some embodiments, the storage device 140 may store data and / or instructions that can be executed or used by the processing device 120 to perform the exemplary methods described in this specification.

[0031] In some embodiments, the storage device 140 may include one or a combination of mass storage, removable storage, volatile read-write memory, read-only memory (ROM), etc. In some embodiments, the storage device 140 may be implemented by the cloud platform described in this specification.

[0032] In some embodiments, the storage device 140 may be connected to the network 150 to implement communication with one or more components (e.g., the processing device 120, the terminal 130) in the scene 100. One or more components in the scene 100 may read data or instructions in the storage device 140 through the network 150. In some embodiments, the storage device 140 may be part of the processing device 120, or may be independent and directly or indirectly connected to the processing device 120.

[0033] The network 150 may include any suitable network capable of facilitating information and / or data exchange of the scene 100. In some embodiments, one or more components of the scene 100 (e.g., the imaging device 110, the processing device 120, the terminal 130, the storage device 140) may exchange information and / or data with one or more components of the scene 100 via the network 150. For example, the processing device 120 may obtain multiple channel images corresponding to multiple channel coils, multiple filtered channel images, and channel merged images from the imaging device 110 via the network 150. In some embodiments, the network 150 may include one or more network access points. For example, the network 150 may include wired and / or wireless network access points.

[0034] It should be noted that the above description of the application scenario 100 is provided for illustrative purposes only and is not intended to limit the scope of the present embodiment. For those of ordinary skill in the art, various changes and modifications can be made under the guidance of the present embodiment. For example, the components and / or functions of the application scenario 100 can be changed or altered according to the specific implementation scheme. As an example only, some other components can be added to the application scenario 100, such as a power module that can supply power to one or more components of the application scenario 100, and other devices or modules.

[0035] Figure 2 is an exemplary flow chart of an image processing method according to some embodiments of the present specification. In some embodiments, process 200 may be executed by processing device 120. For example, process 200 may be stored in a storage device (such as a built-in storage unit of the processing device or an external storage device) in the form of a program or instruction, and when the program or instruction is executed, process 200 may be implemented. In some embodiments, process 200 may include the following operations.

[0036] Step 202 , acquiring multiple channel images corresponding to multiple channel coils of the magnetic resonance system. In some embodiments, step 202 may be performed by the processing device 120 or the channel image acquisition module 510 .

[0037] The magnetic resonance system is a medical imaging technology used for medical diagnosis, which can generate magnetic resonance images of the target object to be scanned (for example, tissues or organs in the human body) by using strong magnetic fields, magnetic field gradients and radio waves. The magnetic resonance imaging equipment mainly includes a magnet, a channel coil, a receiving link, etc. The channel coil can also be called an imaging coil, which can include a gradient coil, a radio frequency coil, a receiving coil, etc. During magnetic resonance imaging, a uniform static magnetic field is generated by a superconducting magnet, and a radio frequency transmitting coil is used to excite the precession of hydrogen nuclear spins to generate a magnetic resonance signal, and the gradient coil is used to encode the spatial information of the signal. The above magnetic resonance signal is collected by the radio frequency receiving coil, converted into a digital signal through the receiving link, and finally reconstructed by a computer to obtain a magnetic resonance image. As an important component of the function realization of the magnetic resonance system, the channel coil plays a decisive role in the quality of the magnetic resonance system image. In some embodiments, the channel coil may include one or more of a gradient coil, a radio frequency coil, a receiving coil, etc.

[0038] A channel image may refer to an image formed by processing signals corresponding to each channel coil. In some embodiments, each channel coil may correspond to a channel image.

[0039] In some embodiments, each channel image may contain information unique to the corresponding channel. For example, assuming there are n channels, the channel images of n channels can be expressed as G = {g1, g2, ..., gn}, where G represents a set of multiple channel images, g1 represents the image corresponding to the first channel coil, g2 represents the image corresponding to the second channel coil, ..., gn represents the image corresponding to the nth channel coil. Since the spatial sensitivity distribution, acquisition, noise and other parameters of the multi-channel coil are different and vary greatly, the information of each channel during imaging will be different. For example, the coil placement position of the first channel coil is different from the placement position of the second channel coil, and the signals of the target object that can be collected are also different. For example, the first channel coil can collect some information that the second channel coil cannot collect. This information can be understood as information unique to the channel coil.

[0040] In some embodiments, a magnetic resonance device may be used to scan a target object, and corresponding signals may be collected through each channel coil of the magnetic resonance device. Then, imaging (eg, image reconstruction) may be performed based on these signals to obtain a channel image corresponding to each channel coil.

[0041] In some embodiments, the processing device may also obtain the channel image by reading from a storage device, a database, or an imaging device. For example, the channel image corresponding to each channel coil may be pre-stored in a storage device, a database, or an imaging device, and the channel image may be obtained after reading.

[0042] Step 204 , input the multiple channel images into the image processing model for processing to obtain the target image. In some embodiments, step 202 may be performed by the processing device 120 or the target image acquisition module 520 .

[0043] The target image refers to an image after processing multiple channel images. The target image is an image with improved image quality that is desired to be obtained. For example, the target image may be an image after converting a low-resolution image to a high-resolution image, and / or an image after noise reduction.

[0044] The image processing model can be used to process multiple channel images to determine the target image. In some embodiments, the image processing model can be a convolutional neural network model (CNN), a deep neural network model, or other combination models. This specification does not limit the specific type of the image processing model.

[0045] In some embodiments, the input data of the image processing model is a plurality of channel images, and the output data is a target image. The image processing model can perform a series of processing on the input plurality of channel images within the model to obtain the target image and output it.

[0046] In some embodiments, the process of inputting multiple channel images into the image processing model for processing can be as shown in steps 2042-2048.

[0047] Step 2042: Perform filtering and merging processing based on the multiple channel images to obtain multiple filtered channel images corresponding to the multiple channel coils.

[0048] Filter merging processing refers to the process of filtering and merging multiple channel images.

[0049] The number of the filtered channel images is the same as the number of the filtered channel images, and each filtered channel image has a corresponding relationship with the channel coil. For example, assuming that there are n channel coils, the channel images of the n channel coils can be expressed as G = {g1, g2, ..., gn}, and after filtering and merging G = {g1, g2, ..., gn}, multiple filtered channel images C' can be obtained, where C' = {c1, c2, ..., cn}, c1 corresponds to g1, c2 corresponds to g2, and cn corresponds to gn.

[0050] In some embodiments, the filtering and merging process may include a filtering operation and a merging operation. The filtering operation may filter the channel image corresponding to each channel coil, for example, filtering g1, g2, ..., gn respectively. After filtering, multiple intermediate images (also referred to as filtered images) may be obtained. For example, after filtering g1, g11, g12, g13, and g14 may be obtained. The number of intermediate images depends on the number of filters used or the filtering algorithm. Afterwards, g11, g12, g13, and g14 may be merged (that is, a merging operation) to obtain the filtered channel image c1.

[0051] In some embodiments, multiple channel images may be filtered by a first neural network, and intermediate images obtained by filtering may be merged by a Sum Of Square (SOS) algorithm.

[0052] In some embodiments, the first neural network may be a convolutional neural network model (CNN). For more information about the first neural network, see Figure 3 Related description.

[0053] It should be noted that the method used for filtering and merging processing disclosed in this embodiment is for illustrative purposes only, and filtering and merging can also be performed in other ways. For example, SOS can be replaced by any channel merging method, such as a channel merging method using coil sensitivity.

[0054] Step 2044: perform channel merging processing based on the multiple channel images to obtain a channel merging image.

[0055] Channel merging refers to the process of merging multiple channel images. A channel merged image is an image obtained by merging multiple channel images. For example, multiple channel images can be merged into one image through channel merging, and the image obtained is the channel merged image. For example, assuming there are n channels, the n channel images can be represented by G = {g1, g2, ..., gn}. By performing channel merging on G = {g1, g2, ..., gn}, a channel merged image G' can be obtained, where G' = {g}.

[0056] In some embodiments, multiple channel images may be merged by a preset merging method to obtain a channel merged image. The preset merging method may include an SOS merging method and a channel merging method based on coil sensitivity. For more information about the SOS and coil sensitivity-based channel merging methods, please refer to the description in step 2042, which will not be repeated here.

[0057] In some embodiments, when acquiring a channel merged image, the processing device will input the result of merging multiple channel images through a preset merging method into a third neural network for filtering, and use the image output by the third neural network as the channel merged image.

[0058] In some embodiments, the preset merging method may be the same as the channel merging method of step 2042, or may be different from the channel merging method of step 2042. For example, the preset merging method may also include channel merging based on signal-to-noise ratio weighting. For another example, the merging method in step 2042 is SOS, and the merging method here may be a channel merging method based on coil sensitivity.

[0059] Step 2046 , fusing the multiple filtered channel images and the channel merged image to generate a multi-channel fused image. In some embodiments, step 202 may be performed by the processing device 120 or the target image acquisition module 520 .

[0060] Fusion may refer to the process of merging multiple filtered channel images and channel merged images through certain processing.

[0061] A multi-channel fusion image refers to an image generated by fusing multiple filtered channel images and channel merged images.

[0062] In some embodiments, a multi-channel fused image can be generated based on linear fusion of multiple filtered channel images and channel merged images. Linear fusion can be superimposing multiple filtered channel images and channel merged images in the image dimension. For example, multiple filtered channel images are C'={c1, c2, ..., cn}, and the channel merged image G'={g}. Linear fusion of G'={g} and C'={c1, c2, ..., cn} can obtain a multi-channel fused image G", where G"={g, c1, c2, ..., cn}.

[0063] Linear fusion can be to superimpose multiple filtered channel images and channel merged images in dimension. For example, the number of multiple filtered channel images is 16, and the number of channel merged images is 1. After dimensional superposition, it can become a 17-dimensional multi-channel fused image. For example, the image size of the filtered channel image and the channel merged image is 16*16, and the multi-channel fused image obtained after dimensional superposition is 17*16*16.

[0064] Step 2048 , obtaining the target image using the multi-channel fused image. In some embodiments, step 202 may be performed by the processing device 120 or the target image acquisition module 520 .

[0065] In some embodiments, the quality of the multi-channel fused image may be improved to obtain a target image. For example, the multi-channel fused image may be processed as input data by an image denoising algorithm, a neural network model, etc. to obtain a target image.

[0066] In some embodiments, the image processing model further includes a second neural network. The multi-channel fused image can be used as input data of the second neural network, and the second neural network outputs a target image.

[0067] The second neural network can be used to improve the resolution and reduce the noise of the image. Super-resolution refers to restoring a low-resolution image to a high-resolution image, while denoising refers to reducing the noise information contained in the image.

[0068] In some embodiments, the second neural network can be a deep neural network model (DNN), a convolutional network model, or other combination models, and this specification does not limit its specific type.

[0069] In some embodiments, the input data of the second neural network is a multi-channel fused image, and the output is a target image.

[0070] The parameters of the second neural network can be obtained through training. In some embodiments, the second neural network can be obtained through training of multiple labeled training samples. For example, multiple labeled training samples can be input into the initial second neural network, and a loss function can be constructed by using the labels and the output results of the initial second neural network, and the parameters of the second neural network can be iteratively updated based on the loss function. When the loss function of the initial second neural network meets the preset conditions, the model training is completed and a trained second neural network is obtained. Among them, the preset conditions can be that the loss function converges, the number of iterations reaches a threshold, etc.

[0071] In some embodiments, the training sample may include a sample multi-channel fusion image. The label may be a sample target image. In some embodiments, the training sample may be obtained based on historical data (e.g., historical multi-channel fusion images), and the label may be obtained through manual annotation.

[0072] In some embodiments, the first neural network and the second neural network can be trained separately or jointly, that is, the image processing model is trained end-to-end. For more information about joint training, see Figure 4 Related description.

[0073] In some embodiments, the process of multi-channel image processing by the image processing model is an end-to-end processing process, that is, after the multi-channel image is input into the image processing model, processing such as steps 2042-2048 is performed within the image processing model, and after processing, the target image is directly output by the image processing model.

[0074] In some embodiments of the present specification, a plurality of channel images corresponding to a plurality of channel coils of a magnetic resonance system are obtained; the plurality of channel images are input into an image processing model for filtering, merging, and fusion processing to obtain a target image. In this process, the plurality of channel images corresponding to the plurality of channel coils are filtered and merged, so that the multi-channel information of the MRI image can be fully utilized, and the multi-channel images are subjected to channel merging processing to obtain a channel merged image, and then the plurality of filtered channel images and the channel merged image are fused, so that the multi-channel information and the single-channel information of the channel coils can be utilized at the same time, and the image super-resolution and noise reduction are realized by using multi-layer information, and the quality of the magnetic resonance image is effectively improved.

[0075] Compared with the method of directly improving the image quality of a single-channel image output by a magnetic resonance system, the image processing method disclosed in some embodiments of this specification has better overall multi-channel details. Even in the case of poor image quality, for example, the image area in the single-channel image is severely damaged, the result of quality improvement using multi-channel information is relatively stable. At the same time, the complexity of the algorithm can be simplified without losing multi-channel information, with better algorithm performance and space complexity. In some embodiments, the multi-channel information of the MRI image is fused as a channel merged image of the secondary branch and the main branch, which can make full use of the multi-channel information of the MRI image coil, achieve super-resolution and noise reduction of the MRI image with maximum performance, and also have good computing performance.

[0076] The experiments on the technical solutions disclosed in some embodiments of this specification have achieved satisfactory results, some of which are as follows: Figure 6-Figure 8 As shown. Among them, Figure 6-Figure 8 It is a graph of experimental comparison results shown in some embodiments of this specification. Figure 6 It can be seen that compared with the single-channel image processing method, the multi-channel overall details are better. In addition, from Figure 7 It can be seen that in the worst case, although the overall GT (gold standard) effect cannot be restored, the single-channel processing results are seriously damaged, and the multi-channel results are relatively stable. Figure 8 The virtual channel fusion result obtained by adaptively learning the network model in the embodiment of this specification is shown ( Figure 8The first image in the figure is the result of real coil channel fusion, i.e. the original image, and the following is the result of virtual coil channel fusion). From the above results, it can be seen that compared with single-channel data, multi-channel overall detail recovery has certain advantages, multi-channel detail performance is better, and the performance is more stable under low signal-to-noise ratio and low image quality conditions. The technical solutions disclosed in some embodiments of this specification can make full use of the multi-channel information of MRI image coils, and through the image processing model, achieve super-resolution and noise reduction of MRI images with maximum performance.

[0077] Figure 3 An exemplary process for obtaining a filtered channel image is shown in some embodiments of this specification.

[0078] In some embodiments, the image processing model may include a first neural network, and filtering and merging processing is performed based on multiple channel images. The step of acquiring multiple filtered channel images corresponding to the channel coils may include the following operations.

[0079] Optionally, the first neural network includes a plurality of filters arranged in parallel, and the first neural network has an input end and a plurality of output ends, corresponding to inputting a channel image, obtaining a plurality of filtered channel images, and the characteristic quality parameters of the plurality of filtered channel images are different. The characteristic quality parameters can be, for example, different structural contrasts, edge fat signal intensity, vascular tissue signal intensity, image blurring effect, cerebrospinal fluid signal intensity, so as to achieve the functions of specifically improving the tissue structure information of the output image, improving the edge contour structure, improving the flow tissue such as blood, etc.

[0080] In one embodiment, multiple filtered channel images include: filtered channel image 1, which has more uniform fat signal suppression in the entire scanning field of view relative to the input image; filtered channel image 2, which reduces the cerebrospinal fluid signal intensity relative to the input image to more clearly display lesions near the cerebrospinal fluid; filtered channel image 3, which has improved soft tissue contrast relative to the input image; filtered channel image 3, which has reduced blur relative to the input image; filtered channel image 4, which has reduced fat signal intensity at the edge contour of the input image and improved edge uniformity relative to the input image; filtered channel image 5, which can visualize blood vessels, arteries and veins relative to the input image.

[0081] Optionally, the aforementioned first neural network can be obtained by training plain scan and enhanced scan images of the same detection object; or, training the frequency selective fat saturation method and short inversion time inversion recovery (STIR) method of the same detection object to obtain fat suppressed images; or, training the rapid inversion recovery sequence and the self-selected echo sequence images of the same detection object to obtain images; or, training the images of the same detection object with and without contrast agent injection; or, training the images of the same detection object with and without artifacts.

[0082] Step 302: Process the multiple channel images through a first neural network to obtain multiple filtered images corresponding to each channel coil.

[0083] In some embodiments, the number of the plurality of filtered images may correspond to the number of filters of the first neural network. The filtered image is the intermediate image obtained by filtering the plurality of channel images as described above.

[0084] The first neural network can be used to process multiple channel images to obtain multiple filtered images corresponding to each channel coil. In some embodiments, multiple channel images can be input into the first neural network, and after the first neural network processes the channel images, it outputs a filtered image of the channel image corresponding to each channel coil.

[0085] In some embodiments, the input of the first neural network may be a plurality of channel images, and the output may be a plurality of filtered channel images corresponding to each channel coil.

[0086] The filtered image refers to the intermediate image obtained after the first neural network processes the input channel image. In some embodiments, the first neural network can process each input channel image and output a corresponding plurality of filtered images. For example, assuming there are n channels, the plurality of channel images can be represented as G={g1, g2, ..., gn}. After G={g1, g2, ..., gn} is input into the first neural network, the first neural network can first filter g1 and output a plurality of filtered images, and then filter g2 and output a plurality of filtered images. The number of output filtered images is the same as the number of filters of the first neural network.

[0087] In this embodiment, when the first neural network processes the input multi-channel model, it adopts a method of filtering each channel image in turn, so it can adapt to the one-time input of different numbers of channel images and has universal applicability.

[0088] In some embodiments, the first neural network may be a convolutional neural network model (CNN), a deep neural network model, or other combination models, which is not limited in this specification.

[0089] The parameters of the first neural network can be obtained through training. In some embodiments, the first neural network can be obtained through training of multiple labeled training samples. For example, multiple labeled training samples can be input into the initial first neural network, and a loss function can be constructed by using the labels and the output results of the initial first neural network, and the parameters of the second neural network can be iteratively updated based on the loss function. When the loss function of the initial first neural network meets the preset conditions, the model training is completed, and a trained first neural network is obtained. Among them, the preset conditions can be that the loss function converges, the number of iterations reaches a threshold, etc.

[0090] In some embodiments, the training sample may include a sample multi-channel image. The label may be a multi-channel image after sample filtering. In some embodiments, the training sample may be obtained based on historical data (e.g., a historical multi-channel fusion image), and the label may be obtained by manual annotation.

[0091] Step 304: Process the multiple filtered images corresponding to each channel coil by a preset merging method to determine a filtered channel image corresponding to each channel coil.

[0092] In some embodiments, the preset merging methods may include an SOS method, a merging method based on coil sensitivity, a merging method based on signal-to-noise ratio weighting, and the like.

[0093] In some embodiments, the input of the preset merging method may be a plurality of filtered images corresponding to each channel coil, and the output may be a filtered channel image corresponding to each channel coil.

[0094] Figure 4 It is an exemplary structural diagram of the image processing model system shown in some embodiments of this specification.

[0095] like Figure 4 As shown, the image processing model system may include a first neural network 410 , an image merging module 420 , an image fusion module 430 , and a second neural network 440 .

[0096] The first neural network 410 can be used to process multiple channel images to obtain multiple filtered images corresponding to each channel coil.

[0097] The image merging module 420 may be configured to perform channel merging processing based on the multiple channel images to obtain a channel merging image.

[0098] The image fusion module 430 may be configured to fuse the plurality of filtered channel images and the channel merged image to generate a multi-channel fused image.

[0099] The second neural network 440 can be used to obtain the target image using the multi-channel fused image.

[0100] For a detailed description of each module of the image processing model, please refer to Figure 2 and Figure 3 The relevant description will not be repeated here.

[0101] In some embodiments, the image processing model can be obtained by training in an end-to-end training manner. For example, multiple sample channel images obtained historically can be used as training samples of the initial image processing model, and the high quality corresponding to the multiple sample channel images (for example, super-resolution and denoising and MRI images) can be used as labels for model training to obtain the image processing model. The training method can be to use multiple sample channel images as input data, construct a loss function with the output of the initial image processing model and the label, adjust the parameters of the model based on the loss function value, and obtain the image processing model when the loss function value converges or the number of iterative training reaches a preset value.

[0102] In this embodiment, the first neural network and the second neural network are jointly trained as part of the image processing model, which can reduce the complexity of model training and improve the efficiency of model training.

[0103] In order to facilitate understanding of the structure of the image processing model, the following embodiment illustrates the process of inputting a low-resolution multi-channel image into the image processing model for processing to obtain a high-resolution target image. Fig. 9 As shown, Fig. 9 It is an exemplary schematic diagram of obtaining a high-resolution image based on a low-resolution image according to some embodiments of the present specification.

[0104] It should be noted that Fig. 9 The illustrated process can be used as an example of a network algorithm for an input multi-channel image by an image processing model. For example, the input data is a low-resolution image G = {g1, g2, ..., gn}, G = {g1, g2, ..., gn} is sent to the first branch 910, the input data is filtered by the second SOS and then by the third neural network to obtain a new G' = {g}, that is, a merged image, where n'≤n; wherein the third neural network is an optional module.

[0105] And the input low-resolution image G={g1, g2, ..., gn} is sent to the second branch 920, the input data is filtered by the first neural network and then passed through the first SOS to obtain a new C'={c1, c2, ..., cn}, that is, the filtered image.

[0106] Afterwards, G'={g} and C'={c1, c2, ..., cn} are linearly fused to obtain the output G"={g, c1, c2, ..., cn}, i.e., the fused image.

[0107] Finally, G″={g, c1, c2, ..., cn} is sent to the second neural network to obtain a high-resolution image.

[0108] In some embodiments, the input low-resolution image is an MRI image, and after being processed by the above process, a super-resolution and denoised image of the MRI image can be obtained.

[0109] It should be noted that the above descriptions are only for illustration and explanation, and do not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to each process under the guidance of this specification. However, these modifications and changes are still within the scope of this specification. For example, changes to the process steps of this specification, such as adding a pre-processing step and a storage step, etc.

[0110] Figure 5 is an exemplary module diagram of an image processing system according to some embodiments of this specification. Figure 5 As shown, the image processing system 500 may include a channel image acquisition module 510 and a target image acquisition module 520. In some embodiments, one or more modules in the image processing system 500 may be interconnected. The connection may be wireless or wired. At least a portion of the image processing system 500 may be configured as follows: Figure 1 The method is implemented on the imaging device 110, the processing device 120 or the terminal 130 shown.

[0111] In some embodiments, the channel image acquisition module 510 may be configured to acquire a plurality of channel images corresponding to a plurality of channel coils of a magnetic resonance system.

[0112] In some embodiments, the target image acquisition module 520 can be used to input multiple channel images into the image processing model for processing to obtain the target image. In some embodiments, the target image acquisition module 520 can be further used to perform filtering and merging processing based on multiple channel images to obtain multiple filtered channel images corresponding to multiple channel coils. In some embodiments, the target image acquisition module 520 can be further used to perform channel merging processing based on multiple channel images to obtain channel merged images. In some embodiments, the target image acquisition module 520 can be further used to fuse multiple filtered channel images and channel merged images to generate a multi-channel fused image. In some embodiments, the target image acquisition module 520 can be used to obtain the target image using the multi-channel fused image.

[0113] In some embodiments, the image processing model includes a first neural network. In some embodiments, the target image acquisition module 520 can be further used to process multiple channel images through the first neural network to obtain multiple filtered images corresponding to each channel coil; wherein the number of the multiple filtered images corresponds to the number of filters of the first neural network. In some embodiments, the target image acquisition module 520 can be further used to process the multiple filtered images corresponding to each channel coil through a preset merging method to determine the filtered channel image corresponding to each channel coil.

[0114] It should be noted that the above description of the image processing system 500 is for illustrative purposes only and is not intended to limit the scope of this specification. For those of ordinary skill in the art, various variations and modifications may be made according to this specification. However, these variations and modifications do not depart from the scope of this specification. For example, one or more modules of the above-mentioned image processing system 500 may be omitted or integrated into a single module. For another example, the image processing system 500 may include one or more additional modules, such as a storage module for data storage, etc.

[0115] It should be understood that Figure 5 The system and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or a dedicated design hardware. Those skilled in the art will understand that the above methods and systems can be implemented using computer executable instructions and / or included in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the system and its modules of this specification. Not only can the hardware circuits such as ultra-large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc. be implemented, it can also be implemented with software such as executed by various types of processors, and it can also be implemented by a combination of the above hardware circuits and software (for example, firmware).

[0116] It should be noted that the above description of the image processing system and its modules is only for convenience of description, and this specification can not be limited to the scope of the embodiments. It can be understood that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the various modules without deviating from this principle, or to form a subsystem connected with other modules. For example, in some embodiments, the channel image acquisition module 510 and the target image acquisition module 520 can be different modules in a system, or a module can realize the functions of the above-mentioned two or more modules. For example, each module can share a storage module, and each module can also have its own storage module. Such deformations are all within the protection scope of this specification.

[0117] The embodiment of the present specification also provides a device, which includes at least a processor and a memory. The memory is used to store instructions. When the instructions are executed by the processor, the device implements the above-mentioned image processing method.

[0118] The embodiment of the present specification also provides a computer-readable storage medium. The storage medium stores computer instructions, and when a computer reads the computer instructions in the storage medium, the computer implements the above-mentioned image processing method.

[0119] The beneficial effects that may be brought about by this application include but are not limited to: (1) making full use of the multi-channel information of the MRI image coil to maximize the improvement of the quality of MRI images; (2) simplifying the complexity of the algorithm without losing multi-channel information, and having better model performance.

[0120] It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other beneficial effects that may be obtained.

[0121] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0122] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0123] In addition, it will be understood by those skilled in the art that various aspects of this specification may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of this specification may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of this specification may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0124] A computer storage medium may include a propagated data signal containing computer program code, for example, in baseband or as part of a carrier wave. The propagated signal may be in a variety of forms, including electromagnetic, optical, etc., or a suitable combination. A computer storage medium may be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, device or apparatus to communicate, propagate or transmit the program for use. The program code on the computer storage medium may be transmitted via any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.

[0125] The computer program code required for the operation of each part of this specification can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., conventional procedural programming languages ​​such as C language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy, or other programming languages. The program code can be run entirely on the user's computer, or run on the user's computer as an independent software package, or run partially on the user's computer and partially on a remote computer, or run entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0126] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0127] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0128] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0129] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0130] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. An image processing method, the method comprising: Acquiring multiple channel images corresponding to multiple channel coils of the magnetic resonance system; Inputting the multiple channel images into an image processing model for processing to obtain a target image; The processing process includes: Perform filtering and merging processing based on the multiple channel images to obtain multiple filtered channel images corresponding to the multiple channel coils; Perform channel merging processing based on the multiple channel images to obtain a channel merging image; Fusing the plurality of filtered channel images and the channel merged image to generate a multi-channel fused image; The target image is obtained by using the multi-channel fused image.

2. The method according to claim 1, wherein: The image processing model includes a first neural network; the filtering and merging process based on the multiple channel images to obtain multiple filtered channel images corresponding to the channel coils includes: Processing the plurality of channel images through the first neural network to obtain a plurality of filtered images corresponding to each channel coil; wherein the number of the plurality of filtered images corresponds to the number of filters of the first neural network; The multiple filtered images corresponding to each channel coil are processed by a preset merging method to determine a filtered channel image corresponding to each channel coil.

3. The method according to claim 2, wherein fusing the plurality of filtered channel images and the channel merged image to generate a multi-channel fused image comprises: The multi-channel fused image is generated by linearly fusion based on the multiple filtered channel images and the channel merged image.

4. The method according to claim 2, wherein: The image processing model also includes a second neural network; the step of obtaining the target image by using the multi-channel fused image includes: The multi-channel fusion image is processed by the second neural network to determine the target image.

5. The method according to claim 1, wherein the performing channel merging processing based on the plurality of channel images to obtain a channel merged image comprises: The multiple channel images are merged by a preset merging method to obtain the channel merged image.

6. An image processing system, the system comprising: A channel image acquisition module, used for acquiring a plurality of channel images corresponding to a plurality of channel coils of the magnetic resonance system; A target image acquisition module, used for inputting the multiple channel images into an image processing model for processing to acquire a target image; Wherein, the target image acquisition module is further used for: Perform filtering and merging processing based on the multiple channel images to obtain multiple filtered channel images corresponding to the multiple channel coils; Perform channel merging processing based on the multiple channel images to obtain a channel merging image; Fusing the plurality of filtered channel images and the channel merged image to generate a multi-channel fused image; The target image is obtained by using the multi-channel fused image.

7. The system according to claim 6, wherein: The image processing model includes a first neural network; the target image acquisition module is further used for: Processing the plurality of channel images through the first neural network to obtain a plurality of filtered images corresponding to each channel coil; wherein the number of the plurality of filtered images corresponds to the number of filters of the first neural network; The multiple filtered images corresponding to each channel coil are processed by a preset merging method to determine a filtered channel image corresponding to each channel coil.

8. An image processing model, the image processing model comprising: A first neural network is used to perform filtering and merging processing on a plurality of channel images corresponding to a plurality of channel coils of the magnetic resonance system to obtain a plurality of filtered channel images corresponding to the channel coils; An image merging module, used for performing channel merging processing based on the multiple channel images to obtain a channel merged image; An image fusion module, used for fusing the plurality of filtered channel images and the channel merged image to generate a multi-channel fused image; The second neural network is used to obtain the target image using the multi-channel fused image.

9. An image processing device, comprising at least one storage medium and at least one processor, wherein the at least one storage medium is used to store computer instructions; and the at least one processor is used to execute the computer instructions to implement the image processing method according to any one of claims 1 to 5.

10. A computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the image processing method according to any one of claims 1 to 5.

Images