Method, device and storage medium for identifying magnetic resonance imaging modality

Abstract

This application discloses a method, device, and storage medium for identifying the modality of a nuclear magnetic resonance image. The method comprises: acquiring a nuclear magnetic resonance image; performing multi-stage adaptive preprocessing on the nuclear magnetic resonance image to obtain a preprocessed nuclear magnetic resonance image; performing multimodal feature extraction on the preprocessed nuclear magnetic resonance image using a trained hybrid residual network to obtain a target feature result; and identifying the final modality of the nuclear magnetic resonance image based on the target feature result. Utilizing the solution of this application, the accuracy and robustness of nuclear magnetic resonance image modality recognition can be significantly improved, thereby enhancing the accuracy and efficiency of modality recognition.

Description

Technical Field

[0001] The present application generally relates to the field of image processing technology. More specifically, the present application relates to a method, device, and computer-readable storage medium for identifying magnetic resonance imaging modalities. Background Art

[0002] Magnetic resonance imaging (MRI) is a commonly used medical imaging technique that, depending on the imaging modality, can provide a wealth of biological tissue information. Common MRI modalities include T1-weighted, T2-weighted, and fluid-attenuated inversion recovery (FLAIR). T1-weighted provides anatomical structural information, T2-weighted provides tumor information, and FLAIR provides information on peritumoral edema. Therefore, identifying the different MRI modalities is essential.

[0003] Traditional modality recognition is often performed manually, which is time-consuming and error-prone. For example, in modality recognition of medical images, radiologists need to examine numerous images one by one and manually annotate different modalities (such as CT, MRI, and ultrasound). This not only increases the workload of radiologists but can also easily lead to misjudgments due to fatigue or subjective factors. Furthermore, the speed of manual annotation cannot keep up with the ever-increasing demand for medical imaging data. Currently, some existing automated modality recognition methods rely primarily on rule-based systems or simple machine learning algorithms. However, rule-based methods typically rely on a set of predefined rules and thresholds to identify and classify modalities. For example, patent application CN202111675248.4 discloses a method for recognizing multiple medical images, which describes recognition based on edge or morphological features. These rules are often static and difficult to adapt to complex and changing real-world situations. In particular, rule-based methods can easily fail when encountering similar modality features, resulting in low accuracy. Furthermore, the application also describes the use of algorithms such as support vector machines (SVMs) or decision trees, which, while somewhat capable of improving the automation of recognition, are limited in effectiveness. Specifically, algorithms such as support vector machines (SVMs) and decision trees rely on hand-crafted features that may not fully capture the complex information in an image. They also perform poorly when processing high-dimensional and nonlinear data, making it difficult to capture deep patterns and relationships within an image. Consequently, recognition accuracy is often suboptimal when complex patterns or noise are present in image data.

[0004] In view of this, there is an urgent need to provide a solution for identifying the modality of MRI images. First, by performing multi-level adaptive preprocessing operations on the MRI images, the preprocessing parameters can be dynamically adjusted multiple times to better preserve the detailed information of the image. Then, multimodal feature extraction is performed on the preprocessed MRI images through a hybrid residual network, which fully utilizes the complementary information of the multimodal images, significantly improves the accuracy and robustness of MRI image modality recognition, and improves the accuracy of modality recognition. Furthermore, the hybrid residual network is trained through the total loss function of the adaptive modality weight, so that the hybrid residual network can adaptively learn the importance of different modalities, thereby improving the recognition accuracy and reliability of the hybrid residual network. Summary of the Invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present application proposes solutions for identifying magnetic resonance imaging modalities in multiple aspects.

[0006] In a first aspect, the present application provides a method for identifying the modality of a magnetic resonance imaging (MRI), comprising: acquiring a MRI image; performing a multi-level adaptive preprocessing operation on the MRI image to obtain a preprocessed MRI image; performing multimodal feature extraction on the preprocessed MRI image using a trained hybrid residual network to obtain a target feature result; and identifying the final modality of the MRI image based on the target feature result.

[0007] In one embodiment, performing a multi-stage adaptive preprocessing operation on the nuclear magnetic resonance image includes: performing an adaptive filtering operation and an adaptive contrast enhancement operation on the nuclear magnetic resonance image in sequence, so as to perform a multi-stage adaptive preprocessing operation on the nuclear magnetic resonance image.

[0008] In another embodiment, the adaptive filtering operation is performed on the nuclear magnetic resonance image by the following operations: calculating the local variance of the nuclear magnetic resonance image; comparing the local variance with a first threshold; and adaptively selecting Gaussian filtering or median filtering based on the comparison result of the local variance and the first threshold to perform the adaptive filtering operation on the nuclear magnetic resonance image.

[0009] In yet another embodiment, an adaptive contrast enhancement operation is performed on the nuclear magnetic resonance image by: performing local contrast analysis on a filtering result of the adaptive filtering operation to obtain a local contrast result; comparing the local contrast result with a second threshold; and adaptively selecting a contrast limiting histogram algorithm or a global histogram algorithm based on a result of the comparison between the local contrast result and the second threshold to perform the adaptive contrast enhancement operation on the nuclear magnetic resonance image.

[0010] In another embodiment, the trained hybrid residual network includes at least an initial convolution module, a multi-scale residual module and a cross-attention module, and the trained hybrid residual network is used to perform multimodal feature extraction on the preprocessed magnetic resonance image to obtain a target feature result, which includes: using the initial convolution module to perform an initial convolution operation on the preprocessed magnetic resonance image to obtain an initial multimodal feature; using the cross-attention module to fuse the features of different modalities in the initial multimodality, and combining them with the preprocessed magnetic resonance image once to obtain an intermediate fusion result; using the multi-scale residual module to extract the multi-scale residual features of the intermediate fusion result, and performing feature splicing on the multi-scale residual features to obtain a feature splicing result; and combining the feature splicing result with the preprocessed magnetic resonance image for a second time to obtain the target feature result.

[0011] In another embodiment, identifying the modality of the magnetic resonance image based on the target feature result includes: mapping the target feature result to different modality categories using a fully connected layer, and calculating the probability of each modality category using a classification function; and identifying the modality corresponding to the maximum probability value as the final modality of the magnetic resonance image.

[0012] In yet another embodiment, the final modalities of the MRI image include at least a T1-weighted modality, a T2-weighted modality, and a fluid-attenuated inversion recovery modality.

[0013] In another embodiment, a hybrid residual network is trained by the following operations to obtain the trained hybrid residual network: magnetic resonance images of different modalities are input into the hybrid residual network for feature extraction to obtain multimodal features; uncertainty weights are set for the multimodal features; and a total loss function of adaptive modal weights is calculated based on the uncertainty weights to train the hybrid residual network to obtain the trained hybrid residual network.

[0014] In a second aspect, the present application provides a device for identifying a nuclear magnetic resonance imaging modality, comprising: a processor; and a memory storing program instructions for identifying a nuclear magnetic resonance imaging modality, wherein when the program instructions are executed by the processor, the device implements one or more embodiments of the aforementioned first aspect.

[0015] In a third aspect, the present application provides a computer-readable storage medium having stored thereon computer-readable instructions for identifying a magnetic resonance imaging modality. When the computer-readable instructions are executed by one or more processors, one or more embodiments of the aforementioned first aspect are implemented.

[0016] Through the scheme for identifying the modality of nuclear magnetic resonance images provided above, the embodiment of the present application first performs multi-level adaptive preprocessing operations on the nuclear magnetic resonance images to dynamically adjust the preprocessing parameters multiple times, thereby better retaining the detailed information of the image. Then, the preprocessed nuclear magnetic resonance images are subjected to multimodal feature extraction through the trained hybrid residual network, which can fully utilize the complementary information and multi-scale features of the multimodal images to extract accurate target feature results, thereby significantly improving the accuracy and robustness of the nuclear magnetic resonance image modality recognition and improving the accuracy of the final modality recognition of the nuclear magnetic resonance images. Further, in some embodiments, the embodiment of the present application trains the hybrid residual network through the total loss function of the adaptive modality weight, so that the hybrid residual network can adaptively learn the importance of different modalities, thereby improving the recognition accuracy and reliability of the hybrid residual network for different modalities of nuclear magnetic resonance images. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0018] Figure 1 is an exemplary flowchart illustrating a method for identifying a nuclear magnetic resonance imaging modality according to an embodiment of the present application;

[0019] Figure 2 is an exemplary flow chart illustrating a multi-stage adaptive pre-processing operation according to an embodiment of the present application;

[0020] Figure 3 is a flowchart illustrating an overall exemplary process for identifying a nuclear magnetic resonance imaging modality according to an embodiment of the present application;

[0021] Figure 4 is an exemplary schematic diagram illustrating a hybrid residual network according to an embodiment of the present application;

[0022] Figure 5 is an exemplary structural block diagram showing a device for identifying a nuclear magnetic resonance imaging modality according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0024] It should be understood that the terms "include" and "comprising" used in the description and claims of this application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0025] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this specification and claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" as used in this specification and claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0026] As used in this specification and claims, the term “if” can be interpreted as “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context. Similarly, the phrase “if it is determined” or “if [described condition or event] is detected” can be interpreted as meaning “upon determination” or “in response to determining” or “upon detection of [described condition or event]” or “in response to detecting [described condition or event],” depending on the context.

[0027] The specific implementation of the present application will be described in detail below with reference to the accompanying drawings.

[0028] Figure 1 FIG. 1 is an exemplary flow chart illustrating a method 100 for identifying a magnetic resonance imaging modality according to an embodiment of the present application. Figure 1 As shown in , at step S101, a nuclear magnetic resonance image is acquired. In one implementation scenario, the nuclear magnetic resonance image can be acquired by, for example, acquiring a magnetic resonance acquisition device. Based on the nuclear magnetic resonance image acquired above, at step S102, a multi-level adaptive preprocessing operation is performed on the nuclear magnetic resonance image to obtain a preprocessed nuclear magnetic resonance image. In one embodiment, an adaptive filtering operation and an adaptive contrast enhancement operation can be performed on the nuclear magnetic resonance image in sequence to perform a multi-level adaptive preprocessing operation on the nuclear magnetic resonance image. That is, an adaptive filtering operation is first performed on the nuclear magnetic resonance image to obtain a filtering result of the adaptive filtering operation, and then an adaptive contrast enhancement operation is performed on the filtering result of the adaptive filtering operation to implement a multi-level adaptive preprocessing operation. In some embodiments, an adaptive contrast enhancement operation can also be performed on the nuclear magnetic resonance image first, and then an adaptive filtering operation is performed.

[0029] Through the aforementioned multi-level adaptive preprocessing operation of the embodiment of the present application, the variance and contrast of the local area can be adaptively calculated according to the noise level or image quality of the magnetic resonance image, so as to dynamically adjust the filter parameters and contrast intensity, better retain the detailed information of the magnetic resonance image, so as to improve the subsequent recognition accuracy.

[0030] In one implementation scenario, the local variance of the MRI image can be calculated, the local variance can be compared with a first threshold, and then Gaussian filtering or median filtering can be adaptively selected based on the comparison result between the local variance and the first threshold to perform an adaptive filtering operation on the MRI image. The local variance of the MRI image can be calculated using the following formula:

[0031] (1)

[0032] in, represents the local variance of the local region R in the MRI image, Indicates the first i The pixel value of pixels, represents the mean value of pixels in the local area R, Indicates the number of pixels in the local area R. In some embodiments, the size of the local area R can be set according to filtering requirements, such as 2×2, 3×3, 5×5, etc., and the embodiment of the present application does not impose any limitation in this regard.

[0033] Based on the above local variance, it can be compared with the first threshold to adaptively select Gaussian filtering or median filtering to perform adaptive filtering operations on the MRI image. Specifically, when the local variance is greater than the first threshold, Gaussian filtering can be selected; when the local variance is less than or equal to the first threshold, median filtering can be selected. As an example, the adaptive filtering operation can be implemented by the following formula:

[0034] (2)

[0035] in, represents the above local variance, represents the first threshold, Represents magnetic resonance imaging, and represent Gaussian filtering and median filtering respectively, and Represent the window sizes of Gaussian filtering and median filtering respectively.

[0036] In one implementation scenario, a local contrast analysis can be performed on the filtering result of the adaptive filtering operation to obtain a local contrast result, and the local contrast result is compared with a second threshold. Then, based on the comparison result between the local contrast result and the second threshold, a contrast-limited histogram algorithm or a global histogram algorithm is adaptively selected to perform an adaptive contrast enhancement operation on the magnetic resonance imaging. In some embodiments, the local contrast analysis can be performed on the filtering result of the adaptive filtering operation to obtain a local contrast result using the following formula:

[0037] (3)

[0038] in, represents the local contrast result, Represents the local area standard deviation of the filtering result of the adaptive filtering operation, Represents the local area average of the filter results of the adaptive filtering operation.

[0039] Based on the above local contrast result, it can be compared with the second threshold to adaptively select the corresponding algorithm to perform adaptive contrast enhancement on the MRI image. Specifically, when the local contrast result is less than the second threshold, the contrast limited histogram algorithm ("CLAHE") is selected for adaptive contrast enhancement; when the local contrast result is greater than or equal to the second threshold, the global histogram algorithm (" ”) for adaptive contrast enhancement. It can be understood that the contrast-limited histogram algorithm is an improved global histogram algorithm, which adaptively processes different areas of the image and limits the degree of contrast enhancement, thereby avoiding the problem of over-enhancement, improving the local contrast of the image and reducing noise amplification.

[0040] In an exemplary scenario, an adaptive contrast enhancement operation can be performed on a magnetic resonance imaging image using the following formula:

[0041] (4)

[0042] in, represents the contrast-enhanced MRI image, I represents the pixel value in the local area, represents the limited contrast of the CLAHE algorithm, represents the local contrast result, In some embodiments, the aforementioned global histogram algorithm can be based on Calculate, where represents the Dirac function, m represents the gray level, Represents pixel value.

[0043] In some implementations, in addition to the aforementioned multi-level adaptive preprocessing, other preprocessing operations can be performed on the MRI images, including image format conversion, image resizing, and image normalization. For example, the NumpyToTensor class can be used to convert the MRI images from NumPy array format to PyTorch tensors and then normalize them. The Resize class can be used to resize the MRI images to a size suitable for the hybrid residual network input (e.g., to 224x224 pixels), and the Normalize class can be used to normalize the MRI images to improve the stability and efficiency of subsequent network recognition.

[0044] After obtaining the preprocessed nuclear magnetic resonance image, at step S103, the trained hybrid residual network is used to perform multimodal feature extraction on the preprocessed nuclear magnetic resonance image to obtain a target feature result. In one embodiment, the trained hybrid residual network may include at least an initial convolution module, a cross-attention module and a multi-scale residual module. In the process of obtaining the target feature result, the initial convolution module is used to perform an initial convolution operation on the preprocessed nuclear magnetic resonance image to obtain an initial multimodal feature, and the cross-attention module is used to fuse the features of different modalities in the initial multimodality, and the features are combined with the preprocessed nuclear magnetic resonance image once to obtain an intermediate fusion result. Then, the multi-scale residual module is used to extract the multi-scale residual features of the intermediate fusion result, and the multi-scale residual features are feature spliced to obtain a feature splicing result; and the feature splicing result is combined with the preprocessed nuclear magnetic resonance image for a second time to obtain the target feature result.

[0045] In some embodiments, the initial convolution module can be configured to include, for example, 1 input channel, 64 output channels, a 7x7 convolution kernel, a step size of 2, and no initial convolution parameters such as bias. It can be expressed as ,in represents the initial convolution, represents the input MRI image, represents the i-th mode.

[0046] In some embodiments, the aforementioned cross attention module may include attention calculation and modality fusion calculation. For attention calculation, it first calculates the attention weights between different modalities. For example, based on Calculate attention weights between different modalities , and Represent the query ("Query") and key ("Key") in the attention mechanism respectively. represents the i-th mode, Represents the jth modality other than i. Then, the attention features under each modality are obtained based on the product of the attention weight and the value (“Value”) in the attention mechanism, such as the attention features under each modality , Represents the value in the attention mechanism. Furthermore, for modality fusion calculation, the attention features under each modality are combined with the preprocessed MRI image (that is, modality fusion) to obtain the intermediate fusion result .

[0047] In some embodiments, the aforementioned multi-scale residual module may include a multi-scale convolution layer, which includes convolution kernels of different scales. Specifically, first, by using convolution kernels of different scales in each residual module to extract multi-scale residual features of the intermediate fusion result in parallel, that is, to obtain convolution results of different scales. For example, , , Corresponding to different scales , , Then, the multi-scale residual features are spliced to obtain the feature splicing result Furthermore, the feature splicing results are combined with the pre-processed MRI images to obtain the target feature results. , that is, based on the fusion of multi-scale features, the residual connection is used to enhance the model learning ability, and the final output feature In some embodiments, before performing feature concatenation on the multi-scale residual features, a batch normalization layer may be included to normalize the layer outputs and improve the stability and convergence speed of the model training.

[0048] Based on this, the embodiment of the present application performs multimodal feature extraction on the preprocessed magnetic resonance imaging by using a hybrid residual network including an initial convolution module, a cross-attention module and a multi-scale residual module. It can fully integrate the features of different modalities, capture the complementary information and multi-scale information between different modalities, and significantly improve the accuracy and robustness of magnetic resonance imaging modality recognition, thereby being able to extract accurate target feature results.

[0049] Furthermore, at step S104, the final modality of the MRI image is identified based on the target feature results. In one embodiment, a fully connected layer can be used to map the target feature results to different modality categories, and a classification function can be used to calculate the probability of each modality category, thereby identifying the modality corresponding to the maximum probability value as the final modality of the MRI image. That is, the modality corresponding to the maximum probability value output by the hybrid residual network is used as the final modality of the MRI image. As previously described, the final modality of the MRI image can include at least a T1-weighted modality, a T2-weighted modality, and a fluid-attenuated inversion recovery modality.

[0050] In one embodiment, a hybrid residual network can be trained to obtain a trained hybrid residual network by inputting MRI images of different modalities into the hybrid residual network for feature extraction to obtain multimodal features, then setting uncertainty weights for the multimodal features, and calculating a total loss function for adaptive modal weights based on the uncertainty weights to train the hybrid residual network to obtain a trained hybrid residual network. In some embodiments, the different modalities of the MRI images can include, for example, T1-weighted modalities, T2-weighted modalities, and fluid-attenuated inversion recovery modalities.

[0051] In one implementation scenario, the total loss function of the adaptive modal weight can be calculated as follows: :

[0052] (5)

[0053] in, represents the uncertainty weight, and ,and represents the confidence variance of modal identification, N represents the number of samples, and denote the true mode and the predicted mode, respectively. It can be, for example, the cross entropy loss.

[0054] Through the above-mentioned total loss function of the adaptive modal weights in the embodiment of the present application, the loss weights can be dynamically adjusted according to the classification difficulty of each modality to increase the attention to difficult modalities, so that the hybrid residual network can adaptively learn the importance of different modalities and improve the recognition accuracy of the hybrid residual network for multimodal data.

[0055] In combination with the above description, it can be seen that the embodiment of the present application performs multi-level adaptive preprocessing operations on the nuclear magnetic resonance image to dynamically adjust the preprocessing parameters multiple times, thereby better retaining the detailed information of the image. Then, the multimodal feature extraction of the preprocessed nuclear magnetic resonance image is performed through the trained hybrid residual network, which can fully utilize the complementary information and multi-scale features of the multimodal image to extract accurate target feature results, thereby significantly improving the accuracy and robustness of the nuclear magnetic resonance image modality recognition, and improving the accuracy of the final modality recognition of the nuclear magnetic resonance image. Further, in some embodiments, the embodiment of the present application trains the hybrid residual network through the total loss function of the adaptive modality weight, so that the hybrid residual network can adaptively learn the importance of different modalities, thereby improving the recognition accuracy and reliability of the hybrid residual network for different modalities of nuclear magnetic resonance images. In some embodiments, the scheme of the present application can also be used for modality recognition of CT images, such as identifying whether it is standard CT, enhanced CT or low-dose CT.

[0056] Figure 2 1 is an exemplary flow chart illustrating a multi-stage adaptive pre-processing operation according to an embodiment of the present application. It should be understood that Figure 2 is the above Figure 1 is a specific embodiment of step S102 in method 100, so the above Figure 1 The description of step S102 in method 100 is also applicable to Figure 2 .

[0057] like Figure 2 As shown in FIG, at step S201, a nuclear magnetic resonance image is acquired. In one implementation scenario, the nuclear magnetic resonance image can be acquired by, for example, a magnetic resonance acquisition device. Then, at step S202, the local variance of the nuclear magnetic resonance image is calculated. Specifically, the local variance of the MRI image can be calculated based on the above formula (1): Based on the local variance At step S203, the local variance is determined Is it greater than the first threshold T, that is, judging >T is established. If >T, in step S204, Gaussian filtering is selected. Otherwise, In step S205 , median filtering is selected to perform an adaptive filtering operation on the magnetic resonance image to obtain a filtering result of the adaptive filtering operation.

[0058] Next, at step S206, a local contrast analysis is performed on the filtering result after Gaussian filtering or median filtering to obtain a local contrast result Specifically, the local contrast result can be obtained based on the above formula (3): Then, in step S207, the local contrast result is determined. Is it greater than or equal to the second threshold? , that is, judgment Is it established? If the value of the global histogram is true, then in step S208, the global histogram algorithm is selected for contrast enhancement. In step S209, the CLAHE algorithm is selected to perform contrast enhancement, so as to obtain a pre-processed MRI image in step S210.

[0059] Figure 3 FIG. 1 is a flow chart showing an overall exemplary process for identifying a magnetic resonance imaging modality according to an embodiment of the present application. Figure 3 As shown in FIG, at step S301, a nuclear magnetic resonance image is acquired. Based on the acquired nuclear magnetic resonance image, at step S302, a multi-level adaptive preprocessing operation is performed on the nuclear magnetic resonance image to obtain a preprocessed nuclear magnetic resonance image. For more details about the multi-level adaptive preprocessing operation, please refer to the above Figure 1 and Figure 2 Next, in step S303, the pre-processed MRI image is input into the trained hybrid residual network.

[0060] In some embodiments, the trained hybrid residual network may include at least an initial convolution module, a cross-attention module, and a multi-scale residual module. Correspondingly, in step S303-1, step S303-2, and step S303-3, the target feature result is obtained in sequence through the initial convolution, attention operation, modality fusion, and multi-scale feature fusion and residual learning of the hybrid residual network. Furthermore, at step S304, the modality corresponding to the feature with the largest probability value in the target feature result is identified as the final modality of the magnetic resonance imaging. Specifically, the final modality of the magnetic resonance imaging may include at least T1-weighted modality, T2-weighted modality, and fluid-attenuated inversion recovery modality.

[0061] Figure 4 : is an exemplary schematic diagram showing a hybrid residual network according to an embodiment of the present application. Figure 4As shown in , the hybrid residual network may include an initial convolution module 401, a cross attention module 402 and a multi-scale residual module 403. Among them, the aforementioned initial convolution module 401 may include an initial convolution 401-1, and the pre-processed magnetic resonance image obtains initial multimodal features after the initial convolution 401-1, such as initial multimodal features 401-11, initial multimodal features 401-12, initial multimodal features 401-13 and initial multimodal features 401-14. Taking the initial multimodal features 401-11 and the initial multimodal features 401-12 as examples, by calculating the attention weights between different modalities , and then multiply the attention weight by the value in the attention mechanism (for example, As shown in FIG4 , attention features for each modality can be obtained, such as attention features 402-1, 402-2, and 402-3 for each modality. By combining the attention features for each modality with the preprocessed MRI image 404 (i.e., modality fusion), an intermediate fusion result 405 is obtained.

[0062] Based on the obtained intermediate fusion result 405, the multi-scale convolution layer in the multi-scale residual module 403, such as the 1×1 convolution layer 403-11, the 2×2 convolution layer 403-21, the 3×3 convolution layer 403-31 and the 4×4 convolution layer 403-41, is used to extract the multi-scale residual features of the intermediate fusion result. In some embodiments, the multi-scale residual module 403 is further provided with a batch normalization layer, such as a batch normalization layer 403-12, a batch normalization layer 403-22, a batch normalization layer 403-32 and a batch normalization layer 403-42, to obtain normalized residual features of different scales. Furthermore, the aforementioned multi-scale residual features are feature spliced to obtain a feature splicing result 406, and the feature splicing result 406 is combined with the pre-processed magnetic resonance image 404 for a second time to obtain a target feature result 407.

[0063] As can be seen from the previous article, the target feature results are mapped to different modality categories by using a fully connected layer, and the probability of each modality category is calculated using a classification function, and then the modality corresponding to the maximum probability value is identified as the final modality of the magnetic resonance imaging.

[0064] Figure 5 is a block diagram illustrating an exemplary structure of a device 500 for identifying magnetic resonance imaging modalities according to an embodiment of the present application. It is understood that the device 500 may include the apparatus of the embodiment of the present application, and the device implementing the solution of the present application may be a single device (e.g., a computing device) or a multifunctional device including various peripheral devices.

[0065] like Figure 5As shown in , the device of the present application may also include a central processing unit ("CPU") 511, which can be a general-purpose CPU, a dedicated CPU, or other execution unit for information processing and program execution. Furthermore, the device 500 may also include a mass storage 512 and a read-only memory ("ROM") 513. The mass storage 512 can be configured to store various data, including various data related to MRI images, target feature results, the final modality of the MRI images, algorithm data, intermediate results, and various programs required to operate the device 500. The ROM 513 can be configured to store data and instructions required for power-on self-test of the device 500, initialization of various functional modules in the system, drivers for the system's basic input / output, and booting the operating system.

[0066] Optionally, device 500 may also include other hardware platforms or components, such as the illustrated tensor processing unit ("TPU") 514, graphics processing unit ("GPU") 515, field programmable gate array ("FPGA") 516, and machine learning unit ("MLU") 517. It will be appreciated that while various hardware platforms or components are shown in device 500, these are merely exemplary and non-limiting, and those skilled in the art may add or remove corresponding hardware as needed. For example, device 500 may include only a CPU, associated storage devices, and interface devices to implement the method for identifying magnetic resonance imaging modalities of the present application.

[0067] In some embodiments, to facilitate data transmission and interaction with external networks, the device 500 of the present application further includes a communication interface 518, which allows the device 500 to connect to a local area network / wireless local area network ("LAN / WLAN") 505 via the communication interface 518, and further connect to a local server 506 or the Internet 507 via the LAN / WLAN. Alternatively or additionally, the device 500 of the present application may also connect directly to the Internet or a cellular network via the communication interface 518 using wireless communication technology, such as third generation ("3G"), fourth generation ("4G"), or fifth generation ("5G") wireless communication technology. In some application scenarios, the device 500 of the present application may also access a server 508 and a database 509 on an external network as needed to obtain various known algorithms, data, and modules, and may remotely store various data, such as various data or instructions used to present, for example, MRI images, target feature results, and the final modality of the MRI images.

[0068] Peripheral devices of device 500 may include a display device 502, an input device 503, and a data transmission interface 504. In one embodiment, display device 502 may include, for example, one or more speakers and / or one or more visual displays, configured to provide voice prompts and / or display images and videos of the MRI modality used to identify the MRI imaging modality. Input device 503 may include, for example, a keyboard, a mouse, a microphone, a gesture capture camera, or other input buttons or controls configured to receive audio data input and / or user commands. Data transmission interface 504 may include, for example, a serial interface, a parallel interface, a Universal Serial Bus (USB), a Small Computer System Interface (SCSI), Serial ATA, FireWire, PCI Express, or a High-Definition Multimedia Interface (HDMI), configured to transmit and interact with other devices or systems. According to the present invention, data transmission interface 504 can receive MRI images acquired by an MRI device and transmit the MRI images or various other types of data or results to device 500.

[0069] The CPU 511, mass storage 512, ROM 513, TPU 514, GPU 515, FPGA 516, MLU 517, and communication interface 518 of the device 500 of the present application can be interconnected via a bus 519 and can interact with peripheral devices via the bus. In one embodiment, the CPU 511 can control other hardware components in the device 500 and its peripheral devices via the bus 519.

[0070] Combination of the above Figure 5 The present invention describes a device for identifying magnetic resonance imaging modalities that can be used to implement the present invention. It should be understood that the device structure or architecture described herein is merely exemplary, and the implementation and implementation of the present invention are not limited thereto and may be modified without departing from the spirit of the present invention.

[0071] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by software programs. Therefore, the present application also provides a computer-readable storage medium, which stores computer-readable instructions for identifying magnetic resonance imaging modalities. When the computer-readable instructions are executed by one or more processors, they can be used to implement the present application in combination with the accompanying drawings. Figure 1 A method for identifying magnetic resonance imaging modalities is described.

[0072] It should be noted that although the operations of the present method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in that particular order, or that all of the operations shown must be performed to achieve the desired results. Rather, the steps depicted in the flowcharts may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into a single step, and / or a single step may be broken down into multiple steps.

[0073] It should be understood that when the terms "first," "second," "third," and "fourth," etc., are used in the claims, specification, and drawings of this application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "comprise" and "comprising" used in the specification and claims of this application indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0074] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this specification and claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" as used in this specification and claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0075] Although the implementation methods of this application are as described above, the contents are only examples adopted to facilitate understanding of this application and are not intended to limit the scope and application scenarios of this application. Any technician in the technical field described in this application can make any modifications and changes in the form and details of implementation without departing from the spirit and scope disclosed in this application, but the scope of patent protection of this application shall still be based on the scope defined by the attached claims.

[0076] In addition, the collection and acquisition of various data in this application complies with relevant laws and regulations and is authorized by the data provider. Any organization or individual that needs to obtain external data must obtain authorization in accordance with the law and ensure data security. They must not illegally collect, use, process, or transmit unauthorized or unprotected data, nor illegally buy, sell, provide, or disclose unauthorized or unprotected data.

Claims

1. A method for identifying a magnetic resonance imaging modality, characterized in that: include: Acquisition of magnetic resonance imaging; performing a multi-stage adaptive preprocessing operation on the nuclear magnetic resonance image to obtain a preprocessed nuclear magnetic resonance image; Using the trained hybrid residual network to perform multimodal feature extraction on the preprocessed magnetic resonance imaging to obtain target feature results; as well as identifying a final modality of the magnetic resonance imaging according to the target feature result; The trained hybrid residual network includes at least an initial convolution module, a multi-scale residual module, and a cross-attention module, and the trained hybrid residual network is used to perform multimodal feature extraction on the preprocessed magnetic resonance imaging to obtain target feature results, including: Using the initial convolution module to perform an initial convolution operation on the preprocessed nuclear magnetic resonance image to obtain initial multimodal features; Using the cross attention module to fuse the features of different modalities in the initial multimodality, and combining them with the preprocessed magnetic resonance imaging to obtain an intermediate fusion result; Extracting multi-scale residual features of the intermediate fusion result using the multi-scale residual module, and performing feature splicing on the multi-scale residual features to obtain a feature splicing result; and The feature stitching result is combined with the preprocessed nuclear magnetic resonance image for a second time to obtain the target feature result.

2. The method according to claim 1, characterized in that The multi-level adaptive preprocessing operation on the magnetic resonance imaging includes: An adaptive filtering operation and an adaptive contrast enhancement operation are sequentially performed on the nuclear magnetic resonance image to perform a multi-stage adaptive preprocessing operation on the nuclear magnetic resonance image.

3. The method according to claim 2, characterized in that The adaptive filtering operation is performed on the magnetic resonance imaging by the following operations: calculating the local variance of the magnetic resonance image; comparing the local variance to a first threshold; and Gaussian filtering or median filtering is adaptively selected according to a comparison result between the local variance and the first threshold value to perform the adaptive filtering operation on the nuclear magnetic resonance image.

4. The method according to claim 2, characterized in that The adaptive contrast enhancement operation is performed on the magnetic resonance imaging by the following operations: performing a local contrast analysis on a filtering result of the adaptive filtering operation to obtain a local contrast result; comparing the local contrast result with a second threshold; as well as A contrast limiting histogram algorithm or a global histogram algorithm is adaptively selected according to a comparison result between the local contrast result and the second threshold value to perform the adaptive contrast enhancement operation on the nuclear magnetic resonance image.

5. The method according to claim 1, wherein Wherein identifying the modality of the magnetic resonance imaging according to the target feature result includes: Mapping the target feature results to different modal categories using a fully connected layer, and calculating the probability of each modal category using a classification function; and The modality corresponding to the maximum probability value is identified as the final modality of the nuclear magnetic resonance image.

6. The method according to claim 5, characterized in that The final modality of the nuclear magnetic resonance image includes at least T1-weighted modality, T2-weighted modality and fluid-attenuated inversion recovery modality.

7. The method according to claim 1, characterized in that The hybrid residual network is trained by the following operations to obtain the trained hybrid residual network: Input MRI images of different modalities into a hybrid residual network for feature extraction to obtain multimodal features; Setting uncertainty weights for the multimodal features; as well as A total loss function of the adaptive modal weights is calculated based on the uncertainty weights to train the hybrid residual network to obtain the trained hybrid residual network.

8. A device for identifying magnetic resonance imaging modalities, characterized in that: include: processor; as well as A memory storing program instructions for identifying a nuclear magnetic resonance imaging modality, wherein when the program instructions are executed by the processor, the device implements the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that Computer-readable instructions for identifying a nuclear magnetic resonance imaging modality are stored thereon, and when the computer-readable instructions are executed by one or more processors, the method according to any one of claims 1 to 7 is implemented.

Images