Method and device for identifying nuclear magnetic resonance image modality and storage medium

Abstract

The invention discloses a method and equipment for identifying a nuclear magnetic resonance image mode and a storage medium. The method comprises the following steps: acquiring a nuclear magnetic resonance image; performing multi-stage adaptive preprocessing operation on the nuclear magnetic resonance image to obtain a preprocessed nuclear magnetic resonance image; performing multi-modal feature extraction on the preprocessed nuclear magnetic resonance image by using a trained mixed residual network to obtain a target feature result; and identifying a final mode of the nuclear magnetic resonance image according to the target feature result. By using the scheme of the invention, the precision and robustness of nuclear magnetic resonance image modal identification can be significantly improved, and the accuracy and efficiency of modal identification can be improved.

Description

Technical Field

[0001] The present application generally relates to the field of image processing technology. More specifically, the present application relates to a method, device and computer-readable storage medium for identifying a nuclear magnetic resonance imaging modality. Background Art

[0002] Magnetic Resonance Imaging (MRI) is a commonly used medical imaging technology that can provide rich biological tissue information depending on the imaging modality. Common MRI modalities include T1-weighted modality, T2-weighted modality, and fluid-attenuated inversion recovery ("FLAIR") modality, among which T1-weighted modality can provide anatomical structure information, T2-weighted modality can provide tumor information, and FLAIR modality can provide peritumoral edema area information. Therefore, it is necessary to identify the different modalities of MRI.

[0003] Traditional modality recognition is often done manually, which is not only time-consuming but also prone to errors. For example, in the modality recognition of medical images, radiologists need to check a large number of images one by one and manually annotate different modalities (such as CT, MRI, and ultrasound), which not only increases the workload of doctors, but also easily leads to misjudgment due to fatigue or subjective factors. In addition, the speed of manual annotation cannot meet the growing demand for medical image data. At present, some existing automated modality recognition methods mainly rely on rule-based systems or simple machine learning algorithms. However, rule-based methods usually rely on a predefined series of rules and thresholds to identify and classify modalities. For example, patent application CN202111675248.4 discloses a method for identifying multiple medical images, which records the recognition based on the edge or morphological features of the image. These rules are often static and difficult to adapt to complex and changeable actual situations. In particular, when encountering some situations where modality features are similar, rule-based methods are easy to fail, resulting in low accuracy. In addition, the application also records that by using algorithms such as support vector machines ("SVM") or decision trees, although the degree of automation of recognition can be improved to a certain extent, its effect is also limited. Specifically, algorithms such as SVM or decision trees rely on manually extracted features that may not fully capture the complex information in the image, and perform poorly when processing high-dimensional and nonlinear data, making it difficult to capture deep patterns and relationships in the image. As a result, when there are complex patterns or noise in the image data, the recognition accuracy is often not ideal.

[0004] In view of this, it is urgent to provide a solution for identifying the modality of MRI images. First, by performing multi-level adaptive preprocessing operations on MRI images, the preprocessing parameters can be dynamically adjusted multiple times to better retain the detailed information of the image. Then, multimodal feature extraction is performed on the preprocessed MRI images through a hybrid residual network, which fully utilizes the complementary information of multimodal images, significantly improves the accuracy and robustness of MRI image modality recognition, and improves the accuracy of modality recognition. Furthermore, the hybrid residual network is trained through the total loss function of the adaptive modality weight, so that the hybrid residual network can adaptively learn the importance of different modalities, thereby improving the recognition accuracy and reliability of the hybrid residual network. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present application proposes solutions for identifying magnetic resonance imaging modalities in multiple aspects.

[0006] In a first aspect, the present application provides a method for identifying the modality of a magnetic resonance imaging (MRI), comprising: acquiring a MRI image; performing a multi-level adaptive preprocessing operation on the MRI image to obtain a preprocessed MRI image; performing multimodal feature extraction on the preprocessed MRI image using a trained hybrid residual network to obtain a target feature result; and identifying the final modality of the MRI image based on the target feature result.

[0007] In one embodiment, performing a multi-stage adaptive preprocessing operation on the nuclear magnetic resonance image includes: sequentially performing an adaptive filtering operation and an adaptive contrast enhancement operation on the nuclear magnetic resonance image, so as to perform a multi-stage adaptive preprocessing operation on the nuclear magnetic resonance image.

[0008] In another embodiment, the adaptive filtering operation is performed on the nuclear magnetic resonance image by the following operations: calculating the local variance of the nuclear magnetic resonance image; comparing the local variance with a first threshold; and adaptively selecting Gaussian filtering or median filtering based on the comparison result of the local variance with the first threshold to perform the adaptive filtering operation on the nuclear magnetic resonance image.

[0009] In yet another embodiment, the adaptive contrast enhancement operation is performed on the nuclear magnetic resonance image by the following operations: performing local contrast analysis on the filtering result of the adaptive filtering operation to obtain a local contrast result; comparing the local contrast result with a second threshold; and adaptively selecting a contrast limiting histogram algorithm or a global histogram algorithm based on the comparison result between the local contrast result and the second threshold to perform the adaptive contrast enhancement operation on the nuclear magnetic resonance image.

[0010] In another embodiment, the trained hybrid residual network includes at least an initial convolution module, a multi-scale residual module and a cross-attention module, and the trained hybrid residual network is used to perform multimodal feature extraction on the preprocessed magnetic resonance image to obtain a target feature result, including: using the initial convolution module to perform an initial convolution operation on the preprocessed magnetic resonance image to obtain an initial multimodal feature; using the cross-attention module to fuse the features of different modalities in the initial multimodality, and combining them with the preprocessed magnetic resonance image once to obtain an intermediate fusion result; using the multi-scale residual module to extract the multi-scale residual features of the intermediate fusion result, and performing feature splicing on the multi-scale residual features to obtain a feature splicing result; and combining the feature splicing result with the preprocessed magnetic resonance image for a second time to obtain the target feature result.

[0011] In another embodiment, identifying the modality of the magnetic resonance image based on the target feature result includes: mapping the target feature result to different modality categories using a fully connected layer, and calculating the probability of each modality category using a classification function; and identifying the modality corresponding to the maximum probability value as the final modality of the magnetic resonance image.

[0012] In yet another embodiment, the final modality of the magnetic resonance image includes at least a T1-weighted modality, a T2-weighted modality, and a fluid-attenuated inversion recovery modality.

[0013] In another embodiment, a hybrid residual network is trained by the following operations to obtain the trained hybrid residual network: magnetic resonance images of different modalities are input into the hybrid residual network for feature extraction to obtain multimodal features; uncertainty weights are set for the multimodal features; and a total loss function of adaptive modal weights is calculated based on the uncertainty weights to train the hybrid residual network to obtain the trained hybrid residual network.

[0014] In a second aspect, the present application provides a device for identifying a nuclear magnetic resonance imaging modality, comprising: a processor; and a memory, wherein program instructions for identifying a nuclear magnetic resonance imaging modality are stored, and when the program instructions are executed by the processor, the device implements one or more embodiments of the aforementioned first aspect.

[0015] In a third aspect, the present application provides a computer-readable storage medium having stored thereon computer-readable instructions for identifying a magnetic resonance imaging modality. When the computer-readable instructions are executed by one or more processors, one or more embodiments of the aforementioned first aspect are implemented.

[0016] Through the scheme for identifying the modality of nuclear magnetic resonance images provided above, the embodiment of the present application first performs multi-level adaptive preprocessing operations on the nuclear magnetic resonance images to dynamically adjust the preprocessing parameters multiple times, thereby better retaining the detailed information of the image. Then, the preprocessed nuclear magnetic resonance images are subjected to multimodal feature extraction through the trained hybrid residual network, which can fully utilize the complementary information and multi-scale features of the multimodal images to extract accurate target feature results, thereby significantly improving the accuracy and robustness of the nuclear magnetic resonance image modality recognition, and improving the accuracy of the final modality recognition of the nuclear magnetic resonance images. Further, in some embodiments, the embodiment of the present application trains the hybrid residual network through the total loss function of the adaptive modality weight, so that the hybrid residual network can adaptively learn the importance of different modalities, thereby improving the recognition accuracy and reliability of the hybrid residual network for different modalities of nuclear magnetic resonance images. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 is an exemplary flowchart showing a method for identifying a nuclear magnetic resonance imaging modality according to an embodiment of the present application; Figure 2 is an exemplary flowchart showing a multi-stage adaptive preprocessing operation according to an embodiment of the present application; Figure 3 is an exemplary flowchart showing an overall process for identifying a nuclear magnetic resonance imaging modality according to an embodiment of the present application; Figure 4 is an exemplary schematic diagram showing a hybrid residual network according to an embodiment of the present application; Figure 5 It is an exemplary structural block diagram showing a device for identifying a nuclear magnetic resonance imaging modality according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0019] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0021] As used in this specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0022] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0023] Figure 1 FIG. 1 is an exemplary flowchart of a method 100 for identifying a nuclear magnetic resonance imaging modality according to an embodiment of the present application. Figure 1 As shown in , at step S101, a nuclear magnetic resonance image is acquired. In an implementation scenario, the nuclear magnetic resonance image can be acquired by, for example, acquiring a magnetic resonance acquisition device. Based on the nuclear magnetic resonance image acquired above, at step S102, a multi-level adaptive preprocessing operation is performed on the nuclear magnetic resonance image to obtain a preprocessed nuclear magnetic resonance image. In one embodiment, an adaptive filtering operation and an adaptive contrast enhancement operation can be performed on the nuclear magnetic resonance image in sequence to perform a multi-level adaptive preprocessing operation on the nuclear magnetic resonance image. That is, an adaptive filtering operation is first performed on the nuclear magnetic resonance image to obtain a filtering result of the adaptive filtering operation, and then an adaptive contrast enhancement operation is performed on the filtering result of the adaptive filtering operation to implement a multi-level adaptive preprocessing operation. In some embodiments, an adaptive contrast enhancement operation can also be performed on the nuclear magnetic resonance image first, and then an adaptive filtering operation is performed.

[0024] Through the aforementioned multi-level adaptive preprocessing operation of the embodiment of the present application, the variance and contrast of the local area can be adaptively calculated according to the noise level or image quality of the magnetic resonance image, so as to dynamically adjust the filter parameters and contrast intensity, better retain the detail information of the magnetic resonance image, so as to improve the subsequent recognition accuracy.

[0025] In one implementation scenario, the local variance of the nuclear magnetic resonance image can be calculated, the local variance can be compared with a first threshold, and then Gaussian filtering or median filtering can be adaptively selected according to the comparison result between the local variance and the first threshold to perform adaptive filtering on the nuclear magnetic resonance image. The local variance of the nuclear magnetic resonance image can be calculated by the following formula: (1) in, represents the local variance of the local region R in the MRI image, Indicates the first i The pixel value of pixels, represents the mean value of pixels in the local area R, Indicates the number of pixels in the local area R. In some embodiments, the size of the local area R can be set according to filtering requirements, such as 2×2, 3×3, 5×5, etc., and the embodiment of the present application does not impose any limitation in this regard.

[0026] Based on the above local variance, it can be compared with the first threshold to adaptively select Gaussian filtering or median filtering to perform adaptive filtering operations on the nuclear magnetic resonance image. Specifically, when the local variance is greater than the first threshold, Gaussian filtering can be selected; when the local variance is less than or equal to the first threshold, median filtering can be selected. As an example, the adaptive filtering operation can be implemented by the following formula: (2) in, represents the above local variance, represents the first threshold, Represents magnetic resonance imaging, and represent Gaussian filtering and median filtering respectively, and Represent the window sizes of Gaussian filtering and median filtering respectively.

[0027] In one implementation scenario, a local contrast analysis may be performed on the filtering result of the adaptive filtering operation to obtain a local contrast result, and the local contrast result may be compared with a second threshold, and then a contrast limiting histogram algorithm or a global histogram algorithm may be adaptively selected according to the comparison result between the local contrast result and the second threshold, so as to perform an adaptive contrast enhancement operation on the magnetic resonance imaging. In some embodiments, a local contrast analysis may be performed on the filtering result of the adaptive filtering operation to obtain a local contrast result by using the following formula: (3) in, represents the local contrast result, Represents the local area standard deviation of the filtering result of the adaptive filtering operation, Represents the local area average of the filter results of the adaptive filtering operation.

[0028] According to the above local contrast result, it can be compared with the second threshold to adaptively select the corresponding algorithm to perform adaptive contrast enhancement on the MRI image. Specifically, when the local contrast result is less than the second threshold, the contrast limited histogram algorithm ("CLAHE") is selected for adaptive contrast enhancement; when the local contrast result is greater than or equal to the second threshold, the global histogram algorithm (" ”) for adaptive contrast enhancement. It can be understood that the contrast-limited histogram algorithm is an improved global histogram algorithm, which adaptively processes different areas of the image and limits the degree of contrast enhancement, thereby avoiding the problem of over-enhancement, improving the local contrast of the image and reducing noise amplification.

[0029] In an exemplary scenario, an adaptive contrast enhancement operation can be performed on a magnetic resonance imaging image by using the following formula: (4) in, represents the contrast-enhanced MRI image, I represents the pixel value in the local area, represents the limiting contrast of the CLAHE algorithm, represents the local contrast result, In some embodiments, the aforementioned global histogram algorithm can be based on Calculate, where represents the Dirac function, m represents the gray level, Represents pixel value.

[0030] In some implementation scenarios, in addition to the above-mentioned multi-level adaptive preprocessing operations, preprocessing operations such as image format conversion, image size adjustment, and image normalization can also be performed on the MRI images. For example, the NumpyToTensor class is used to convert the MRI images from the NumPy array format to the PyTorch tensor and perform normalization. The Resize class is used to adjust the MRI images to a size suitable for the input of the hybrid residual network (for example, to 224x224 pixels) and the Normalize class is used to normalize the MRI images to improve the stability and efficiency of subsequent network recognition.

[0031] After obtaining the preprocessed nuclear magnetic resonance image, at step S103, the trained hybrid residual network is used to perform multimodal feature extraction on the preprocessed nuclear magnetic resonance image to obtain the target feature result. In one embodiment, the trained hybrid residual network may include at least an initial convolution module, a cross-attention module and a multi-scale residual module. In the process of obtaining the target feature result, the initial convolution module is used to perform an initial convolution operation on the preprocessed nuclear magnetic resonance image to obtain the initial multimodal feature, and the cross-attention module is used to fuse the features of different modes in the initial multimodality, and the features are combined with the preprocessed nuclear magnetic resonance image once to obtain an intermediate fusion result. Then, the multi-scale residual module is used to extract the multi-scale residual features of the intermediate fusion result, and the multi-scale residual features are feature spliced ​​to obtain the feature splicing result; and the feature splicing result is combined with the preprocessed nuclear magnetic resonance image for a second time to obtain the target feature result.

[0032] In some embodiments, the aforementioned initial convolution module can be configured to include, for example, 1 input channel, 64 output channels, a 7x7 convolution kernel, a step size of 2, and an initial convolution without using parameters such as bias. It can be expressed as ,in represents the initial convolution, represents the input MRI image, represents the i-th mode.

[0033] In some embodiments, the aforementioned cross-attention module may include attention calculation and modality fusion calculation. For attention calculation, it first calculates the attention weights between different modalities. For example, based on Calculate attention weights between different modalities , and Respectively represent the query ("Query") and key ("Key") in the attention mechanism. Among them, represents the i-th mode, represents the jth modality other than i. Next, the attention features under each modality are obtained based on the product of the attention weight and the value (“Value”) in the attention mechanism, such as the attention features under each modality , Represents the value in the attention mechanism. Furthermore, for modality fusion calculation, the attention features under each modality are combined with the preprocessed MRI images (that is, modality fusion) to obtain the intermediate fusion result .

[0034] In some embodiments, the aforementioned multi-scale residual module may include a multi-scale convolution layer, which includes convolution kernels of different scales. Specifically, firstly, the multi-scale residual features of the intermediate fusion results are extracted in parallel by using convolution kernels of different scales in each residual module, that is, convolution results of different scales are obtained. For example, , , Corresponding to different scales , , Next, the multi-scale residual features are concatenated to obtain the feature concatenation result Furthermore, the feature splicing result is combined with the preprocessed MRI image to obtain the target feature result. That is, based on the fusion of multi-scale features, the residual connection is used to enhance the model learning ability, and the final output feature In some embodiments, before the multi-scale residual features are concatenated, a batch normalization layer may be included to normalize the layer output and improve the stability and convergence speed of the model training.

[0035] Based on this, the embodiment of the present application performs multimodal feature extraction on the preprocessed magnetic resonance imaging by using a hybrid residual network including an initial convolution module, a cross-attention module and a multi-scale residual module. It can fully integrate the features of different modalities, capture the complementary information and multi-scale information between different modalities, and significantly improve the accuracy and robustness of magnetic resonance imaging modality recognition, thereby being able to extract accurate target feature results.

[0036] Further, at step S104, the final modality of the nuclear magnetic resonance image is identified according to the target feature result. In one embodiment, the target feature result can be mapped to different modality categories using a fully connected layer, and the probability of each modality category can be calculated using a classification function, and then the modality corresponding to the maximum probability value is identified as the final modality of the nuclear magnetic resonance image. That is, the modality corresponding to the maximum probability value output by the hybrid residual network is used as the final modality of the nuclear magnetic resonance image. As mentioned above, the final modality of the nuclear magnetic resonance image can include at least T1-weighted modality, T2-weighted modality and fluid-attenuated inversion recovery modality.

[0037] In one embodiment, the hybrid residual network can be trained by the following operations to obtain a trained hybrid residual network: inputting magnetic resonance images of different modalities into the hybrid residual network for feature extraction to obtain multimodal features, and then setting uncertainty weights for the multimodal features to calculate the total loss function of the adaptive modal weights based on the uncertainty weights to train the hybrid residual network to obtain a trained hybrid residual network. In some embodiments, the different modalities of the aforementioned magnetic resonance images may include, for example, T1-weighted modality, T2-weighted modality, and fluid-attenuated inversion recovery modality.

[0038] In one implementation scenario, the total loss function of the adaptive modal weight can be calculated as follows: : (5) in, represents the uncertainty weight, and ,and represents the confidence variance of modal identification, N represents the number of samples, and denote the true mode and the predicted mode respectively, It could be, for example, the cross entropy loss.

[0039] Through the above-mentioned total loss function of the adaptive modal weights in the embodiment of the present application, the loss weights can be dynamically adjusted according to the classification difficulty of each modality to increase the attention to difficult modalities, so that the hybrid residual network can adaptively learn the importance of different modalities and improve the recognition accuracy of the hybrid residual network for multimodal data.

[0040] Combined with the above description, it can be seen that the embodiment of the present application performs multi-level adaptive preprocessing operations on the nuclear magnetic resonance image to dynamically adjust the preprocessing parameters multiple times, thereby better retaining the detailed information of the image. Then, the multimodal feature extraction of the preprocessed nuclear magnetic resonance image is performed through the trained hybrid residual network, which can fully utilize the complementary information and multi-scale features of the multimodal image to extract accurate target feature results, thereby significantly improving the accuracy and robustness of the modality recognition of the nuclear magnetic resonance image, and improving the accuracy of the final modality recognition of the nuclear magnetic resonance image. Further, in some embodiments, the embodiment of the present application trains the hybrid residual network through the total loss function of the adaptive modality weight, so that the hybrid residual network can adaptively learn the importance of different modalities, and improve the recognition accuracy and reliability of the hybrid residual network for different modalities of nuclear magnetic resonance images. In some embodiments, the scheme of the present application can also be used for modality recognition of CT images, such as recognizing whether it is standard CT, enhanced CT or low-dose CT.

[0041] Figure 2 is an exemplary flowchart showing a multi-stage adaptive preprocessing operation according to an embodiment of the present application. It should be understood that, Figure 2 is the above Figure 1 is a specific embodiment of step S102 in method 100, so the above Figure 1 The description of step S102 in method 100 is also applicable to Figure 2 .

[0042] like Figure 2 As shown in FIG. 1 , at step S201, a nuclear magnetic resonance image is acquired. In one implementation scenario, the nuclear magnetic resonance image can be acquired by, for example, a magnetic resonance acquisition device. Next, at step S202, the local variance of the nuclear magnetic resonance image is calculated. Specifically, the local variance of the MRI image can be calculated based on the above formula (1): Based on the local variance , in step S203, the local variance is determined Is it greater than the first threshold T, that is, judging >T is true. >T, in step S204, Gaussian filtering is selected. Otherwise, In step S205, a median filter is selected to perform an adaptive filtering operation on the nuclear magnetic resonance image to obtain a filtering result of the adaptive filtering operation.

[0043] Next, at step S206, a local contrast analysis is performed on the filtering result after Gaussian filtering or median filtering to obtain a local contrast result Specifically, the local contrast result can be obtained based on the above formula (3): Next, in step S207, the local contrast result is determined. Is it greater than or equal to the second threshold? , that is, judge Is it established? If the global histogram algorithm is selected for contrast enhancement in step S208, At step S209, the CLAHE algorithm is selected for contrast enhancement, so as to obtain a preprocessed magnetic resonance image at step S210.

[0044] Figure 3 FIG. 1 is an exemplary flowchart showing an overall process for identifying a nuclear magnetic resonance imaging modality according to an embodiment of the present application. Figure 3 As shown in FIG. 1 , at step S301, a nuclear magnetic resonance image is acquired. Based on the acquired nuclear magnetic resonance image, at step S302, a multi-level adaptive preprocessing operation is performed on the nuclear magnetic resonance image to obtain a preprocessed nuclear magnetic resonance image. For more details about the multi-level adaptive preprocessing operation, please refer to the above Figure 1 and Figure 2 Next, at step S303, the pre-processed MRI image is input into the trained hybrid residual network.

[0045] In some embodiments, the trained hybrid residual network may include at least an initial convolution module, a cross-attention module, and a multi-scale residual module. Correspondingly, in step S303-1, step S303-2, and step S303-3, the target feature result is obtained in sequence through the initial convolution, attention operation, modality fusion, and multi-scale feature fusion and residual learning of the hybrid residual network. Further, at step S304, the modality corresponding to the feature with the largest probability value in the target feature result is identified as the final modality of the magnetic resonance image. Specifically, the final modality of the magnetic resonance image may include at least T1-weighted modality, T2-weighted modality, and fluid-attenuated inversion recovery modality.

[0046] Figure 4 is an exemplary schematic diagram showing a hybrid residual network according to an embodiment of the present application. Figure 4As shown in , the hybrid residual network may include an initial convolution module 401, a cross attention module 402 and a multi-scale residual module 403. The initial convolution module 401 may include an initial convolution 401-1, and the pre-processed magnetic resonance image is subjected to the initial convolution 401-1 to obtain initial multimodal features, such as initial multimodal features 401-11, initial multimodal features 401-12, initial multimodal features 401-13 and initial multimodal features 401-14. Taking the initial multimodal features 401-11 and the initial multimodal features 401-12 as examples, by calculating the attention weights between different modalities , and then multiply the attention weight by the value in the attention mechanism (for example, As shown in FIG. 4 ), attention features under each modality can be obtained, such as attention features 402-1, attention features 402-2, and attention features 402-3 under each modality. By combining the attention features under each modality with the preprocessed magnetic resonance image 404 (i.e., modality fusion), an intermediate fusion result 405 is obtained.

[0047] Based on the obtained intermediate fusion result 405, the multi-scale convolutional layers in the multi-scale residual module 403, such as 1×1 convolutional layer 403-11, 2×2 convolutional layer 403-21, 3×3 convolutional layer 403-31 and 4×4 convolutional layer 403-41, are used to extract the multi-scale residual features of the intermediate fusion result. In some embodiments, the multi-scale residual module 403 is also provided with batch normalization layers, such as batch normalization layer 403-12, batch normalization layer 403-22, batch normalization layer 403-32 and batch normalization layer 403-42, to obtain normalized residual features of different scales. Further, the aforementioned multi-scale residual features are feature spliced ​​to obtain a feature splicing result 406, and the feature splicing result 406 is combined with the pre-processed nuclear magnetic resonance image 404 for a second time to obtain a target feature result 407.

[0048] As can be seen from the previous text, the target feature results are mapped to different modality categories by using a fully connected layer, and the probability of each modality category is calculated using a classification function, and then the modality corresponding to the maximum probability value is identified as the final modality of the magnetic resonance imaging.

[0049] Figure 5 is an exemplary structural block diagram of a device 500 for identifying a nuclear magnetic resonance imaging modality according to an embodiment of the present application. It is understood that the device 500 may include an apparatus of an embodiment of the present application, and the device implementing the solution of the present application may be a single device (such as a computing device) or a multifunctional device including various peripheral devices.

[0050] like Figure 5As shown in , the device of the present application may also include a central processing unit or central processing unit ("CPU") 511, which may be a general-purpose CPU, a dedicated CPU, or other information processing and program execution unit. Further, the device 500 may also include a large-capacity memory 512 and a read-only memory ("ROM") 513, wherein the large-capacity memory 512 may be configured to store various types of data, including various programs related to nuclear magnetic resonance images, target feature results, final modalities of nuclear magnetic resonance images, algorithm data, intermediate results, and various programs required to run the device 500. The ROM 513 may be configured to store the power-on self-test of the device 500, the initialization of each functional module in the system, the basic input / output driver of the system, and the data and instructions required to boot the operating system.

[0051] Optionally, the device 500 may also include other hardware platforms or components, such as the tensor processing unit ("TPU") 514, the graphics processing unit ("GPU") 515, the field programmable gate array ("FPGA") 516, and the machine learning unit ("MLU") 517 shown. It is understood that although a variety of hardware platforms or components are shown in the device 500, they are only exemplary and not restrictive, and those skilled in the art can add or remove corresponding hardware according to actual needs. For example, the device 500 may only include a CPU, related storage devices, and interface devices to implement the method for identifying the nuclear magnetic resonance imaging modality of the present application.

[0052] In some embodiments, in order to facilitate the transmission and interaction of data with an external network, the device 500 of the present application further includes a communication interface 518, so that it can be connected to a local area network / wireless local area network ("LAN / WLAN") 505 through the communication interface 518, and then connected to a local server 506 or to the Internet ("Internet") 507 through the LAN / WLAN. Alternatively or additionally, the device 500 of the present application can also be directly connected to the Internet or a cellular network through the communication interface 518 based on wireless communication technology, such as wireless communication technology based on the third generation ("3G"), the fourth generation ("4G") or the fifth generation ("5G") of the present invention. In some application scenarios, the device 500 of the present application can also access a server 508 and a database 509 of an external network as needed to obtain various known algorithms, data and modules, and can remotely store various data, such as various types of data or instructions for presenting, for example, nuclear magnetic resonance images, target feature results, and the final modality of nuclear magnetic resonance images.

[0053] The peripheral devices of the device 500 may include a display device 502, an input device 503, and a data transmission interface 504. In one embodiment, the display device 502 may include, for example, one or more speakers and / or one or more visual displays, which are configured to perform voice prompts and / or image video display for the present application for identifying the nuclear magnetic resonance imaging modality. The input device 503 may include other input buttons or controls such as a keyboard, a mouse, a microphone, a gesture capture camera, etc., which are configured to receive input of audio data and / or user instructions. The data transmission interface 504 may include, for example, a serial interface, a parallel interface or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), a serial ATA, a FireWire ("FireWire"), a PCI Express, and a high-definition multimedia interface ("HDMI"), etc., which are configured for data transmission and interaction with other devices or systems. According to the solution of the present application, the data transmission interface 504 can receive the nuclear magnetic resonance image collected from the MRI device, and transmit the data or results including the nuclear magnetic resonance image or various other types to the device 500.

[0054] The CPU 511, mass storage 512, ROM 513, TPU 514, GPU 515, FPGA 516, MLU 517 and communication interface 518 of the device 500 of the present application can be interconnected through a bus 519, and data interaction with peripheral devices can be achieved through the bus. In one embodiment, through the bus 519, the CPU 511 can control other hardware components in the device 500 and its peripheral devices.

[0055] Combination of the above Figure 5 The device for identifying the MRI imaging modality that can be used to implement the present application is described. It should be understood that the device structure or architecture here is only exemplary, and the implementation method and implementation entity of the present application are not limited thereto, but can be changed without departing from the spirit of the present application.

[0056] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by software programs. Therefore, the present application also provides a computer-readable storage medium, on which computer-readable instructions for identifying magnetic resonance imaging modalities are stored. When the computer-readable instructions are executed by one or more processors, they can be used to implement the present application in combination with the accompanying drawings. Figure 1 A method for identifying a magnetic resonance imaging modality is described.

[0057] It should be noted that although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0058] It should be understood that when the terms "first", "second", "third" and "fourth" are used in the claims, the specification and the drawings of the present application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0059] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0060] Although the implementation methods of the present application are as above, the contents described are only examples adopted to facilitate the understanding of the present application, and are not intended to limit the scope and application scenarios of the present application. Any technician in the technical field described in the present application can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present application, but the scope of patent protection of the present application shall still be subject to the scope defined in the attached claims.

[0061] In addition, the collection and acquisition of various data in this application complies with relevant laws and regulations and is authorized by the data provider. Any organization or individual that needs to obtain external data must obtain authorization in accordance with the law and ensure data security. It is not allowed to illegally collect, use, process, or transmit unauthorized or unprotected data, or to illegally buy, sell, provide, or disclose unauthorized or unprotected data.

Claims

1. A method for identifying a magnetic resonance imaging modality, characterized in that: include: Acquisition of magnetic resonance imaging; Performing a multi-level adaptive preprocessing operation on the nuclear magnetic resonance image to obtain a preprocessed nuclear magnetic resonance image; Using the trained hybrid residual network to perform multimodal feature extraction on the preprocessed magnetic resonance imaging to obtain target feature results; as well as A final modality of the magnetic resonance imaging is identified based on the target feature result.

2. The method according to claim 1, characterized in that The multi-level adaptive preprocessing operation on the nuclear magnetic resonance image includes: An adaptive filtering operation and an adaptive contrast enhancement operation are sequentially performed on the nuclear magnetic resonance image, so as to perform a multi-level adaptive preprocessing operation on the nuclear magnetic resonance image.

3. The method according to claim 2, characterized in that The adaptive filtering operation is performed on the magnetic resonance imaging by the following operations: Calculating the local variance of the magnetic resonance image; comparing the local variance to a first threshold; and Gaussian filtering or median filtering is adaptively selected according to the comparison result between the local variance and the first threshold value to perform the adaptive filtering operation on the nuclear magnetic resonance image.

4. The method according to claim 2, characterized in that: The adaptive contrast enhancement operation is performed on the magnetic resonance imaging by the following operations: Performing local contrast analysis on the filtering result of the adaptive filtering operation to obtain a local contrast result; comparing the local contrast result with a second threshold; as well as A contrast limiting histogram algorithm or a global histogram algorithm is adaptively selected according to a comparison result between the local contrast result and the second threshold value to perform the adaptive contrast enhancement operation on the nuclear magnetic resonance image.

5. The method according to claim 1 or 2, characterized in that: The trained hybrid residual network at least includes an initial convolution module, a multi-scale residual module and a cross-attention module, and the pre-processed magnetic resonance imaging is subjected to multimodal feature extraction using the trained hybrid residual network to obtain target feature results, including: Using the initial convolution module to perform an initial convolution operation on the preprocessed nuclear magnetic resonance image to obtain initial multimodal features; Using the cross attention module to fuse the features of different modalities in the initial multimodality, and combining them with the preprocessed magnetic resonance imaging to obtain an intermediate fusion result; Extracting multi-scale residual features of the intermediate fusion result using the multi-scale residual module, and performing feature splicing on the multi-scale residual features to obtain a feature splicing result; and The feature stitching result is combined with the preprocessed nuclear magnetic resonance image for a second time to obtain the target feature result.

6. The method according to claim 1, characterized in that Wherein identifying the modality of the magnetic resonance imaging according to the target feature result includes: Mapping the target feature results to different modal categories using a fully connected layer, and calculating the probability of each modal category using a classification function; and The modality corresponding to the maximum probability value is identified as the final modality of the nuclear magnetic resonance image.

7. The method according to claim 6, characterized in that The final modality of the nuclear magnetic resonance image includes at least T1-weighted modality, T2-weighted modality and fluid-attenuated inversion recovery modality.

8. The method according to claim 5, characterized in that The hybrid residual network is trained by the following operations to obtain the trained hybrid residual network: Magnetic resonance images of different modalities are input into a hybrid residual network for feature extraction to obtain multimodal features; Setting uncertainty weights for the multimodal features; as well as A total loss function of the adaptive modal weights is calculated based on the uncertainty weights to train the hybrid residual network to obtain the trained hybrid residual network.

9. A device for identifying a nuclear magnetic resonance imaging modality, characterized in that: include: processor; as well as A memory storing program instructions for identifying a nuclear magnetic resonance imaging modality, wherein when the program instructions are executed by the processor, the device implements the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions for identifying a nuclear magnetic resonance imaging modality are stored thereon, and when the computer-readable instructions are executed by one or more processors, the method according to any one of claims 1 to 8 is implemented.

Images