A deep learning-based MRI image recognition method and device

Abstract

The application discloses a kind of MRI image recognition method and device based on deep learning, comprising: obtaining MRI image dataset;For MRI image dataset is preprocessed, obtain the preprocessed MRI image data;The preprocessed MRI image data is convolved and pooled, and the recovered MRI image data is obtained;The recovered MRI image data is output to MSB module, and the output first feature map is obtained;First feature map is input to FSA module and is processed, and the output second feature map is obtained;Second feature map is input to FPN module and is processed, and the output third feature map is obtained;Third feature map is input to FCOS HEAD module and is processed, and the output ROI region is obtained;For ROI region is target detection operation, and the output target detection result image is obtained, can effectively deblurring processing to fuzzy MRI image, improve the accuracy and efficiency of diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a MRI image recognition method based on deep learning, a MRI image recognition device based on deep learning, a computer device and a storage medium. BACKGROUND

[0002] Image processing and analysis play a crucial role in medical diagnosis. Among them, magnetic resonance imaging (MRI) provides a non-invasive, non-radiation examination method for clinical diagnosis, especially for soft tissue imaging. However, limited by hardware equipment and imaging conditions, the MRI image often appears blurred, which affects the quality of the image and the accuracy of the diagnosis. Therefore, how to improve the clarity of the MRI image in order to more accurately identify diseases is an urgent problem to be solved.

[0003] The existing image deblurring algorithm mainly adopts the classical machine learning method, which processes by extracting the edge, texture and other features of the image. However, these traditional methods mainly rely on hand-crafted features, and for complex MRI images, the accuracy, robustness and efficiency need to be improved. Especially when encountering overlapping occlusion, illumination change and complex background, their performance will be greatly reduced. SUMMARY

[0004] In view of the above problems, the present application is proposed to provide a MRI image recognition method based on deep learning, a MRI image recognition device based on deep learning, a computer device and a storage medium which overcome the above problems or at least partially solve the above problems.

[0005] To achieve the above purpose, the present application provides a MRI image recognition method based on deep learning, which comprises:

[0006] obtaining an MRI image data set;

[0007] preprocessing the MRI image data set to obtain preprocessed MRI image data;

[0008] convolving and pooling the preprocessed MRI image data to obtain restored MRI image data;

[0009] outputting the restored MRI image data to an MSB module to obtain an output first feature map;

[0010] inputting the first feature map into an FSA module for processing to obtain an output second feature map;

[0011] The second feature map is input into an FPN module for processing to obtain an output third feature map;

[0012] The third feature map is input into an FCOS HEAD module for processing to obtain an output ROI region;

[0013] A target detection operation is performed on the ROI region to obtain an output target detection result image.

[0014] Preferably, the pre-processing of the MRI image dataset to obtain pre-processed MRI image data comprises:

[0015] The MRI image dataset is subjected to standardization and intensity normalization processing to obtain pre-processed MRI image data.

[0016] Preferably, the convolutional pooling of the pre-processed MRI image data to obtain restored MRI image data comprises:

[0017] A first convolution operation is performed on the pre-processed MRI image data to obtain a first convolution feature map;

[0018] A first pooling operation is performed on the first convolution feature map to obtain a first pooled feature map;

[0019] A second convolution operation is performed on the first pooled feature map to obtain a second convolution feature map;

[0020] A second pooling operation is performed on the second convolution feature map to obtain a second pooled feature map;

[0021] A third convolution operation is performed on the second pooled feature map to obtain a third convolution feature map;

[0022] A third pooling operation is performed on the third convolution feature map to obtain a third pooled feature map;

[0023] A fourth convolution operation is performed on the third pooled feature map to obtain a fourth convolution feature map;

[0024] A fourth pooling operation is performed on the fourth convolution feature map to obtain a fourth pooled feature map;

[0025] A first deconvolution operation is performed on the fourth pooled feature map to obtain a first deconvolution feature map;

[0026] A fifth convolution operation is performed on the first deconvolution feature map to obtain a fifth convolution feature map;

[0027] performing a second deconvolution operation on the fifth convolution feature map to obtain a second deconvolution feature map;

[0028] performing a sixth convolution operation on the second deconvolution feature map to obtain a sixth convolution feature map;

[0029] performing a third deconvolution operation on the sixth convolution feature map to obtain a third deconvolution feature map;

[0030] performing a seventh convolution operation on the third deconvolution feature map to obtain a seventh convolution feature map;

[0031] performing a fourth deconvolution operation on the seventh convolution feature map to obtain a fourth deconvolution feature map;

[0032] performing an eighth convolution operation on the fourth deconvolution feature map to obtain an eighth convolution feature map;

[0033] performing a ninth convolution operation on the eighth convolution feature map to obtain the restored MRI image data.

[0034] Preferably, the outputting the restored MRI image data to the MSB module to obtain an output first feature map comprises:

[0035] processing the restored MRI image data into a feature map to obtain a convolution layer output image;

[0036] calculating a residual connection output according to the restored MRI image data and the convolution layer output image;

[0037] performing channel dimension splicing according to the convolution layer output image and the residual connection output to obtain the first feature map.

[0038] Preferably, the inputting the first feature map into the FSA module for processing to obtain an output second feature map comprises:

[0039] performing Fourier transform on the convolution layer output image to convert the convolution layer output image to a frequency domain to obtain a complex frequency matrix, performing twice frequency pooling operation on the complex frequency matrix to obtain two channel perception attention maps;

[0040] connecting the two channel perception attention maps, performing convolution operation and sigmoid activation function to obtain a channel attention distribution;

[0041] applying the channel attention distribution to the original first feature map to obtain a channel attention feature map;

[0042] performing average pooling and maximum pooling on the channel attention feature map to obtain an average pooling feature map and a maximum pooling feature map.

[0043] connecting the average-pooled feature map and the max-pooled feature map, performing a convolution operation and a sigmoid activation function to obtain a spatial attention distribution;

[0044] applying the spatial attention distribution to the channel attention feature map to obtain a second feature map.

[0045] Preferably, the second feature map includes a first sub-feature map, a second sub-feature map, a third sub-feature map, and a fourth sub-feature map; the third feature map includes a first final feature map, a second final feature map, a third final feature map, a fourth final feature map, and a fifth final feature map; the second feature map is input into an FPN module for processing to obtain an output third feature map, including:

[0046] performing a first convolution operation on the first sub-feature map to obtain a first intermediate feature map;

[0047] performing a first convolution operation on the second sub-feature map to obtain a first intermediate sub-feature map, and adding the first intermediate feature map and the first intermediate sub-feature map to obtain a second intermediate feature map;

[0048] performing a first convolution operation on the third sub-feature map to obtain a second intermediate sub-feature map, and adding the second intermediate feature map and the second intermediate sub-feature map to obtain a third intermediate feature map;

[0049] performing a first convolution operation on the fourth sub-feature map to obtain a third intermediate sub-feature map, and adding the third intermediate feature map and the third intermediate sub-feature map to obtain a fourth intermediate feature map;

[0050] performing a second convolution operation on the first intermediate feature map, the second intermediate feature map, the third intermediate feature map, and the fourth intermediate feature map, respectively, to obtain an output first final feature map, a second final feature map, a third final feature map, and a fourth final feature map;

[0051] performing a third convolution operation and a max-pooling operation on the fourth final feature map to obtain a fifth final feature map.

[0052] Preferably, the third feature map is input into an FCOS HEAD module for processing to obtain an output ROI region, including:

[0053] performing a convolution operation, a normalization processing operation, and an activation function operation on the third feature map in sequence and repeating the above operations several times to obtain a deep feature;

[0054] According to the deep feature, a convolution operation is performed to generate a class prediction map, and a sigmoid function is applied to the class prediction map to obtain a prediction probability of each class at each position;

[0055] According to the deep feature, a convolution operation is performed to generate a bounding box prediction map, wherein the bounding box prediction map contains coordinates of a bounding box predicted at each position;

[0056] According to the deep feature, a convolution operation is performed to generate a center prediction map, and a sigmoid function is applied to the center prediction map to obtain a center prediction value at each position;

[0057] According to the prediction probability, the coordinates of the bounding box and the center prediction value, a prediction box meeting a preset condition is screened out as an ROI region through threshold screening and non-maximum suppression.

[0058] Preferably, the target detection operation is performed on the ROI region to obtain an output target detection result image, including:

[0059] A feature map is extracted from the ROI region, and a first convolution feature map is obtained by performing a convolution operation on the feature map;

[0060] The correlation between pixels is calculated according to the feature map;

[0061] The spatial information of the feature map is extracted by a pooling operation according to the feature map, and a second convolution feature map is obtained by performing an operation and a convolution operation on the spatial information of the feature map;

[0062] A final attention feature map is calculated according to the first convolution feature map, the correlation between pixels and the second convolution feature map;

[0063] The final attention feature map is subjected to a second layer convolution to obtain a target detection result image.

[0064] An MRI image recognition device based on deep learning is provided in the embodiment of the application, and the device includes:

[0065] A data set acquisition module is configured to acquire an MRI image data set;

[0066] A preprocessing module is configured to preprocess the MRI image data set to obtain preprocessed MRI image data;

[0067] A convolution pooling module is configured to perform convolution pooling on the preprocessed MRI image data to obtain recovered MRI image data;

[0068] The first feature map output module is configured to output the recovered MRI image data to an MSB module to obtain an output first feature map.

[0069] The second feature map output module is configured to input the first feature map to an FSA module for processing to obtain an output second feature map.

[0070] The third feature map output module is configured to input the second feature map to an FPN module for processing to obtain an output third feature map.

[0071] The ROI region output module is configured to input the third feature map to an FCOS HEAD module for processing to obtain an output ROI region.

[0072] The target detection operation module is configured to perform a target detection operation on the ROI region to obtain an output target detection result image.

[0073] Preferably, the preprocessing module comprises:

[0074] The preprocessing submodule is configured to perform standardization and intensity normalization processing on the MRI image dataset to obtain preprocessed MRI image data.

[0075] Preferably, the convolution pooling module comprises:

[0076] The first convolution feature map obtaining submodule is configured to perform a first convolution operation on the preprocessed MRI image data to obtain a first convolution feature map.

[0077] The first pooling feature map obtaining submodule is configured to perform a first pooling operation on the first convolution feature map to obtain a first pooling feature map.

[0078] The second convolution feature map obtaining submodule is configured to perform a second convolution operation on the first pooling feature map to obtain a second convolution feature map.

[0079] The second pooling feature map obtaining submodule is configured to perform a second pooling operation on the second convolution feature map to obtain a second pooling feature map.

[0080] The third convolution feature map obtaining submodule is configured to perform a third convolution operation on the second pooling feature map to obtain a third convolution feature map.

[0081] The third pooling feature map obtaining submodule is configured to perform a third pooling operation on the third convolution feature map to obtain a third pooling feature map.

[0082] The fourth convolution feature map obtaining submodule is configured to perform a fourth convolution operation on the third pooling feature map to obtain a fourth convolution feature map.

[0083] a fourth pooling feature map obtaining submodule configured to perform a fourth pooling operation on the fourth convolution feature map to obtain a fourth pooling feature map;

[0084] a first deconvolution feature map obtaining submodule configured to perform a first deconvolution operation on the fourth pooling feature map to obtain a first deconvolution feature map;

[0085] a fifth convolution feature map obtaining submodule configured to perform a fifth convolution operation on the first deconvolution feature map to obtain a fifth convolution feature map;

[0086] a second deconvolution feature map obtaining submodule configured to perform a second deconvolution operation on the fifth convolution feature map to obtain a second deconvolution feature map;

[0087] a sixth convolution feature map obtaining submodule configured to perform a sixth convolution operation on the second deconvolution feature map to obtain a sixth convolution feature map;

[0088] a third deconvolution feature map obtaining submodule configured to perform a third deconvolution operation on the sixth convolution feature map to obtain a third deconvolution feature map;

[0089] a seventh convolution feature map obtaining submodule configured to perform a seventh convolution operation on the third deconvolution feature map to obtain a seventh convolution feature map;

[0090] a fourth deconvolution feature map obtaining submodule configured to perform a fourth deconvolution operation on the seventh convolution feature map to obtain a fourth deconvolution feature map;

[0091] an eighth convolution feature map obtaining submodule configured to perform an eighth convolution operation on the fourth deconvolution feature map to obtain an eighth convolution feature map;

[0092] an MRI image data obtaining submodule configured to perform a ninth convolution operation on the eighth convolution feature map to obtain recovered MRI image data.

[0093] Preferably, the first feature map output module comprises:

[0094] a convolution layer output image obtaining submodule configured to process the recovered MRI image data into a feature map to obtain a convolution layer output image;

[0095] a residual connection output calculating submodule configured to calculate a residual connection output according to the recovered MRI image data and the convolution layer output image;

[0096] The first feature map output sub-module is configured to perform channel dimension splicing on the convolutional layer output image and the residual connection output to obtain the first feature map.

[0097] Preferably, the second feature map output module comprises:

[0098] The frequency pooling operation sub-module is configured to convert the convolutional layer output image to a frequency domain through Fourier transform to obtain a complex frequency matrix, and perform twice frequency pooling operation on the complex frequency matrix to obtain two channel-aware attention maps.

[0099] The channel attention distribution obtaining sub-module is configured to connect the two channel-aware attention maps, and obtain a channel attention distribution through convolution operation and sigmoid activation function.

[0100] The channel attention feature map obtaining sub-module is configured to apply the channel attention distribution to the original first feature map to obtain a channel attention feature map.

[0101] The average pooling and maximum pooling sub-module is configured to obtain an average pooling feature map and a maximum pooling feature map through average pooling and maximum pooling on the channel attention feature map.

[0102] The spatial attention distribution sub-module is configured to connect the average pooling feature map and the maximum pooling feature map, and obtain a spatial attention distribution through convolution operation and sigmoid activation function.

[0103] The second feature map obtaining sub-module is configured to apply the spatial attention distribution to the channel attention feature map to obtain a second feature map.

[0104] Preferably, the second feature map comprises a first sub-feature map, a second sub-feature map, a third sub-feature map and a fourth sub-feature map; the third feature map comprises a first final feature map, a second final feature map, a third final feature map, a fourth final feature map and a fifth final feature map; and the third feature map output module comprises:

[0105] The first intermediate feature map obtaining sub-module is configured to perform first convolution operation on the first sub-feature map to obtain a first intermediate feature map.

[0106] The second intermediate feature map obtaining sub-module is configured to perform first convolution operation on the second sub-feature map to obtain a first intermediate sub-feature map, and add the first intermediate feature map and the first intermediate sub-feature map to obtain a second intermediate feature map.

[0107] The third intermediate feature map obtaining sub-module is configured to perform a first convolution operation on the third sub-feature map to obtain a second intermediate sub-feature map, and add the second intermediate feature map and the second intermediate sub-feature map to obtain a third intermediate feature map.

[0108] The fourth intermediate feature map obtaining sub-module is configured to perform a first convolution operation on the fourth sub-feature map to obtain a third intermediate sub-feature map, and add the third intermediate feature map and the third intermediate sub-feature map to obtain a fourth intermediate feature map.

[0109] The final feature map obtaining sub-module is configured to perform a second convolution operation on the first intermediate feature map, the second intermediate feature map, the third intermediate feature map and the fourth intermediate feature map respectively to obtain output first final feature map, second final feature map, third final feature map and fourth final feature map.

[0110] The fifth final feature map obtaining sub-module is configured to perform a third convolution operation and a maximum pooling operation on the fourth final feature map to obtain a fifth final feature map.

[0111] Preferably, the ROI region output module comprises:

[0112] The depth feature obtaining sub-module is configured to sequentially perform a convolution operation, a normalization processing operation and an activation function operation on the third feature map and repeat the above operations for several times to obtain a depth feature.

[0113] The prediction probability obtaining sub-module is configured to perform a convolution operation on the depth feature to generate a class prediction map, and apply a sigmoid function to the class prediction map to obtain a prediction probability of each class at each position.

[0114] The bounding box prediction map generating sub-module is configured to perform a convolution operation on the depth feature to generate a bounding box prediction map, wherein the bounding box prediction map contains coordinates of a bounding box predicted at each position.

[0115] The center prediction map generating sub-module is configured to perform a convolution operation on the depth feature to generate a center prediction map, and apply a sigmoid function to the center prediction map to obtain a center prediction value at each position.

[0116] The ROI region screening sub-module is configured to screen a prediction box meeting a preset condition as an ROI region through threshold screening and non-maximum suppression according to the prediction probability, the coordinates of the bounding box and the center prediction value.

[0117] Preferably, the target detection operation module comprises:

[0118] The first convolution feature map obtaining sub-module is configured to extract a feature map from the ROI region, and perform convolution operation on the feature map to obtain a first convolution feature map.

[0119] The correlation calculating sub-module is configured to calculate a correlation between pixels according to the feature map.

[0120] The second convolution feature map obtaining sub-module is configured to extract spatial information of a feature map from the feature map through a pooling operation, and perform operation and convolution operation on the spatial information of the feature map to obtain a second convolution feature map.

[0121] The final attention feature map calculating sub-module is configured to calculate a final attention feature map according to the first convolution feature map, the correlation between pixels and the second convolution feature map.

[0122] The target detection result image obtaining sub-module is configured to obtain a target detection result image by performing second layer convolution on the final attention feature map.

[0123] The embodiment of the present application discloses a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor realizes the steps of the MRI image recognition method based on deep learning.

[0124] The embodiment of the present application discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the MRI image recognition method based on deep learning.

[0125] In the embodiment of the present application, the DPDNet and the BCNet algorithm are combined, the DPDNet reads two MRI images IL and IR as a 6-channel cube, processes them through an encoder, a bottleneck and a decoder stage to obtain a final clear image, the BCNet sends a feature map to a detection head after extracting the feature, adds a segmentation branch to realize segmentation of the target, and introduces two networks for different problems to process blur and target recognition respectively. In the aspect of target recognition, the FSA module is added as an innovation point, the MSB module is used as a feature extraction module, the MRI image can be effectively deblurred, accurate disease recognition and region segmentation can be performed, and therefore the accuracy and efficiency of diagnosis are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0126] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0127] Figure 1 is a step flow chart of an MRI image recognition method based on deep learning in an embodiment of the present application;

[0128] Figure 2 is a flow chart of an MRI image acquisition step after recovery in an embodiment of the present application;

[0129] Figure 3 is a schematic diagram of an MRI image recognition system in an embodiment of the present application;

[0130] Figure 4 is a schematic diagram of an MSB module in an embodiment of the present application;

[0131] Figure 5 is a flow chart of a first feature map acquisition step in an embodiment of the present application;

[0132] Figure 6 is a schematic diagram of an FSA module in an embodiment of the present application;

[0133] Figure 7 is a flow chart of a second feature map output step in an embodiment of the present application;

[0134] Figure 8 is a schematic diagram of an FPN module in an embodiment of the present application;

[0135] Figure 9 is a flow chart of a third feature map output step in an embodiment of the present application;

[0136] Figure 10 is a schematic diagram of an FCOS HEAD module in an embodiment of the present application;

[0137] Figure 11 is a flow chart of an ROI region output step in an embodiment of the present application;

[0138] Figure 12 is a flow chart of a target detection result image output step in an embodiment of the present application;

[0139] Figure 13 is a structural block diagram of an MRI image recognition device based on deep learning in an embodiment of the present application;

[0140] Figure 14Fig. 1 is a block diagram of an internal structure of a computer device according to an embodiment. DETAILED DESCRIPTION

[0141] In order to make the technical problems solved by the embodiments of the present application, technical solutions and beneficial effects clearer, the embodiments of the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0142] Referring to Figure 1 Fig. 1 is a flowchart of steps of an embodiment of a deep learning-based MRI image recognition method according to the present application, which can specifically include the following steps:

[0143] Step 101: Obtain an MRI image dataset.

[0144] In the embodiments of the present application, the method can be applied to a terminal, which can be a tablet computer, a personal computer, an all-in-one computer, etc. The specific type of the terminal is not limited in the embodiments of the present application. The operating system of the terminal can include Android, IOS, Windows Phone, Windows, etc. The present application does not make too many limitations on this.

[0145] Specifically in the embodiments of the present application, the method can be applied to a Pytorch framework. The specific parameters of the terminal are a processor Intel(R) Xeon(R) CPU i7-12700, a memory of 64G, an operating system of 64-bit windows11, and a GPU model of GEFORCERTX 2080Ti. The method runs in GPU mode.

[0146] Firstly, the terminal can obtain an MRI image dataset from a database. The MRI image dataset can include a pre-acquired out-of-focus blurred MRI image dataset. The present application does not make too many limitations on this.

[0147] Step 102: Preprocess the MRI image dataset to obtain preprocessed MRI image data.

[0148] After obtaining the MRI image dataset, the terminal can preprocess the MRI image dataset,

[0149] In practical application in the embodiments of the present application, the preprocessing of the MRI image dataset to obtain preprocessed MRI image data includes: performing standardization and intensity normalization processing on the MRI image dataset to obtain preprocessed MRI image data.

[0150] In addition to standardization and intensity normalization, preprocessing operations such as color channel adjustment can be performed. Since MRI images obtained from different devices or studies may have different sizes, it is necessary to resize all images to the same size for subsequent processing. This step can be achieved by image scaling or cropping, as shown in the following formula:

[0151] ;

[0152] Where X is the original image and resize is the image scaling function.

[0153] For intensity normalization, pixel intensities in MRI image data may vary due to differences in equipment or acquisition conditions. Intensity normalization unifies the range of pixel intensities in an image to a specific range, such as [0, 1] or [-1, 1]. The intensity normalization formula is as follows:

[0154]

[0155] in, and These are the minimum and maximum pixel intensities of the original image, respectively.

[0156] MRI image data is typically grayscale, meaning it has only one channel.

[0157] The input to this model will be a three-dimensional MRI image, assuming its dimensions are (width, height, depth). Since MRI images are usually grayscale, the color channels do not need to be processed in the same way as color images in this embodiment of the invention.

[0158] Step 103: Perform convolutional pooling on the preprocessed MRI image data to obtain the recovered MRI image data;

[0159] In this embodiment of the invention, a convolutional pooling module is first constructed. The preprocessed MRI image data is then convolved and pooled by the convolutional pooling module to obtain the restored MRI image data. The restored MRI image data is then input into the MSB module.

[0160] The convolutional pooling module consists of nine convolutional layers, which are mainly responsible for extracting features from the input image; four pooling layers, which are used to reduce the spatial dimension of the feature image while retaining the most important feature information; and four deconvolutional layers, which are used in the upsampling part of the network to restore the spatial dimension of the image and generate a deblurred image.

[0161] In one specific embodiment, reference is made to... Figure 2, a flow chart of an MRI image acquisition step after recovery according to an embodiment of the present application is shown, the pre-processed MRI image data is convolved and pooled to obtain recovered MRI image data, comprising:

[0162] Step 11, a first convolution operation is performed on the pre-processed MRI image data to obtain a first convolution feature map;

[0163] For example, the pre-processed MRI image data is an out-of-focus image The final output is three different spatial scale feature maps extracted ;

[0164] First, the out-of-focus image is subjected to a first convolution operation, and the feature map after convolution (the first convolution feature map) is represented as:

[0165] ;

[0166] Wherein, represents the weight of the convolution operation at this step, represents the bias of the convolution operation at this step, the convolution operation at this step uses n 3*3 convolution kernels, .

[0167] Step 12, a first pooling operation is performed on the first convolution feature map to obtain a first pooling feature map;

[0168] The feature map is subjected to a 2*2 pooling operation, and the feature map after pooling (the first pooling feature map) is represented as:

[0169] ;

[0170] Wherein, Pooling represents a 2*2 pooling operation, .

[0171] Step 13, a second convolution operation is performed on the first pooling feature map to obtain a second convolution feature map;

[0172] The feature map is subjected to a convolution operation again, and the feature map after convolution (the second convolution feature map) is represented as:

[0173] ;

[0174] Wherein, represents the weight of the convolution operation at this step, This indicates the bias of the convolution operation, which uses n 3x3 convolution kernels. .

[0175] Step 14: Perform a second pooling operation on the second convolutional feature map to obtain a second pooled feature map;

[0176] For feature maps Perform a 2x2 pooling operation, and the resulting feature map (The second pooling feature map) is represented as:

[0177] ;

[0178] Where Pooling represents a 2x2 pooling operation. .

[0179] Step 15: Perform a third convolution operation on the second pooling feature map to obtain the third convolution feature map;

[0180] For feature maps Perform another convolution operation, and the resulting feature map... (The third convolutional feature map) is represented as:

[0181] ;

[0182] in, This indicates the weights of the convolution operation in this step. This indicates the bias of the convolution operation, which uses n 3x3 convolution kernels. .

[0183] Step 16: Perform a third pooling operation on the third convolutional feature map to obtain a third pooled feature map;

[0184] For feature maps Perform a 2x2 pooling operation, and the resulting feature map (The third pooling feature map) is represented as:

[0185] ;

[0186] Where Pooling represents a 2x2 pooling operation. .

[0187] Step 17: Perform a fourth convolution operation on the third pooling feature map to obtain a fourth convolution feature map;

[0188] For feature maps Perform another convolution operation, and the resulting feature map (The fourth convolutional feature map) is represented as:

[0189] ;

[0190] wherein, represents the weight of the step convolution operation, represents the bias of the step convolution operation, the step convolution operation adopts n 3*3 convolution kernels, .

[0191] Step 18, performing a fourth pooling operation on the fourth convolution feature map to obtain a fourth pooling feature map;

[0192] performing a pooling operation on the feature map , the feature map after the pooling operation (the fourth pooling feature map) is :

[0193] ;

[0194] wherein, Pooling represents a pooling operation, .

[0195] Step 19, performing a first deconvolution operation on the fourth pooling feature map to obtain a first deconvolution feature map;

[0196] performing a deconvolution operation on the feature map , the feature map after the deconvolution operation (the first deconvolution feature map) is :

[0197] ;

[0198] wherein, represents the weight of the step deconvolution operation, represents the bias of the step deconvolution operation, the step deconvolution operation adopts deconvolution kernels, .

[0199] Step 20, performing a fifth convolution operation on the first deconvolution feature map to obtain a fifth convolution feature map;

[0200] performing a convolution operation on the feature map again, the feature map after the convolution operation (the fifth convolution feature map) is :

[0201] ;

[0202] wherein, represents the weight of the step convolution operation,​​​ bias of the step of convolution operation, the step of convolution operation adopts convolution kernel, .

[0203] Step 21, a second deconvolution operation is performed on the fifth convolution feature map to obtain a second deconvolution feature map;

[0204] a deconvolution operation is performed on the feature map , and the feature map after the deconvolution is (second deconvolution feature map):

[0205] ;

[0206] wherein, weight of the step of deconvolution operation, bias of the step of deconvolution operation, the step of deconvolution operation adopts deconvolution kernel, .

[0207] Step 22, a sixth convolution operation is performed on the second deconvolution feature map to obtain a sixth convolution feature map;

[0208] a convolution operation is performed on the feature map again, and the feature map after the convolution is (sixth convolution feature map):

[0209] ;

[0210] wherein, weight of the step of convolution operation, bias of the step of convolution operation, the step of convolution operation adopts convolution kernel, .

[0211] Step 23, a third deconvolution operation is performed on the sixth convolution feature map to obtain a third deconvolution feature map;

[0212] a deconvolution operation is performed on the feature map , and the feature map after the deconvolution is (third deconvolution feature map):

[0213] ;

[0214] wherein, weight of the step of deconvolution operation,​​​​​ denotes the bias of this step of deconvolution operation, and the deconvolution operation adopts a deconvolution kernel of .

[0215] Step 24, a seventh convolution operation is performed on the third deconvolution feature map to obtain a seventh convolution feature map;

[0216] The feature map is further subjected to a convolution operation, and the feature map after the convolution is (the seventh convolution feature map):

[0217] ;

[0218] wherein, w7 denotes the weight of this step of convolution operation, denotes the bias of this step of convolution operation, and the convolution operation adopts a convolution kernel of .

[0219] Step 25, a fourth deconvolution operation is performed on the seventh convolution feature map to obtain a fourth deconvolution feature map;

[0220] The feature map is subjected to a deconvolution operation, and the feature map after the deconvolution is (the fourth deconvolution feature map):

[0221] ;

[0222] wherein, w8 denotes the weight of this step of deconvolution operation, denotes the bias of this step of deconvolution operation, and the deconvolution operation adopts a deconvolution kernel of .

[0223] Step 26, an eighth convolution operation is performed on the fourth deconvolution feature map to obtain an eighth convolution feature map;

[0224] The feature map is further subjected to a convolution operation, and the feature map after the convolution is (the eighth convolution feature map):

[0225] ;

[0226] wherein, w8 denotes the weight of this step of convolution operation, denotes the bias of this step of convolution operation, and the convolution operation adopts a convolution kernel of ​​​​​​​​​a bias of the convolution operation, the convolution operation adopting a convolution kernel of .

[0227] Step 27, a ninth convolution operation is performed on the eighth convolution feature map, and a restored MRI image data is obtained.

[0228] The feature map is further subjected to a convolution operation, and the convolution feature map is (restore MRI image data):

[0229] ;

[0230] wherein, a weight of the convolution operation, a bias of the convolution operation, the convolution operation adopting a convolution kernel of .

[0231] The neural network model in the embodiment of the application introduces Pconvolution convolution, reduces redundant calculation and memory access, and can more effectively extract spatial features.

[0232] Step 104, outputting the restored MRI image data to an MSB module to obtain an output first feature map;

[0233] Referring to Figure 3 , a schematic diagram of an MRI image recognition system is shown, after convolution and pooling are completed, the feature map after convolution and pooling can be taken as input, and is first input to an MSB module, as shown in Figure 3 , the MSB module is further connected with an FSA module, an FPN module and an FCOS HEAD module, the FCOS HEAD module is connected to a module for outputting an ROI region, and the module for outputting the ROI region can also be connected with a related module of target segmentation and detection, so as to obtain a target detection result image

[0234] Referring to Figure 4 , a schematic diagram of an MSB module is shown, which can include a plurality of convolution layers; specifically applied to the embodiment of the application, referring to Figure 5 , a flow chart of a first feature map acquisition step is shown, the output of the restored MRI image data to the MSB module to obtain the output first feature map includes the following sub-steps:

[0235] ​​​​​Step 31, processing the recovered MRI image data into a feature map to obtain a convolution layer output image;

[0236] Step 32, calculating a residual connection output according to the recovered MRI image data and the convolution layer output image;

[0237] Step 33, performing channel dimension splicing according to the convolution layer output image and the residual connection output to obtain the first feature map.

[0238] The input recovered MRI image data is F, and each convolution layer uses a convolution operation to process the image into a feature map. The working process of each convolution layer can be expressed as:

[0239] ;

[0240] Here, , is the output of the previous convolution layer, is the convolution operation, is a nonlinear activation function. Note that for the first convolution layer, the input image is regarded as the input image (i.e. the convolution layer output image), that is .

[0241] Then, the residual connection is realized by adding the input image and the output of the third convolution layer , that is:

[0242] ;

[0243] Finally, the outputs of all convolution layers (convolution layer output images) and the output of the residual connection are combined to obtain the output of the MSB module (i.e. the first feature map):

[0244] ;

[0245] In the embodiment of the application Operation is to splice all feature maps together along the channel dimension. In this way, the MSB module can process and learn the multi-scale features of the input image at the same time, and some information of the original input is preserved through the residual connection.

[0246] Step 105, inputting the first feature map into the FSA module for processing to obtain an output second feature map;

[0247] Referring to Figure 6 , a schematic diagram of an FSA module in an embodiment of the application is shown, and in a preferred embodiment, referring to Figure 7, a flow chart of a second feature map output step of an embodiment of the application is shown, the first feature map is input to the FSA module for processing to obtain an output second feature map, comprising:

[0248] Step 41, the convolutional layer output image is converted to the frequency domain by Fourier transform to obtain a complex frequency matrix, and two frequency pooling operations are performed on the complex frequency matrix to obtain two channel-aware attention maps;

[0249] For the input of each layer , it is first converted to the frequency domain by Fourier transform to obtain a complex frequency matrix . Then, two frequency pooling operations are performed on each channel in the frequency domain to obtain two channel-aware attention maps and . The two attention maps can be calculated by the following formula:

[0250] ;

[0251] ;

[0252] Wherein, represents the amplitude of the frequency matrix, represents the phase of the frequency matrix, is the pooling operation.

[0253] Step 42, the two channel-aware attention maps are connected, and a channel attention distribution is obtained by convolution operation and sigmoid activation function;

[0254] and are connected, and a channel attention distribution is obtained by a 1x1 convolutional layer and a sigmoid activation function:

[0255] ;

[0256] Wherein, represents the 1x1 convolution operation, represents the sigmoid activation function.

[0257] Step 43, the channel attention distribution is applied to the original first feature map to obtain a channel attention feature map;

[0258] The channel attention distribution is applied to the original input feature map , i.e. the first feature map, to obtain a channel attention feature map :

[0259] ​ ;

[0260] Step 44, for the channel attention feature map, using average pooling and max pooling, the average pooling feature map and the max pooling feature map are obtained;

[0261] For the channel attention feature map obtained by the channel attention block , using average pooling and max pooling, the average pooling feature map and the max pooling feature map are obtained;

[0262] Step 45, connecting the average pooling feature map and the max pooling feature map, through convolution operation and sigmoid activation function, the spatial attention distribution is obtained;

[0263] Connecting the two feature maps together, through a 3x1 convolution operation and a sigmoid activation function, the spatial attention distribution is obtained ;

[0264] Step 46, applying the spatial attention distribution to the channel attention feature map to obtain a second feature map.

[0265] Applying the spatial attention distribution to the channel attention feature map , the spatial attention feature map is obtained:

[0266] ;

[0267] The MSB output and the output of the residual connection of each layer are processed by the FSA module, and then these outputs can be used for subsequent operations. Specifically, the feature map of each channel is multiplied by the attention value corresponding to the channel. In this way, those channels that show importance in the frequency domain will get higher weights, while those unimportant channels will be suppressed. This process realizes adaptive weighting of channels, thereby improving the model's perception ability of important channels.

[0268] That is, the feature map of the MSB output and is input to the FSA for processing to obtain and . It can be represented as:

[0269] ;

[0270] ;

[0271] ;

[0272] ;

[0273] Then, the second feature map output by the FSA module is taken as the input feature map of the FPN And Then, the FPN is used for processing.

[0274] Step 106: inputting the second feature map into the FPN module for processing to obtain an output third feature map;

[0275] Referring to Figure 8 , a schematic diagram of an FPN module in an embodiment of the present application is shown, which can include multiple convolution layers;

[0276] In a preferred embodiment, referring to Figure 9 , a flowchart of a third feature map output step in an embodiment of the present application is shown,

[0277] The second feature map includes a first sub-feature map (C2), a second sub-feature map (C3), a third sub-feature map (C4) and a fourth sub-feature map (C5); the third feature map includes a first final feature map (P2), a second final feature map (P3), a third final feature map (P4), a fourth final feature map (P5) and a fifth final feature map (P6); and the step of inputting the second feature map into the FPN module for processing to obtain an output third feature map includes the following steps:

[0278] Step 51: performing a first convolution operation on the first sub-feature map to obtain a first intermediate feature map;

[0279] First, The feature map is processed by a 1x1 convolution operation to obtain an intermediate feature map , i.e., a first intermediate feature map. This step can be expressed as:

[0280] ;

[0281] Step 52: performing a first convolution operation on the second sub-feature map to obtain a first intermediate sub-feature map, and adding the first intermediate feature map and the first intermediate sub-feature map to obtain a second intermediate feature map;

[0282] Then, the feature map of the layer is added to the feature map (i.e., the first intermediate sub-feature map) obtained by upsampling the feature map of the layer after 1x1 convolution to obtain an intermediate feature map (second intermediate feature map). This can be expressed as:

[0283] ;

[0284] Step 53, the third sub-feature map is subjected to a first convolution operation to obtain a second intermediate sub-feature map, and the second intermediate feature map and the second intermediate sub-feature map are added to obtain a third intermediate feature map;

[0285] Step 54, the fourth sub-feature map is subjected to a first convolution operation to obtain a third intermediate sub-feature map, and the third intermediate feature map and the third intermediate sub-feature map are added to obtain a fourth intermediate feature map;

[0286] Similarly, the above process is repeated twice to obtain intermediate feature maps (third intermediate feature map) and (fourth intermediate feature map). The corresponding formula can be represented as:

[0287] ;

[0288] ;

[0289] Step 55, the first intermediate feature map, the second intermediate feature map, the third intermediate feature map and the fourth intermediate feature map are subjected to a second convolution operation to obtain output first final feature map (P2), second final feature map (P3), third final feature map (P4) and fourth final feature map (P5);

[0290] that is and , respectively, after a 3x3 convolution operation, to obtain the final output feature map and . The corresponding formula is:

[0291] ;

[0292] ;

[0293] ;

[0294] ;

[0295] Step 56, the fourth final feature map is subjected to a third convolution operation and a maximum pooling operation to obtain a fifth final feature map (P6).

[0296] The feature map is subjected to a 3x3 convolution operation and then subjected to a 2x2 maximum pooling operation to obtain the final output feature map . The corresponding formula is:

[0297] ;

[0298] In this process, the feature map is processed by the FPN module to adapt to different scales while retaining rich information.

[0299] Step 107, input the third feature map into the FCOS HEAD module for processing to obtain the output ROI region;

[0300] Referring to Figure 10 , a schematic diagram of a FCOS HEAD module according to an embodiment of the application is shown, which can include multiple convolution layers; in a preferred embodiment, referring to Figure 11 , a flowchart of a ROI region output step according to an embodiment of the application is shown, which comprises:

[0301] Step 61, sequentially performing convolution operation, normalization processing operation and activation function operation on the third feature map and repeating the above operations several times to obtain deep features;

[0302] The FPN output feature map (third feature map) is input into the FCOS HEAD as input. The input first undergoes a 3x3 convolution operation with a convolution kernel number of 256, then is normalized by a Group Normalization (GN) layer, and finally a ReLU activation function is applied to increase nonlinearity, and this step is repeated four times to further extract deep features.

[0303] Step 62, performing convolution operation according to the deep features to generate a class prediction map, and applying a sigmoid function to the class prediction map to obtain the prediction probability of each class at each position;

[0304] After the above processing, the FCOS HEAD generates a class prediction map (classification) through a 3x3 convolution operation. The convolution kernel number is equal to the number of target classes C, and then a sigmoid function is applied to obtain the prediction probability of each class at each position.

[0305] Step 63, performing convolution operation according to the deep features to generate a bounding box prediction map, wherein the bounding box prediction map contains the coordinates of the bounding box predicted at each position;

[0306] The FCOS HEAD also generates a bounding box prediction map (regression) through a 3x3 convolution operation. The convolution kernel number is 4, and the 4 values represent the coordinates of the bounding box predicted at each position.

[0307] Step 64, a center prediction map is generated by performing a convolution operation according to the depth features, and a sigmoid function is applied to the center prediction map to obtain a center prediction value at each position;

[0308] Similarly, the FCOS HEAD also generates a center prediction map (centerness) through a 3x3 convolution operation. The number of convolution kernels is 1, and then a sigmoid function is applied to obtain a center prediction value at each position.

[0309] Step 65, according to the prediction probability, the coordinates of the bounding box, and the center prediction value, a prediction box meeting a preset condition is screened out as an ROI region through threshold screening and non-maximum suppression.

[0310] Through the output of the FCOS HEAD, multiple predicted target boxes and their class prediction scores and center scores can be obtained. By combining the above information, some high-quality prediction boxes are selected as the ROI region through threshold screening and non-maximum suppression (NMS) and the like. The ROI region can be used for subsequent target classification or target refinement tasks.

[0311] Step 108, performing a target detection operation on the ROI region to obtain an output target detection result image.

[0312] After obtaining the ROI region, a target detection operation can be performed on the ROI region to obtain an output target detection result image. In one specific embodiment, referring to Figure 12 , a flowchart of a target detection result image output step of an embodiment of the present application is shown. The target detection operation on the ROI region to obtain an output target detection result image includes:

[0313] Step 71, extracting a feature map from the ROI region, and performing a convolution operation on the feature map to obtain a first convolution feature map;

[0314] Step 72, calculating a correlation between pixels according to the feature map;

[0315] Step 73, extracting spatial information of the feature map through a pooling operation according to the feature map, and performing a operation and a convolution operation on the spatial information of the feature map to obtain a second convolution feature map;

[0316] Step 74, calculating a final attention feature map according to the first convolution feature map, the correlation between pixels, and the second convolution feature map;

[0317] Step 75, the final attention feature map is passed through a second layer of convolution to obtain a target detection result image.

[0318] The feature map extracted from the ROI is A, with dimensions where is the batch size, is the number of channels, is the height of the feature map, is the width of the feature map.

[0319] The first layer Layer:

[0320] ;

[0321] where represents the convolution operation, represents the feature map after the first layer of convolution (i.e., the first convolution feature map), which also has dimensions .

[0322] The module for target detection operation includes a PCA layer:

[0323] Processing through the PCA layer can be divided into pixel attention and coordinate attention calculation;

[0324] Pixel attention calculation: let and be two linearly transformed feature spaces, define the correlation between pixels as follows:

[0325] ;

[0326] where, represents the transpose, is a scaling factor, so that each element value of is between , the dimension of .

[0327] Coordinate attention calculation: first, extract the spatial information of the feature map through the pooling operation:

[0328] ;

[0329] ;

[0330] and represent the height and width direction pooling operation respectively, and have dimensions .

[0331] Then, and conduct Operation, and through Convolution yields a new feature map v (i.e., the second convolutional feature map):

[0332] ;

[0333] in, Represents a non-linear activation function. express and In spatial dimension operate, This represents a 1×1 convolution operation. The dimension is .

[0334] By combining pixel attention and coordinate attention, the final attention feature map is obtained. :

[0335] ;

[0336] Second floor layer:

[0337] Attention feature map The final output feature map is obtained through the second convolution layer. :

[0338] ;

[0339] in, This represents the convolution operation. This is the image of the target detection result.

[0340] In this embodiment of the invention, the DPDNet and BCNet algorithms are combined. DPDNet acquires two MRI images, IL and IR, as a 6-channel cube and processes them through encoder, bottleneck, and decoder stages to obtain a final clear image. BCNet extracts features, feeds the feature map into the detection head for recognition, and adds a segmentation branch to segment the target. Two networks are introduced to handle blurring and target recognition respectively. For target recognition, an attention mechanism FSA module is added as an innovation, and the MSB module is used as the feature extraction module. This enables effective deblurring of blurry MRI images, while simultaneously performing accurate disease identification and region segmentation, thereby significantly improving the accuracy and efficiency of diagnosis.

[0341] It should be noted that, for the method embodiments, the series of acts complement each other to achieve the purpose of this embodiment, therefore, the sequence of the acts should not be construed as a limitation of the embodiments of this application. In addition, those skilled in the art should understand that the acts of the embodiments described in the specification are only preferred acts, and not all acts are necessary for the embodiments of this application.

[0342] Referring to Figure 6 , a structural block diagram of an embodiment of a deep learning-based MRI image recognition device of the application is shown, which can specifically include the following modules:

[0343] The data set acquisition module 301 is configured to acquire an MRI image data set.

[0344] The preprocessing module 302 is configured to preprocess the MRI image data set to obtain preprocessed MRI image data.

[0345] The convolutional pooling module 303 is configured to perform convolutional pooling on the preprocessed MRI image data to obtain recovered MRI image data.

[0346] The first feature map output module 304 is configured to output the recovered MRI image data to the MSB module to obtain an output first feature map.

[0347] The second feature map output module 305 is configured to input the first feature map to the FSA module for processing to obtain an output second feature map.

[0348] The third feature map output module 306 is configured to input the second feature map to the FPN module for processing to obtain an output third feature map.

[0349] The ROI region output module 307 is configured to input the third feature map to the FCOS HEAD module for processing to obtain an output ROI region.

[0350] The target detection operation module 308 is configured to perform target detection operation on the ROI region to obtain an output target detection result image.

[0351] Preferably, the preprocessing module includes:

[0352] The preprocessing submodule is configured to perform standardization and intensity normalization processing on the MRI image data set to obtain preprocessed MRI image data.

[0353] Preferably, the convolutional pooling module includes:

[0354] The first convolution feature map obtaining submodule is configured to perform a first convolution operation on the preprocessed MRI image data to obtain a first convolution feature map.

[0355] The first pooling feature map obtaining submodule is configured to perform a first pooling operation on the first convolution feature map to obtain a first pooling feature map.

[0356] The second convolution feature map obtaining submodule is configured to perform a second convolution operation on the first pooling feature map to obtain a second convolution feature map.

[0357] The second pooling feature map obtaining submodule is configured to perform a second pooling operation on the second convolution feature map to obtain a second pooling feature map.

[0358] The third convolution feature map obtaining submodule is configured to perform a third convolution operation on the second pooling feature map to obtain a third convolution feature map.

[0359] The third pooling feature map obtaining submodule is configured to perform a third pooling operation on the third convolution feature map to obtain a third pooling feature map.

[0360] The fourth convolution feature map obtaining submodule is configured to perform a fourth convolution operation on the third pooling feature map to obtain a fourth convolution feature map.

[0361] The fourth pooling feature map obtaining submodule is configured to perform a fourth pooling operation on the fourth convolution feature map to obtain a fourth pooling feature map.

[0362] The first deconvolution feature map obtaining submodule is configured to perform a first deconvolution operation on the fourth pooling feature map to obtain a first deconvolution feature map.

[0363] The fifth convolution feature map obtaining submodule is configured to perform a fifth convolution operation on the first deconvolution feature map to obtain a fifth convolution feature map.

[0364] The second deconvolution feature map obtaining submodule is configured to perform a second deconvolution operation on the fifth convolution feature map to obtain a second deconvolution feature map.

[0365] The sixth convolution feature map obtaining submodule is configured to perform a sixth convolution operation on the second deconvolution feature map to obtain a sixth convolution feature map.

[0366] The third deconvolution feature map obtaining submodule is configured to perform a third deconvolution operation on the sixth convolution feature map to obtain a third deconvolution feature map.

[0367] The seventh convolution feature map obtaining submodule is configured to perform a seventh convolution operation on the third deconvolution feature map to obtain a seventh convolution feature map.

[0368] a fourth deconvolution feature map obtaining sub-module configured to perform a fourth deconvolution operation on the seventh convolution feature map to obtain a fourth deconvolution feature map;

[0369] an eighth convolution feature map obtaining sub-module configured to perform an eighth convolution operation on the fourth deconvolution feature map to obtain an eighth convolution feature map;

[0370] an MRI image data obtaining sub-module configured to perform a ninth convolution operation on the eighth convolution feature map to obtain recovered MRI image data.

[0371] Preferably, the first feature map output module comprises:

[0372] a convolution layer output image obtaining sub-module configured to process the recovered MRI image data into a feature map to obtain a convolution layer output image;

[0373] a residual connection output calculating sub-module configured to calculate a residual connection output according to the recovered MRI image data and the convolution layer output image;

[0374] a first feature map output sub-module configured to perform channel dimension splicing on the convolution layer output image and the residual connection output to obtain the first feature map.

[0375] Preferably, the second feature map output module comprises:

[0376] a frequency pooling operation sub-module configured to convert the convolution layer output image into a complex frequency matrix in a frequency domain through Fourier transform, and perform twice frequency pooling operation on the complex frequency matrix to obtain two channel-aware attention maps;

[0377] a channel attention distribution obtaining sub-module configured to connect the two channel-aware attention maps, and obtain a channel attention distribution through convolution operation and sigmoid activation function;

[0378] a channel attention feature map obtaining sub-module configured to apply the channel attention distribution to the original first feature map to obtain a channel attention feature map;

[0379] an average pooling and maximum pooling sub-module configured to obtain an average pooling feature map and a maximum pooling feature map through average pooling and maximum pooling on the channel attention feature map;

[0380] a spatial attention distribution sub-module configured to connect the average pooling feature map and the maximum pooling feature map, and obtain a spatial attention distribution through convolution operation and sigmoid activation function;

[0381] The second feature map obtaining submodule is configured to apply a spatial attention distribution to the channel attention feature map to obtain a second feature map.

[0382] Preferably, the second feature map comprises a first sub-feature map, a second sub-feature map, a third sub-feature map, and a fourth sub-feature map; the third feature map comprises a first final feature map, a second final feature map, a third final feature map, a fourth final feature map, and a fifth final feature map; and the third feature map output module comprises:

[0383] The first intermediate feature map obtaining submodule is configured to perform a first convolution operation on the first sub-feature map to obtain a first intermediate feature map.

[0384] The second intermediate feature map obtaining submodule is configured to perform a first convolution operation on the second sub-feature map to obtain a first intermediate sub-feature map, and add the first intermediate feature map and the first intermediate sub-feature map to obtain a second intermediate feature map.

[0385] The third intermediate feature map obtaining submodule is configured to perform a first convolution operation on the third sub-feature map to obtain a second intermediate sub-feature map, and add the second intermediate feature map and the second intermediate sub-feature map to obtain a third intermediate feature map.

[0386] The fourth intermediate feature map obtaining submodule is configured to perform a first convolution operation on the fourth sub-feature map to obtain a third intermediate sub-feature map, and add the third intermediate feature map and the third intermediate sub-feature map to obtain a fourth intermediate feature map.

[0387] The final feature map obtaining submodule is configured to perform a second convolution operation on the first intermediate feature map, the second intermediate feature map, the third intermediate feature map, and the fourth intermediate feature map, respectively, to obtain the output first final feature map, the second final feature map, the third final feature map, and the fourth final feature map.

[0388] The fifth final feature map obtaining submodule is configured to perform a third convolution operation and a maximum pooling operation on the fourth final feature map to obtain a fifth final feature map.

[0389] Preferably, the ROI region output module comprises:

[0390] The depth feature obtaining submodule is configured to sequentially perform a convolution operation, a normalization processing operation, and an activation function operation on the third feature map, and repeat the above operations several times to obtain a depth feature.

[0391] The prediction probability obtaining submodule is configured to perform a convolution operation on the depth feature to generate a class prediction map, and apply a sigmoid function to the class prediction map to obtain a prediction probability of each class at each position.

[0392] The bounding box prediction map generation submodule is configured to perform convolution operation on the depth feature to generate a bounding box prediction map, wherein the bounding box prediction map contains coordinates of a bounding box predicted at each position;

[0393] The centrality prediction map generation submodule is configured to perform convolution operation on the depth feature to generate a centrality prediction map, and apply a sigmoid function to the centrality prediction map to obtain a centrality prediction value at each position.

[0394] The ROI region screening submodule is configured to screen a prediction box meeting a preset condition as an ROI region through threshold screening and non-maximum suppression according to the prediction probability, the coordinates of the bounding box and the centrality prediction value.

[0395] Preferably, the target detection operation module comprises:

[0396] The first convolution feature map obtaining submodule is configured to extract a feature map from the ROI region, and perform convolution operation on the feature map to obtain a first convolution feature map.

[0397] The correlation calculation submodule is configured to calculate a correlation between pixels according to the feature map.

[0398] The second convolution feature map obtaining submodule is configured to extract spatial information of a feature map from the feature map through a pooling operation, and perform operation and convolution operation on the spatial information of the feature map to obtain a second convolution feature map.

[0399] The final attention feature map calculation submodule is configured to calculate a final attention feature map according to the first convolution feature map, the correlation between pixels and the second convolution feature map.

[0400] The target detection result image obtaining submodule is configured to obtain a target detection result image by performing second-layer convolution on the final attention feature map.

[0401] The above-mentioned various modules in the MRI image recognition device based on deep learning can be realized by software, hardware and combinations thereof in whole or in part. The above-mentioned various modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to the above-mentioned various modules.

[0402] The MRI image recognition device based on deep learning provided above can be used to execute the MRI image recognition method based on deep learning provided in any of the above-mentioned embodiments, and has corresponding functions and beneficial effects.

[0403] ​In one embodiment, a computer device, which can be a terminal, is provided, and an internal structure diagram of the computer device can be as shown in FIG. 1. Figure 14 The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement a deep learning-based MRI image recognition method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0404] Those skilled in the art can understand that Figure 14 the structure shown in FIG. 1 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0405] In one embodiment, a computer device is provided, including a memory and a processor, and the memory stores a computer program, and the processor executes the computer program to implement the steps as Figures 1 to 12 described above.

[0406] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps as Figures 1 to 12 described above.

[0407] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other.

[0408] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, embodiments of the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) embodying computer program instructions.

[0409] Embodiments of the present application are described herein with reference to the drawings, which are as follows: Figure 1 one or more processes and / or blocks Figure 1 means for performing the functions specified in the one or more processes and / or blocks

[0410] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 one or more processes and / or blocks Figure 1 means for performing the functions specified in the one or more processes and / or blocks

[0411] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 one or more processes and / or blocks Figure 1 means for performing the functions specified in the one or more processes and / or blocks

[0412] While preferred embodiments of the present application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the foregoing description. Therefore, the appended claims are intended to encompass within their scope all such variations and modifications as are included within the scope of the present application.

[0413] Finally, it needs to be pointed out that in this article, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or terminal device. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of other identical elements in the process, method, article or terminal device including the element.

[0414] The above provides a detailed introduction to the MRI image recognition method based on deep learning, the MRI image recognition device based on deep learning, the computer device and the storage medium provided by the present application. The principle and implementation mode of the present application are described by applying specific examples in this article. The above example is only used to help understand the method and core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A deep learning-based MRI image recognition method, characterized in that, The method includes: Obtain the MRI image dataset; The MRI image dataset is preprocessed to obtain preprocessed MRI image data. The preprocessed MRI image data is then convolved and pooled to obtain the recovered MRI image data. The recovered MRI image data is output to the MSB module to obtain the first feature map. The first feature map is input into the FSA module for processing to obtain the output second feature map; The second feature map is input into the FPN module for processing to obtain the output third feature map; The third feature map is input into the FCOS HEAD module for processing to obtain the output ROI region; Perform target detection on the ROI region to obtain the output target detection result image; The step of outputting the recovered MRI image data to the MSB module to obtain the first feature map includes: The recovered MRI image data is processed into feature maps to obtain the output image of the convolutional layer; Based on the recovered MRI image data and the output image of the convolutional layer, the residual connection output is calculated; The first feature map is obtained by concatenating the channel dimensions of the convolutional layer output image and the residual connection output. The step of inputting the first feature map into the FSA module for processing to obtain the output second feature map includes: The output image of the convolutional layer is transformed to the frequency domain through Fourier transform to obtain a complex frequency matrix. Two frequency pooling operations are performed on the complex frequency matrix to obtain attention maps for two channels. The attention maps of the two channels are connected, and the channel attention distribution is obtained by convolution operation and sigmoid activation function. The channel attention distribution is applied to the original first feature map to obtain the channel attention feature map; For the channel attention feature map, average pooling and max pooling are used to obtain average pooling feature map and max pooling feature map; The average pooling feature map and the max pooling feature map are concatenated, and the spatial attention distribution is obtained by convolution operation and sigmoid activation function; The spatial attention distribution is applied to the channel attention feature map to obtain the second feature map.

2. The MRI image recognition method according to claim 1, characterized in that, The preprocessing of the MRI image dataset to obtain preprocessed MRI image data includes: The MRI image dataset is standardized and intensity normalized to obtain preprocessed MRI image data.

3. The MRI image recognition method according to claim 1, characterized in that, The step of performing convolutional pooling on the preprocessed MRI image data to obtain the recovered MRI image data includes: The preprocessed MRI image data is subjected to a first convolution operation to obtain the first convolution feature map; A first pooling operation is performed on the first convolutional feature map to obtain a first pooling feature map; A second convolution operation is performed on the first pooling feature map to obtain the second convolution feature map; A second pooling operation is performed on the second convolutional feature map to obtain the second pooling feature map; A third convolution operation is performed on the second pooling feature map to obtain the third convolution feature map; A third pooling operation is performed on the third convolutional feature map to obtain the third pooling feature map; A fourth convolution operation is performed on the third pooling feature map to obtain a fourth convolution feature map; A fourth pooling operation is performed on the fourth convolutional feature map to obtain the fourth pooled feature map; A first deconvolution operation is performed on the fourth pooling feature map to obtain a first deconvolution feature map; A fifth convolution operation is performed on the first deconvolution feature map to obtain the fifth convolution feature map; A second deconvolution operation is performed on the fifth convolution feature map to obtain a second deconvolution feature map; A sixth convolution operation is performed on the second deconvolution feature map to obtain the sixth convolution feature map; A third deconvolution operation is performed on the sixth convolution feature map to obtain the third deconvolution feature map; A seventh convolution operation is performed on the third deconvolution feature map to obtain the seventh convolution feature map; A fourth deconvolution operation is performed on the seventh convolution feature map to obtain a fourth deconvolution feature map; An eighth convolution operation is performed on the fourth deconvolution feature map to obtain the eighth convolution feature map; A ninth convolution operation is performed on the eighth convolution feature map to obtain the restored MRI image data.

4. The MRI image recognition method according to claim 1, characterized in that, in, The second feature map includes a first sub-feature map, a second sub-feature map, a third sub-feature map, and a fourth sub-feature map; the third feature map includes a first final feature map, a second final feature map, a third final feature map, a fourth final feature map, and a fifth final feature map. The step of inputting the second feature map into the FPN module for processing to obtain the output third feature map includes: Perform a first convolution operation on the first sub-feature map to obtain the first intermediate feature map; The second sub-feature map is convolved for the first time to obtain the first intermediate sub-feature map, and the first intermediate feature map is added to the first intermediate sub-feature map to obtain the second intermediate feature map. The third sub-feature map is subjected to a first convolution operation to obtain a second intermediate sub-feature map, and the second intermediate feature map is added to the second intermediate sub-feature map to obtain a third intermediate feature map; The fourth sub-feature map is convolved for the first time to obtain the third intermediate sub-feature map, and the third intermediate feature map is added to the third intermediate sub-feature map to obtain the fourth intermediate feature map. The first intermediate feature map, the second intermediate feature map, the third intermediate feature map, and the fourth intermediate feature map are each subjected to a second convolution operation to obtain the first final feature map, the second final feature map, the third final feature map, and the fourth final feature map. The fourth final feature map is subjected to a third convolution operation and a max pooling operation to obtain the fifth final feature map.

5. The MRI image recognition method according to claim 1, characterized in that, The process of inputting the third feature map into the FCOS HEAD module for processing to obtain the output ROI region includes: The third feature map is subjected to convolution, normalization and activation function operations in sequence, and the above operations are repeated several times to obtain deep features; Convolution operations are performed based on the depth features to generate a category prediction map. The sigmoid function is then applied to the category prediction map to obtain the predicted probability of each category at each position. A convolution operation is performed based on the depth features to generate a bounding box prediction map, wherein the bounding box prediction map contains the coordinates of the bounding box predicted at each location; A convolution operation is performed based on the depth features to generate a centrality prediction map. The sigmoid function is then applied to the centrality prediction map to obtain the centrality prediction value at each position. Based on the predicted probability, bounding box coordinates, and centrality prediction value, prediction boxes that meet preset conditions are selected as ROI regions through threshold filtering and non-maximum suppression.

6. The MRI image recognition method according to claim 1, characterized in that, The target detection operation performed on the ROI region to obtain the output target detection result image includes: Feature maps are extracted from the ROI region, and convolution operations are performed on the feature maps to obtain the first convolutional feature map; The correlation between defined pixels is calculated based on the feature mapping. Based on the feature mapping, spatial information of the feature map is extracted through pooling operations, and the spatial information of the feature map is then processed. The operation and convolution operation are performed to obtain the second convolutional feature map; The final attention feature map is calculated based on the first convolutional feature map, the correlation between defined pixels, and the second convolutional feature map. The final attention feature map is passed through a second convolutional layer to obtain the target detection result image.

7. A deep learning-based MRI image recognition device, characterized in that, The device includes: The dataset acquisition module is used to acquire MRI image datasets; The preprocessing module is used to preprocess the MRI image dataset to obtain preprocessed MRI image data; The convolutional pooling module is used to perform convolutional pooling on the preprocessed MRI image data to obtain the restored MRI image data. The first feature map output module is used to output the recovered MRI image data to the MSB module to obtain the output first feature map. The second feature map output module is used to input the first feature map into the FSA module for processing to obtain the output second feature map; The third feature map output module is used to input the second feature map into the FPN module for processing, and obtain the output third feature map; The ROI region output module is used to input the third feature map into the FCOS HEAD module for processing to obtain the output ROI region; The target detection operation module is used to perform target detection operations on the ROI region and obtain the output target detection result image. The first feature map output module includes: The convolutional layer output image acquisition submodule is used to process the recovered MRI image data into a feature map to obtain the convolutional layer output image. The residual connection output calculation submodule is used to calculate the residual connection output based on the recovered MRI image data and the output image of the convolutional layer; The first feature map output submodule is used to perform channel dimension concatenation based on the convolutional layer output image and the residual connection output to obtain the first feature map. The second feature map output module includes: The frequency pooling operation submodule is used to transform the output image of the convolutional layer to the frequency domain through Fourier transform to obtain a complex frequency matrix, and perform two frequency pooling operations on the complex frequency matrix to obtain two channel-aware attention maps. The channel attention distribution acquisition submodule is used to connect the attention maps perceived by the two channels and obtain the channel attention distribution through convolution operations and the sigmoid activation function; The channel attention feature map acquisition submodule is used to apply the channel attention distribution to the original first feature map to obtain the channel attention feature map. The average pooling and max pooling submodules are used to obtain average pooling feature maps and max pooling feature maps from the channel attention feature maps by using average pooling and max pooling. The spatial attention distribution submodule is used to connect the average pooling feature map and the max pooling feature map, and obtain the spatial attention distribution through convolution operation and sigmoid activation function; The second feature map acquisition submodule is used to apply the spatial attention distribution to the channel attention feature map to obtain the second feature map.

8. The MRI image recognition device according to claim 7, characterized in that, The preprocessing module includes: The preprocessing submodule is used to perform standardization and intensity normalization on the MRI image dataset to obtain preprocessed MRI image data.

9. The MRI image recognition device according to claim 7, characterized in that, The convolutional pooling module includes: The first convolutional feature map acquisition submodule is used to perform the first convolution operation on the preprocessed MRI image data to obtain the first convolutional feature map. The first pooling feature map acquisition submodule is used to perform a first pooling operation on the first convolutional feature map to obtain the first pooling feature map. The second convolutional feature map acquisition submodule is used to perform a second convolution operation on the first pooling feature map to obtain the second convolutional feature map. The second pooling feature map acquisition submodule is used to perform a second pooling operation on the second convolutional feature map to obtain the second pooling feature map; The third convolutional feature map acquisition submodule is used to perform a third convolution operation on the second pooling feature map to obtain the third convolutional feature map. The third pooling feature map acquisition submodule is used to perform a third pooling operation on the third convolutional feature map to obtain the third pooling feature map. The fourth convolutional feature map acquisition submodule is used to perform a fourth convolution operation on the third pooling feature map to obtain the fourth convolutional feature map. The fourth pooling feature map acquisition submodule is used to perform a fourth pooling operation on the fourth convolutional feature map to obtain the fourth pooling feature map. The first deconvolution feature map acquisition submodule is used to perform the first deconvolution operation on the fourth pooling feature map to obtain the first deconvolution feature map. The fifth convolutional feature map acquisition submodule is used to perform a fifth convolution operation on the first deconvolutional feature map to obtain the fifth convolutional feature map. The second deconvolution feature map acquisition submodule is used to perform a second deconvolution operation on the fifth convolution feature map to obtain a second deconvolution feature map. The sixth convolutional feature map acquisition submodule is used to perform a sixth convolution operation on the second deconvolutional feature map to obtain the sixth convolutional feature map. The third deconvolution feature map acquisition submodule is used to perform a third deconvolution operation on the sixth convolution feature map to obtain the third deconvolution feature map. The seventh convolutional feature map acquisition submodule is used to perform a seventh convolution operation on the third deconvolutional feature map to obtain the seventh convolutional feature map. The fourth deconvolution feature map acquisition submodule is used to perform a fourth deconvolution operation on the seventh convolution feature map to obtain the fourth deconvolution feature map. The eighth convolutional feature map acquisition submodule is used to perform an eighth convolution operation on the fourth deconvolutional feature map to obtain the eighth convolutional feature map. The MRI image data acquisition submodule is used to perform a ninth convolution operation on the eighth convolution feature map to obtain the recovered MRI image data.

10. The MRI image recognition device according to claim 7, characterized in that, in, The second feature map includes a first sub-feature map, a second sub-feature map, a third sub-feature map, and a fourth sub-feature map; the third feature map includes a first final feature map, a second final feature map, a third final feature map, a fourth final feature map, and a fifth final feature map. The third feature map output module includes: The first intermediate feature map acquisition submodule is used to perform a first convolution operation on the first sub-feature map to obtain the first intermediate feature map. The second intermediate feature map acquisition submodule is used to perform a first convolution operation on the second sub-feature map to obtain a first intermediate sub-feature map, and add the first intermediate feature map to the first intermediate sub-feature map to obtain a second intermediate feature map. The third intermediate feature map acquisition submodule is used to perform a first convolution operation on the third sub-feature map to obtain a second intermediate sub-feature map, and add the second intermediate feature map to the second intermediate sub-feature map to obtain a third intermediate feature map; The fourth intermediate feature map acquisition submodule is used to perform a first convolution operation on the fourth sub-feature map to obtain a third intermediate sub-feature map, and add the third intermediate feature map to the third intermediate sub-feature map to obtain a fourth intermediate feature map. The final feature map acquisition submodule is used to perform a second convolution operation on the first intermediate feature map, the second intermediate feature map, the third intermediate feature map and the fourth intermediate feature map respectively to obtain the output first final feature map, the second final feature map, the third final feature map and the fourth final feature map; The fifth final feature map acquisition submodule is used to perform a third convolution operation and a max pooling operation on the fourth final feature map to obtain the fifth final feature map.

11. The MRI image recognition device according to claim 7, characterized in that, The ROI region output module includes: The deep feature acquisition submodule is used to sequentially perform convolution, normalization and activation function operations on the third feature map, and repeat the above operations several times to obtain deep features. The prediction probability acquisition submodule is used to perform convolution operations based on the depth features to generate a category prediction map. The sigmoid function is applied to the category prediction map to obtain the prediction probability of each category at each position. The bounding box prediction map generation submodule is used to perform convolution operations based on the depth features to generate a bounding box prediction map, wherein the bounding box prediction map contains the coordinates of the predicted bounding box at each location; The centrality prediction map generation submodule is used to perform convolution operations based on the depth features to generate a centrality prediction map. The sigmoid function is applied to the centrality prediction map to obtain the centrality prediction value at each position. The ROI region filtering submodule is used to filter out predicted boxes that meet preset conditions as ROI regions based on the predicted probability, the coordinates of the bounding box, and the predicted centrality value through threshold filtering and non-maximum suppression.

12. The MRI image recognition device according to claim 7, characterized in that, The target detection operation module includes: The first convolutional feature map acquisition submodule is used to extract feature maps from the ROI region, and perform convolution operation on the feature maps to obtain the first convolutional feature map; The correlation calculation submodule is used to calculate the correlation between defined pixels based on the feature map; The second convolutional feature map acquisition submodule is used to extract spatial information of the feature map through pooling operations based on the feature map, and then perform... The operation and convolution operation are performed to obtain the second convolutional feature map; The final attention feature map calculation submodule is used to calculate the final attention feature map based on the first convolutional feature map, the correlation between defined pixels, and the second convolutional feature map. The target detection result image acquisition submodule is used to obtain the target detection result image by passing the final attention feature map through a second convolution layer.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the deep learning-based MRI image recognition method according to any one of claims 1 to 6.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the deep learning-based MRI image recognition method according to any one of claims 1 to 6.

Images