Pituitary micro-adenoma detection system based on dynamically enhanced MR image

Abstract

The invention belongs to the field of medical and industrial crossing, particularly relates to a pituitary micro-adenoma detection system based on a dynamic enhanced MR image, and aims to solve the problem that pituitary micro-adenoma is difficult to judge and find timely and accurately in clinical diagnosis. The system provided by the invention comprises a preprocessing module, a feature extraction module, a pituitary segmentation module, an image reconstruction module, a pituitary micro-adenoma segmentation module and a diagnosis module. On the basis of the system, MR images in the same layer and multiple periods in an MR image sequence are fused, the contrast ratio of an image concerned area and a general area is enhanced, intensity change information in a dynamic enhancement sequence is reserved to the maximum extent, and the pituitary micro-adenoma segmentation and recognition precision is improved through super-resolution reconstruction. For medical images with low resolution and simple image semantics, pituitary micro-adenoma can still be segmented and detected accurately.

Description

Background Art

[0002] The most common benign tumor of the central nervous system in the pituitary is pituitary adenoma, accounting for about 9% to 16% of all intracranial tumors, and the incidence rate ranks third among intracranial tumor diseases, with a prevalence rate of about 0.08% to 0.09%. In recent years, with the development of medical equipment and clinical medicine, the incidence of pituitary adenoma has been on the rise, and the number of asymptomatic cases has increased significantly.

[0003] Pituitary microadenomas are a special type of pituitary adenoma. Because of their small size, they are difficult to detect, making it difficult for doctors to make timely and accurate clinical judgments. In addition, different types of pituitary microadenomas have different endocrine characteristics and clinical symptoms. For example, growth hormone-type pituitary microadenomas can cause headaches, vision loss, arthritis, etc.; adrenocorticotropic hormone-type pituitary microadenomas mostly manifest as weight gain, muscle weakness, etc. Summary of the invention

[0004] In order to solve the above-mentioned problem in the prior art, that is, the problem that the prior art is difficult to detect pituitary microadenomas in a timely and accurate manner, the present invention, in a first aspect, proposes a pituitary microadenoma detection system based on dynamic enhanced MR images, the system comprising:

[0005] A preprocessing module, configured to acquire dynamically enhanced MR images and perform preprocessing, wherein the data preprocessing module uses a preset image processing algorithm to perform image fusion processing on MR images of the same layer at different time points to improve the contrast between a region of interest and a general region in the MR image, wherein the region of interest is a region in the MR image enhanced by an enhancer, and the general region is a region in the MR image not enhanced by an enhancer;

[0006] A feature extraction module is configured to perform multi-scale feature extraction on the preprocessed MR image through a preset convolutional neural network to obtain a first feature map group;

[0007] a pituitary segmentation module, configured to input the first feature map group into a preset segmentation model to perform image segmentation, and determine a pituitary region segmentation result output by the segmentation model;

[0008] An image reconstruction module is configured to perform super-resolution reconstruction on the pituitary region segmentation result to obtain a super-resolution image of the pituitary region;

[0009] a pituitary microadenoma segmentation module, configured to input the pituitary region super-resolution image into the feature extraction module, so that the feature extraction module outputs a second feature map group according to the pituitary region super-resolution image, and performs image segmentation on the second feature map group, and outputs a pituitary microadenoma segmentation result;

[0010] The diagnosis module is configured to classify the pituitary microadenoma according to the segmentation result and output a diagnosis result.

[0011] In some preferred embodiments, the MR image is composed of multiple dynamic sequences, including patient MR images and normal MR images, and the patient MR images and normal MR images both include corresponding classification labels and diagnostic information, and the classification labels and diagnostic information are used to enable the diagnostic module to classify pituitary microadenomas.

[0012] In some preferred embodiments, the preprocessing module performs fusion processing on the MR images through a variety of image preprocessing algorithms, specifically including:

[0013] The preprocessing module performs maximum value-based image preprocessing to satisfy:

[0014]

[0015] In the formula, are MR images at different time points in the same layer of the MR image sequence, Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image, respectively, L n represents all MR image sequences;

[0016] The preprocessing module performs minimum-based image preprocessing to satisfy:

[0017]

[0018] Where Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image respectively, and L n represents all MR image sequences;

[0019] The preprocessing module performs mean-based image preprocessing to meet the following requirements:

[0020]

[0021] Where Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image respectively. is the first phase MR image in the same layer, L n represents all MR image sequences;

[0022] The preprocessing module performs image preprocessing based on maximum and minimum values, satisfying:

[0023]

[0024] Where Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image respectively, and L n represents all MR image sequences;

[0025] The preprocessing module performs image preprocessing based on maximum value and bilateral filtering, satisfying:

[0026]

[0027] Where S is the neighborhood around pixel q, G σ is the spatial weight function, p is the neighborhood pixel, q is the center pixel, and q x ,q y Respectively, images The x and y coordinates of the center pixel q.

[0028] In some preferred embodiments, the training process of the segmentation model includes:

[0029] Acquire a dataset of dynamic enhanced MR images;

[0030] Dividing the data set according to a preset ratio to determine a training set;

[0031] Preprocessing the MR images in the training set to determine a pituitary training set, wherein the true value label of the pituitary training set corresponds to the pituitary region representing the dynamic enhancement sequence image;

[0032] Inputting the pituitary training set into the feature extraction module to obtain a multi-scale feature map group, and determining the pituitary region segmentation result, and calculating the loss value according to the true value label corresponding to the pituitary region, performing back propagation parameter update, and after the training is completed, using the model weight of the model under the optimal result as the pre-training weight of the pituitary microadenoma segmentation task;

[0033] According to the pituitary region segmentation result, updating the data set so that the true value label of the data set corresponds to the pituitary microadenoma region of the dynamic enhanced sequence image, and performing super-resolution reconstruction on the data set and the true value label;

[0034] The pituitary microadenoma training set is input into the feature extraction module to obtain a multi-scale feature map group, and the preliminary segmentation result of the pituitary microadenoma area is determined, and the loss value is calculated according to the true value label, and the parameters are updated by back propagation until the training is completed.

[0035] In some preferred embodiments, during the training process of the segmentation model, the parameters of the loss function are updated by an SGD optimizer, and the SGD optimizer satisfies:

[0036]

[0037] In the formula, η is the learning rate that controls the parameter update step, θ is the model parameter, J is the loss function, and x (i:i+n) ,y (i:i+n) are the x and y coordinates of the pixel in the image.

[0038] In some preferred embodiments, during the training process of the segmentation model, a momentum term is introduced into the momentum to improve the convergence speed and stability of the algorithm, and the momentum term is:

[0039]

[0040] In the formula, θ is the model parameter, v is the momentum, and γ is the attenuation coefficient;

[0041] Introducing the momentum term into the SGD optimizer satisfies:

[0042] θ=θ-v t ;

[0043] Where θ is the model parameter and t is the time coordinate of the image sequence.

[0044] In some preferred embodiments, the feature extraction module performs multi-scale feature extraction on the MR image, including:

[0045] By using a preset convolutional neural network, the MR image is subjected to multi-scale parallel convolution to obtain a feature map group at multiple scales, wherein the feature map group includes a low-resolution high-level multi-scale feature map group and a high-resolution low-level multi-scale feature map group;

[0046] Multi-scale cross convolution is performed on the feature map groups at the multiple scales to fuse the high-resolution feature map groups with the low-resolution feature map groups to obtain a new multi-scale feature map.

[0047] In some preferred embodiments, the convolutional neural network includes 4 stages, wherein the first stage is used to perform multi-scale parallel convolution, and the second, third, and fourth stages are used to perform multi-scale parallel convolution and multi-scale cross convolution. The multi-scale parallel convolution introduces a compression-excitation channel attention mechanism and a dual-pooling attention mechanism. In the compression stage of the compression-excitation channel attention mechanism, the input feature map F is firstly globally averaged pooled:

[0048]

[0049] Where F is the input feature map, H and W are the height and width of the feature map, and F(i,j) is the pixel element in the feature map;

[0050] In the excitation phase of the compression-excitation channel attention mechanism, the f obtained in the compression phase is sq (F) Dimensionality reduction is performed through a preset fully connected layer. After activation by a preset ReLU activation function, it is input into a preset fully connected layer to restore the dimension.

[0051] In some preferred embodiments, the system uses a TecoGAN model based on adversarial generation and cyclic training to reconstruct the MR image, and the TecoGAN obtains the super-resolution pituitary region image by changing the dimension of the pituitary region image.

[0052] In some preferred embodiments, the TecoGAN includes a cycle generator, a flow estimation network and a spatiotemporal discriminator. The cycle generator cyclically generates a high-resolution pituitary region image based on a low-resolution pituitary region image. The flow estimation network is used to learn dynamic compensation between different periods of the same layer of the dynamically enhanced image, so that the game between the cycle generator and the spatiotemporal discriminator fuses spatial and temporal features.

[0053] Beneficial effects of the present invention:

[0054] (1) The system proposed in the present invention has a strong feature extraction capability. For medical images with low resolution and simple image semantics, it can still accurately segment pituitary microadenomas, a small target, and has a high accuracy in the task of pituitary microadenomas diagnosis.

[0055] (2) The system proposed in the present invention fuses images of the same layer in multiple periods through a preprocessing process, thereby enhancing the contrast between the area of ​​interest and the general area of ​​the image, retaining the intensity change information in the dynamic enhancement sequence to the maximum extent. The area with drastic intensity changes is the pituitary microadenoma area, and provides a strong guarantee for the accuracy of subsequent pituitary segmentation, segmentation and diagnosis of pituitary microadenomas.

[0056] (3) The system proposed in the present invention utilizes super-resolution reconstruction to solve the problem of uneven distribution of target and background pixels of pituitary microadenomas, as well as the problem that fuzzy semantic information and position information at low resolution may lead to unsatisfactory segmentation results, thereby making the final segmentation results more accurate.

[0057] (4) When faced with an unknown dynamic enhancement image sequence, the system proposed in the present invention can diagnose whether the dynamic enhancement sequence contains pituitary microadenoma from data preprocessing to segmentation and classification, thereby efficiently assisting doctors in diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0059] Figure 1 It is a module schematic diagram of a pituitary microadenoma detection system based on dynamic enhanced MR images proposed in an embodiment of the present invention;

[0060] Figure 2 is a schematic diagram of a data preprocessing process proposed in an embodiment of the present invention;

[0061] Figure 3 is a schematic diagram of a data preprocessing result proposed by an embodiment of the present invention;

[0062] Figure 4 It is a flowchart of a method for fusing low-resolution and high-resolution feature maps proposed in an embodiment of the present invention;

[0063] Figure 5 is a schematic diagram of a compression-excitation channel attention mechanism proposed in an embodiment of the present invention;

[0064] Figure 6 It is a schematic diagram of a dual-pooling attention mechanism proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0066] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0067] Reference Figure 1 ,like Figure 1 As shown, the present invention provides a pituitary microadenoma detection system based on dynamic enhanced MR images, the system comprising:

[0068] A preprocessing module, configured to acquire dynamically enhanced MR images and perform preprocessing, wherein the data preprocessing module uses a preset image processing algorithm to perform image fusion processing on MR images of the same layer at different time points to improve the contrast between a region of interest and a general region in the MR image, wherein the region of interest is a region in the MR image enhanced by an enhancer, and the general region is a region in the MR image not enhanced by an enhancer;

[0069] A feature extraction module is configured to perform multi-scale feature extraction on the preprocessed MR image through a preset convolutional neural network to obtain a first feature map group;

[0070] a pituitary segmentation module, configured to input the first feature map group into a preset segmentation model to perform image segmentation, and determine a pituitary region segmentation result output by the segmentation model;

[0071] An image reconstruction module is configured to perform super-resolution reconstruction on the pituitary region segmentation result to obtain a super-resolution image of the pituitary region;

[0072] a pituitary microadenoma segmentation module, configured to input the pituitary region super-resolution image into the feature extraction module, so that the feature extraction module outputs a second feature map group according to the pituitary region super-resolution image, and performs image segmentation on the second feature map group, and outputs a pituitary microadenoma segmentation result;

[0073] The diagnosis module is configured to classify the pituitary microadenoma according to the segmentation result and output a diagnosis result.

[0074] To facilitate a better understanding of the present invention, the application scenarios and related technologies involved in this system are explained:

[0075] Magnetic resonance imaging (MRI) is the most commonly used imaging technique for clinical diagnosis of pituitary microadenomas. MRI has many technical characteristics and advantages, such as multi-parameter, multi-sequence, multi-directional imaging, high soft tissue resolution, and the ability to use flow effects for vascular imaging. Dynamic enhanced scanning is a very important technical technique in the diagnosis of pituitary microadenomas. Even in the diagnosis process of imaging physicians, the dynamic enhanced MR image sequence is the "gold standard" for judgment.

[0076] Nowadays, with the development of deep learning in the biomedical field, medical image processing has become a very important research direction in computer vision. Many researchers and clinicians also hope to achieve computer-aided diagnosis in the clinical medicine field through deep learning or machine learning algorithms, thereby effectively improving the accuracy and timeliness of medical imaging analysis and clinical diagnosis.

[0077] Based on the above conditions, this system aims to combine the relevant algorithm models of deep learning with the diagnosis of pituitary microadenomas. By extracting the temporal information and spatial information features in the dynamic enhancement sequence, combined with the relevant models of deep learning, it can finally achieve accurate segmentation and characterization of pituitary microadenomas, assist clinical diagnosis, and explore the interpretability of deep learning in the medical field.

[0078] In this embodiment, the dynamically enhanced MR image data is firstly acquired. During the acquisition process, the inclusion and exclusion criteria of the MR image data are as follows:

[0079] The inclusion criteria include:

[0080] (1) Patients diagnosed with pituitary microadenoma after preliminary diagnosis, with a maximum diameter of less than 10 mm and consistent with the dynamic enhancement sequence performance; (2) For patients initially diagnosed with functional pituitary microadenoma, the corresponding endocrine hormone index text data are provided; (3) Patients without a history of sellar region surgery; (4) Complete clinical and pathological characteristics.

[0081] The exclusion criteria included: (1) MRI imaging errors, low image resolution, significant motion or susceptibility artifacts; and (2) postoperative recurrence of pituitary adenoma or history of sellar region surgery.

[0082] Specifically, the above-mentioned MR image is composed of multiple dynamic sequences, including patient MR images and normal MR images, and the patient MR images and normal MR images both include corresponding classification labels and diagnostic information, and the classification labels and diagnostic information are used to enable the diagnostic module to classify pituitary microadenomas.

[0083] It is easy to understand that since dynamic enhanced scanning has a fixed layer thickness, accompanied by the injection of contrast agent, each layer is scanned multiple times and in sequence, and finally a dynamic enhanced scanning sequence is formed. The blood flow velocity of normal pituitary tissue and pituitary microadenoma tissue is inconsistent, which also leads to inconsistent enhancement of the two tissues by contrast agents, that is, in a short period of time after injection, the pituitary shows high signal and the pituitary microadenoma shows low signal.

[0084] Therefore, if Figure 2 As shown in the figure, during the data preprocessing process, this system fuses the MR images of the same layer at different time points to enhance the contrast between the image area of ​​interest and the general area, and retains the intensity change information in the dynamic enhancement sequence to the maximum extent. The purpose of selecting the same layer at different times is that the MR images of the same layer have the same intracranial tissue structure, and there will be no deviation after image fusion. In terms of data dimension, since the MR image format is DICOM, it is a grayscale image, and its data dimension is Where c, w, and h are the number of channels, width, and height of the image respectively. The image L' i In terms of data dimension, it is similar to the original image L i Be consistent.

[0085] In this embodiment, the image fusion method for data preprocessing mainly includes: maximum filtering, minimum filtering, mean filtering and subtraction of the first-phase image, subtraction of maximum filtering and minimum filtering, and bilateral filtering after maximum filtering.

[0086] Specifically, the preprocessing module performs a preprocessing process of fusing the MR images through the above-mentioned multiple image preprocessing algorithms, which specifically includes:

[0087] The preprocessing module performs maximum-based image preprocessing to meet the following requirements:

[0088]

[0089] In the formula, is the MR image at different time points in each layer of the MR image sequence, Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image respectively, L n Represents all MR image sequences. The preprocessing module processes the same layer MR image sequence L i Take the maximum grayscale value of the corresponding pixel, put the maximum grayscale value at the position of the pixel, and traverse the entire image to get the final result.

[0090] The corresponding pixel points refer to the alignment between images in the MR image sequence of the same layer. The first pixel point in the upper left corner of the first image corresponds to the first pixel point in the upper left corner of all subsequent images, and the other pixel points are corresponding pixel points by analogy.

[0091] The preprocessing module performs image preprocessing based on the minimum value, that is, the same layer MR image sequence L i Take the minimum grayscale value of the corresponding pixel, put the minimum grayscale value at the position of the pixel, traverse the entire image to get the result, satisfying:

[0092]

[0093] In the formula, is the MR image at different time points in each layer of the MR image sequence, Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image respectively, L n represents all MR image sequences;

[0094] The preprocessing module performs mean-based image preprocessing, that is, the same layer MR image sequence L i Take the mean of the corresponding pixels and subtract the MR image of the first phase, that is, In this way, the grayscale values ​​of the pixels in the MR image that have not been enhanced by contrast agents are basically the same as those of the pixels in the corresponding positions in the first-phase MR image after the mean calculation, while the grayscale values ​​of the pixels in the MR image that have been enhanced by contrast agents will be greater than the grayscale values ​​of the pixels in the first-phase MR image after the mean calculation. After that, the grayscale values ​​of the unenhanced areas will tend to 0, while the grayscale values ​​of the enhanced areas will remain at a certain value, satisfying:

[0095]

[0096] In the formula, is the MR image at different time points in each layer of the MR image sequence, m is the time range of each layer in each layer of the MR image sequence, Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image, respectively. is the first phase MR image in the same layer, L n represents all MR image sequences;

[0097] The preprocessing module performs image preprocessing based on maximum and minimum values, that is, subtracts the maximum value filtering from the minimum value filtering to satisfy:

[0098]

[0099] Where, L iTj is the MR image at different time points in each layer of the MR image sequence, Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image respectively, L n represents all MR image sequences;

[0100] The preprocessing module performs image preprocessing based on maximum value and bilateral filtering, that is, smoothing the noise of the image obtained after maximum value filtering by bilateral filtering, satisfying:

[0101]

[0102] Where S is the neighborhood around pixel q, G σ is the spatial weight function, p is the neighborhood pixel, q is the center pixel, and q x ,q y Respectively, images The x and y coordinates of the center pixel q.

[0103] For easier understanding, please refer to Figure 3 ,like Figure 3 As shown, Figure 3 (a) is the original image of the coronal slice at a certain time point; Figure 3 (b) is the image after maximum filtering; Figure 3 (c) is the image after minimum filtering; Figure 3 (d) is the image after mean filtering and subtracting the first phase; Figure 3 (e) is the image after the maximum value filtering minus the minimum value filtering; Figure 3 (f) is the image after bilateral filtering. The preprocessed image retains the information of signal intensity changes in the pituitary region of interest, blurs the information of other structures in the sella turcica, and improves the contrast between the region of interest and the general area.

[0104] Furthermore, the training process of the segmentation model includes: obtaining a data set of dynamically enhanced MR images; dividing the data set according to a preset ratio to determine a training set; preprocessing the MR images in the training set to determine a pituitary training set, wherein the true value label of the pituitary training set corresponds to the pituitary region representing the dynamically enhanced sequence image; inputting the pituitary training set into the feature extraction module to obtain a multi-scale feature map group, and determining the pituitary region segmentation result, while calculating the loss value according to the true value label corresponding to the pituitary region, performing back propagation parameter update, and after the training is completed, using the model weight of the model under the optimal result as the pre-training weight of the pituitary microadenoma segmentation task; updating the data set according to the pituitary region segmentation result so that the true value label of the data set corresponds to the pituitary microadenoma region of the dynamically enhanced sequence image, and super-resolution reconstruction of the data set and the true value label; inputting the pituitary microadenoma training set into the feature extraction module to obtain a multi-scale feature map group, and determining the preliminary segmentation result of the pituitary microadenoma region, and calculating the loss value according to the true value label, performing back propagation parameter update until the training is completed.

[0105] Among them, the preset ratio is used to indicate the division ratio of the data set. According to the division ratio, the data set can be divided into a training set, a test set and a validation set. The training set is used to train the model, and the test set is used to train the model. After the model training is completed, the test set is used to make an unbiased estimate of the performance of the final model. The validation set is used to evaluate the performance of the model during the model training process and adjust the hyperparameters of the model.

[0106] Preferably, the ratio of the training set, the test set, and the validation set is 6:2:2.

[0107] Furthermore, during the training of the segmentation model, the parameters of the loss function are updated by the SGD optimizer, which satisfies:

[0108]

[0109] In the formula, η is the learning rate that controls the parameter update step, θ is the model parameter, J is the loss function, and x (i:i+n) ,y (i:i+n) are the x and y coordinates of the pixel in the image.

[0110] This embodiment uses the SGD optimizer in the process of model training, so as to guide the parameters of the loss function to update the appropriate size in the correct direction during the back propagation process, thereby obtaining the minimized loss function and the optimal model parameters. Regarding the learning rate η, its parameters are specifically set by the implementer, and its value should match the performance of the model. If the learning rate is set too small, the parameters of the model will be updated slowly during the training process, so that a lot of training time is required to achieve convergence, which increases the time cost; and if the learning rate is set too large, the iteration of the parameters cannot guarantee the final convergence of the model.

[0111] In addition, during the training process of the segmentation model, a momentum term is introduced into the momentum to improve the convergence speed and stability of the algorithm. The momentum term is:

[0112]

[0113] In the formula, θ is the model parameter, v is the momentum, and γ is the attenuation coefficient;

[0114] Introducing the momentum term into the SGD optimizer satisfies:

[0115] θ=θ-v t ;

[0116] Where θ is the model parameter and t is the time coordinate of the image sequence.

[0117] In this embodiment, the momentum concept is introduced into the calculation process of gradient descent. Momentum gradient descent is an optimization algorithm in gradient descent, which is used to accelerate the convergence of the gradient descent algorithm. In this algorithm, momentum refers to a proportion of the last parameter update added during the gradient update process, similar to the concept of inertial force. Its purpose is to avoid oscillation and help accelerate the descent on the slope while reducing fluctuations during the convergence process.

[0118] Furthermore, in the above system, multi-scale feature extraction is performed on the MR image through a feature extraction module, including: performing multi-scale parallel convolution on the MR image through a preset convolutional neural network to obtain a feature map group at multiple scales, the feature map group including a low-resolution high-level multi-scale feature map group and a high-resolution bottom-level multi-scale feature map group; performing multi-scale cross convolution on the feature map group at multiple scales to fuse the high-resolution and low-resolution feature map groups to obtain a new multi-scale feature map.

[0119] The convolutional neural network includes 4 stages, the first stage is used to perform multi-scale parallel convolution, the second, third and fourth stages are used to perform multi-scale parallel convolution and multi-scale cross convolution. The multi-scale parallel convolution introduces the compression-excitation channel attention mechanism and the double pooling attention mechanism. In the compression stage of the compression-excitation channel attention mechanism, the input feature map F is firstly globally averaged pooled:

[0120]

[0121] Where F is the input feature map, H and W are the height and width of the feature map, and F(i,j) is the pixel element in the feature map;

[0122] In the excitation phase of the compression-excitation channel attention mechanism, the f obtained in the compression phase is sq (F) Dimensionality reduction is performed through a preset fully connected layer. After activation by a preset ReLU activation function, it is input into a preset fully connected layer to restore the dimension.

[0123] Specifically, a preferred embodiment of the present invention selects the HRNet network in the deep convolutional network to obtain the feature map of the image to be processed. The backbone structure of the network consists of 4 stages, of which the first module only contains multi-scale parallel convolution, and the remaining 3 modules all contain multi-scale parallel convolution and multi-scale convolution. Multi-scale parallel convolution can always maintain high-resolution representation, instead of convolution from high resolution to low resolution like serial convolution, which can reduce the loss of scale features. Multi-scale cross convolution refers to the fusion of feature maps of multiple scales so that the fused feature map has a stronger feature representation. After connecting the high-resolution to low-resolution convolution streams in parallel between the 4 stages and repeatedly exchanging multi-scale feature information, HRNet can maintain high-resolution representation throughout the process. The resolutions of different feature maps in the 4 stages are 4 times, 8 times, 16 times, and 32 times the downsampling multiples of the input image, respectively.

[0124] Among them, two attention mechanisms are introduced in the parallel convolution operation: channel attention and double pooling attention. They aim to enhance the neural network's ability to represent features in channel and spatial dimensions, and better obtain key information in feature maps of different scales.

[0125] It is easy to understand that the channel attention mechanism focuses on the channel dimension of the feature map, which is used to capture the correlation information between different channels to enhance the representation ability of the features. It can also help the network focus on the most important channels, thereby improving the performance of the model. Figure 4In the compression-excitation channel attention of this embodiment, the input feature map is F, and the elements in the feature map are represented by F(i,j). In the compression stage, the input feature map is firstly subjected to global average pooling, as shown in Formula 1. After this operation, the feature map of each channel will be replaced by a value, and the dimension is changed from C×W×H to C×1×1. The feature map can be understood as having a global receptive field. In the excitation stage, the f in the compression stage is sq (F) Through a fully connected layer with 1 / 16 of the number of input nodes, the dimension is reduced to (C / 16)×1×1. After the ReLU activation function, it is restored to its original dimension through a fully connected layer with 16 times the number of input nodes. The purpose of this is to reduce the number of channels and thus the amount of calculation. Finally, through a Sigmoid activation layer, it becomes a value between 0 and 1, and these values ​​are regarded as the importance weights of each channel.

[0126] Correspondingly, spatial attention focuses on the spatial dimension of the feature map, which is used to capture the relationship between different spatial positions in order to better understand the structural information of the image. It also helps the network to better focus on local areas of the image when recognizing objects or performing pixel-level tasks. Please refer to Figure 5 The dual pooling attention in this embodiment is very similar to the compression-excitation channel attention in general, but there are some differences in the early stages. The dual pooling attention uses global average pooling and maximum pooling in the early stages to obtain two channel features of dimension C×1×1, where the output of the maximum pooling layer is the maximum value of all elements in a feature map. Afterwards, the outputs of the two pooling layers will further pass through the same "fully connected layer-ReLU-fully connected layer" three-layer structure as the compression-excitation channel attention module to capture the correlation between channels. The two features are then added together and passed through the Sigmoid activation layer to obtain the weight coefficient, and finally the weight M is used. c Multiplying with the original feature can complete the channel calibration and obtain the new feature.

[0127] Furthermore, in this embodiment, based on the above feature extraction module, this embodiment proposes a method for fusing low-resolution and high-resolution feature maps, the steps of which are:

[0128] Step S100, performing multi-scale parallel convolution on the preprocessed dynamic enhanced magnetic resonance data to obtain feature map groups at multiple scales;

[0129] Step S200, obtaining a new multi-scale feature map by multi-scale cross convolution of the multi-scale feature map group. This step combines the high-resolution and low-resolution feature maps to accurately segment pituitary microadenomas;

[0130] Step S300, based on the segmentation and diagnosis tasks, inputting the extracted multi-scale feature map group into the segmentation task head and the classification task head respectively to obtain the desired segmentation and classification results;

[0131] Among them, step S100 and step S200 will be reused in the two tasks of pituitary segmentation and pituitary microadenoma segmentation;

[0132] Among them, two different channel attention mechanisms are added to the multi-scale parallel convolution, aiming to help the neural network acquire more valuable features during the training process.

[0133] Furthermore, the system uses a TecoGAN model based on adversarial generation and cyclic training to reconstruct the MR image, and the TecoGAN obtains the super-resolution pituitary region image by changing the dimension of the pituitary region image.

[0134] The TecoGAN includes a cycle generator, a flow estimation network and a spatiotemporal discriminator. The cycle generator cyclically generates a high-resolution pituitary region image based on a low-resolution pituitary region image. The flow estimation network is used to learn dynamic compensation between different periods of the same layer of the dynamically enhanced image, so that the game between the cycle generator and the spatiotemporal discriminator fuses spatial and temporal features.

[0135] A MR image super-resolution method according to a third embodiment of the present invention uses a pre-trained model TecoGAN based on adversarial generation and cyclic training to more accurately reconstruct MR images in terms of supervising spatial high-frequency details and temporal relationships.

[0136] It should be noted that the pre-trained TecoGAN model changes the dimension of the pituitary region image from (W / 3)×(H / 3)×3 to 512×512×3, and obtains a super-resolution pituitary region image for the subsequent segmentation of pituitary microadenomas.

[0137] The pre-trained TecoGAN model contains three components: a loop generator, a flow estimation network, and a spatiotemporal discriminator. The loop generator cyclically generates high-resolution pituitary region images based on low-resolution pituitary region image inputs. The flow estimation network learns dynamic compensation between different periods of the same layer of dynamically enhanced images to help the game between the loop generator and the spatiotemporal discriminator better consider temporal continuity, integrate spatial and temporal features, and capture the grayscale change details in the dynamically enhanced sequence.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the system described above can refer to the corresponding process in the aforementioned embodiment, and will not be repeated here.

[0139] It should be noted that the pituitary microadenoma detection system based on dynamic enhanced MR images provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.

[0140] The terms "first", "second", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.

[0141] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus / device.

[0142] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A pituitary microadenoma detection system based on dynamic enhanced MR images, characterized in that: include: A preprocessing module is configured to acquire a dynamically enhanced MR image and perform preprocessing, wherein the preprocessing comprises: using a preset image processing algorithm to perform image fusion processing on MR images of the same layer at different time points to improve the contrast between a region of interest and a general region in the MR image, wherein the region of interest is a region in the MR image enhanced by an enhancer, and the general region is a region in the MR image not enhanced by an enhancer; a feature extraction module configured to perform multi-scale feature extraction on the preprocessed MR image through a pre-built convolutional neural network to obtain a first feature map group; a pituitary segmentation module, configured to input the first feature map group into a trained segmentation model to perform image segmentation, and determine a pituitary region segmentation result output by the segmentation model; An image reconstruction module is configured to perform super-resolution reconstruction on the pituitary region segmentation result to obtain a super-resolution image of the pituitary region; a pituitary microadenoma segmentation module, configured to input the pituitary region super-resolution image into the convolutional neural network to obtain a second feature map group, perform image segmentation on the second feature map group, and output a pituitary microadenoma segmentation result; The diagnosis module is configured to classify the pituitary microadenoma according to the segmentation result, and output the classification result as the detection result.

2. The system according to claim 1, characterized in that The MR image is composed of multiple dynamic sequences, including a patient MR image sequence and a normal MR image sequence. The patient MR image and the normal MR image both include corresponding classification labels and diagnostic information. The classification labels and diagnostic information are used to enable the diagnostic module to classify pituitary microadenomas.

3. The system according to claim 1, characterized in that The preprocessing module performs fusion processing on the MR images through a variety of image preprocessing algorithms, specifically including: The preprocessing module performs maximum value-based image preprocessing to satisfy: In the formula, are MR images at different time points in the same layer of the MR image sequence, Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image respectively, L n represents all MR image sequences; The preprocessing module performs minimum-based image preprocessing to satisfy: Where Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image respectively, and L n represents all MR image sequences; The preprocessing module performs mean-based image preprocessing to meet the following requirements: Where Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image respectively. is the first phase MR image in the same layer, L n represents all MR image sequences; The preprocessing module performs image preprocessing based on maximum and minimum values, satisfying: Where Ω is the image sequence space, h, w, c are the height, width and number of channels of the MR image respectively, and L n represents all MR image sequences; The preprocessing module performs image preprocessing based on maximum value and bilateral filtering, satisfying: Where S is the neighborhood around pixel q, G σ is the spatial weight function, p is the neighborhood pixel, q is the center pixel, and q x ,q y Respectively, images The x and y coordinates of the center pixel q.

4. The system according to claim 1, characterized in that The training process of the segmentation model includes: Acquire a dataset of dynamic enhanced MR images; Dividing the data set according to a preset ratio to determine a training set; Preprocessing the MR images in the training set to determine a pituitary training set, wherein the true value label of the pituitary training set corresponds to the pituitary region representing the dynamic enhancement sequence image; Inputting the pituitary training set into the convolutional neural network to obtain a multi-scale feature map group, and inputting the multi-scale feature map group into the segmentation model to obtain a pituitary region segmentation result; Based on the pituitary region segmentation result and its corresponding true value label, the loss value is calculated, the back propagation parameter is updated, and after the training is completed, the model weight of the model under the optimal result is used as the pre-training weight of the pituitary microadenoma segmentation task; According to the pituitary region segmentation result, the data set is updated, the true value label of the updated data set corresponds to the pituitary microadenoma region of the dynamic enhanced sequence image, and super-resolution reconstruction is performed on the images in the updated data set; The result of super-resolution reconstruction is input into the convolutional neural network and the segmentation model to obtain a preliminary segmentation result of the pituitary microadenoma area, and the loss value is calculated based on the preliminary segmentation result of the pituitary microadenoma area and its corresponding true value label, and back propagation is performed to update the parameters until the training is completed.

5. The system according to claim 4, characterized in that During the training process of the segmentation model, the segmentation model updates the parameters of the loss function through the SGD optimizer during training, and the SGD optimizer satisfies: In the formula, η is the learning rate that controls the parameter update step, θ is the model parameter, J is the loss function, and x (i:i+n) ,y (i:i+n) are the x and y coordinates of the pixel in the MR image.

6. The system according to claim 5, characterized in that During the training process of the segmentation model, a momentum term is introduced into the SGD optimizer to improve the convergence speed and stability of the algorithm. The momentum term is: In the formula, θ is the model parameter, v is the momentum, and γ is the attenuation coefficient; The SGD optimizer after introducing the momentum term satisfies: θ=θ-v t ; Where θ is the model parameter and t is the time coordinate of the MR image sequence.

7. The system according to claim 1, characterized in that The feature extraction module performs multi-scale feature extraction on the MR image, including: By using a preset convolutional neural network, multi-scale parallel convolution is performed on the MR image to obtain a feature map group at multiple scales, wherein the feature map group includes a low-resolution high-level multi-scale feature map group and a high-resolution low-level multi-scale feature map group; Multi-scale cross convolution is performed on the feature map groups at the multiple scales to fuse the high-resolution feature map groups with the low-resolution feature map groups to obtain a new multi-scale feature map.

8. The system according to claim 7, characterized in that The convolutional neural network includes 4 stages, wherein the first stage is used to perform multi-scale parallel convolution, and the second, third and fourth stages are used to perform multi-scale parallel convolution and multi-scale cross convolution, and a compression-excitation channel attention mechanism and a double-pooling attention mechanism are introduced in the process of the multi-scale parallel convolution; In the compression stage of the compression-excitation channel attention mechanism, the input feature map F is first globally average pooled: Where F is the input feature map, H and W are the height and width of the feature map, and F(i,j) is the pixel element in the feature map; In the excitation phase of the compression-excitation channel attention mechanism, the f obtained in the compression phase is sq (F) Dimensionality reduction is performed through a preset fully connected layer. After activation by a preset ReLU activation function, it is input into a preset fully connected layer to restore the dimension.

9. The system according to claim 1, characterized in that The system uses a TecoGAN model based on adversarial generation and cyclic training to reconstruct the MR image. The TecoGAN model obtains the super-resolution pituitary region image by changing the dimension of the pituitary region image.

10. The system according to claim 9, characterized in that The TecoGAN includes a cycle generator, a flow estimation network and a spatiotemporal discriminator. The cycle generator cyclically generates a high-resolution pituitary region image based on a low-resolution pituitary region image. The flow estimation network is used to learn dynamic compensation between different periods of the same layer of the dynamically enhanced image, so that the game between the cycle generator and the spatiotemporal discriminator fuses spatial and temporal features.

Images