Real-time lightweight magnetic resonance image enhancement method and system based on region grading

Abstract

The present application relates to the technical field of image enhancement processing. Provided are a real-time lightweight magnetic resonance image enhancement method and system based on region grading. The method comprises: first, acquiring a magnetic resonance image having a first resolution, and performing automatic region segmentation processing on the magnetic resonance image having the first resolution, wherein the processed magnetic resonance image having the first resolution comprises a plurality of regions; then, performing classification processing on images from the plurality of regions, so as to obtain three regions; performing sparse dilated convolution processing on the images in the three regions, and then inputting the processed images into different residual groups for separate image processing; and performing pixel-wise addition on the images processed by means of the different residual groups, so as to obtain an enhanced magnetic resonance image. By means of automatic region segmentation and classification processing, personalized processing can be accurately performed on different regions in a magnetic resonance image, thereby improving the image quality. Sparse dilated convolution and residual group processing are used to effectively enhance detailed features of the magnetic resonance image, thereby improving the image clarity and the diagnostic accuracy.

Description

A Real-Time Lightweight NMR Image Enhancement Method and System Based on Region Hierarchy

[0001] This invention claims priority to Chinese Patent Application No. 202411609698.7, filed on November 12, 2024, with the title "Real-time Lightweight NMR Image Enhancement Method and System Based on Regional Hierarchy". Technical Field

[0002] This application relates to the field of image enhancement processing technology, and in particular to a real-time lightweight nuclear magnetic resonance image enhancement method and system based on region hierarchical classification. Background Technology

[0003] Magnetic Resonance Imaging (MRI) offers high contrast, high resolution, and is radiation-free, making it the optimal choice for tissue imaging. However, compared to other medical imaging methods, its acquisition speed is relatively slow, making it less suitable as an image servoing method for surgical robots. For MRI-based brain puncture surgical robots, real-time MRI images are needed to monitor the needle's interventional navigation process and provide corresponding control feedback based on the images. Therefore, temporal resolution is a prerequisite for realizing such robotic interventional surgery. To ensure this prerequisite, relevant designs are implemented for MRI sequence acquisition. However, existing image enhancement algorithms are not only time-consuming but also produce unsatisfactory image enhancement results. Summary of the Invention

[0004] To address the aforementioned problems, this application provides a real-time lightweight NMR image enhancement method and system based on region hierarchies, comprising the following:

[0005] In a first aspect, this application provides a real-time lightweight NMR image enhancement method based on region hierarchies, the method comprising:

[0006] An MRI image with a first resolution is acquired, and the MRI image with the first resolution is subjected to automatic region segmentation processing. The processed MRI image with the first resolution includes multiple regions.

[0007] The images of the multiple regions are classified to obtain three regions;

[0008] The images in the three regions are subjected to sparse dilation convolution processing and then input into different residual groups for image processing.

[0009] The pixel results of the images processed by different residual groups are added together to obtain the enhanced NMR image.

[0010] Optionally, the classification process of the images of the multiple regions to obtain three regions includes:

[0011] Calculate the priority score of each region in the processed first resolution image;

[0012] Based on the priority score, each region in the processed first resolution image is classified to obtain a first region, a second region, and a third region.

[0013] Optionally, classifying each region in the processed first resolution image according to the priority score to obtain a first region, a second region, and a third region includes:

[0014] Based on the priority scores, the k regions with the highest scores are selected as the regions of most interest.

[0015] The region of most interest is matched with the corresponding NMR image with a second resolution to obtain the region of most interest in the NMR image with a second resolution, which is taken as the first region; the NMR image with a second resolution is obtained by downsampling pixel matching based on the NMR image with a first resolution, and the resolution of the NMR image with a second resolution is lower than the resolution of the NMR image with a first resolution;

[0016] The MRI image with the second resolution is filtered using pixel filtering to obtain salient information. Based on the salient information and the MRI image with the second resolution, a general region of interest is obtained, which is used as the second region.

[0017] The remaining regions in the MRI image with the second resolution, excluding the first and second regions, are designated as the third region.

[0018] Optionally, performing sparse dilated convolution processing on the images in the three regions respectively includes:

[0019] Obtain the minimum and maximum horizontal and vertical coordinates of the effective region in the partitioned image, perform rectangular expansion on the features of the effective region, and obtain the region enclosed by the four axes of the minimum and maximum horizontal and vertical coordinates as the expanded region.

[0020] Perform an outermost expansion operation on the expanded region;

[0021] The image features, including the outermost ring, are convolved to obtain the convolved features.

[0022] A mask is applied to filter the convolutional features to obtain deep features of the same size as the original region;

[0023] Alternatively, an 8-neighbor dilation convolution can be performed on the features of the effective region in the partitioned image to obtain the dilated region; before each convolution, an 8-neighbor dilation is performed on the features of the effective region in the partitioned image, and the convolved features are the same size as the original region.

[0024] A mask is applied to filter the convolutional features to obtain deep features of the same size as the original region.

[0025] Optionally, the step of performing sparse dilation convolution on the images in the three regions and then inputting them into different residual groups for image processing includes:

[0026] The image of the first region is processed by sparse dilation convolution and then input into the first residual group for image processing.

[0027] After performing sparse dilated convolution on the images of the second region and the third region, the images are input into the second residual group for image processing.

[0028] The complexity of the first residual set is higher than that of the second residual set.

[0029] Optionally, the automatic region segmentation processing of the MRI image with the first resolution specifically includes:

[0030] The MRI image with a first resolution is automatically segmented using an automatic mask generator component, resulting in multiple regions.

[0031] Optionally, the automatic mask generator component can be a pre-trained Segment All Model (SAM), a second-generation Segment All Model derivative (SAM2), or a medical-specific Segment All Model (MedSAM).

[0032] Optionally, calculating the priority score of each region in the processed first resolution image includes:

[0033] The Region Aware Priority Score Calculation (RAPC) component is used to calculate scores for each region in the first resolution image, and the regions are then classified based on the calculated scores.

[0034] Optionally, the method further includes:

[0035] During convolution and sliding windowing, it is determined whether the pixels within the window meet the preset feature conditions;

[0036] If the preset feature conditions are not met, the convolution calculation of that window is skipped, and subsequent convolution operations continue. For pixels that meet the preset feature conditions, they are divided into two regions. Pixels in one region participate in the convolution calculation and produce the convolution result, while pixels in the other region only participate in the convolution calculation of the previous region.

[0037] Optionally, when the images in the three regions are processed by sparse dilation convolution and then input into different residual groups for image processing, different levels of network structures are used to process image regions of different levels; the different levels of network structures include hierarchical modulation expert network RME and spatial modulation expert network SME.

[0038] The step of performing sparse dilation convolution on the image of the first region and then inputting it into the first residual group for image processing includes:

[0039] The image of the first region is processed by sparse dilation convolution and then input into the first residual group composed of RME and SME for image processing.

[0040] The image processing of the second region and the third region images after sparse dilation convolution processing and input into the second residual group includes: the second region and the third region images after sparse dilation convolution processing and input into the second residual group composed of SME and SME for image processing, wherein the second residual group shares parameters.

[0041] Secondly, this application provides a region-level-based real-time lightweight MRI image enhancement system, which includes:

[0042] An acquisition unit is used to acquire an MRI image with a first resolution, and to perform automatic region segmentation processing on the MRI image with the first resolution, wherein the processed MRI image with the first resolution includes multiple regions.

[0043] The first processing unit is used to classify the images of the multiple regions to obtain three regions;

[0044] The second processing unit is used to perform sparse dilation convolution on the images in the three regions respectively, and then input them into different residual groups for image processing respectively.

[0045] The third processing unit is used to add the pixel results of the images processed by different residual groups to obtain the enhanced NMR image.

[0046] Optionally, the first processing unit is specifically used for:

[0047] Calculate the priority score of each region in the processed first resolution image;

[0048] Based on the priority score, each region in the processed first resolution image is classified to obtain a first region, a second region, and a third region.

[0049] Optionally, classifying each region in the processed first resolution image according to the priority score to obtain a first region, a second region, and a third region includes:

[0050] Based on the priority scores, the k regions with the highest scores are selected as the regions of most interest.

[0051] The region of most interest is matched with the corresponding NMR image with a second resolution to obtain the region of most interest in the NMR image with a second resolution, which is taken as the first region; the NMR image with a second resolution is obtained by downsampling pixel matching based on the NMR image with a first resolution, and the resolution of the NMR image with a second resolution is lower than the resolution of the NMR image with a first resolution;

[0052] The MRI image with the second resolution is filtered using pixel filtering to obtain salient information. Based on the salient information and the MRI image with the second resolution, a general region of interest is obtained, which is used as the second region.

[0053] The remaining regions in the MRI image with the second resolution, excluding the first and second regions, are designated as the third region.

[0054] Optionally, the second processing unit performs sparse dilated convolution processing on the images in the three regions respectively, including:

[0055] Obtain the minimum and maximum horizontal and vertical coordinates of the effective region in the partitioned image, perform rectangular expansion on the features of the effective region, and obtain the region enclosed by the four axes of the minimum and maximum horizontal and vertical coordinates as the expanded region.

[0056] Perform an outermost expansion operation on the expanded region;

[0057] The image features, including the outermost ring, are convolved to obtain the convolved features.

[0058] A mask is applied to filter the convolutional features to obtain deep features of the same size as the original region;

[0059] Alternatively, an 8-neighbor dilation convolution can be performed on the features of the effective region in the partitioned image to obtain the dilated region; before each convolution, an 8-neighbor dilation is performed on the features of the effective region in the partitioned image, and the convolved features are the same size as the original region.

[0060] A mask is applied to filter the convolutional features to obtain deep features of the same size as the original region.

[0061] Optionally, the second processing unit performs sparse dilation convolution on the images in the three regions respectively, and then inputs them into different residual groups for image processing, including:

[0062] The image of the first region is processed by sparse dilation convolution and then input into the first residual group for image processing.

[0063] After performing sparse dilated convolution on the images of the second region and the third region, the images are input into the second residual group for image processing.

[0064] The complexity of the first residual set is higher than that of the second residual set.

[0065] Optionally, the acquisition unit performs automatic region segmentation processing on the MRI image with the first resolution, specifically including:

[0066] The MRI image with a first resolution is automatically segmented using an automatic mask generator component, resulting in multiple regions.

[0067] Optionally, the automatic mask generator component can be a pre-trained Segment All Model (SAM), a second-generation Segment All Model derivative (SAM2), or a medical-specific Segment All Model (MedSAM).

[0068] Optionally, the first processing unit calculates the priority score for each region in the processed first resolution image, including:

[0069] The Region Aware Priority Score Calculation (RAPC) component is used to calculate scores for each region in the first resolution image, and the regions are then classified based on the calculated scores.

[0070] Optionally, the second processing unit is further configured to:

[0071] During convolution and sliding windowing, it is determined whether the pixels within the window meet the preset feature conditions;

[0072] If the preset feature conditions are not met, the convolution calculation of that window is skipped, and subsequent convolution operations continue. For pixels that meet the preset feature conditions, they are divided into two regions. Pixels in one region participate in the convolution calculation and produce the convolution result, while pixels in the other region only participate in the convolution calculation of the previous region.

[0073] Optionally, when the images in the three regions are processed by sparse dilation convolution and then input into different residual groups for image processing, different levels of network structures are used to process image regions of different levels; the different levels of network structures include hierarchical modulation expert network RME and spatial modulation expert network SME.

[0074] The step of performing sparse dilation convolution on the image of the first region and then inputting it into the first residual group for image processing includes:

[0075] The image of the first region is processed by sparse dilation convolution and then input into the first residual group composed of RME and SME for image processing.

[0076] The image processing of the second region and the third region images after sparse dilation convolution processing and input into the second residual group includes: the second region and the third region images after sparse dilation convolution processing and input into the second residual group composed of SME and SME for image processing, wherein the second residual group shares parameters.

[0077] Thirdly, this application provides an apparatus comprising a memory and a processor, the memory for storing instructions or code, and the processor for executing the instructions or code to cause the apparatus to perform the region-level-based real-time lightweight NMR image enhancement method described in any of the implementations of the first aspect.

[0078] Fourthly, this application provides a computer-readable storage medium storing code, wherein when the code is executed, a device running the code implements the region-level-based real-time lightweight NMR image enhancement method described in any of the implementations of the first aspect.

[0079] This application provides a real-time lightweight NMR image enhancement method based on region hierarchical classification. In executing the method, first, an NMR image with a first resolution is acquired. Automatic region segmentation is then performed on the NMR image with the first resolution, resulting in multiple regions. These regions are then classified into three regions. The images in each of the three regions are then subjected to sparse dilated convolution and input into different residual groups for further image processing. The pixel results of the images processed by the different residual groups are summed to obtain the enhanced NMR image. Through automatic region segmentation and classification, personalized processing can be precisely applied to different regions of the NMR image, improving image quality. The use of sparse dilated convolution and residual group processing effectively enhances the detailed features of the NMR image, improving image clarity and diagnostic accuracy. Attached Figure Description

[0080] Figure 1 is a flowchart of a real-time lightweight NMR image enhancement method based on region hierarchical classification provided in an embodiment of this application;

[0081] Figure 2 is a schematic diagram of an image sparse dilated convolution provided in an embodiment of this application;

[0082] Figure 3 is a schematic diagram of another image sparse dilated convolution provided in an embodiment of this application;

[0083] Figure 4 is a schematic diagram of a convolution method provided in an embodiment of this application;

[0084] Figure 5 is a schematic diagram of a real-time lightweight nuclear magnetic resonance image enhancement system based on regional hierarchical classification provided in an embodiment of this application. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0086] Figure 1 is a flowchart of a real-time lightweight NMR image enhancement method based on region hierarchical classification provided in an embodiment of this application. Referring to Figure 1, the real-time lightweight NMR image enhancement method based on region hierarchical classification provided in this embodiment of the application may include:

[0087] S101. Obtain an NMR image with a first resolution, and perform automatic region segmentation processing on the NMR image with the first resolution. The processed NMR image with the first resolution includes multiple regions.

[0088] The image with the first resolution is a high-resolution MRI image. The image source is the T1, T2 and PD type MRI images from the public dataset IXI. The dataset only contains relatively high-resolution MRI images. The high-resolution image is obtained by scaling and has a size of 240×240, which is used as the image with the first resolution.

[0089] When performing automatic region segmentation on the image with the first resolution, i.e., the high-resolution image, an Automatic Mask Generator (AMG) component is used to automatically segment the high-resolution MRI image, resulting in several brain regions. The segmentation model / automatic mask generator used in image segmentation can be a pre-trained Segment Anything Model (SAM), its derivative second-generation model SAM2, or the medical-specific Segment Anything Model MedSAM. Because the training process of the augmentation algorithm does not need to prioritize model inference speed, some relatively universal but not lightweight models can be used. Furthermore, models like SAM are plug-and-play, eliminating the need for additional training. In this application, it is not necessary to know the specific tissue identified by the segmentation; it is only necessary to know that the segmentation represents different regions and different tissues. That is, the segmentation effect is relatively coarse-grained and does not need to reach the level of specifically identifying tissue categories.

[0090] The AMG component can also use other fast segmentation models. The models do not need to be retrained; they only need to meet the requirements of real-time processing and be able to clearly distinguish the different contrasts between regions.

[0091] S102. The images of the multiple regions are classified to obtain three regions.

[0092] After the image segmentation process described above, multiple regions are obtained. If each region is processed separately, the workload will increase significantly. In this application, a score is calculated based on the priority of each region. Based on the score, the multiple regions are further classified into three regions. Based on the priority score, each region in the processed first resolution image is classified to obtain a first region, a second region, and a third region.

[0093] The process of calculating priority scores for each region uses a Region-Aware Priority Calculator (RAPC) to calculate scores for multiple segmented regions and classify them according to their priority scores. The score calculation considers various factors, both across and within regions, including the distance between regions, the complexity of the content within each region, and the size of the region.

[0094] Based on the calculation results, the k regions with the highest scores are selected as regions of interest (ROIs). These ROIs are then matched with the corresponding low-resolution NMR images to obtain the R1 region of interest in the low-resolution NMR image. Additionally, pixel filtering is used to filter the low-resolution NMR image to obtain salient information. This salient information is multiplied by the low-resolution image, and the R2 region of interest is subtracted from it to obtain the general R2 region of interest. The remaining regions of the low-resolution image are used as R3.

[0095] Specifically, the first step is to introduce statistical characteristics between regions. Based on spatial statistical analysis, this application uses a distance-based spatial weighting matrix W to measure the information correlation between regions. We define that the closer a region is to other regions, the higher its correlation, the greater the inter-regional entanglement, and the higher the degree of mutual influence. That is, enhancing a region will have a greater impact on other regions due to the shared and invariant nature of the convolution kernel. Furthermore, the closer several regions are, the higher the probability that they belong to similar organizations. This also reduces the potential consequence of the segmentation model misclassifying an organizational region into two regions, leading to poor performance in subsequent downstream tasks.

[0096] Suppose that AMG segments the image into n regions, and the i-th region S is defined as follows: i The centroid coordinates are (x i ,y i ). Then S i With the j-th region S j The Euclidean distance d between them i,j It can be defined as:

[0097] Therefore, the spatial weighting matrix W can be written as:

[0098] The priority score weights for each region based on distance can be written as:

[0099] In the formula S dis,i For the i-th region S i The score.

[0100] This application also introduces statistical features within the region. The complexity of the image content within the region is considered as one of the factors to be taken into account, because the more significant the contrast difference or gradient change within a region, the more details the content contains, requiring more attention. Furthermore, the area of ​​the region is also a factor to consider; the larger the region, the more prominent its features (such as edges, textures, etc.). These features will be captured and enhanced multiple times during the convolution operation, contributing more to the gradient of the convolution kernel parameters during backpropagation. Therefore, the i-th region S i The priority scores for content complexity and area can be written as s.var,i and s area,i As shown in the following formula:

[0101] Among them, region S i Mean and variance of pixel values Defined as:

[0102] Where p k,i It is region S i The k-th pixel of the region is thus obtained, yielding the final region-aware priority score s. i :s i =ω1·s dis,i +ω2·s var,i +ω3·s area,i (7)

[0103] Where ω1, ω2, and ω3 are adjustable weights for each score, representing score preferences.

[0104] The score for each region can be calculated by combining the statistical characteristics between regions and the statistical characteristics within regions.

[0105] Then, based on the scores, the multiple regions are further classified into three regions, resulting in Region 1, Region 2, and Region 3, which include:

[0106] The regions in a two-dimensional magnetic resonance imaging (MRI) image need to be graded based on their scores, and different network designs are applied to the selected regions. The k regions with the highest scores are selected as the regions of most interest (R1), where:

[0107] Where J represents the set of index values ​​of the top k regions with the highest scores. Since this application uses a high-resolution MRI image I... HR Perform segmentation, instead of processing low-resolution image I LR The segmentation is performed on the image itself, meaning the region of most interest, R1, corresponds to the high-resolution image, R 1H=R1. However, what actually needs enhancement is the low-resolution image. Because the resolution of low-resolution images is relatively small, the partitioning effect after directly applying the segmentation model may not be excellent, especially during training, so it is necessary to make the image partitioning as detailed as possible. That is, it is necessary to perform region matching between the region of interest and the corresponding NMR image with a second resolution to obtain the region of interest in the NMR image with a second resolution, which is taken as the first region. The NMR image with a second resolution is the low-resolution image obtained from the high-resolution image. The NMR image with a second resolution is obtained by downsampling pixel matching based on the NMR image with a first resolution. The resolution of the NMR image with a second resolution is lower than that of the NMR image with a first resolution. Assume that the resolution scaling ratio between the high-resolution image and the low-resolution image is r>1. R 1H The coordinates of the i-th pixel (x) H,i ,y H,i ) corresponding to R 1L The i / r 2 pixel coordinates (x) L,i ,y H,i This can be represented as:

[0108] At this point, region matching has been completed, and the R-values ​​of the low-resolution MRI image have been obtained. 1L , which serves as the first region R1.

[0109] To obtain the region of general interest R2, it is necessary to differentiate based on pixel values ​​(such as brightness and contrast). Therefore, some regions of general interest in the low-resolution NMR image are considered as R2. Specifically, the NMR image with the second resolution is filtered using pixel filtering to obtain salient information. The salient information is multiplied by the low-resolution image, and then the region of most interest R1 is subtracted to obtain the region of general interest, which is used as the second region. Pixel values ​​exceeding the pixel threshold T can be used to obtain the salient information map M, where M(x,y) are elements of M. Therefore, it can filter the effective regions in the image. Thus, R2 can be written as:

[0110] It is the multiplication of corresponding elements, and M1 is a matrix of all ones with the same size as M.

[0111] The remaining regions in the MRI image with the second resolution, excluding the first and second regions, are designated as the third region. The calculation process for R3 in the other regions is as follows:

[0112] S103. After performing sparse dilation convolution on the images in the three regions, input them into different residual groups for image processing.

[0113] After obtaining the images of the three regions, further processing of the first, second, and third regions is required. The image enhancement method in this application needs to meet real-time requirements, therefore, convolution cannot be performed repeatedly on image regions that do not require convolution. A sparse, block-based convolution approach is adopted to improve efficiency. When processing different partitions, some extreme cases may be encountered, where some pixels do not meet the most basic convolution requirements of the image; that is, the region may not be perfectly covered by the convolution kernel for the sliding window operation. In such cases, a zero-filling operation might be used. However, in this embodiment, the original features are dilated, or pixel values ​​corresponding to pixels that do not originally belong to the region are filled in before convolution is completed, and then pixels are removed using a mask. Therefore, this situation is called sparse dilated convolution.

[0114] This application provides two sparse dilated convolution methods, both based on a 3×3 kernel size. These are described below with reference to Figure 2, which is a schematic diagram of an image sparse dilated convolution method provided in this application embodiment.

[0115] [Detailed Rule 91, 30.10.2025] The first method is the minimum-maximum coordinate dilation principle. Figure 2 shows the original partition features. The minimum and maximum coordinates of the effective region in the partition image are obtained horizontally and vertically. Rectangular dilation is then performed on the features of this effective region, as shown by the diagonal lines in Figure 2. This is equivalent to drawing four axes around the minimum and maximum coordinates to define the desired region. Then, the outermost dilation operation is performed on this region to facilitate convolution, which is the padding step of a typical convolution operation, as shown by the dashed lines. At this point, the diagonal and dashed regions, along with the original black region, all participate in the convolution process. However, the convolved features have the same size as the black plus diagonal region. Finally, a mask can be used to filter out deep features of the same size as the black region. The final feature filtering should be performed after all convolutions are completed.

[0116] [Detailed Rules 91, 30.10.2025] The second method is to perform 8-neighborhood dilation convolution on the original region, as shown in Figure 3. Figure 3 is a schematic diagram of another image sparse dilation convolution provided by the embodiment of this application. After 8-neighborhood dilation of the features of the effective region in the partitioned image, it is as shown in the diagonal region. The convolution in this method does not need to further dilate the outer circle of the features in the diagonal region. The features after each convolution are the same size as the black region, but 8-neighborhood dilation needs to be performed before each convolution. Finally, a mask is also needed for feature selection.

[0117] Furthermore, when there are cases where all pixels within the window are present but the feature does not meet the requirements for participating in convolution and sliding window operations, this part of the convolution is skipped for subsequent convolution operations, as shown in Figure 4. Figure 4 is a schematic diagram of a convolution method provided in an embodiment of this application, where the light gray part represents the skipped pixels, and the dark gray part still participates in the convolution calculation, but not at its original position; it only participates in the convolution calculation of the black area. The feature size after the final convolution will be reduced, and the size of the subsequent convolution window is the black area plus the dark gray area. The final mask size is the size of the black area. Although the area involved in the calculation is smaller, the actual restoration effect is still excellent, and the temporal resolution of the algorithm is further improved. The above methods all handle the irregularity of the partitioned region well, while avoiding excessive addition of other unnecessary regions to participate in the convolution calculation, ensuring that the model as a whole remains lightweight.

[0118] After performing sparse dilation convolution processing on the first, second, and third regions, the images in the three regions are further processed by sparse dilation convolution and then input into different residual groups for image processing. This includes: processing the image of the first region by sparse dilation convolution and then inputting it into the first residual group for image processing; processing the images of the second and third regions by sparse dilation convolution and then inputting them into the second residual group for image processing; the complexity of the first residual group is higher than that of the second residual group.

[0119] Different levels of image regions are processed using network structures of varying scales, such as Rank Modulating Expert (RME) and Spatial Modulating Expert (SME). This application uses RME and SME networks as examples. It should be noted that the RME and SME networks involved in feature interaction and fusion can be replaced with others to meet the task requirements. A relatively complex network can be used for region R1, while a simpler network can be used for regions R2 and R3. However, all of these networks should be designed with lightweight principles.

[0120] The former contains a low-rank hybrid module to extract global texture; its model is a dynamic extraction model with a relatively complex network structure. The latter utilizes a spatial augmentation expert module to supplement the RME with spatial information; its network structure is relatively simple. In this embodiment, different networks are used for different regions to selectively focus on key areas while maintaining a lightweight overall network structure. For example, an RME+SME combination is used for region R1, while a simple SME+SME combination is sufficient for R2 and R3. Furthermore, the SME+SME combination does not use separately trained parameters but shares parameters, further reducing the number of network parameters. Thus, the shallow and deep features of the forward propagation can be represented as: F shallow,i =Shallow Conv (R i )F deep,i =RG-r i (RG-r i (…(RG-r i (R i (12)

[0121] For each layer of an MRI image region, as convolutional pooling and nonlinear activation operations are performed, feature details are further lost. It is important to enhance the detailed information of shallow and deep features, rather than simply adding or fusing them into a series. Referring to the methods of binocular feature fusion and edge / salient information fusion, this application proposes a self- and cross-attention mechanism for the R1 and R2 regions. The fusion weight is obtained by calculating channel similarity position by position, rather than based on the feature itself. This fusion method allows for the fusion of more similar information with lower weights and the fusion of information with greater differences with higher weights, thereby fusing information with significant differences between shallow and deep features and dynamically supplementing the information lost by the network. q and k represent the shallow and deep features based on the channel dimension at coordinates x and y, respectively, where the subscripts x and y are the specific coordinates of the features, and γ is the cosine similarity between the two features at coordinates x and y.

[0122] Finally, γ forms the Γ similarity matrix, and M1 is an all-1 matrix.

[0123] The fusion process is as follows:

[0124] Among them, F mix,i＝1,2 This represents the feature fusion method for regions R1 and R2. F mix,3 This represents the feature fusion method for the R3 region.

[0125] S104. The pixel results of the images processed by different residual groups are added together to obtain the enhanced NMR image.

[0126] Images from different residual groups are added at the pixel level; specifically, pixel results based on the mask region are added to obtain the enhanced super-resolution NMR image.

[0127] The real-time lightweight region-level MRI image enhancement method described in the above embodiments of this application can perform hierarchical processing on real-time acquired MRI images. It uses a region-aware priority scoring component to calculate scores for each region for hierarchical classification, rather than other network-based adaptive partitioning methods. This component allows the model to more effectively focus on restoring important regions of the MRI image. Furthermore, the method employs a block-based / sparse convolution approach combined with feature dilation, reducing the number of network parameters involved in computation during inference, resulting in faster model inference. Additionally, a self-attention mechanism is designed, enabling the model to not only focus on details within image features but also enhance interactions between features. Thus, the low-quality problem of real-time MRI images is solved while ensuring that the overall temporal resolution of the real-time MRI imaging process is not significantly reduced.

[0128] The above are some specific implementations of a region-level-based real-time lightweight NMR image enhancement method provided in this application. Based on this, this application also provides a corresponding system. The system provided in this application will be described below from the perspective of functional modularity.

[0129] Figure 5 is a schematic diagram of a region-level-based real-time lightweight NMR image enhancement system provided in an embodiment of this application. Referring to Figure 5, the region-level-based real-time lightweight NMR image enhancement system 500 provided in this embodiment includes:

[0130] The acquisition unit 510 is used to acquire an MRI image with a first resolution, and to perform automatic region segmentation processing on the MRI image with the first resolution. The processed MRI image with the first resolution includes multiple regions.

[0131] The first processing unit 520 is used to classify the images of the multiple regions to obtain three regions.

[0132] The second processing unit 530 is used to perform sparse dilation convolution processing on the images in the three regions respectively, and then input them into different residual groups for image processing respectively.

[0133] The third processing unit 540 is used to add the pixel results of the images processed by different residual groups to obtain the enhanced NMR image.

[0134] In one implementation of this application embodiment, the first processing unit is specifically used for:

[0135] Calculate the priority score of each region in the processed first resolution image;

[0136] Based on the priority score, each region in the processed first resolution image is classified to obtain a first region, a second region, and a third region.

[0137] In one implementation of this application, the step of classifying each region in the processed first resolution image according to the priority score to obtain a first region, a second region, and a third region includes:

[0138] Based on the priority scores, the k regions with the highest scores are selected as the regions of most interest.

[0139] The region of most interest is matched with the corresponding NMR image with a second resolution to obtain the region of most interest in the NMR image with a second resolution, which is taken as the first region; the NMR image with a second resolution is obtained by downsampling pixel matching based on the NMR image with a first resolution, and the resolution of the NMR image with a second resolution is lower than the resolution of the NMR image with a first resolution;

[0140] The MRI image with the second resolution is filtered using pixel filtering to obtain salient information. Based on the salient information and the MRI image with the second resolution, a general region of interest is obtained, which is used as the second region.

[0141] The remaining regions in the MRI image with the second resolution, excluding the first and second regions, are designated as the third region.

[0142] In one implementation of this application, the second processing unit performs sparse dilated convolution processing on the images in the three regions respectively, including:

[0143] Obtain the minimum and maximum horizontal and vertical coordinates of the effective region in the partitioned image, perform rectangular expansion on the features of the effective region, and obtain the region enclosed by the four axes of the minimum and maximum horizontal and vertical coordinates as the expanded region.

[0144] Perform an outermost expansion operation on the expanded region;

[0145] The image features, including the outermost ring, are convolved to obtain the convolved features.

[0146] A mask is applied to filter the convolutional features to obtain deep features of the same size as the original region;

[0147] Alternatively, an 8-neighbor dilation convolution can be performed on the features of the effective region in the partitioned image to obtain the dilated region; before each convolution, an 8-neighbor dilation is performed on the features of the effective region in the partitioned image, and the convolved features are the same size as the original region.

[0148] A mask is applied to filter the convolutional features to obtain deep features of the same size as the original region.

[0149] In one implementation of this application, the second processing unit performs sparse dilation convolution on the images in the three regions and then inputs them into different residual groups for image processing, including:

[0150] The image of the first region is processed by sparse dilation convolution and then input into the first residual group for image processing.

[0151] After performing sparse dilated convolution on the images of the second region and the third region, the images are input into the second residual group for image processing.

[0152] The complexity of the first residual set is higher than that of the second residual set.

[0153] This application also provides corresponding devices and computer storage media for implementing the solutions provided in this application.

[0154] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to cause the device to perform the method described in any embodiment of this application.

[0155] The computer storage medium stores code, and when the code is run, the device running the code implements the method described in any embodiment of this application.

[0156] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0157] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0158] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0159] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A real-time lightweight magnetic resonance image enhancement method based on regional grading, characterized in that, The method includes: An MRI image with a first resolution is acquired, and the MRI image with the first resolution is subjected to automatic region segmentation processing. The processed MRI image with the first resolution includes multiple regions. The images of the multiple regions are classified to obtain three regions; The images in the three regions are subjected to sparse dilation convolution processing and then input into different residual groups for image processing. The pixel results of the images processed by different residual groups are added together to obtain the enhanced NMR image.

2. The method of claim 1, wherein, The image classification process of the multiple regions yields three regions, including: Calculate the priority score of each region in the processed first resolution image; Based on the priority score, each region in the processed first resolution image is classified to obtain a first region, a second region, and a third region.

3. The method of claim 2, wherein, The step of classifying each region in the processed first resolution image according to the priority score to obtain a first region, a second region, and a third region includes: Based on the priority scores, the k regions with the highest scores are selected as the regions of most interest. The region of most interest is matched with the corresponding NMR image with a second resolution to obtain the region of most interest in the NMR image with a second resolution, which is taken as the first region; the NMR image with a second resolution is obtained by downsampling pixel matching based on the NMR image with a first resolution, and the resolution of the NMR image with a second resolution is lower than the resolution of the NMR image with a first resolution; The MRI image with the second resolution is filtered using pixel filtering to obtain salient information. Based on the salient information and the MRI image with the second resolution, a general region of interest is obtained, which is used as the second region. The remaining regions in the MRI image with the second resolution, excluding the first and second regions, are designated as the third region.

4. The method of claim 1, wherein, The step of performing sparse dilation convolution processing on the images in the three regions includes: Obtain the minimum and maximum horizontal and vertical coordinates of the effective region in the partitioned image, perform rectangular expansion on the features of the effective region, and obtain the region enclosed by the four axes of the minimum and maximum horizontal and vertical coordinates as the expanded region. Perform an outermost expansion operation on the expanded region; The image features, including the outermost ring, are convolved to obtain the convolved features. A mask is applied to filter the convolutional features to obtain deep features of the same size as the original region; Alternatively, an 8-neighbor dilation convolution can be performed on the features of the effective region in the partitioned image to obtain the dilated region; before each convolution, an 8-neighbor dilation is performed on the features of the effective region in the partitioned image, and the convolved features are the same size as the original region. A mask is applied to filter the convolutional features to obtain deep features of the same size as the original region.

5. The method of claim 3, wherein, The step of performing sparse dilation convolution on the images in the three regions and then inputting them into different residual groups for image processing includes: The image of the first region is processed by sparse dilation convolution and then input into the first residual group for image processing. After performing sparse dilation convolution on the images of the second region and the third region, the images are input into the second residual group for image processing. The complexity of the first residual set is higher than that of the second residual set.

6. The method of claim 1, wherein, The automatic region segmentation process for the MRI image with the first resolution specifically includes: The MRI image with a first resolution is automatically segmented using an automatic mask generator component, resulting in multiple regions.

7. The method of claim 6, wherein, The automatic mask generator component is a pre-trained Segment All Model (SAM), a second-generation Segment All Model derivative (SAM2), or a medical-specific Segment All Model (MedSAM).

8. The method of claim 2, wherein, The calculation of priority scores for each region in the processed first-resolution image includes: The Region Aware Priority Score Calculation (RAPC) component is used to calculate scores for each region in the first resolution image, and the regions are then classified based on the calculated scores.

9. The method of claim 4, wherein, The method further includes: During convolution and sliding windowing, it is determined whether the pixels within the window meet the preset feature conditions; If the preset feature conditions are not met, the convolution calculation of that window is skipped, and subsequent convolution operations continue. For pixels that meet the preset feature conditions, they are divided into two regions. Pixels in one region participate in the convolution calculation and produce the convolution result, while pixels in the other region only participate in the convolution calculation of the previous region.

10. The method of claim 5, wherein, When the images in the three regions are processed by sparse dilation convolution and then input into different residual groups for image processing, different levels of network structures are used to process image regions of different levels; the different levels of network structures include hierarchical modulation expert network RME and spatial modulation expert network SME. The step of performing sparse dilation convolution on the image of the first region and then inputting it into the first residual group for image processing includes: The image of the first region is processed by sparse dilation convolution and then input into the first residual group composed of RME and SME for image processing. The image processing of the second region and the third region images after sparse dilation convolution processing and input into the second residual group includes: the second region and the third region images after sparse dilation convolution processing and input into the second residual group composed of SME and SME for image processing, wherein the second residual group shares parameters.

11. A real-time lightweight MRI image enhancement system based on region hierarchical, characterized in that, The system includes: An acquisition unit is used to acquire an MRI image with a first resolution, and to perform automatic region segmentation processing on the MRI image with the first resolution, wherein the processed MRI image with the first resolution includes multiple regions. The first processing unit is used to classify the images of the multiple regions to obtain three regions; The second processing unit is used to perform sparse dilation convolution on the images in the three regions respectively, and then input them into different residual groups for image processing respectively. The third processing unit is used to add the pixel results of the images processed by different residual groups to obtain the enhanced NMR image.

12. The system of claim 11, wherein, The first processing unit is specifically used for: Calculate the priority score of each region in the processed first resolution image; Based on the priority score, each region in the processed first resolution image is classified to obtain a first region, a second region, and a third region.

13. The system of claim 12, wherein, The step of classifying each region in the processed first resolution image according to the priority score to obtain a first region, a second region, and a third region includes: Based on the priority scores, the k regions with the highest scores are selected as the regions of most interest. The region of most interest is matched with the corresponding NMR image with a second resolution to obtain the region of most interest in the NMR image with a second resolution, which is taken as the first region; the NMR image with a second resolution is obtained by downsampling pixel matching based on the NMR image with a first resolution, and the resolution of the NMR image with a second resolution is lower than the resolution of the NMR image with a first resolution; The MRI image with the second resolution is filtered using pixel filtering to obtain salient information. Based on the salient information and the MRI image with the second resolution, a general region of interest is obtained, which is used as the second region. The remaining regions in the MRI image with the second resolution, excluding the first and second regions, are designated as the third region.

14. The system of claim 11, wherein, The second processing unit performs sparse dilated convolution processing on the images in the three regions respectively, including: Obtain the minimum and maximum horizontal and vertical coordinates of the effective region in the partitioned image, perform rectangular expansion on the features of the effective region, and obtain the region enclosed by the four axes of the minimum and maximum horizontal and vertical coordinates as the expanded region. Perform an outermost expansion operation on the expanded region; The image features, including the outermost ring, are convolved to obtain the convolved features. A mask is applied to filter the convolutional features to obtain deep features of the same size as the original region; Alternatively, an 8-neighbor dilation convolution can be performed on the features of the effective region in the partitioned image to obtain the dilated region; before each convolution, an 8-neighbor dilation is performed on the features of the effective region in the partitioned image, and the convolved features are the same size as the original region. A mask is applied to filter the convolutional features to obtain deep features of the same size as the original region.

15. The system of claim 13, wherein, The second processing unit performs sparse dilation convolution on the images in the three regions respectively, and then inputs them into different residual groups for image processing, including: The image of the first region is processed by sparse dilation convolution and then input into the first residual group for image processing. After performing sparse dilation convolution on the images of the second region and the third region, the images are input into the second residual group for image processing. The complexity of the first residual set is higher than that of the second residual set.

16. The system of claim 11, wherein, The acquisition unit performs automatic region segmentation processing on the MRI image with the first resolution, specifically including: The MRI image with a first resolution is automatically segmented using an automatic mask generator component, resulting in multiple regions.

17. The system of claim 16, wherein, The automatic mask generator component is a pre-trained Segment All Model (SAM), a second-generation Segment All Model derivative (SAM2), or a medical-specific Segment All Model (MedSAM).

18. The system of claim 12, wherein, The first processing unit calculates the priority score for each region in the processed first resolution image, including: The Region Aware Priority Score Calculation (RAPC) component is used to calculate scores for each region in the first resolution image, and the regions are then classified based on the calculated scores.

19. The system of claim 14, wherein, The second processing unit is further configured to: During convolution and sliding windowing, it is determined whether the pixels within the window meet the preset feature conditions; If the preset feature conditions are not met, the convolution calculation of that window is skipped, and subsequent convolution operations continue. For pixels that meet the preset feature conditions, they are divided into two regions. Pixels in one region participate in the convolution calculation and produce the convolution result, while pixels in the other region only participate in the convolution calculation of the previous region.

20. The system of claim 15, wherein, When the images in the three regions are processed by sparse dilation convolution and then input into different residual groups for image processing, different levels of network structures are used to process image regions of different levels; the different levels of network structures include hierarchical modulation expert network RME and spatial modulation expert network SME. The step of performing sparse dilation convolution on the image of the first region and then inputting it into the first residual group for image processing includes: The image of the first region is processed by sparse dilation convolution and then input into the first residual group composed of RME and SME for image processing. The image processing of the second region and the third region images after sparse dilation convolution processing and input into the second residual group includes: the second region and the third region images after sparse dilation convolution processing and input into the second residual group composed of SME and SME for image processing, wherein the second residual group shares parameters.

21. A computing device, comprising: The computing device includes: a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method as described in any one of claims 1 to 10 when executing the computer program.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 10.

Images