Real-time lightweight nuclear magnetic image enhancement method and system based on region grading

Abstract

The invention provides a real-time lightweight nuclear magnetic image enhancement method and system based on region grading, and relates to the technical field of image enhancement process.The method comprises the steps that firstly, a nuclear magnetic image with a first resolution is obtained, and automatic region segmentation processing is conducted on the nuclear magnetic image with the first resolution; the processed nuclear magnetic image with the first resolution comprises a plurality of regions, classifying the images of the plurality of regions to obtain three regions, performing sparse expansion convolution processing on the images in the three regions, and inputting the images into different residual groups to perform image processing; and adding pixel results of the images processed by the different residual groups to obtain an enhanced nuclear magnetic image. Through automatic region segmentation and classification processing, personalized processing can be accurately carried out on different regions in the nuclear magnetic image, and the image quality is improved. Sparse expansion convolution and residual group processing are utilized, detail features of the nuclear magnetic image are effectively enhanced, and the definition and diagnosis accuracy of the image are improved.

Description

Technical Field

[0001] The present application relates to the technical field of image enhancement processing, and in particular to a real-time lightweight nuclear magnetic resonance image enhancement method and system based on regional classification. Background Art

[0002] Magnetic resonance imaging (MRI) has the characteristics of high contrast, high resolution and no radiation, and is the best choice for tissue imaging. However, compared with other medical imaging methods, its acquisition speed is relatively slow, making it difficult to be selected as an image servo for surgical robots. For a brain puncture surgical robot based on nuclear magnetic resonance, real-time nuclear magnetic resonance images are required to monitor the interventional navigation process of the needle and give corresponding control feedback based on the image. Therefore, time resolution is a prerequisite for the above-mentioned robotic interventional surgery. In order to ensure this prerequisite, the nuclear magnetic resonance sequence acquisition will be designed accordingly. However, the image enhancement algorithm in the prior art is not only time-consuming, but also the effect of image enhancement processing is not ideal. Summary of the invention

[0003] In view of the above problems, the present application provides a real-time lightweight nuclear magnetic resonance image enhancement method and system based on regional classification, including the following contents:

[0004] In a first aspect, the present application provides a real-time lightweight nuclear magnetic resonance image enhancement method based on regional classification, the method comprising:

[0005] Acquiring a nuclear magnetic resonance image with a first resolution, and performing automatic region segmentation processing on the nuclear magnetic resonance image with the first resolution, wherein the processed nuclear magnetic resonance image with the first resolution includes a plurality of regions;

[0006] Classifying the images of the multiple regions to obtain three regions;

[0007] The images in the three regions are respectively subjected to sparse dilated convolution processing and then input into different residual groups for image processing respectively;

[0008] The pixel results of the images processed by different residual groups are added together to obtain the enhanced nuclear magnetic resonance image.

[0009] Optionally, the classifying the images of the multiple regions to obtain three regions includes:

[0010] Calculating a priority score for each region in the processed image with the first resolution;

[0011] The regions in the processed image with the first resolution are classified according to the priority scores to obtain a first region, a second region and a third region.

[0012] Optionally, classifying the regions in the processed first-resolution image according to the priority scores to obtain the first region, the second region and the third region includes:

[0013] According to the priority scores, k regions with the highest scores are selected as the regions of greatest interest;

[0014] Performing region matching on the region of greatest interest and the corresponding nuclear magnetic image with a second resolution, obtaining the region of greatest interest in the nuclear magnetic image with the second resolution as the first region; the nuclear magnetic image with the second resolution is obtained by performing down-sampling pixel matching based on the nuclear magnetic image with the first resolution, and the resolution of the nuclear magnetic image with the second resolution is lower than the resolution of the nuclear magnetic image with the first resolution;

[0015] Performing filtering processing on the nuclear magnetic resonance image with the second resolution by using pixel filtering to obtain significant information, and obtaining a general region of interest as the second region according to the significant information and the nuclear magnetic resonance image with the second resolution;

[0016] The remaining area of ​​the nuclear magnetic resonance image with the second resolution except the first area and the second area is used as the third area.

[0017] Optionally, performing sparse dilated convolution processing on the images in the three regions respectively includes:

[0018] Acquire the minimum and maximum horizontal and vertical coordinates of the effective area in the partition image, perform rectangular expansion on the features of the effective area, and obtain the area surrounded by the four axes of the minimum and maximum horizontal and vertical coordinates as the expanded area;

[0019] Performing an outermost circle expansion operation on the expanded area;

[0020] Convolving the image features including the outermost circle to obtain convolved features;

[0021] Apply the mask to filter the convolutional features to obtain deep features of the same size as the original area;

[0022] Or, the features of the effective area in the partition image are subjected to 8-neighborhood dilation convolution to obtain the dilated area; before each convolution, the features of the effective area in the partition image are subjected to 8-neighborhood dilation once, and the features after convolution are consistent with the size of the original area;

[0023] Apply the mask to filter the convolved features to obtain deep features of the same size as the original region.

[0024] Optionally, performing sparse dilated convolution processing on the images in the three regions respectively and then inputting the images into different residual groups for performing image processing respectively includes:

[0025] Performing sparse dilated convolution processing on the image of the first region and then inputting the processed image into the first residual group for image processing;

[0026] Performing sparse dilation convolution processing on the second region and the third region images and then inputting the processed images into a second residual group for image processing;

[0027] The complexity of the first residual group is higher than the complexity of the second residual group.

[0028] In a second aspect, the present application provides a real-time lightweight nuclear magnetic resonance image enhancement system based on regional classification, the system comprising:

[0029] an acquisition unit, configured to acquire a nuclear magnetic resonance image with a first resolution, and perform automatic region segmentation processing on the nuclear magnetic resonance image with the first resolution, wherein the processed nuclear magnetic resonance image with the first resolution includes a plurality of regions;

[0030] A first processing unit, configured to classify the images of the multiple regions to obtain three regions;

[0031] A second processing unit is used to perform sparse dilation convolution processing on the images in the three regions respectively and then input the images into different residual groups for image processing respectively;

[0032] The third processing unit is used to add pixel results of the images processed by different residual groups to obtain an enhanced nuclear magnetic resonance image.

[0033] Optionally, the first processing unit is specifically configured to:

[0034] Calculating a priority score for each region in the processed image with the first resolution;

[0035] The regions in the processed image with the first resolution are classified according to the priority scores to obtain a first region, a second region and a third region.

[0036] Optionally, classifying the regions in the processed first-resolution image according to the priority scores to obtain the first region, the second region and the third region includes:

[0037] According to the priority scores, k regions with the highest scores are selected as the regions of greatest interest;

[0038] Performing region matching on the region of greatest interest and the corresponding nuclear magnetic image with a second resolution, obtaining the region of greatest interest in the nuclear magnetic image with the second resolution as the first region; the nuclear magnetic image with the second resolution is obtained by performing down-sampling pixel matching based on the nuclear magnetic image with the first resolution, and the resolution of the nuclear magnetic image with the second resolution is lower than the resolution of the nuclear magnetic image with the first resolution;

[0039] Performing filtering processing on the nuclear magnetic resonance image with the second resolution by using pixel filtering to obtain significant information, and obtaining a general region of interest as the second region according to the significant information and the nuclear magnetic resonance image with the second resolution;

[0040] The remaining area of ​​the nuclear magnetic resonance image with the second resolution except the first area and the second area is used as the third area.

[0041] Optionally, the second processing unit performs sparse dilated convolution processing on the images in the three regions respectively, including:

[0042] Acquire the minimum and maximum horizontal and vertical coordinates of the effective area in the partition image, perform rectangular expansion on the features of the effective area, and obtain the area surrounded by the four axes of the minimum and maximum horizontal and vertical coordinates as the expanded area;

[0043] Performing an outermost circle expansion operation on the expanded area;

[0044] Convolving the image features including the outermost circle to obtain convolved features;

[0045] Apply the mask to filter the convolutional features to obtain deep features of the same size as the original area;

[0046] Or, the features of the effective area in the partition image are subjected to 8-neighborhood dilation convolution to obtain the dilated area; before each convolution, the features of the effective area in the partition image are subjected to 8-neighborhood dilation once, and the features after convolution are consistent with the size of the original area;

[0047] Apply the mask to filter the convolved features to obtain deep features of the same size as the original region.

[0048] Optionally, the second processing unit performs sparse dilated convolution processing on the images in the three regions respectively and then inputs the images into different residual groups for performing image processing respectively, including:

[0049] Performing sparse dilated convolution processing on the image of the first region and then inputting the processed image into the first residual group for image processing;

[0050] Performing sparse dilation convolution processing on the second region and the third region images and then inputting the processed images into a second residual group for image processing;

[0051] The complexity of the first residual group is higher than the complexity of the second residual group.

[0052] In a third aspect, the present application provides a device comprising a memory and a processor, wherein the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the real-time lightweight nuclear magnetic image enhancement method based on regional grading introduced in any implementation method of the first aspect.

[0053] In a fourth aspect, the present application provides a computer-readable storage medium, in which a code is stored. When the code is executed, the device executing the code implements the real-time lightweight nuclear magnetic resonance image enhancement method based on regional grading introduced in any implementation method of the first aspect.

[0054] The present application provides a real-time lightweight nuclear magnetic image enhancement method based on regional classification. When executing the method, the method first obtains a nuclear magnetic image with a first resolution, performs automatic regional segmentation processing on the nuclear magnetic image with the first resolution, and the processed nuclear magnetic image with the first resolution includes multiple regions, and then the images of the multiple regions are classified to obtain three regions, and the images in the three regions are respectively subjected to sparse expansion convolution processing and input into different residual groups for image processing, and the images processed by different residual groups are added to obtain enhanced nuclear magnetic images. Through automatic regional segmentation and classification processing, personalized processing can be accurately performed on different regions in the nuclear magnetic image to improve image quality. Using sparse expansion convolution and residual group processing, the detail features of the nuclear magnetic image are effectively enhanced, and the clarity and diagnostic accuracy of the image are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A flowchart of a real-time lightweight nuclear magnetic resonance image enhancement method based on regional classification provided in an embodiment of the present application;

[0057] Figure 2 A schematic diagram of an image sparse dilated convolution provided in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of another image sparse dilated convolution provided in an embodiment of the present application;

[0059] Figure 4 A schematic diagram of a convolution method provided in an embodiment of the present application;

[0060] Figure 5 A structural schematic diagram of a real-time lightweight nuclear magnetic resonance image enhancement system based on regional classification provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] Figure 1 A flowchart of a real-time lightweight nuclear magnetic resonance image enhancement method based on regional classification provided in an embodiment of the present application. Figure 1 As shown, the real-time lightweight nuclear magnetic resonance image enhancement method based on regional classification provided in the embodiment of the present application may include:

[0063] S101, acquiring a nuclear magnetic resonance image with a first resolution, and performing automatic region segmentation processing on the nuclear magnetic resonance image with the first resolution, wherein the processed nuclear magnetic resonance image with the first resolution includes a plurality of regions.

[0064] The image with the first resolution is a high-resolution nuclear magnetic resonance image, and the image source is the T1, T2 and PD types of nuclear magnetic resonance images of the public data set IXI. The data set only contains relatively high-resolution nuclear magnetic resonance images. The size of the high-resolution image obtained by scaling is 240×240, which is used as the image with the first resolution.

[0065] When the automatic regional segmentation process is performed on the image with the first resolution, i.e., the high-resolution image, an automatic mask generator (Automatic Mask Generator, AMG) component is used to automatically segment the high-resolution nuclear magnetic image, and several brain regions are obtained after segmentation. The segmentation model / automatic mask generator used in the image segmentation process can select the pre-trained segmentation model (Segment Anything Model, SAM), or its derivative second-generation model SAM2 or the segmentation model MedSAM for medicine. Because the enhanced algorithm training process does not have to pursue the reasoning speed of the model, some relatively universal and not lightweight models can be used. In addition, models such as SAM are plug-and-play, eliminating the need for additional training processes. In this application, it is not necessary to know what the specific tissue segmented is, it is only necessary to know that the segmentation represents different tissues in different regions, that is, the effect of the segmentation is relatively coarse-grained and does not need to reach the level of specifically identifying the tissue category.

[0066] The AMG component can also use other fast segmentation models. The model does not need to be retrained. It only needs to meet the requirements of real-time processing and can clearly distinguish different contrasts between regions.

[0067] S102: Classify the images of the multiple regions to obtain three regions.

[0068] After the image segmentation processing in the above steps, multiple regions are obtained. If each region is processed separately, the workload will be greatly increased. In the present application, the scores are calculated by the priority of each region, and the multiple regions are further classified into three regions according to the score results. The regions in the processed first-resolution image are classified according to the priority scores to obtain the first region, the second region and the third region.

[0069] The process of calculating the priority score for each region uses the Region-Aware Priority Calculator (RAPC) to score the multiple segmented regions and classify them according to the priority score. The score calculation takes into account multiple cross-region and intra-region factors such as the distance between regions, the complexity of the content within the region, and the size of the region.

[0070] Based on the calculation results, the k regions with the highest scores are selected as the regions of most interest, and the regions are matched with the corresponding low-resolution nuclear magnetic images to obtain the most interesting region R1 of the low-resolution nuclear magnetic image. In addition, the low-resolution nuclear magnetic image is filtered by pixel filtering to obtain significant information, and the significant information is multiplied by the low-resolution image and then the most interesting region is subtracted to obtain the general region of interest R2, and the rest of the low-resolution image is used as R3.

[0071] Specifically, the statistical characteristics between regions are first introduced. Based on spatial statistical analysis, the distance-based spatial weighted matrix W is used in this application to measure the information association between regions. The closer a region is to other regions, the higher the degree of association, the greater the bond between regions, and the higher the degree of mutual influence, that is, the greater the impact of the super-resolution result on other regions when the region is enhanced, because the convolution kernel is essentially common and unchanged. And the closer the regions are, the higher the possibility that they belong to similar tissues. This also reduces the possible consequences of the poor effect of subsequent downstream tasks caused by the segmentation model misdividing a tissue region into two regions.

[0072] Assume that n image regions are segmented by AMG, and define the i-th region S i The coordinates of the center of mass are (x i ,y i ). Then S i With the jth region S j The Euclidean distance d between i,j It can be defined as:

[0073]

[0074] Therefore, the spatial weighting matrix W can be written as:

[0075]

[0076] The priority score weight of each area with respect to distance can be written as:

[0077]

[0078] Where s dis,i is the i-th region S i score.

[0079] This application also introduces statistical features within the region. The complexity of the image content in the region is taken as one of the factors to be considered, because the more obvious the contrast difference or gradient change in a region is, the more details the content contains, and more attention needs to be paid. In addition, the area of ​​the region is also one of the factors that need to be paid attention to. The larger the region, the more prominent its features (such as edges, textures, etc.). These features will be captured and enhanced multiple times during the convolution operation, and contribute more to the gradient of the convolution kernel parameters during the back propagation process. Therefore, the i-th region S i The priority scores of content complexity and area can be written as s var,i and area,i , as follows:

[0080]

[0081] Among them, area S i The mean and variance of the pixel values ​​in Defined as:

[0082]

[0083] where p k,i It is area S i The kth pixel in the area, so far the final area perception priority score s is obtained i :

[0084] s i =ω1·s dis,i +ω2·s var,i +ω3·s area,i (7)

[0085] Where ω1, ω2, ω3 are the weighted score preferences for each score that can be adjusted.

[0086] Based on the above statistical features between regions and statistical features within a region, the score of each region can be obtained.

[0087] Then, the multiple regions are further classified into three regions according to the scoring results, i.e., the scores, and the first region, the second region, and the third region include:

[0088] It is necessary to grade the regions of two-dimensional magnetic resonance imaging (MRI) according to the scores, and use different network designs for the screened regions. The k regions with the highest scores are selected as the most interesting regions R1, where:

[0089] R1={S i |i∈J},

[0090] |J|=k,

[0091]

[0092] Where J represents the index value set of the top k regions with the highest scores. HR Segmentation is performed instead of the low-resolution image I LR The region of most interest, R1, corresponds to the high-resolution image. 1H =R1. However, the content that actually needs to be enhanced is the low-resolution image. Since the resolution of the low-resolution image is relatively small, the partitioning effect may not be excellent after directly applying the segmentation model, especially in the training process, the image partitioning should be as detailed as possible. That is, it is necessary to perform regional matching between the region of greatest interest and the corresponding nuclear magnetic image with the second resolution, and obtain the region of greatest interest in the nuclear magnetic image with the second resolution as the first region, wherein the nuclear magnetic image with the second resolution is the low-resolution image obtained through the high-resolution image, and the nuclear magnetic image with the second resolution is obtained based on the nuclear magnetic image with the first resolution by downsampling pixel matching, and the resolution of the nuclear magnetic image with the second resolution is lower than that of the nuclear magnetic image with the first resolution, assuming that the resolution scaling ratio between the high-resolution image and the low-resolution image is r>1. R 1H The i-th pixel coordinate (x H,i ,y H,i ) corresponds to mapping to R 1L The i / r 2 Pixel coordinates (x L,i ,y H,i ) can be expressed as:

[0093]

[0094] So far, the regional matching has been completed and the R 1L , as the first region R1.

[0095] If you need to get the general area of ​​interest R2, you need to distinguish it according to the pixel value (such as brightness and contrast). Therefore, some areas of general interest in the low-resolution nuclear magnetic resonance image are regarded as R2. Specifically, the nuclear magnetic resonance image with the second resolution is filtered using pixel filtering to obtain significant information. The significant information is multiplied by the low-resolution image and then the most interesting area R1 is subtracted to obtain the general area of ​​interest as the second area. Pixel values ​​exceeding the pixel threshold T can be used to obtain the saliency information map M, where M(x,y) is an element of M. Therefore, it can filter the valid area in the image. Therefore, R2 can be written as:

[0096]

[0097] is the multiplication of corresponding elements, and M1 is a matrix of all 1s of the same size as M.

[0098] The remaining regions except the first region and the second region in the nuclear magnetic resonance image with the second resolution are taken as the third region, and the calculation process of the other regions R3 is as follows:

[0099]

[0100] S103, performing sparse dilated convolution processing on the images in the three regions respectively, and then inputting the images into different residual groups for image processing respectively.

[0101] After obtaining the images of the three regions, the images of the first region, the second region, and the third region need to be further processed. The image enhancement method in this application needs to meet the real-time requirements, so the image area that does not need convolution cannot be repeatedly convolved. A sparse, block convolution method is adopted to improve efficiency. When processing different partitions, some extreme cases may be encountered, that is, some pixels do not meet the most basic convolution requirements of the image, that is, the area may not allow the convolution kernel to perfectly cover the sliding window operation. Generally, in this case, a 0 operation may be filled, but in the embodiment of the present application, the original features are selected to be expanded, or the pixel values ​​corresponding to the pixels that do not originally belong to the area are filled in, and the pixels are removed by using a mask after the convolution is completed. Therefore, this situation is called sparse dilated convolution.

[0102] In the present application, two sparse dilated convolution methods are provided, both of which are demonstrated based on a convolution kernel size of 3×3. Figure 2 Introduce them respectively. Figure 2 A schematic diagram of an image sparse dilated convolution provided in an embodiment of the present application.

[0103] The first is the minimum and maximum coordinate expansion principle. Figure 2 The original partition features are given in , and the minimum and maximum horizontal and vertical coordinates of the effective area in the partition image are obtained, and the features of the effective area are rectangularly expanded, such as Figure 2 As shown in the red part, it is equivalent to drawing the area enclosed by four axes for the minimum and maximum coordinates. Then the outermost circle of the area is expanded to facilitate convolution, which is the padding step of the general convolution operation, as shown in orange. So far, the above color area plus the original black area are all involved in the convolution process. However, the features after convolution are the same size as the black plus red area. Finally, the mask can be used to filter out deep features of the same size as the black area. The mask should be applied for final feature screening after all convolutions are completed.

[0104] The second method is to perform 8-neighborhood dilation convolution on the original area, such as Figure 3 As shown, Figure 3 A schematic diagram of another image sparse dilated convolution provided in an embodiment of the present application, in which the features of the valid area in the partitioned image are expanded by 8 neighborhoods as shown in red. This type of convolution does not need to further expand the outer circle of the red features. The features after each convolution are the same size as the black area, but it is necessary to perform an 8-neighborhood expansion before each convolution, and finally a mask is required for feature screening.

[0105] In addition, when there are unsatisfactory features participating in the convolution and sliding window process, and all the pixels in the window exist, we choose to skip this part of the convolution and perform subsequent convolution operations, such as Figure 4 The skipping principle is not satisfied. Figure 4 A schematic diagram of a convolution method provided for an embodiment of the present application, wherein the light gray part is a skipped pixel, and the dark gray part still participates in the convolution calculation, but is not the convolution result of its position, but only participates in the convolution calculation of the black area. The feature size after the final convolution will be reduced, and the subsequent convolution window size is the black area plus the dark gray area. The final mask size is the size of the black area. In this way, although the area involved in the calculation becomes smaller, the actual restoration effect is still excellent and the algorithm's temporal resolution is further improved. The above methods all deal with the irregularity of the partitioned area well, while avoiding excessive increase of other unnecessary areas to participate in the convolution calculation, ensuring that the overall model is in a lightweight state.

[0106] After completing the sparse dilated convolution processing on the first area, the second area, and the third area, further performing sparse dilated convolution processing on the images in the three areas and then inputting them into different residual groups for image processing respectively, including: performing sparse dilated convolution processing on the image of the first area and then inputting them into the first residual group for image processing; performing sparse dilated convolution processing on the images of the second area and the third area and then inputting them into the second residual group for image processing; the complexity of the first residual group is higher than the complexity of the second residual group.

[0107] Image regions of different levels are processed by using network structures of different magnitudes, for example, by using a Rank Modulating Expert (RME) network and a Spatial Modulating Expert (SME) network. In the embodiments of the present application, the use of RME and SME networks is used as an example for introduction. It should be noted that the RME and SME networks involved in feature interaction and fusion can be replaced by other methods to meet the task requirements, such as using a relatively complex network for the R1 region and a relatively simple network for the R2 and R3 regions. However, the above networks should all be based on lightweight design.

[0108] The former contains a low-rank hybrid module to extract global texture, and its model is a dynamic extraction model with a relatively complex network structure. The latter uses a spatial enhancement expert module to supplement the spatial information of RME, and its network structure is relatively simple. In the embodiment of the present application, different networks are used for different areas to focus on key areas while ensuring the lightweight structure of the overall network. For example, the combination of RME+SME is used for area R1, and the network combination of SME+SME is simply used for R2 and R3, and the combination of SME+SME is not a parameter trained separately, but a shared parameter, which can further reduce the network parameters. At this point, the shallow and deep features of the forward propagation can be expressed as:

[0109] F shallow,i =Shallow Conv (R i )

[0110] F deep,i =RG-r i (RG-r i (…(RG-r i (R i )))) (12)

[0111] For each layer of the MRI image area, as the convolution pooling and nonlinear activation operations are gradually carried out, the feature details are further lost. It is important to enhance the detailed information of shallow features and deep features, rather than simply adding or fusing them into a series. Referring to the binocular fusion feature and edge and significant information fusion method, the embodiment of the present application proposes a fusion method of self- and mutual cross-attention mechanism for the R1 and R2 regions. The fusion weight is obtained based on the channel similarity calculation position by position rather than based on the feature itself. This fusion method can make the closer information low-weight fusion, and make the information with a larger gap high-weight, so as to fuse the information with a large gap between deep and shallow features, and dynamically supplement the information lost by the network. q, k represent the shallow and deep features based on the channel dimension at a certain coordinate x, y, respectively. The subscripts x, y are the specific coordinates of the features, and γ is the cosine similarity of the two features at the coordinates x, y.

[0112]

[0113] Finally, γ forms the Γ similarity matrix, and M1 is an all-1 matrix.

[0114] The fusion process is:

[0115]

[0116] F mix,3 =F shallow,3 +F deep,3(14)

[0117] Among them, F mix,i＝1,2 Represents the feature fusion method of R1 and R2 regions. mix,3 Represents the feature fusion method of R3 area.

[0118] S104, adding pixel results of the images processed by different residual groups to obtain an enhanced nuclear magnetic resonance image.

[0119] The images from different residual groups are added at the pixel level. Specifically, the enhanced super-resolution nuclear magnetic resonance image is obtained by adding the pixel results based on the mask area.

[0120] Through the real-time lightweight regional hierarchical nuclear magnetic image enhancement method in the above-mentioned embodiment of the present application, the nuclear magnetic images acquired in real time can be graded and processed separately, and the corresponding scores of the regions are calculated for grading through the regional perception priority scoring component, rather than other network-based adaptive screening partitions. The component enables the model to more effectively restore the important areas of the nuclear magnetic image. In addition, the method adopts a block / sparse convolution-based approach combined with feature expansion, which can allow fewer network parameters to participate in the calculation during the reasoning process, making the model reasoning faster. In addition, by designing a self-attention mechanism, the model not only pays attention to the details inside the image features, but also enhances the interaction between features. In this way, the low quality problem of real-time nuclear magnetic images is solved, while ensuring that the temporal resolution of the overall process of real-time nuclear magnetic imaging is not significantly reduced.

[0121] The above are some specific implementations of a real-time lightweight nuclear magnetic image enhancement method based on regional classification provided in the embodiment of the present application. Based on this, the present application also provides a corresponding system. The system provided in the embodiment of the present application will be introduced from the perspective of functional modularization.

[0122] Figure 5 A schematic diagram of the structure of a real-time lightweight nuclear magnetic resonance image enhancement system based on regional classification provided in an embodiment of the present application. Figure 5 As shown, the real-time lightweight nuclear magnetic resonance image enhancement system 500 based on regional classification provided in the embodiment of the present application includes:

[0123] An acquisition unit 510 is used to acquire a nuclear magnetic resonance image with a first resolution, and perform automatic region segmentation processing on the nuclear magnetic resonance image with the first resolution, wherein the processed nuclear magnetic resonance image with the first resolution includes a plurality of regions;

[0124] A first processing unit 520 is used to classify the images of the multiple regions to obtain three regions;

[0125] A second processing unit 530 is used to perform sparse dilated convolution processing on the images in the three regions respectively and then input the images into different residual groups for image processing respectively;

[0126] The third processing unit 540 is used to add pixel results of the images processed by different residual groups to obtain an enhanced nuclear magnetic resonance image.

[0127] In an implementation manner of the embodiment of the present application, the first processing unit is specifically configured to:

[0128] Calculating a priority score for each region in the processed image with the first resolution;

[0129] The regions in the processed image with the first resolution are classified according to the priority scores to obtain a first region, a second region and a third region.

[0130] In an implementation of the embodiment of the present application, the classifying the regions in the processed first-resolution image according to the priority scores to obtain the first region, the second region, and the third region includes:

[0131] According to the priority scores, k regions with the highest scores are selected as the regions of greatest interest;

[0132] Performing region matching on the region of greatest interest and the corresponding nuclear magnetic image with a second resolution, obtaining the region of greatest interest in the nuclear magnetic image with the second resolution as the first region; the nuclear magnetic image with the second resolution is obtained by performing down-sampling pixel matching based on the nuclear magnetic image with the first resolution, and the resolution of the nuclear magnetic image with the second resolution is lower than the resolution of the nuclear magnetic image with the first resolution;

[0133] Performing filtering processing on the nuclear magnetic resonance image with the second resolution by using pixel filtering to obtain significant information, and obtaining a general region of interest as the second region according to the significant information and the nuclear magnetic resonance image with the second resolution;

[0134] The remaining area of ​​the nuclear magnetic resonance image with the second resolution except the first area and the second area is used as the third area.

[0135] In an implementation of the embodiment of the present application, the second processing unit performs sparse dilation convolution processing on the images in the three areas respectively, including:

[0136] Acquire the minimum and maximum horizontal and vertical coordinates of the effective area in the partition image, perform rectangular expansion on the features of the effective area, and obtain the area surrounded by the four axes of the minimum and maximum horizontal and vertical coordinates as the expanded area;

[0137] Performing an outermost circle expansion operation on the expanded area;

[0138] Convolving the image features including the outermost circle to obtain convolved features;

[0139] Apply the mask to filter the convolutional features to obtain deep features of the same size as the original area;

[0140] Or, the features of the effective area in the partition image are subjected to 8-neighborhood dilation convolution to obtain the dilated area; before each convolution, the features of the effective area in the partition image are subjected to 8-neighborhood dilation once, and the features after convolution are consistent with the size of the original area;

[0141] Apply the mask to filter the convolved features to obtain deep features of the same size as the original region.

[0142] In an implementation of the embodiment of the present application, the second processing unit performs sparse dilated convolution processing on the images in the three regions respectively and then inputs the images into different residual groups for image processing respectively, including:

[0143] Performing sparse dilated convolution processing on the image of the first region and then inputting the processed image into the first residual group for image processing;

[0144] Performing sparse dilation convolution processing on the second region and the third region images and then inputting the processed images into a second residual group for image processing;

[0145] The complexity of the first residual group is higher than the complexity of the second residual group.

[0146] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the solutions provided by the embodiments of the present application.

[0147] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the method described in any embodiment of the present application.

[0148] The computer storage medium stores codes, and when the codes are executed, a device executing the codes implements the method described in any embodiment of the present application.

[0149] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0150] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0151] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same and similar parts between the various embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0152] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A real-time lightweight nuclear magnetic resonance image enhancement method based on regional classification, characterized in that: The method comprises: Acquiring a nuclear magnetic resonance image with a first resolution, and performing automatic region segmentation processing on the nuclear magnetic resonance image with the first resolution, wherein the processed nuclear magnetic resonance image with the first resolution includes a plurality of regions; Classifying the images of the multiple regions to obtain three regions; The images in the three regions are respectively subjected to sparse dilated convolution processing and then input into different residual groups for image processing respectively; The pixel results of the images processed by different residual groups are added together to obtain the enhanced nuclear magnetic resonance image.

2. The method according to claim 1, characterized in that The three regions obtained by classifying the images of the multiple regions include: Calculating a priority score for each region in the processed image with the first resolution; The regions in the processed image with the first resolution are classified according to the priority scores to obtain a first region, a second region and a third region.

3. The method according to claim 2, characterized in that The classifying the regions in the processed first-resolution image according to the priority scores to obtain the first region, the second region and the third region comprises: According to the priority scores, k regions with the highest scores are selected as the regions of greatest interest; Performing region matching on the region of greatest interest and the corresponding nuclear magnetic image with a second resolution, obtaining the region of greatest interest in the nuclear magnetic image with the second resolution as the first region; the nuclear magnetic image with the second resolution is obtained by performing down-sampling pixel matching based on the nuclear magnetic image with the first resolution, and the resolution of the nuclear magnetic image with the second resolution is lower than the resolution of the nuclear magnetic image with the first resolution; Performing filtering processing on the nuclear magnetic resonance image with the second resolution by using pixel filtering to obtain significant information, and obtaining a general region of interest as the second region according to the significant information and the nuclear magnetic resonance image with the second resolution; The remaining area of ​​the nuclear magnetic resonance image with the second resolution except the first area and the second area is used as the third area.

4. The method according to claim 1, characterized in that: The performing sparse dilated convolution processing on the images in the three regions respectively comprises: Acquire the minimum and maximum horizontal and vertical coordinates of the effective area in the partition image, perform rectangular expansion on the features of the effective area, and obtain the area surrounded by the four axes of the minimum and maximum horizontal and vertical coordinates as the expanded area; Performing an outermost circle expansion operation on the expanded area; Convolving the image features including the outermost circle to obtain convolved features; Apply the mask to filter the convolutional features to obtain deep features of the same size as the original area; Or, the features of the effective area in the partition image are subjected to 8-neighborhood dilation convolution to obtain the dilated area; before each convolution, the features of the effective area in the partition image are subjected to 8-neighborhood dilation once, and the features after convolution are consistent with the size of the original area; Apply the mask to filter the convolved features to obtain deep features of the same size as the original region.

5. The method according to claim 3, characterized in that: The steps of performing sparse dilated convolution processing on the images in the three regions and then inputting the images into different residual groups for performing image processing respectively include: Performing sparse dilated convolution processing on the image of the first region and then inputting the processed image into the first residual group for image processing; Performing sparse dilation convolution processing on the second region and the third region images and then inputting the processed images into a second residual group for image processing; The complexity of the first residual group is higher than the complexity of the second residual group.

6. A real-time lightweight nuclear magnetic resonance image enhancement system based on regional classification, characterized in that: The system comprises: an acquisition unit, configured to acquire a nuclear magnetic resonance image with a first resolution, and perform automatic region segmentation processing on the nuclear magnetic resonance image with the first resolution, wherein the processed nuclear magnetic resonance image with the first resolution includes a plurality of regions; A first processing unit, configured to classify the images of the multiple regions to obtain three regions; A second processing unit is used to perform sparse dilation convolution processing on the images in the three regions respectively and then input the images into different residual groups for image processing respectively; The third processing unit is used to add pixel results of the images processed by different residual groups to obtain an enhanced nuclear magnetic resonance image.

7. The system according to claim 6, characterized in that The first processing unit is specifically configured to: Calculating a priority score for each region in the processed image with the first resolution; The regions in the processed image with the first resolution are classified according to the priority scores to obtain a first region, a second region and a third region.

8. The system according to claim 7, characterized in that The classifying the regions in the processed first-resolution image according to the priority scores to obtain the first region, the second region and the third region comprises: According to the priority scores, k regions with the highest scores are selected as the regions of greatest interest; Performing region matching on the region of greatest interest and the corresponding nuclear magnetic image with a second resolution, obtaining the region of greatest interest in the nuclear magnetic image with the second resolution as the first region; the nuclear magnetic image with the second resolution is obtained by performing down-sampling pixel matching based on the nuclear magnetic image with the first resolution, and the resolution of the nuclear magnetic image with the second resolution is lower than the resolution of the nuclear magnetic image with the first resolution; Performing filtering processing on the nuclear magnetic resonance image with the second resolution by using pixel filtering to obtain significant information, and obtaining a general region of interest as the second region according to the significant information and the nuclear magnetic resonance image with the second resolution; The remaining area of ​​the nuclear magnetic resonance image with the second resolution except the first area and the second area is used as the third area.

9. A computing device, characterized in that The computing device includes: a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method according to any one of claims 1 to 5 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images