Image resolution enhancement method and system based on selective state space model

Abstract

The invention discloses an image resolution enhancement method and system based on a selective state space model, which can improve the automation degree and improve the precision of medical image super-resolution reconstruction. According to the technical scheme, the method comprises the steps that S1, an original image is preprocessed; s2, a feature coding module based on a Swin-Transform network is adopted, and multi-scale coding is carried out on the preprocessed image features; s3, performing up-sampling on the image subjected to feature coding by using an image interpolation algorithm; s4, performing feature decoding on the up-sampled image by using a selective state space model decoding module; and S5, on the basis of training the coding-decoding module, carrying out super-resolution reconstruction by using a network, and outputting an image with a higher resolution than an original image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a multi-scale medical image resolution enhancement method based on a selective state space model. BACKGROUND

[0002] In the modern medical diagnosis system, medical images such as computed tomography (CT) and magnetic resonance imaging (MRI) have become the core technical means for disease screening and diagnosis, and the quality of the images directly affects the accuracy of clinical decision-making, and the spatial resolution is a key indicator of image quality. The single-layer resolution and layer thickness of medical images are key factors in determining spatial resolution. Thinner slices can provide significantly higher spatial resolution, which is crucial for accurately identifying small abnormalities, evaluating complex anatomical structures, and accurately characterizing lesion properties. For example, in the early diagnosis of lung nodules, current clinical guidelines clearly recommend using continuous thin-layer chest CT images for reconstruction, with a thickness of ≤1.5 mm, usually 1 mm. However, despite the significant advantages of high-resolution thin-layer CT in diagnosis, its clinical application in many developing countries faces serious challenges: the cost of acquiring high-performance CT equipment and building a large-scale data storage infrastructure is high; many existing CT devices have the ability to collect thin-layer data, but are limited by the operation process and storage strategy, and the reconstruction and storage are usually set to thick layers (such as 5 mm) by default, and the adjustment process is complex and device-dependent. Therefore, 5 mm thick layer CT images are still the mainstream choice in many clinical practices. The spatial resolution of thick layer medical images is relatively coarse, which may result in unclear or blurred display of subtle anatomical structures, increasing the risk of missing small lesions or misjudging lesion characteristics, which may lead to adverse clinical consequences.

[0003] Image quality issues also limit computer-aided medical image analysis, especially artificial intelligence (AI)-based diagnostic applications. For example, most of the currently marketed chest AI diagnostic products are developed and optimized for high-quality thin-layer CT images (1 mm layer thickness), and when applied to low-resolution CT images (thick layer 5 mm), their diagnostic performance often decreases significantly. This results in patients in areas relying on thick layer images being unable to effectively benefit from these advanced AI-assisted diagnostic tools, further exacerbating the inequality of medical resources and diagnostic levels.

[0004] As a cutting-edge technology in the field of artificial intelligence, deep learning has become a core driving force for medical image analysis and is widely used in disease diagnosis, lesion detection, and target region segmentation. With the continuous development of deep learning technology, its application in the field of medical image conversion has also become increasingly in-depth. How to use this technology to overcome the aforementioned shortcomings is a problem that needs to be solved in the industry. SUMMARY

[0005] The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.

[0006] The purpose of the present application is to solve the above problems, provide an image resolution enhancement method and system based on selective state space model, which can improve the automation degree and improve the accuracy of medical image super-resolution reconstruction.

[0007] The technical scheme of the present application is: the present application discloses an image resolution enhancement method based on selective state space model, the method comprising:

[0008] Step S1: preprocessing the original image;

[0009] Step S2: using a feature encoding module based on Swin-Transformer network to perform multi-scale encoding on the features of the preprocessed image;

[0010] Step S3: using an image interpolation algorithm to upsample the image after feature encoding;

[0011] Step S4: using a selective state space model decoding module to perform feature decoding on the upsampled image;

[0012] Step S5: based on the training of the encoding-decoding module, using the network to perform super-resolution reconstruction, and outputting an image with higher resolution than the original image.

[0013] According to an embodiment of the image resolution enhancement method based on selective state space model of the present application, step S1 further comprises:

[0014] Step S11: input the original image;

[0015] Step S12: standardizing the image gray value of the input image, selecting a suitable gray value standardization method, and adjusting the image gray value to [0, 1];

[0016] Step S13: output the standardized image I Low .

[0017] According to an embodiment of the image resolution enhancement method based on selective state space model of the present application, step S2 further comprises:

[0018] Step S21: input the standardized low-resolution image I Low;

[0019] Step S22: using an image convolution layer as a linear embedding layer, setting the convolution kernel size and step length to correspond to the output image block size, and inputting the divided image block I Low into the image block I Patch ;

[0020] Step S23: inputting the divided image block I Patch into a Swin-Transformer network trunk to extract multi-scale image features;

[0021] Step S24: using a convolution network as a feature convolution layer to further extract the extracted multi-scale image features to obtain an encoded feature map.

[0022] According to an embodiment of the image resolution enhancement method based on the selective state space model, the Swin-Transformer network trunk in step S23 includes a plurality of Swin-Transformer feature extraction modules, and each Swin-Transformer feature extraction module performs the following processing steps:

[0023] Step S231: inputting the divided image block I Patch into a layer normalization module to standardize the hidden layer of each sample;

[0024] Step S232: using a window multi-head self-attention network to divide the input feature sequence into a plurality of non-overlapping windows, and independently applying multi-head self-attention in each window to extract local spatial features F W-MHSA ;

[0025] Step S233: using a residual connection module to superimpose the image block I Patch and the local spatial features F W-MHSA to obtain primary residual features F1;

[0026] Step S234: using a layer normalization module to standardize the primary residual features F1;

[0027] Step S235: inputting the standardized image features into a multi-layer perceptron module to extract deep-level abstract features F MLP1 ;

[0028] Step S236: using a residual connection module to superimpose the primary residual features F1 and the deep-level abstract features F MLP1 to obtain secondary residual features F2;

[0029] Step S237: using a layer normalization module to standardize the secondary residual features F2;

[0030] Step S238: The shift window multi-head self-attention network is used to solve the information isolation problem of the local window, to enhance the global perception ability to establish long-distance dependence, and to extract global dependence features F SW-MHSA ;

[0031] Step S239: The secondary residual features F2 are superimposed with the global dependence features F SW-MHSA by using a residual connection module to obtain tertiary residual features F3;

[0032] Step S2310: The tertiary residual features F3 are standardized by using a layer normalization module;

[0033] Step S2311: The standardized image features are input into a multi-layer perception module to extract high-order semantic features F MLP2 ;

[0034] Step S2312: The tertiary residual features F3 are superimposed with the high-order semantic features F MLP2 by using a residual connection module to obtain image features F Transformer .

[0035] According to an embodiment of the image resolution enhancement method based on the selective state space model, step S3 further includes:

[0036] Step S31: An image block F Transformer is input after feature encoding;

[0037] Step S32: An image upsampling resolution is set, and the image block F Transformer is upsampled by using an image interpolation algorithm;

[0038] Step S33: A resampled image I UpSample is output.

[0039] According to an embodiment of the image resolution enhancement method based on the selective state space model, step S4 further includes:

[0040] Step S41: The upsampled image I UpSample is subjected to a spatial sorting operation to obtain a coronal image I Cor and a sagittal image I Sag , respectively;

[0041] Step S42: A residual selective state space module is used to respectively decode the features of the coronal and sagittal images to recover image information, to obtain coronal reconstructed image features F Cor and sagittal reconstructed image features F Sag ;

[0042] Step S43: The coronal reconstructed image features F Cor are superimposed with the sagittal reconstructed image features F Sag by using a residual connection module to obtain a reconstructed image FCor and sagittal reconstruction image features F Sag fused to generate a multi-planar reconstruction feature set;

[0043] Step S44: applying a two-dimensional convolution layer to process the multi-planar reconstruction features to obtain a reconstruction image with higher resolution than the original image.

[0044] According to an embodiment of the image resolution enhancement method based on the selective state space model, the residual selective state space module in step S42 includes a plurality of residual state space module feature extraction modules, and each individual residual state space module feature extraction module performs the following processing steps:

[0045] Step S421: applying a layer normalization module to perform standardization processing on image features, wherein the image features include coronal reconstruction image I Cor and sagittal reconstruction image I Sag ;

[0046] Step S422: using a visual state space module to extract visual state space features F VSSM ;

[0047] Step S423: introducing a channel-level scaling factor to scale the reconstruction image features and fuse them with the visual state space features F VSSM to obtain primary fusion features F VF after channel fusion;

[0048] Step S424: applying a layer normalization module to perform standardization processing on the primary fusion features F VF ;

[0049] Step S425: using a convolution layer to further extract the image features after standardization processing to obtain local convolution features F Conv after local convolution;

[0050] Step S426: selecting a channel attention layer to select key channels and screen the local convolution features F Conv after local convolution to obtain channel attention features F Gate ;

[0051] Step S427: introducing a channel-level scaling factor to scale the primary fusion features F VF and fuse them with the channel attention features F Gate to obtain fused image features.

[0052] According to an embodiment of the image resolution enhancement method based on the selective state space model, step S422 further includes the following processing:

[0053] Step S4221: input the image feature after the standard post-processing of step S421, expand the channel number to a predefined channel number through a linear layer, and obtain first branch expansion feature F L1 ;

[0054] Step S4222: process the first branch expansion feature F L1 using a Sigmoid linear unit, and obtain first branch activation feature F Path1 ;

[0055] Step S4223: input the image feature after the standard post-processing of step S421, expand the channel number to a predefined channel number through a linear layer, and obtain second branch expansion feature F L2 ;

[0056] Step S4224: input the second branch expansion feature F L2 into a depth separable convolution layer, and obtain depth separable convolution feature F DC2 ;

[0057] Step S4225: process the depth separable convolution feature F DC2 through a Sigmoid linear unit, and obtain second branch activation feature F SiLU2 ;

[0058] Step S4226: perform multiple direction scanning on the second branch activation feature F SiLU2 using a 2D selective scanning layer, establish longitudinal and transverse bidirectional dependence, and obtain multiple scanning features F 2D-SSM2 ;

[0059] Step S4227: perform standardization processing on the multiple scanning features F 2D-SSM2 using a layer standardization module, and obtain second branch standardized feature F Path2 ;

[0060] Step S4228: aggregate the first branch activation feature F Path1 and the second branch standardized feature F Path2 using Hadamard product, and obtain two-way aggregation feature F H ;

[0061] Step S4229: expand the channel number of the two-way aggregation feature F H to a predefined channel number through a linear layer, and obtain visual state space feature F VSSM .

[0062] According to an embodiment of the image resolution enhancement method based on the selective state space model, step S5 further includes the following processing steps:

[0063] Step S51: using online data augmentation technology to expand the training sample;

[0064] Step S52: selecting L1 loss function as the loss function of network training;

[0065] Step S53: using Adam as the optimizer, setting the appropriate learning rate, and training the encoding-decoding network;

[0066] Step S54: inputting the original image into the trained encoding-decoding network model to obtain an image with higher resolution.

[0067] The application also discloses an image resolution enhancement system based on a selective state space model, which comprises:

[0068] A preprocessing module configured to preprocess an original image;

[0069] A multi-scale encoding module configured to adopt a feature encoding module based on a Swin-Transformer network to perform multi-scale encoding on the features of the preprocessed image;

[0070] An up-sampling module configured to use an image interpolation algorithm to perform up-sampling on the image after feature encoding;

[0071] A feature decoding module configured to use a selective state space model decoding module to perform feature decoding on the up-sampled image;

[0072] A resolution reconstruction module configured to use a network to perform super-resolution reconstruction on the basis of training the encoding-decoding module, and output an image with higher resolution than the original image.

[0073] The application also discloses a computer device for image resolution enhancement based on a selective state space model, which comprises a memory, a processor and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to implement the steps of the image resolution enhancement method based on the selective state space model.

[0074] The application also discloses a computer readable storage medium storing program instructions executable by a processor to implement the steps of the image resolution enhancement method based on the selective state space model.

[0075] The application also discloses a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the image resolution enhancement method based on the selective state space model.

[0076] The present application has the following beneficial effects compared with the prior art: in the method of the present application, first, the original low-resolution medical image (such as CT, MRI, etc.) is preprocessed; second, the Swin-Transformer feature encoding module is used to encode the image features in multiple scales; then, the image interpolation algorithm is used to upsample the image after feature encoding to achieve the expected resolution; finally, based on the training of the encoding-decoding module, the network is used for super-resolution reconstruction, and the high-resolution medical image is output. Compared with the prior art, the present application not only can break through the performance and cost limitations of the equipment, significantly improve the diagnostic accuracy of radiologists, but also can optimize the application effect of existing and future AI diagnosis products in the thick layer medical image environment, and help to solve the problem of uneven distribution of medical resources. BRIEF DESCRIPTION OF DRAWINGS

[0077] The above features and advantages of the present application can be better understood after reading the detailed description of embodiments of the present application in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components having similar related properties or features can have the same or similar reference numerals.

[0078] Figure 1 A flowchart of an embodiment of the image resolution enhancement method based on the selective state space model of the present application is shown.

[0079] Figure 2 An image feature encoding network flowchart in the method embodiment shown. Figure 1

[0080] An image feature decoding network flowchart in the method embodiment shown. Figure 3 Figure 1 A Swin-Transformer feature extraction module flowchart in the method embodiment shown.

[0081] Figure 4 Figure 1 A residual state space module feature extraction module flowchart in the method embodiment shown.

[0082] Figure 5 A visual state space module flowchart in the method embodiment shown. Figure 1

[0083] A visual state space module flowchart in the method embodiment shown. Figure 6 Figure 1 A visual state space module flowchart in the method embodiment shown.

[0084] Figure 7 A principle diagram of an embodiment of the image resolution enhancement system based on the selective state space model of the present application is shown. DETAILED DESCRIPTION ​​​

[0085] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments. Note that the aspects described below in conjunction with the accompanying drawings and specific embodiments are only exemplary and should not be understood as limiting the scope of protection of the present application in any way.

[0086] Figure 1 The flow of an embodiment of the image resolution enhancement method based on the selective state space model of the present application is shown. Please refer to Figure 1 , the implementation steps of the method of the present embodiment are described in detail as follows.

[0087] Step S1: Preprocessing the original low-resolution image.

[0088] Step S1 further includes the following processing steps.

[0089] Step S11: Input the original low-resolution image, which can be CT or MRI;

[0090] Step S12: Standardize the image gray value of the input image, select a suitable gray value standardization method, adjust the image gray value to [0, 1], so as to ensure that the image gray value is within a certain scale range, and facilitate subsequent analysis. For example, the maximum and minimum value normalization technique can be used to adjust the image gray value for CT image, and the Z-Score standardization technique can be used to adjust the gray value for MRI image.

[0091] Step S13: Output the standardized low-resolution image I Low for subsequent processing and analysis.

[0092] Step S2: Use the feature encoding module based on the Swin-Transformer network to encode the features of the preprocessed image in multiple scales.

[0093] Please also refer to Figure 2 , Step S2 further includes the following processing steps.

[0094] Step S21: Input the standardized low-resolution image I Low .

[0095] Step S22: Use the image convolution layer as the linear embedding layer, set the convolution kernel size and step length to correspond to the output image block size, and divide I Low into fixed-size image blocks I Patch . This operation not only reduces the high-dimensional data scale of the original image, but also captures effective information in the local area.

[0096] Step S23: Input the divided image blocks into the Swin-Transformer network backbone to extract multi-scale image features.

[0097] Step S24: Further extract the extracted multi-scale image features using a convolutional network as a feature convolutional layer to obtain an encoded feature map.

[0098] For the above step S23, N Swin-Transformer feature extraction modules are included, as shown in Figure 3 Each individual module includes the following steps.

[0099] Step S231: Divide the image block I Patch The input layer normalization module normalizes the hidden layer of each sample to reduce the influence of internal covariate shift.

[0100] Step S232: Divide the input feature sequence into multiple non-overlapping windows using a window multi-head self-attention network, and independently apply multi-head self-attention to extract local spatial features F W-MHSA Windowing design can significantly reduce computational complexity while preserving the spatial local structure of features.

[0101] Step S233: Use a residual connection module to superimpose the image block feature I Patch and the local spatial feature F W-MHSA to obtain a primary residual feature F1. The residual structure can prevent gradient vanishing in deep networks, reduce overfitting risk, and ensure that key bottom-level visual information is not lost.

[0102] Step S234: Standardize the primary residual feature F1 using a layer normalization module.

[0103] Step S235: Input the standardized image feature into a multi-layer perceptron module to extract deep-level abstract features F MLP1 .

[0104] Step S236: Use a residual connection module to superimpose the primary residual feature F1 and the deep-level abstract feature F MLP1 to obtain a secondary residual feature F2.

[0105] Step S237: Standardize the secondary residual feature F2 using a layer normalization module.

[0106] Step S238: Use a shift window multi-head self-attention network to solve the information isolation problem of local windows, enhance global perception ability, establish long-distance dependence, and extract global dependence features F SW-MHSA .

[0107] Step S239: Use a residual connection module to superimpose the secondary residual feature F2 and the global dependence feature F SW-MHSA to obtain a tertiary residual feature F3.

[0108] Step S2310: standardize the third-level residual feature F3 using a layer normalization module.

[0109] Step S2311: input the standardized image feature into a multi-layer perception module to extract a high-order semantic feature F MLP2 .

[0110] Step S2312: superimpose the third-level residual feature F3 and the high-order semantic feature F MLP2 using a residual connection module to obtain an image feature F Transformer .

[0111] Step S3: use an image interpolation algorithm to up-sample the image after feature encoding to achieve the expected resolution.

[0112] Step S3 further includes the following processing steps.

[0113] Step S31: input the image block F Transformer after feature encoding.

[0114] Step S32: set the image up-sampling resolution, and use an image interpolation algorithm (such as nearest neighbor interpolation, linear interpolation, bilinear interpolation, bicubic interpolation, etc.) to up-sample F Transformer .

[0115] Step S33: output the resampled image I UpSample .

[0116] Step S4: use a selective state space model decoding module to perform feature decoding on the up-sampled image.

[0117] Please refer to Figure 4 , Step S4 further includes the following processing steps.

[0118] Step S41: perform spatial sorting operation on the up-sampled image I UpSample to obtain a coronal image I Cor and a sagittal image I Sag respectively.

[0119] Step S42: use a residual selective state space module to perform feature decoding on the coronal and sagittal images respectively to recover image information, and obtain a coronal reconstructed image feature F Cor and a sagittal reconstructed image F Sag . The residual selective state space module combines the long sequence modeling capability of the state space model and the stability of residual learning, while maintaining the integrity of the anatomical structure, it realizes the deep integration of cross-plane features, and is particularly suitable for multi-plane analysis of medical images.

[0120] Step S43: Feature F Cor and sagittal reconstruction image feature F Sag are fused to generate a multi-planar reconstruction feature set.

[0121] Step S44: The multi-planar reconstruction feature is processed by using a two-dimensional convolution layer to obtain a high-resolution reconstruction image.

[0122] For the above step S42, the residual selective state space module internally includes a plurality of residual state space module feature extraction modules. The processing steps of each individual residual state space module feature extraction module are as shown in Figure 5 and described in detail as follows.

[0123] Step S421: The image features (such as the coronal reconstruction image I Cor and the sagittal reconstruction image I Sag ) are standardized by using a layer normalization module.

[0124] Step S422: The visual state space feature F VSSM of the visual state space is extracted by using a visual state space module. The visual state space module fuses the sequence modeling capability of the visual state space model and the characteristics of the visual task.

[0125] Step S423: A channel-level scaling factor is introduced to scale the reconstruction image feature, and the visual state space feature F VSSM is fused to obtain a primary fusion feature F VF after channel fusion.

[0126] Step S424: The primary fusion feature F VF is standardized by using a layer normalization module.

[0127] Step S425: The standardized image feature is further extracted by using a convolution layer to obtain a local convolution feature F Conv after local convolution.

[0128] Step S426: A channel attention layer is selected to select a key channel and screen the local convolution feature F Conv to obtain a channel attention feature F Gate , so as to improve the expression ability of the module and reduce channel redundancy.

[0129] Step S427: A channel-level scaling factor is introduced to scale the primary fusion feature F VF , and the channel attention feature F Gate is fused to obtain a fused image feature.

[0130] For the above step S422, the further processing process is as shown in Figure 6As shown, detailed as follows.

[0131] Step S4221: input the image features after the normalization processing in step S421, expand the channel number to a predefined channel number through a linear layer, and obtain first branch expansion features F L1 .

[0132] Step S4222: process the first branch expansion features F L1 through a Sigmoid linear unit, and obtain first branch activation features F Path1 .

[0133] Step S4223: input the image features after the normalization processing in step S421, expand the channel number to a predefined channel number through a linear layer, and obtain second branch expansion features F L2 .

[0134] Step S4224: input the second branch expansion features F L2 into a depth separable convolution layer, and obtain depth separable convolution features F DC2 . This step can independently perform spatial convolution operation on each input channel without cross-channel information fusion, which can significantly reduce the calculation amount and parameter amount.

[0135] Step S4225: process the depth separable convolution F DC2 through a Sigmoid linear unit, and obtain second branch activation features F SiLU2 .

[0136] Step S4226: perform multiple direction scanning on the second branch activation features F SiLU2 using a 2D selective scanning layer, establish longitudinal and transverse bidirectional dependence, and obtain multiple scanning features F 2D-SSM2 .

[0137] Step S4227: perform normalization processing on the multiple scanning features F 2D-SSM2 using a layer normalization module, and obtain second branch normalized features F Path2 .

[0138] Step S4228: aggregate the first branch activation features F Path1 and the second branch normalized features F Path2 using Hadamard product, and obtain two-way aggregation features F H .

[0139] Step S4229: expand the channel number of the two-way aggregation features F H to a predefined channel number through a linear layer, and obtain visual state space features.

[0140] Step S5: Based on the training of the encoding-decoding module, the network is used for super-resolution reconstruction, and a high-resolution medical image is output.

[0141] Step S5 further comprises the following processing steps.

[0142] Step S51: The training sample is expanded by using an online data enhancement technology.

[0143] Step S52: An L1 loss function is selected as a loss function Loss of network training, and the calculation formula is as follows:

[0144]

[0145] Wherein, SR represents a medical image after network reconstruction; HQ is a high-resolution medical image, that is, a "gold standard"; N is the total number of pixels in the image; i is a pixel code; and Loss represents a loss function.

[0146] Step S53: Adam is used as an optimizer, and a suitable learning rate is set to train the encoding-decoding network.

[0147] Step S54: The low-resolution image is input into the trained encoding-decoding network model to obtain a higher-resolution image.

[0148] Figure 7 The principle of an embodiment of the image resolution enhancement system based on the selective state space model of the application is shown. Please refer to Figure 7 The system of the embodiment comprises a preprocessing module, a multi-scale encoding module, an upsampling module, a feature decoding module, and a resolution reconstruction module.

[0149] The preprocessing module is configured to preprocess the original image. The internal processing of the module is the same as that of step S1 in the method embodiment shown in Figure 1 , and will not be repeated here.

[0150] The multi-scale encoding module is configured to use a feature encoding module based on the Swin-Transformer network to perform multi-scale encoding on the features of the preprocessed image. The internal processing of the module is the same as that of step S2 in the method embodiment shown in Figure 1 , and will not be repeated here.

[0151] The upsampling module is configured to use an image interpolation algorithm to perform upsampling on the image after feature encoding. The internal processing of the module is the same as that of step S3 in the method embodiment shown in Figure 1 , and will not be repeated here.

[0152] The feature decoding module is configured to use the selective state space model decoding module to perform feature decoding on the up-sampled image. The internal processing of this module is the same as that of the Figure 1 Step S4 in the method embodiment shown in the figure is the same, and will not be repeated here.

[0153] The resolution reconstruction module is configured to use the network to perform super-resolution reconstruction on the basis of training the encoding-decoding module, and output an image with higher resolution than the original image. The internal processing of this module is the same as that of the Figure 1 Step S5 in the method embodiment shown in the figure is the same, and will not be repeated here.

[0154] In addition, the present application also discloses a computer device for image resolution enhancement based on a selective state space model, comprising a memory, a processor and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to implement the steps of the image resolution enhancement method embodiment based on a selective state space model as shown in the figure. Figure 1 The steps of the image resolution enhancement method embodiment based on a selective state space model as shown in the figure.

[0155] In addition, the present application also discloses a computer readable storage medium storing program instructions executable by a processor to implement the steps of the image resolution enhancement method embodiment based on a selective state space model as shown in the figure. Figure 1 The steps of the image resolution enhancement method embodiment based on a selective state space model as shown in the figure.

[0156] In addition, the present application also discloses a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the image resolution enhancement method embodiment based on a selective state space model as shown in the figure. Figure 1 The steps of the image resolution enhancement method embodiment based on a selective state space model as shown in the figure.

[0157] The following uses an example to verify the beneficial effects of the above-mentioned method of the present application.

[0158] 1. Experimental conditions:

[0159] The training set data of this experiment comes from the Affiliated Tumor Hospital of Fudan University, a total of 2494 CT images. The test data set comes from the RPLHR-CT-tiny public data set (https: / / aistudio.baidu.com / datasetdetail / 239394), a total of 50 chest CT images are collected as test set samples; all images are thick layer CT images (layer thickness 5mm) and thin layer CT images (layer thickness 1mm).

[0160] 2. Experimental results and analysis of results

[0161] By constructing the super-resolution enhancement model for the chest CT image, the model obtains an average peak signal-to-noise ratio (PSNR) of 35.10, a mean square error (MSE) of 0.016, and a structural similarity (SSIM) of 46.37 on the test set, which are all better than those of the Swin-Transformer network (https: / / doi.org / 10.1038 / s41746-024-01338-8, PSNR is 34.49, MSE is 0.018, and SSIM is 46.12) and the three-linear interpolation algorithm (PSNR is 34.26, MSE is 0.019, and SSIM is 45.64). Therefore, the selective state space model multi-scale medical image resolution enhancement method has higher signal-to-noise ratio and lower mean square error.

[0162] Although the above-described methods are illustrated and described as a series of acts for purposes of simplicity, it should be appreciated and understood that the methods are not limited in terms of the order of acts, as some acts can, in accordance with one or more embodiments, occur simultaneously or in different order than that depicted and described herein.

[0163] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality, without limitation. Such functionality can be implemented in a variety of ways using electronic hardware, computer software, or any combination thereof. It will be appreciated that one of ordinary skill in the art, having the benefit of the present disclosure, can implement the described functionality in a manner that differs from the variety of implementations described herein without departing from the scope of the present application.

[0164] The various illustrative logical blocks, modules, and circuits described in connection with the embodiments disclosed herein can be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general purpose processor can be a microprocessor, but in the alternative, the processor can be any conventional processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0165] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as discrete components in a user terminal.

[0166] In one or more exemplary embodiments, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0167] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for image resolution enhancement based on a selective state space model, characterized in that, The method comprises: Step S1: preprocessing the original image; Step S2: using a feature encoding module based on a Swin-Transformer network to perform multi-scale encoding on the features of the preprocessed image; Step S3: using an image interpolation algorithm to perform up-sampling on the image after feature encoding; Step S4: using a selective state space model decoding module to perform feature decoding on the up-sampled image; Step S5: based on the training of the encoding-decoding module, using the network to perform super-resolution reconstruction, and outputting an image with higher resolution than the original image.

2. The method of claim 1, wherein the method is based on a selective state space model. Step S1 further comprises: Step S11: inputting the original image; Step S12: standardizing the image gray value of the input image, selecting a suitable gray value standardization method, and adjusting the image gray value to [0, 1]; Step S13: output the standardized image I Low .

3. The method of claim 2, wherein the method is based on a selective state space model. Step S2 further comprises: Step S21: input the standardized low-resolution image I Low ; Step S22: Set the convolution kernel size and step length corresponding to the size of the output image block by using the image convolution layer as the linear embedding layer, and I Low is divided into fixed-size image blocks I Patch ; Step S23: dividing the image block I Patch Input the Swin-Transformer network backbone to extract multi-scale image features; Step S24: using a convolution network as a feature convolution layer to further extract the extracted multi-scale image features to obtain the encoded feature map.

4. The method of claim 3, wherein the selective state space model is based on a Markov Random Field (MRF) model. The Swin-Transformer network in step S23 comprises a plurality of Swin-Transformer feature extraction modules, wherein each Swin-Transformer feature extraction module performs the following processing steps: Step S231: dividing the image I into image blocks I Patch The input layer normalization module normalizes the hidden layer of each sample. Step S232: using a window multi-head self-attention network, dividing the input feature sequence into multiple non-overlapping windows, and independently applying multi-head self-attention in each window to extract local spatial features F W-MHSA ; Step S233: applying the residual connection module to the image block I Patch with the local spatial feature F W-MHSA superimposition, obtaining the primary residual feature F1; Step S234: using a layer normalization module to standardize the primary residual feature F1; Step S235: input the normalized image features into a multi-layer perception module to extract deep-level abstract features F by introducing a nonlinear transformation MLP1 ; Step S236: using the residual connection module to connect the primary residual feature F1 with the deep-level abstract feature F MLP1 to obtain the secondary residual feature F2; Step S237: using a layer normalization module to standardize the secondary residual feature F2; Step S238: use the shift window multi-head self-attention network to solve the information isolation problem of the local window, enhance the global perception ability to establish long-distance dependence, and extract global dependence features F SW-MHSA ; Step S239: The secondary residual feature F2 is connected with the global dependency feature F by using a residual connection module to obtain a tertiary residual feature F3. SW-MHSA superimposition. Step S2310: using a layer normalization module to standardize the tertiary residual feature F3; Step S2311: input the normalized image features into the multi-layer perception module to extract high-order semantic features F MLP2 ; Step S2312: adopt the residual connection module to connect the third-level residual feature F3 with the high-order semantic feature F MLP2 superimposed, the image feature F Transformer .

5. The method of claim 4, wherein the method is based on a selective state space model. Step S3 further comprises: Step S31: input the feature-encoded image block F Transformer ; Step S32: set the image up-sampling resolution, and perform image interpolation algorithm on the image block F Transformer to perform up-sampling; Step S33: output the resampled image I UpSample .

6. The method of claim 5, wherein the method is based on a selective state space model. Step S4 further comprises: Step S41: up-sampling the image I UpSample The spatial ordering operation is performed to obtain the coronal image I Cor and the sagittal image I Sag , respectively. Step S42: using the residual selective state space module, respectively, the coronal and sagittal images are feature decoded to recover image information, and the coronal reconstruction image feature F Cor and the sagittal reconstruction image F Sag ; Step S43: Reconstruct the coronal image features F Cor and sagittal reconstructed image features F Sag The features are fused to generate a multi-plane reconstruction feature set; Step S44: using a two-dimensional convolution layer to process the multi-planar reconstruction features to obtain a reconstructed image with higher resolution than the original image.

7. The method of claim 6, wherein the method is based on a selective state space model. The residual selective state space module in step S42 comprises a plurality of residual state space module feature extraction modules, and each individual residual state space module feature extraction module performs the following processing steps: Step S421: standardize the image features, including the coronal reconstructed image I Cor and the sagittal reconstructed image I Sag ; Step S422: Extract visual state space features F using the visual state space module VSSM ; Step S423: Introducing a channel-level scaling factor to scale the reconstructed image features, and combining with the visual state space features F VSSM Fusion, obtaining the primary fusion features F VF ; Step S424: applying a layer normalization module to the primary fusion feature F VF standardization processing; Step S425: further extracting the image features after the standardization processing by using a convolution layer to obtain local convolution features F Conv ; Step S426: Selecting the channel attention layer to select the key channel and screening the local convolution feature F Conv , to obtain the channel attention feature F Gate ; Step S427: Introducing a channel-level scaling factor to scale the primary fusion feature F VF Perform scaling and fuse with the channel attention feature F Gate to obtain the fused image feature.

8. The method of claim 7, wherein the method is based on a selective state space model. Step S422 further comprises the following processing: Step S4221: input the image features after the standard post-processing of step S421, expand the channel number to a predefined channel number through a linear layer, and obtain first branch expanded features F L1 ; Step S4222: processing the first branch extended feature F using a Sigmoid linear unit L1 , to obtain a first branch activation feature F Path1 ; Step S4223: input the image features after the standard post-processing of step S421, expand the channel number to a predefined channel number through a linear layer, and obtain second branch expansion features F L2 ; Step S4224: extend the second branch feature F L2 Input the depth separable convolution layer to obtain the depth separable convolution feature F DC2 ; Step S4225: separating the deep convolutional features F DC2 processed by a Sigmoid linear unit to obtain second branch activation features F SiLU2 ; Step S4226: Activate the feature F with 2D selective scanning layer SiLU2 Multiple direction scanning is performed to establish longitudinal and transverse bidirectional dependence, and multiple scanning features F are obtained 2D-SSM2 ; Step S4227: applying a layer normalization module to the plurality of scanning features F 2D-SSM2 The standardization processing is performed to obtain second branch standardized features F Path2 ; Step S4228: aggregate the first branch activation feature F Path1 and the second branch normalized feature F Path2 to obtain a two-way aggregated feature F H ; Step S4229: Utilize linear layer to double-path aggregate feature F H The number of channels is expanded to a predefined number of channels to obtain a visual state space feature F VSSM .

9. The method of claim 8, wherein the method further comprises: Step S5 further comprises the following processing steps: Step S51: using online data augmentation technology to expand the training samples; Step S52: selecting an L1 loss function as the loss function for network training; Step S53: using Adam as an optimizer, setting a suitable learning rate, and training the encoding-decoding network; Step S54: inputting the original image into the trained encoding-decoding network model to obtain an image with higher resolution.

10. An image resolution enhancement system based on a selective state space model, characterized by, The system comprises: a preprocessing module configured to preprocess the original image; a multi-scale encoding module configured to use a feature encoding module based on a Swin-Transformer network to perform multi-scale encoding on the features of the preprocessed image; an up-sampling module configured to use an image interpolation algorithm to perform up-sampling on the image after feature encoding; a feature decoding module configured to use a selective state space model decoding module to perform feature decoding on the up-sampled image; The resolution reconstruction module is configured to perform super-resolution reconstruction using the network on the basis of the training of the encoding-decoding module, and output an image with a higher resolution than the original image.

11. A computer device for image resolution enhancement based on a selective state-space model, characterized in that, A computer program product comprising a memory, a processor and program instructions stored in the memory for execution by the processor, wherein the processor executes the program instructions to implement the steps of the method for image resolution enhancement based on a selective state space model according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, A computer program product comprising a memory, a processor and program instructions stored in the memory for execution by the processor, wherein the processor executes the program instructions to implement the steps of the method for image resolution enhancement based on a selective state space model according to any one of claims 1 to 9.

13. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method for image resolution enhancement based on a selective state space model according to any one of claims 1 to 9. The computer program is executed by the processor to implement the steps of the method for image resolution enhancement based on a selective state space model according to any one of claims 1 to 9.

Images