Image super-resolution reconstruction method and system based on dense residual network

Abstract

The present disclosure provides an image super-resolution reconstruction method and system based on a dense residual network, including: obtaining image information: obtaining an image super-resolution reconstruction result according to the obtained image information and a preset image super-resolution reconstruction model; wherein the image super-resolution reconstruction model is obtained by combining a dense residual network and an improved Transformer network; in the improved Transformer network, new weights are continuously assigned to update the dot product result of the Query and the Key, thereby obtaining a weighted output; the present disclosure proposes a novel network combining a W-Transformer with a dense residual network for improving the super-resolution of an image, and the network adopts a combination of the two networks to achieve the purpose of improving the overall performance.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular, relates to an image super-resolution reconstruction method and system based on a dense residual network. Background Art

[0002] With the widespread development of computer vision and the continuous innovation in the field of artificial intelligence, the development of single image super-resolution technology (SISR) is getting faster and faster. In 2006, Hinton et al. proposed the concept of deep learning. Its main idea is to extract high-level semantic features of images through multi-layer nonlinear transformations, focus on learning the underlying distribution of data, and gain the ability to judge and predict new data through continuous training. As a result, deep learning has officially entered the big stage of the computer field. With its powerful fitting and learning capabilities, it has quickly occupied a place in many computer vision fields such as image processing. The rapid development of deep learning has prompted more and more researchers to try to introduce deep learning into the field of super-resolution.

[0003] Dong et al. first applied the concept of deep learning to single image super-resolution (SISIR) and proposed the SRCNN super-resolution algorithm. Although the algorithm uses a relatively simple residual network, it has a significant improvement in performance and computing speed. With the advent of SRCNN, more and more super-resolution algorithms based on deep learning have emerged, such as RCN, DRRN, LapSRN, SRResNet and EDSR. However, these models still have a lot of room for improvement in computational complexity and performance.

[0004] The inventors of the present disclosure have discovered that the existing image super-resolution reconstruction methods have the following problems: after the input image passes through many layers, before reaching the end of the network or the beginning of the next network, many key information features will disappear or be filtered, resulting in the output image lacking many key information feature points, ultimately making the output image too blurry or even causing image reconstruction errors. Summary of the invention

[0005] In order to solve the above problems, the present disclosure proposes an image super-resolution reconstruction method and system based on a dense residual network. The present disclosure proposes a new network combining a W-Transformer and a dense residual network to improve the super-resolution of the image. The network adopts a combination of the two networks to achieve the purpose of improving the overall performance.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0007] In a first aspect, the present disclosure provides an image super-resolution reconstruction method based on a dense residual network, comprising:

[0008] Get image information:

[0009] Obtaining an image super-resolution reconstruction result according to the acquired image information and a preset image super-resolution reconstruction model;

[0010] The image super-resolution reconstruction model is obtained by combining a dense residual network and an improved Transformer network; in the improved Transformer network, new weights are continuously assigned to update the dot product result of the Query and the Key, thereby obtaining a weighted output.

[0011] Furthermore, the dense residual network adds a dense layer on the basis of the existing direct path and performs jump connections on each convolutional block.

[0012] Furthermore, each convolution block generates a new branch path, each convolution block of an even layer is cross-connected with the convolution blocks of the even layers below it, and each convolution block of an odd layer is cross-connected with the convolution blocks of the odd layers below it.

[0013] Furthermore, each branch adopts nonlinear transformation; in the branch connection between the first layer and the last layer, a feature pyramid module is added.

[0014] Furthermore, the improved Transformer network includes an encoding layer and a decoding layer, the encoding layer includes 6 encoders, and the decoding layer includes 6 decoders.

[0015] Furthermore, the encoder includes a weighted self-attention mechanism and a feedforward neural network; the decoder includes a weighted self-attention mechanism, encoding-decoding attention and a feedforward neural network.

[0016] Furthermore, the weighted self-attention mechanism is a weight parameter allocation mechanism that adds weight blocks and continuously assigns new weights to update the dot product results of Query and Key to obtain a weighted output that includes all visible input sequence information. It then passes through a normalized exponential function to obtain a new vector matrix, and then multiplies each Value vector by the value in the new vector matrix.

[0017] In a second aspect, the present disclosure further provides an image super-resolution reconstruction system based on a dense residual network, including a data collection module and a super-resolution reconstruction module;

[0018] The data collection module is configured to: obtain image information:

[0019] The super-resolution reconstruction module is configured to: obtain an image super-resolution reconstruction result according to the acquired image information and a preset image super-resolution reconstruction model;

[0020] The image super-resolution reconstruction model is obtained by combining a dense residual network and an improved Transformer network; in the improved Transformer network, new weights are continuously assigned to update the dot product result of the Query and the Key, thereby obtaining a weighted output.

[0021] In a third aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image super-resolution reconstruction method based on a dense residual network described in the first aspect.

[0022] In a fourth aspect, the present disclosure also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the image super-resolution reconstruction method based on a dense residual network described in the first aspect are implemented.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. This paper proposes a new backbone network for super-resolution reconstruction; a global block is added to the multi-branch serial block sequence, and repeated jumps and crosses with convolution are performed to enhance the ability to extract key semantic features. When processing low-resolution images with complex scenes, the backbone network of this paper shows excellent peak signal-to-noise ratio (PSNR) performance;

[0025] 2. The present invention adopts an improved Transformer to make up for the shortcomings of convolutional neural networks in parallel computing capabilities and feature extraction. The proposed weighted self-attention mechanism (Weighted Self-Attention) is used to discover deep feature correspondences, so that images can better restore features such as texture and details, and thus output higher-resolution images. Subsequent experiments show that the improved W-Transformer (Weighted Self-Attention-Transformer) of the present invention has significant improvements in both quantitative and qualitative image evaluation.

[0026] 3. In the present invention, a new network (TransDRN) combining W-Transformer and dense residual network (DRN) has excellent network performance and faster processing speed; experimental results show that the algorithm of the present invention is superior to existing advanced super-resolution algorithms, such as EDSR, OISR-RK3, MDCN, LW-CSC, and IPT, in both processing speed and network performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings in the specification that constitute a part of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments of this embodiment and their descriptions are used to explain this embodiment and do not constitute improper limitations on this embodiment.

[0028] Figure 1 This is a specific flow chart of Example 1 of the present disclosure;

[0029] Figure 2 It is the overall network framework of Embodiment 1 of the present disclosure;

[0030] Figure 3 This is a block diagram of a dense residual network according to Embodiment 1 of the present disclosure;

[0031] Figure 4 It is a global block detail diagram of Embodiment 1 of the present disclosure;

[0032] Figure 5 This is a W-Transformer framework diagram of Embodiment 1 of the present disclosure;

[0033] Figure 6 Detailed diagram of the weighted self-attention module of Embodiment 1 of the present disclosure;

[0034] Figure 7 A visual comparison diagram of the network performance of five models in Example 1 of the present disclosure;

[0035] Figure 8 This is an image comparison example 1 of the W-Trans DRN of Embodiment 1 of the present disclosure and 6 other advanced algorithms;

[0036] Fig. 9 This is the second image comparison example between the W-Trans DRN of Embodiment 1 of the present disclosure and the other 6 advanced algorithms;

[0037] Fig.10 This is the third image comparison example between the W-Trans DRN of Embodiment 1 of the present disclosure and the other 6 advanced algorithms;

[0038] Fig.11 This is the fourth image comparison example of W-Trans DRN of Embodiment 1 of the present disclosure and 6 other advanced algorithms;

[0039] Fig.12 is a graph showing the relationship between the parameter quantity and the peak signal-to-noise ratio (PSNR) of Example 1 of the present disclosure;

[0040] Fig.13 Refer to the figure for the relationship between PSNR and model running time of Example 1 of the present disclosure;

[0041] Fig.14This is a comparison chart of the network performance of W-TransDRN of Example 1 of the present disclosure and the other three models under different data amounts. DETAILED DESCRIPTION

[0042] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0044] The super-resolution model is an important basis for studying image super-resolution. As mentioned in the background technology, although the super-resolution model has achieved some effects to a certain extent, the single convolutional neural network has obvious deficiencies in parallel computing power and feature extraction capabilities. The network model combining the W-Transformer and the dense residual network proposed in this disclosure can make up for the parallel computing power of the single convolutional neural network, improve the ability to extract key information features, and have better expressiveness in network performance. In this model, the following three technical problems will be solved in this disclosure, and the problems are as follows:

[0045] 1. Whether the network model can significantly improve the resolution of the output image;

[0046] 2. Can the combination of Transformer and convolutional neural network produce a good balance between running time and network performance?

[0047] 3. When the amount of pre-training data is large, whether the network model in the present disclosure has good image processing capabilities.

[0048] Embodiment 1:

[0049] like Figure 1 As shown, this embodiment provides an image super-resolution reconstruction method based on a dense residual network, including:

[0050] Get image information:

[0051] Obtaining an image super-resolution reconstruction result according to the acquired image information and a preset image super-resolution reconstruction model;

[0052] The image super-resolution reconstruction model is obtained by combining a dense residual network and an improved Transformer network; in the improved Transformer network, new weights are continuously given to update the dot product result of the Query and the Key, thereby obtaining a weighted output.

[0053] Specifically, firstly, the dense residual network (DRN) is used as the backbone in this embodiment, aiming to increase the network depth by adding branch layers, thereby improving the network's information and gradient flow capabilities; the dense residual network (DRN) maintains direct connections while increasing branch connections, so as to further enhance the ability to extract key information features and avoid the phenomenon of "information loss or filtering" caused by the network being too deep. Secondly, a new self-attention mechanism, the weighted self-attention mechanism (Weighted Self-Attention), is proposed in this embodiment, which continuously assigns new weights (Weight) to update the dot product results of Query and Key, thereby obtaining a better weighted output. The Weighted Self-Attention aims to focus on key image information features and weaken irrelevant features by continuously updating weights (Weight). The introduction of the weighted self-attention mechanism is conducive to further improving the modeling ability of the self-attention mechanism and expanding the range of the receptive field. Finally, in this embodiment, the improved Transformer (W-Transformer) is combined with DRN to solve the parallel computing power and feature extraction problems brought by the convolutional neural network. In addition, W-Transformer has the ability to mine long-distance dependencies and is not limited by the inductive bias, so it has strong expressiveness. Through experiments, it is further proved that the network structure proposed in this embodiment has better PSNR (peak signal-to-noise ratio) / SSIM (structural similarity) performance and faster convergence speed.

[0054] The overall network diagram is as follows: Figure 1 As shown in the figure, the network framework is introduced in detail. In this embodiment, a residual network is combined with a Transformer to further improve the super-resolution of the image; the dense residual network (DRN) proposed in this embodiment adopts a new form of the traditional residual network, adds branch connections on the basis of the existing direct path, and jumps each convolution block to better extract the semantic features of interest and avoid the problem of key information being filtered or lost during the information extraction process; since the residual network has weak parallel computing and feature extraction capabilities, the dense residual network (DRN) is combined with the Transformer in this embodiment. In order to better improve the network performance and obtain higher quality images, a new form of W-Transformer is proposed in this embodiment. A weighted self-attention mechanism is adopted on the original basis. The weighted self-attention mechanism has good performance in extracting information features such as image texture.

[0055] like Figure 3As shown, the design method of the dense residual network is introduced in detail; in order to improve the feature extraction capability of the attention information, this embodiment adopts a new connection form: on the basis of the traditional residual network (ResNet), on the basis of the original direct connection, each convolution block will generate a new branch path, and each convolution block of the even layer is cross-connected with the convolution blocks of the even layer in the layer below its position, and each convolution block of the odd layer is cross-connected with the convolution blocks of the odd layer below its position, aiming to further improve the information flow capability between the layers. Through the branch connection of the odd and even convolution blocks, the ability to extract the attention information can be better improved, and the attention to irrelevant information can be weakened. In the dense residual network (DRN), this embodiment uses M convolution layers, each of which has a 3×3 convolution block. Each convolution layer will generate M branches, with a total of Even-numbered layers, resulting in Even-numbered branches, odd layers, producing There are an odd number of branches, each of which has a 1×1 convolution block and a global block. Figure 4 Each branch adopts nonlinear transformation H M (·), where H M (·) is a composite function that performs batch normalization, rectified linear units, pooling, or convolution. Finally, the Mth layer receives all the previous layers (x0,…,x M-1 ), namely:

[0056] x M =H M ([x0,…,x M-1 ])(1)

[0057] Among them, [x0,…,x M-1 ] represents the concatenation of the 0th, ..., M-1th branch feature maps; M th represents the Mth layer of convolution; H M Represents the composite function used by M convolutions (the composite function includes operations such as batch normalization, linear correction unit, pooling, and convolution).

[0058] In addition, in the branch connection between the first layer and the last layer, in order to weaken the position feature and enhance the semantic feature, a feature pyramid module (Feature Pyramid Networks: Feature Pyramid Networks, FPN) is added to the branch in this embodiment to fuse low-level features with high-level features, reduce the loss of attention information, thereby obtaining more accurate information features and improving the output capability of high-quality images.

[0059] like Figure 5As shown, the improved Transformer, namely, W-Transformer, is introduced in detail. In this embodiment, one W-Transformer and a dense residual network (DRN) are used to build the overall network, wherein the encoding layer (Encoder Block) in the W-Transformer is composed of 6 encoders (Encoders) stacked, and the decoding layer (Decoder Block) is composed of 6 decoders (Decoders) stacked, and each Encoder-Decoder block diagram is the same. Figure 5 An Encoder-Decoder is shown. The main process of W-Transformer is: input-encoding-decoding-output. Before using W-Transformer, each information point on the image needs to be embedded into a vector with a model sample dimension equal to 512, that is: d model =512. The encoder mainly consists of two parts: weighted self-attention mechanism and feed forward neural network. The decoder mainly consists of three parts: weighted self-attention mechanism, encoder-decoder attention and feed forward neural network. Unlike the encoder, the encoder-decoder attention is added to the decoder, and its main purpose is to help the current node obtain the key semantic features that need to be paid attention to. After obtaining Z1, it is not directly passed into the feed forward neural network, but first passes through the normalization layer, in order to prevent degradation problems in deep neural network training. After that, a multi-layer perceptron (MLP) is added, and the query and key are concatenated and passed through a fully connected layer with the activation function of the hyperbolic tangent function (tanh). The result is then multiplied with the weight matrix (Formula 2) to better complete the interactive task of channel information.

[0060]

[0061] Among them, a is a vector matrix; q is the vector Query in the weighted self-attention mechanism; k is the vector Key in the weighted self-attention mechanism; is the weight matrix; W1 is the weight of the weighted self-attention mechanism; T is the weight.

[0062] Figure 5In the , position information is added to improve the disadvantage that the weighted self-attention mechanism cannot directly obtain position information. The position information uses sine and cosine position encoding, which is generated by using sine and cosine functions of different frequencies. The formula used is as follows:

[0063]

[0064]

[0065] Among them, i is the dimension of the feature point vector; pos is the absolute position of the image feature point in the image.

[0066] like Figure 6 The figure shows the details of the weighted self-attention mechanism. The self-attention mechanism is mainly a weight parameter allocation mechanism, which aims to assist the network model in capturing attention information. The weighted self-attention mechanism introduces three elements: Query, Key, and Value. The similarity between the Query and each group of Keys is calculated to obtain the weight coefficient of each Key, and then the Value is weighted and summed to obtain the final output value.

[0067] Right now:

[0068]

[0069] Assume X∈R n×d is the feature of an input sample sequence, where n is the number of input samples (sequence length), d is the dimension of a single sample, and L x is the length of Source; R n×d is a natural sequence matrix; Query, Key and Value are defined as follows:

[0070] Query: Q = X·W Q ,in

[0071] Key: K=X·W K ,in

[0072] Value: V = X·W V ,in

[0073] Among them, d q =d v =d k =d model =512.

[0074] Figure 6In this embodiment, a weighted block is added to update the dot product result of Query and Key by continuously assigning new weights (Weight) to obtain a better weighted output, which mainly contains all visible input sequence information, and then passes through a normalized exponential function (Softmax) to obtain a new vector matrix L, and then multiply each Value vector with the value in L. The purpose is to focus on key image information features and weaken irrelevant features by continuously updating weights (Weight). Finally, the obtained results are added and summed and then output.

[0075] Step 1: Calculate the score between different input vectors, that is:

[0076] S=Q·K T

[0077] Among them, S represents Scores, which is a vector matrix;

[0078] Step 2: Normalize the Scorces matrix after gradient stabilization, that is:

[0079]

[0080] Among them, S n Represents an n-dimensional matrix;

[0081] Step 3: Use the softmax function for further processing, namely:

[0082] P = softmax(Sn)

[0083] Among them, P represents the vector value after the Softmax function processing;

[0084] Step 4: Finally, the weighted value matrix is ​​obtained, namely:

[0085] Z=V·P

[0086] Among them, Z represents the output weighted value matrix.

[0087] The overall formula for the process is as follows:

[0088]

[0089] In order to further prove that the algorithm (W-Trans DRN) proposed in this embodiment has good network performance and excellent image processing capabilities in super-resolution applications, this embodiment carries out the following five experiments, which aim to show the advantages and competitiveness of the algorithm in this embodiment through comparison of experimental data and visual comparison of images.

[0090] 1. As Figure 7As shown in FIG. 1 , a visual comparison experiment of network performance between four existing algorithms and the algorithm in this embodiment is performed:

[0091] Figure 7 The visual comparison of the W-Trans DRN model and the dense residual network (DRN) model in this embodiment with the other three algorithms (VDSR, LapSRN and IPT) on the BSDS100 dataset when the scale factor is 2 is shown. Figure 7 It is further shown that: (1) the dense residual network (DRN) proposed in this embodiment is significantly better than the traditional ResNet network (VDSR and LapSRN) in terms of PSNR / SSIM performance; (2) compared with the IPT model, it is found that the PSNR value of DRN is not significantly improved (PSNR is only improved by 0.01dB), while its SSIM is significantly better than IPT, indicating that the single DRN network still lacks excellent performance; (3) based on DRN combined with the W-Transformer structure, that is, the algorithm in this paper (W-Trans DRN), compared with VDSR, LapSRN and IPT, the PSNR / SSIM performance is significantly improved, indicating that W-Trans DRN has higher network performance.

[0092] 2. Data comparison experiment between the existing 11 algorithms and the algorithm in this embodiment:

[0093] This embodiment is experimentally compared with 11 existing advanced algorithms: LapSRN, EDSR, SRMDNF, MSRN, MRFN, OISR-RK3, SeaNet, MDCN, MSCN, LW-CSC, and IPT. In this section, this embodiment uses two evaluation indicators, PSNR and SSIM, to comprehensively evaluate the network performance, as shown in Table 1:

[0094] Table 1: Comparison data of the proposed algorithm and existing advanced algorithms

[0095]

[0096]

[0097]

[0098] The experimental data are summarized in Table 1, and all suboptimal results are underlined in the table. It can be seen from Table 1 that when the scale factor is 2, in the four datasets of SET5, SET14, BSDS100, and Urban100, the PSNR values ​​of W-Trans DRN are respectively higher than the suboptimal results by 0.51dB, 1.01dB, 1.13dB, and 1.12dB, and the SSIM values ​​of W-Trans DRN are respectively higher by 0.009dB, 0.0094dB, 0.0104dB, and 0.0047dB; when the scale factor is 3, in the four datasets of SET5, SET14, BSDS100, and Urban100, the PSNR values ​​of W-Trans DRN are respectively higher than the suboptimal results by 0.3dB, 0.48dB, 0.84dB, and 0.49dB, and the SSIM values ​​of W-Trans DRN are respectively higher than the suboptimal results by The SSIM values ​​of DRN are higher by 0.0014dB, 0.0096dB, 0.0098dB and 0.0112dB respectively. When the scale factor is 4, in the four datasets of SET5, SET14, BSDS100 and Urban100, compared with the suboptimal results, the PSNR values ​​of W-Trans DRN are higher by 1.37dB, 1.1dB, 1.17dB and 1.27dB respectively, and the SSIM values ​​of W-Trans DRN are higher by 0.0130dB, 0.0023dB, 0.0116dB and 0.0163dB respectively.

[0099] In summary, the W-Trans DRN algorithm proposed in this embodiment achieves good results when the scale factors are 2, 3, and 4.

[0100] Figure 8 , Fig. 9 , Fig.10 and Fig.11 The images reconstructed by the algorithm model proposed in this embodiment and six other advanced algorithms are shown. This embodiment uses the BSDS100 and Urban100 benchmark datasets to reconstruct images when the scale factors are 2 and 4. Figure 8 , Fig. 9 , Fig.10 and Fig.11 It can be seen that compared with other methods, the method in this embodiment is successful in improving the clarity of image details, further indicating that the W-Trans DRN algorithm is highly reliable in recovering complex textures from low-resolution images.

[0101] 3. Model parameter comparison and analysis experiment:

[0102] exist Fig.12In this embodiment, the balance between performance and model size is demonstrated. When the scale factor is 2 on the BSDS100 benchmark dataset, the algorithm proposed in this embodiment is tested and compared with the other 6 algorithms (LapSRN, DRCN, MDSR, EDSR, OISR-RK3 and IPT) on the same configuration. As can be seen from the figure, compared with LapSRN, DRCN and MDSR, although the algorithm proposed in this embodiment has a higher number of parameters, its network performance is significantly better than these three algorithms (PSNR W-Trans DRN =33.61dB); Compared with EDSR, OISR-RK3 and IPT, the algorithm proposed in this embodiment is superior to these three algorithms in terms of parameter quantity and network performance. Therefore, the algorithm in this embodiment can achieve a better balance between network performance and parameter quantity.

[0103] 4. Model running time comparison analysis experiment:

[0104] Fig.13 In the figure, the balance between performance and running time is shown. During the experiment, this embodiment uses the SET14 benchmark data set with a scale factor of 4 and performs tests and comparisons on the same configuration. The algorithms involved in the comparison include: LapSRN, LW-CSC, MDCN, OISR-RK3, EDSR, IPT and W-Trans DRN (ours). As can be seen from the figure, LapSRN and LW-CSC are superior to the algorithm in this embodiment in terms of running time, but their PSNR values ​​are significantly lower than W-Trans DRN (PSNRW-Trans DRN=30.11dB); compared with the other four algorithms, the algorithm in this embodiment is superior to the other four algorithms in terms of running time and network performance. Therefore, the algorithm proposed in this embodiment shows a better balance between network performance and running time.

[0105] 5. Data percentage analysis experiment:

[0106] In this example, an experiment on the impact of data percentage was conducted to evaluate the effectiveness of the W-Transformer architecture. The experimental results analyzed the performance improvement of the pre-training based on CNN models (EDSR, LW-CSC), Transformer models (IPT), and Trans-CNN models (W-Trans DRN). In this section, this example uses 20%, 40%, 60%, 80%, and 100% synthetic percentages of the ImageNet dataset to conduct an impact analysis experiment on the amount of data. The experimental results are shown in Figure 2. Fig.14 shown. Fig.14 It can be seen that:

[0107] (1) When the model is not pre-trained or pre-trained with a small amount (<60%), the CNN-based model shows better network performance than the Transformer-based and Trans-CNN-based models;

[0108] (2) When the amount of training data increases (>60%), the performance of the Transformer-based model and the Trans-CNN-based model (W-Trans DRN) is significantly better than that of the CNN-based model;

[0109] (3) Compared with the single Transformer model (IPT), the model based on the combination of Transformer and CNN, namely W-Trans DRN, has always had better network performance (PSNR) than the IPT model. Moreover, when the amount of training data is >50%, the W-TransDRN model completely overwhelms the CNN-based model and exhibits more excellent network performance.

[0110] Embodiment 2:

[0111] This embodiment provides an image super-resolution reconstruction system based on a dense residual network, including a data collection module and a super-resolution reconstruction module;

[0112] The data collection module is configured to: obtain image information:

[0113] The super-resolution reconstruction module is configured to: obtain an image super-resolution reconstruction result according to the acquired image information and a preset image super-resolution reconstruction model;

[0114] The image super-resolution reconstruction model is obtained by combining a dense residual network and an improved Transformer network; in the improved Transformer network, new weights are continuously assigned to update the dot product result of the Query and the Key, thereby obtaining a weighted output.

[0115] Embodiment 3:

[0116] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the image super-resolution reconstruction method based on a dense residual network described in Example 1 are implemented.

[0117] Embodiment 4:

[0118] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the image super-resolution reconstruction method based on a dense residual network described in Embodiment 1 are implemented.

[0119] The above description is only a preferred embodiment of the present embodiment and is not intended to limit the present embodiment. For those skilled in the art, the present embodiment may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present embodiment shall be included in the protection scope of the present embodiment.

Claims

1. An image super-resolution reconstruction method based on a dense residual network, characterized in that: include: Get image information; Obtaining an image super-resolution reconstruction result according to the acquired image information and a preset image super-resolution reconstruction model; The image super-resolution reconstruction model is obtained by combining a dense residual network and an improved Transformer network; The dense residual network adds a dense layer on the basis of the existing direct path, and skips each convolution block; Each convolution block generates a new branch path. Each branch has a 1×1 convolution block and a global block. Each even-numbered convolution block is cross-connected with the even-numbered convolution blocks in the layers below it, and each odd-numbered convolution block is cross-connected with the odd-numbered convolution blocks in the layers below it. In the improved Transformer network, new weights are continuously assigned to update the dot product result of Query and Key, so as to obtain a weighted output. Specifically, the improved Transformer network includes an encoding layer and a decoding layer, the encoding layer includes 6 encoders, and the decoding layer includes 6 decoders; the encoder includes a weighted self-attention mechanism and a feedforward neural network; the decoder includes a weighted self-attention mechanism, encoding-decoding attention and a feedforward neural network; the weighted self-attention mechanism is a weight parameter allocation mechanism, which adds weight blocks and continuously assigns new weights to update the dot product result of Query and Key to obtain a weighted output, which includes all visible input sequence information, and then passes through a normalized exponential function to obtain a new vector matrix, and then multiplies each Value vector by the value in the new vector matrix.

2. The image super-resolution reconstruction method based on dense residual network according to claim 1, characterized in that: Each branch adopts nonlinear transformation; in the branch connection between the first layer and the last layer, a feature pyramid module is added.

3. Image super-resolution reconstruction system based on dense residual network, characterized by: It includes a data collection module and a super-resolution reconstruction module; The data collection module is configured to: acquire image information; The super-resolution reconstruction module is configured to: obtain an image super-resolution reconstruction result according to the acquired image information and a preset image super-resolution reconstruction model; The image super-resolution reconstruction model is obtained by combining a dense residual network and an improved Transformer network; The dense residual network adds a dense layer on the basis of the existing direct path, and skips each convolution block; Each convolution block generates a new branch path. Each branch has a 1×1 convolution block and a global block. Each even-numbered convolution block is cross-connected with the even-numbered convolution blocks in the layers below it, and each odd-numbered convolution block is cross-connected with the odd-numbered convolution blocks in the layers below it. In the improved Transformer network, new weights are continuously assigned to update the dot product result of Query and Key, so as to obtain a weighted output. Specifically, the improved Transformer network includes an encoding layer and a decoding layer, the encoding layer includes 6 encoders, and the decoding layer includes 6 decoders; the encoder includes a weighted self-attention mechanism and a feedforward neural network; the decoder includes a weighted self-attention mechanism, encoding-decoding attention and a feedforward neural network; the weighted self-attention mechanism is a weight parameter allocation mechanism, which adds weight blocks and continuously assigns new weights to update the dot product result of Query and Key to obtain a weighted output, which includes all visible input sequence information, and then passes through a normalized exponential function to obtain a new vector matrix, and then multiplies each Value vector by the value in the new vector matrix.

4. A computer-readable storage medium having a computer program stored thereon for calculating fingerprint similarity, characterized in that: When the program is executed by a processor, the steps of the image super-resolution reconstruction method based on a dense residual network as described in any one of claims 1-2 are implemented.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the image super-resolution reconstruction method based on a dense residual network as described in any one of claims 1-2 are implemented.

Images