Lightweight mine image super-resolution reconstruction system and method based on progressive receptive field
By adopting the gradual receptive field design in the lightweight image super-resolution reconstruction system, the receptive field of the convolution chain is gradually adjusted, and the overall structural information and local detail characteristics of the image are effectively integrated, and the receptive field in the existing technology is solved. The problem of limitation improves super-resolution image reconstruction performance.
Patent Information
- Application Number
- CN202410579283.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-11
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing lightweight image super-resolution reconstruction methods are limited in receptive fields when capturing global image information, making it difficult to effectively integrate shallow and deep features, resulting in insufficient super-resolution image reconstruction performance.
The lightweight mine image super-resolution reconstruction system adopts a progressive receptive field, and gradually adjusts the receptive field size through step-by-step upper and lower dual-path convolution chains, combining shallow feature extraction module, deep feature extraction module and multi-dimensional latent feature fusion module , realize the fusion of the overall structure information of the image and the local detail features.
The super-resolution image reconstruction performance is significantly improved, the information flow and interaction between multi-scale convolutional layers is enhanced, the feature utilization rate is improved, and the balance of parameter quantity, calculation quantity and reconstruction effect is achieved.
Smart Images

Figure CN118918005A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image super-resolution reconstruction, and in particular to a lightweight mine image super-resolution reconstruction system and method based on a progressive receptive field. Background Art
[0002] Image super-resolution technology plays an important role in the fields of medical imaging, smart mines, and computational photography. It aims to upgrade low-resolution images to high resolution to enhance the clarity and details of the image. With the development of information technology and the improvement of hardware level, digital images have gradually become one of the main communication media, and people have also put forward higher requirements for image quality. However, in harsh working environments such as underground coal mines, the images collected are often of low resolution due to the influence of image acquisition equipment, network transmission media, and lighting conditions, which greatly interferes with subsequent personnel safety detection and equipment monitoring functions. At the same time, the construction of smart mines has high real-time requirements. Therefore, how to efficiently reconstruct low-resolution (LR) images into high-resolution (HR) images has become one of the current research hotspots.
[0003] Among the main super-resolution reconstruction methods currently used, super-resolution networks often focus on image reconstruction performance and ignore their applications in mobile terminals in real life; therefore, lightweight image super-resolution reconstruction methods have attracted widespread attention due to their advantages of being easy to deploy on mobile edge devices. However, existing lightweight image super-resolution reconstruction methods often use a single large kernel convolution to capture global information or use multiple layers of small-size convolution to extract deeper local features, which limits the overall receptive field and makes it difficult to obtain global information of the image.
[0004] In this regard, in order to solve the problems existing in the prior art, technical personnel in this field have made many efforts. For example, Chinese patent application CN117237190B proposes a lightweight image super-resolution reconstruction system and method for edge mobile devices. By combining distillation feature fusion and pixel attention mechanism, redundant information is reduced, high-frequency information in the image is efficiently extracted, and better super-resolution reconstruction results are provided. However, there is a lack of deep interactive fusion between shallow features and deep features, and the super-resolution image reconstruction performance needs to be further improved. Summary of the invention
[0005] To solve the above problems, the present invention provides a lightweight mine image super-resolution reconstruction method based on progressive receptive field, which gradually adjusts the size of the receptive field through a step-by-step upper and lower dual-path convolution chain, effectively integrating the overall structural information and local detail features of the image, and improving the super-resolution image reconstruction performance.
[0006] To achieve the above object, the present invention proposes the following technical solutions:
[0007] A lightweight mine image super-resolution reconstruction system based on progressive receptive field includes a shallow feature extraction module, a deep feature extraction module, a multi-dimensional latent feature fusion module MLFFB and a reconstruction module; the shallow feature extraction module includes a short-range dense connection block and a convolution layer for extracting a low-resolution image from an input LR The shallow features are extracted from the deep feature module; the deep feature module includes multiple cascaded progressive receptive field distillation blocks, the progressive receptive field distillation blocks include attention modules and multiple progressive step convolution chains; the progressive step convolution includes an upper step convolution chain and a lower step convolution chain; it is used to further process the input shallow features to extract deep features; the attention module includes enhanced spatial attention and spatial adaptive modulation; the multi-dimensional potential feature fusion module is used to fully fuse the potential features of different dimensions in multiple progressive receptive field distillation blocks, and combine the fused features with the shallow features; the reconstruction module is used to reconstruct the output features of the multi-dimensional potential feature fusion module to obtain the final high-resolution reconstructed image I SR .
[0008] A lightweight mine image super-resolution reconstruction method based on progressive receptive field, the method comprising the following steps:
[0009] Step S1: high resolution image I HR The low-resolution image I obtained by bicubic downsampling LR ; Through the shallow feature extraction module, I LR Perform shallow feature extraction, and the extraction process is expressed as: F 0 =H SDB (Conv 3 (I LR )); where H SDB (·) represents a short-range dense connection block, Conv 3 (·) represents a convolutional layer with a kernel size of 3×3;
[0010] Step S2: Process the input shallow features through the deep feature extraction module to extract deep features; for the i-th progressive receptive field distillation block, the deep feature extraction process is expressed as: in, represents the i-th progressive receptive field distillation block, F i is the output feature of the i-th progressive receptive field distillation block. When i=1, F i Represents shallow features, i.e., F 0 ;
[0011] Step S3: The multi-dimensional latent feature fusion module MLFFB fully fuses the latent features of different dimensions in multiple progressive receptive field distillation blocks and combines them with shallow features, which is expressed as: Among them, F MLFFB is the output feature after fusion, H MLFFB (·) represents latent feature fusion modules. They are the fusion results of the potential features of the corresponding dimensions of the progressive receptive field distillation block at different depths;
[0012] Step S4: Reconstruct the output features of the multi-dimensional potential feature fusion module through the reconstruction module to obtain the final high-resolution reconstructed image I SR ; The calculation process is expressed as: I SR =H PixeShuffle (Conv 3 (F MLFFB )); where I SR Represents the final reconstruction result of the network, Conv 3 (·) represents a 3×3 convolution, H PixeShuffle (·) is sub-pixel convolution.
[0013] Furthermore, in step S1, the processing process of the short-range dense connection block for the input image is specifically expressed as follows: Among them, DW i (·) represents the i-th depthwise convolution; is the output feature of the previous depth convolution. When i=1, The output features of the first convolution in the shallow feature extraction module are integrated with feature information extracted from different channels using point-by-point convolution; PW(·) represents point-by-point convolution.
[0014] Furthermore, in step S2, the input shallow features are processed by a deep feature extraction module, and the specific method for extracting deep features is:
[0015] Step S2-1: Extract the output features F of the shallow feature extraction module 0 Input 1×1 convolution and progressive step convolution chain for feature preservation and feature extraction respectively, and repeat the operation many times; it can be expressed as: Among them, F distilled_1 、F distilled_2 、F distilled_3 、F distilled_4 They represent the features of the input feature retained by the 1×1 convolution; F coarse_1 、F coarse_2 、F coarse_3 They represent the features extracted by the progressive step convolution chain of the input features; DL j (·) represents the jth progressive staircase convolution chain, j = 1, 2, 3, 4, RL i(·) represents the i-th 1×1 convolution, which is used for dimensionality reduction and saves the image features at the current depth, i=1,2,3;
[0016] Step S2-2: Concatenate the features retained multiple times and the output features of the last progressive staircase convolution chain according to the channel dimension, and use 1×1 convolution for dimensionality reduction; expressed as: F distilled =Conv 1 (Concat(F distilled_1 ,...,F distilled_4 )); Concat(·) represents concatenation operation according to channel dimension, Conv 1 (·) is a 1×1 convolutional layer for dimensionality reduction;
[0017] Step S2-3: Input the reduced dimension features into the attention module, focusing on the image spatial features first globally and then locally; expressed as: F attention =H SAFM (H ESA (F distilled )); where H SAFM (·) and H ESA (·) denotes spatial adaptive modulation SAFM and enhanced spatial attention ESA, respectively;
[0018] Step S2-4: At the end of the progressive receptive field distillation block, a 3×3 standard convolution is used to further extract the output features of the attention module, and a jump connection is introduced to combine the input features of the progressive receptive field distillation block and the final output features after further extraction, expressed as: F out =Conv 3 (F attention ), where F attention Represents the output features of the attention module.
[0019] Furthermore, in step S2-1, the progressive ladder convolution chain is used to extract the output feature F of the shallow feature extraction module. 0 The specific method for feature extraction is:
[0020] Step S2-1-1: Given input feature F coarse It is divided into two parts according to the channel dimension, expressed as: F s1 ,F s2 =Split(F coarse );where Fs 1 ,Fs 2 Represent the first part of input features and the second part of input features respectively;
[0021] Step S2-1-2: The upper step convolution chain sequentially uses a depthwise separable convolution with a convolution kernel of 5×5, a depthwise separable convolution with a convolution kernel of 3×3, and a 1×1 convolution to increase the receptive field, extract more global image features, and combine the convolution results with the input features of the upper step convolution chain after being processed by the activation function to gradually refine the image features; expressed as: Among them, F S15 is the calculated intermediate value obtained after using a depth-separable convolution with a convolution kernel of 5×5; F S13 is the calculated intermediate value obtained after using a depthwise separable convolution with a convolution kernel of 3×3; σ(·) represents the GELU activation function; DSC 5 (·) indicates a depth-wise separable convolution with a convolution kernel of 5×5, DSC 3 (·) indicates that the convolution is a 3×3 depth-separable convolution, Conv 1 (·) represents 1×1 convolution; F up Represents the output of the upper ladder convolution chain;
[0022] Step S2-1-3: The lower staircase convolution chain uses 1×1 convolution, 3×3 depth-separable convolution and 5×5 depth-separable convolution in sequence to focus on the overall structural information, and concatenates the output results of the upper staircase convolution chain and the lower staircase convolution chain to obtain image features containing local details and overall information; expressed as: Among them, F S21 is the calculated intermediate value obtained after using 1×1 convolution; F S23 is the calculated intermediate value obtained after using a depth-separable convolution with a convolution kernel of 3×3; F down Represents the output result of the lower staircase convolution chain;
[0023] Step S2-1-4: Combine the image features containing local details and overall information with the original input features F coarse Add together, and the calculation formula is: F PRFFM =Concat(F up ,F down )+F coarse ; Concat(·) represents the concatenation operation based on the channel dimension.
[0024] Furthermore, in step S3, the multi-dimensional latent feature fusion module MLFFB fully fuses the latent features of different dimensions in multiple progressive receptive field distillation blocks and combines them with the shallow features. The specific method is:
[0025] Step S3-1: Fuse the features of corresponding dimensions in different progressive receptive field distillation blocks, expressed as: in, represents the x-dimensional latent features F from the first to the fifth x ; and Represents the y-dimensional potential features F in multiple y and z-dimensional latent features F z ;
[0026] Step S3-2: Concatenation, using 1×1 convolution and adaptive pooling layers in sequence to integrate feature information of different dimensions and reduce the spatial dimension of the feature map;
[0027] Step S3-3: Use a set of depth-wise separable convolutions with kernel size 3×3 and Softmax activation function to dynamically assign weights to the two sets of features and combine them with Add together to achieve full integration of multi-dimensional potential features;
[0028] Step S3-4: The fused result F MLFFB With shallow features F 0 After addition, the data are input into the reconstruction module for image reconstruction.
[0029] The technical solution of the present invention has the following beneficial effects:
[0030] 1. The present invention takes the information distillation structure as the framework, designs multiple cascaded progressive receptive field distillation blocks for deep feature extraction, and combines the overall structural information and local detail features of the image by gradually changing the receptive field size through the progressive step convolution chain. It can effectively fuse the overall structural information and local detail features of the image, enhance the information flow and interaction between multi-scale convolutional layers, and greatly improve the performance of super-resolution image reconstruction.
[0031] 2. The extracted features of the present invention can be divided into three dimensions, namely, distillation features F x , attention feature F y And the final output feature F z After multiple distillation operations, the retained features are fused and spliced to obtain F x , the attention mechanism is used to dynamically select representative features, the convolutional layer is used to further extract the deep information of the image, and then the intermediate features of the progressive receptive field distillation block are fused through the multi-dimensional potential feature fusion module, making full use of F x , F y and F z The correlation and distinct dimensionality differences among the three further improve feature utilization and the performance of super-resolution image reconstruction.
[0032] 3. In the present invention, the information distillation structure is used as the framework to design a progressive step convolution chain and a multi-dimensional latent feature fusion module; and the enhanced spatial attention and spatial adaptive modulation mechanism are introduced to achieve a balance between the number of parameters, the amount of calculation and the reconstruction effect, thereby improving the image reconstruction performance while ensuring the stability of the image reconstruction process. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments and in conjunction with the accompanying drawings.
[0034] Figure 1 This is a structural framework diagram of a lightweight mine image super-resolution reconstruction system based on progressive receptive field.
[0035] Figure 2 It is a flow chart of a lightweight mine image super-resolution reconstruction method based on progressive receptive field.
[0036] Figure 3 It is a structural diagram of the progressive receptive field distillation block.
[0037] Figure 4 It is a structural diagram of the progressive step convolution chain.
[0038] Figure 5 It is the structural diagram of the multi-dimensional latent feature fusion module.
[0039] Figure 6 This is a visual comparison chart of the reconstruction method proposed in the present invention and other comparison methods on urban100img015 at 4 times reconstruction.
[0040] Figure 7 This is a visual comparison chart of the reconstruction method proposed in the present invention and other comparison methods on urban100img078 at 4 times reconstruction. DETAILED DESCRIPTION
[0041] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.
[0042] like Figure 1 As shown, this embodiment proposes a lightweight mine image super-resolution reconstruction system based on progressive receptive field, which uses a multi-scene data set to train a network model and applies it to the coal mining field to achieve efficient reconstruction of mine images; the system includes a shallow feature extraction module, a deep feature extraction module, a multi-dimensional potential feature fusion module and a reconstruction module.
[0043] The shallow feature extraction module includes 3×3 convolution and short-range dense connection blocks. The short-range dense connection block consists of 3 depth convolutions with a kernel size of 3×3 and 1 point-by-point convolution through short-range dense connection. The shallow feature extraction module extracts the low-resolution image I from the input by using 3×3 convolution and short-range dense connection blocks. LR Extracting shallow features from the image can effectively enhance the transmission of shallow information and guide deep features to complete the final image reconstruction.
[0044] The deep feature extraction module consists of five cascaded progressive receptive field distillation blocks. The shallow features are input to the deep feature extraction module, and the output of the previous one is used as the input of the next one. The progressive receptive field distillation block contains three progressive step convolution chains and an attention module composed of enhanced spatial attention and spatial adaptive modulation mechanism. Among them, the progressive step convolution chain includes an upper step convolution chain and a lower step convolution chain. The upper branch gradually reduces the receptive field starting from 5×5 convolution, and the lower branch convolution chain gradually increases the receptive field starting from 1×1 convolution. The output results of the upper and lower branches are spliced together to obtain image features containing local details and overall information, thereby gradually refining the image features.
[0045] The multi-dimensional latent feature fusion module is used to fully fuse the latent features of different dimensions in multiple progressive receptive field distillation blocks, and combine the fused features with shallow features.
[0046] The reconstruction module consists of a 3×3 convolution and a sub-pixel convolution, which is used to reconstruct the output features of the multi-dimensional latent feature fusion module to obtain the final high-resolution reconstructed image I SR .
[0047] Preferably, Figure 2 As shown, this embodiment also proposes a lightweight mine image super-resolution reconstruction method based on progressive receptive field, which is implemented based on the above-mentioned lightweight mine image super-resolution reconstruction system based on progressive receptive field, and includes the following steps:
[0048] Step S1: high resolution image I HR The low-resolution image I obtained by bicubic downsampling LR ; Through the shallow feature extraction module, I LR Perform shallow feature extraction, and the extraction process is expressed as: F 0 =H SDB (Conv 3 (I LR )); where H SDB (·) represents a short-range dense connection block, Conv 3(·) represents a convolution layer with a kernel size of 3×3. The specific method is: the input image is processed by a short-range dense connection block consisting of three 3×3 depthwise convolutions and one point-by-point convolution. The processing process is expressed as: Among them, DW i (·) represents the i-th depthwise convolution; is the output feature of the previous depth convolution. When i=1, The output features of the first convolution in the shallow feature extraction module are integrated with feature information extracted from different channels using point-by-point convolution; PW(·) represents point-by-point convolution.
[0049] Step S2: Process the input shallow features through the deep feature extraction module to extract deep features; for the i-th progressive receptive field distillation block, the deep feature extraction process is expressed as: in, represents the i-th progressive receptive field distillation block, F i is the output feature of the i-th progressive receptive field distillation block. When i=1, F i Represents shallow features, i.e., F 0 .
[0050] Specifically, Figure 3 As shown in Figure 1, the specific process of extracting deep features through the deep feature extraction module is as follows:
[0051] S2-1: The output feature F of the shallow feature extraction module 0 Input 1×1 convolution and progressive step convolution chain for feature preservation and feature extraction respectively, and repeat the operation many times; it can be expressed as: Among them, F distilled_1 、F distilled_2 、F distilled_3 、F distilled_4 They represent the features of the input feature retained by the 1×1 convolution; F coarse_1 、F coarse_2 、F coarse_3 They represent the features extracted by the progressive step convolution chain of the input features; DL j (·) represents the jth progressive staircase convolution chain, j = 1, 2, 3, 4, RL i (·) represents the i-th 1×1 convolution, which is used for dimensionality reduction and preserves the image features at the current depth, i = 1, 2, 3.
[0052] Specifically, Figure 4 As shown, the progressive ladder convolution chain is used to extract the output features F of the shallow feature extraction module. 0 The specific method for feature extraction is:
[0053] Step S2-1-1: Given input feature F coarse It is divided into two parts according to the channel dimension, expressed as: F s1 ,F s2 =Split(F coarse );where Fs 1 ,Fs 2 Represent the first part of input features and the second part of input features respectively.
[0054] Step S2-1-2: The upper step convolution chain sequentially uses a depthwise separable convolution with a convolution kernel of 5×5, a depthwise separable convolution with a convolution kernel of 3×3, and a 1×1 convolution to increase the receptive field, extract more global image features, and combine the convolution results with the input features of the upper step convolution chain after being processed by the activation function to gradually refine the image features; expressed as: Among them, F S15 is the calculated intermediate value obtained after using a depth-separable convolution with a convolution kernel of 5×5; F S13 is the calculated intermediate value obtained after using a depthwise separable convolution with a convolution kernel of 3×3; σ(·) represents the GELU activation function; DSC 5 (·) indicates a depth-wise separable convolution with a convolution kernel of 5×5, DSC 3 (·) indicates that the convolution is a 3×3 depth-separable convolution, Conv 1 (·) represents 1×1 convolution; F up Represents the output of the upper ladder convolution chain;
[0055] Step S2-1-3: The lower staircase convolution chain uses 1×1 convolution, 3×3 depth-separable convolution and 5×5 depth-separable convolution in sequence to focus on the overall structural information, and concatenates the output results of the upper staircase convolution chain and the lower staircase convolution chain to obtain image features containing local details and overall information; expressed as: Among them, F S21 is the calculated intermediate value obtained after using 1×1 convolution; F S23 is the calculated intermediate value obtained after using a depth-separable convolution with a convolution kernel of 3×3; F down Represents the output result of the lower staircase convolution chain;
[0056] Step S2-1-4: Combine the image features containing local details and overall information with the original input features F coarse Add together, and the calculation formula is: F PRFFM =Concat(F up ,F down )+F coarse ; Concat(·) represents the concatenation operation based on the channel dimension.
[0057] Step S2-2: Concatenate the features retained multiple times and the output features of the last progressive staircase convolution chain according to the channel dimension, and use 1×1 convolution for dimensionality reduction; expressed as: F distilled =Conv 1 (Concat(F distilled_1 ,...,F distilled_4 )); Concat(·) represents concatenation operation according to channel dimension, Conv 1 (·) is a 1×1 convolutional layer for dimensionality reduction.
[0058] Step S2-3: Input the reduced dimension features into the attention module, focusing on the image spatial features first globally and then locally; expressed as: F attention =H SAFM (H ESA (F distilled )); where H SAFM (·) and H ESA (·) denote spatial adaptive modulation SAFM and enhanced spatial attention ESA, respectively.
[0059] Step S2-4: At the end of the progressive receptive field distillation block, a 3×3 standard convolution is used to further extract the output features of the attention module, and a jump connection is introduced to combine the input features of the progressive receptive field distillation block and the final output features after further extraction, expressed as: F out =Conv 3 (F attention ), where F attention Represents the output features of the attention module.
[0060] Step S3: The multi-dimensional latent feature fusion module MLFFB fully fuses the latent features of different dimensions in multiple progressive receptive field distillation blocks and combines them with shallow features, which is expressed as: Among them, F MLFFB is the output feature after fusion, H MLFFB (·) represents latent feature fusion modules. They are the fusion results of the potential features of the corresponding dimensions of the progressive receptive field distillation block at different depths.
[0061] Specifically, Figure 5 As shown in the figure, the multi-dimensional latent feature fusion module MLFFB fully integrates the latent features of different dimensions in multiple progressive receptive field distillation blocks and combines them with shallow features in the following specific method:
[0062] Step S3-1: Fuse the features of corresponding dimensions in different progressive receptive field distillation blocks, expressed as: in, represents the x-dimensional latent features F from the first to the fifth x ; and Represents the y-dimensional potential features F in multiple y and z-dimensional latent features F z .
[0063] Step S3-2: Concatenation, using 1×1 convolution and adaptive pooling layers in sequence to integrate feature information of different dimensions and reduce the spatial dimension of the feature map.
[0064] Step S3-3: Use a set of depth-wise separable convolutions with kernel size 3×3 and Softmax activation function to dynamically assign weights to the two sets of features and combine them with Add together to achieve full integration of multi-dimensional potential features; expressed as: Among them, σ(·) represents the GELU activation function, DSC 3 (·) is a depth-wise separable convolution with a kernel size of 3×3. and A set of weights generated by the Softmax activation function.
[0065] Step S3-4: The fused result F MLFFB With shallow features F 0 After addition, the image is reconstructed in the reconstruction module.
[0066] Step S4: The reconstruction module uses 3×3 convolution and sub-pixel convolution to reconstruct the output features of the multi-dimensional potential feature fusion module to obtain the final high-resolution reconstructed image I SR ; The calculation process is expressed as: I SR =H PixeShuffle (Conv 3 (F MLFFB )); where I SR Represents the final reconstruction result of the network, Conv 3 (·) represents a 3×3 convolution, H PixeShuffle (·) is sub-pixel convolution.
[0067] In order to evaluate the feasibility and effectiveness of the method of the present invention, this embodiment is experimentally verified, and the specific contents are as follows:
[0068] The method of the present invention is compared with the current mainstream lightweight methods and some classic methods on the Set5, Set14, B100 and Urban100 data sets in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). The comparison methods include: IMDN, RFDN, SMSR, FMEN, BSRN, LatticeNet, SAFMN, FDSCSR, DRSAN. The analysis and comparison results are as follows:
[0069] When comparing the ability to restore the detail texture of the image when the scale is increased, each method is used to perform image super-resolution reconstruction at a 4-fold scaling factor, and a quantitative analysis is performed. The statistical parameter quantity, PSNR and SSIM indicator data are shown in Table 1 below.
[0070] Comparative analysis of the data in Table 1 shows that under a 4-fold scaling factor, the PSNR and SSIM indicators of the method proposed in the present invention are significantly higher than those of other models, and at the same time, it has an advantage in parameter quantity. On the Urban100 dataset with rich details, the method proposed in the present invention is compared with DRSAN with more than 1M parameters. The PSNR of the method proposed in the present invention on Urban100 is improved by 0.21dB, and the SSIM is improved by 0.0054, while the parameter quantity is only 70.6% of DRSAN, indicating that it is significantly better than other models in terms of stripes and mesh patterns with rich image texture details. .
[0071]
[0072]
[0073] like Figure 6 and Figure 7 As shown in Figure 1, in order to further verify the superiority of the super-resolution image reconstruction method proposed in the present invention, when the scaling factor is 4, the reconstructed images of BSRN, SAFMN, LatticeNet, DRSAN, ESRT and NGSwin on the Urban100 dataset are selected for subjective visual comparison. Figure 6 The top three layers of grids have a lot of vertical lines inside the grids, and the grids are divided by horizontal lines. The image reconstructed by the method described in the present invention is closest to the HR image, and the three layers of grids are clearly distinguished, while the grid shapes reconstructed by other methods are fuzzy and difficult to recognize. Figure 7 In the HR image, there are multiple inclined lines. Among the multiple comparison images, only the image reconstructed by the method of the present invention has a clear straight line shape. This further illustrates the advantage of the method of the present invention in restoring image details and having a better visual reconstruction effect. This shows that the method proposed by the present invention has obvious superiority in image super-resolution reconstruction compared with other methods.
[0074] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A lightweight mine image super-resolution reconstruction system based on progressive receptive field, It is characterized in that It includes shallow feature extraction module, deep feature extraction module, multi-dimensional latent feature fusion module MLFFB and reconstruction module; The shallow feature extraction module includes a short-range dense connection block and a convolutional layer, which is used to extract the low-resolution image I from the input LR Extract shallow features from The deep feature module includes a plurality of cascaded progressive receptive field distillation blocks, the progressive receptive field distillation block includes an attention module and a plurality of progressive step convolution chains; the progressive step convolution includes an upper step convolution chain and a lower step convolution chain; used to further process the input shallow features to extract deep features; the attention module includes enhanced spatial attention and spatial adaptive modulation; The multi-dimensional latent feature fusion module is used to fully fuse the latent features of different dimensions in multiple progressive receptive field distillation blocks, and combine the fused features with shallow features; The reconstruction module is used to reconstruct the output features of the multi-dimensional potential feature fusion module to obtain the final high-resolution reconstructed image I SR .
2. A lightweight mine image super-resolution reconstruction method based on progressive receptive field, which is implemented based on the lightweight mine image super-resolution reconstruction system based on progressive receptive field according to claim 1. It is characterized in that The method comprises the following steps: Step S1: high resolution image I HR The low-resolution image I obtained by bicubic downsampling LR ; Through the shallow feature extraction module, I LR Perform shallow feature extraction, and the extraction process is expressed as: F 0 =H SDB (Conv 3 (I LR )); where H SDB (·) represents a short-range dense connection block, Conv 3 (·) represents a convolutional layer with a kernel size of 3×3; Step S2: Process the input shallow features through the deep feature extraction module to extract deep features; for the i-th progressive receptive field distillation block, the deep feature extraction process is expressed as: in, represents the i-th progressive receptive field distillation block, F i is the output feature of the i-th progressive receptive field distillation block. When i=1, F i Represents shallow features, i.e., F 0 ; Step S3: The multi-dimensional latent feature fusion module MLFFB fully fuses the latent features of different dimensions in multiple progressive receptive field distillation blocks and combines them with shallow features, which is expressed as: Among them, F MLFFB is the output feature after fusion, H MLFFB (·) represents latent feature fusion modules. They are the fusion results of the potential features of the corresponding dimensions of the progressive receptive field distillation block at different depths; Step S4: Reconstruct the output features of the multi-dimensional potential feature fusion module through the reconstruction module to obtain the final high-resolution reconstructed image I SR ; The calculation process is expressed as: I SR =H PixeShuffle (Conv 3 (F MLFFB )); where I SR Represents the final reconstruction result of the network, Conv 3 (·) represents a 3×3 convolution, H PixeShuffle (·) is sub-pixel convolution.
3. According to claim 2, the lightweight mine image super-resolution reconstruction method based on progressive receptive field, It is characterized in that In step S1, the processing process of the short-range dense connection block for the input image is specifically expressed as: Among them, DW i (·) represents the i-th depthwise convolution; is the output feature of the previous depth convolution. When i=1, The output features of the first convolution in the shallow feature extraction module are integrated with feature information extracted from different channels using point-by-point convolution; PW(·) represents point-by-point convolution.
4. According to claim 2, the lightweight mine image super-resolution reconstruction method based on progressive receptive field, It is characterized in that In step S2, the input shallow features are processed by the deep feature extraction module. The specific method of extracting deep features is: Step S2-1: Extract the output features F of the shallow feature extraction module 0 Input 1×1 convolution and progressive step convolution chain for feature preservation and feature extraction respectively, and repeat the operation many times; it can be expressed as: Among them, F distilled_1 、F distilled_2 、F distilled_3 , respectively represent the features retained by the input features after 1×1 convolution; F distilled_4 It is the feature preserved by the short-range dense block SDB; F coarse_1 、F coarse_2 、F coarse_3 They represent the features extracted by the progressive step convolution chain of the input features; DL j (·) represents the jth progressive staircase convolution chain, j = 1, 2, 3, 4, RL i (·) represents the i-th 1×1 convolution, which is used for dimensionality reduction and saves the image features at the current depth, i = 1, 2, 3; F in For the given input features, that is, F 0 ; Step S2-2: Concatenate the features retained multiple times and the output features of the last progressive staircase convolution chain according to the channel dimension, and use 1×1 convolution for dimensionality reduction; expressed as: F distilled =Conv 1 (Concat(F distilled_1 ,...,F distilled_4 )); Concat(·) represents concatenation operation according to channel dimension, Conv 1 (·) is a 1×1 convolutional layer for dimensionality reduction; Step S2-3: Input the reduced dimension features into the attention module, focusing on the image spatial features first globally and then locally; expressed as: F attention =H SAFM (H ESA (F distilled )); where H SAFM (·) and H ESA (·) denotes spatial adaptive modulation SAFM and enhanced spatial attention ESA, respectively; Step S2-4: At the end of the progressive receptive field distillation block, a 3×3 standard convolution is used to further extract the output features of the attention module, and a jump connection is introduced to combine the input features of the progressive receptive field distillation block and the final output features after further extraction, expressed as: F out =Conv 3 (F attention ), where F attention Represents the output features of the attention module.
5. According to claim 4, the lightweight mine image super-resolution reconstruction method based on progressive receptive field, It is characterized in that In step S2-1, the progressive ladder convolution chain is used to extract the output features F of the shallow feature extraction module. 0 The specific method for feature extraction is: Step S2-1-1: Given input feature F coarse It is divided into two parts according to the channel dimension, expressed as: F s1 ,F s2 =Split(F coarse );where F s1 ,F s2 Represent the first part of input features and the second part of input features respectively; Step S2-1-2: The upper step convolution chain sequentially uses a depthwise separable convolution with a convolution kernel of 5×5, a depthwise separable convolution with a convolution kernel of 3×3, and a 1×1 convolution to increase the receptive field, extract more global image features, and combine the convolution results with the input features of the upper step convolution chain after being processed by an activation function; To gradually refine the image features; expressed as: Among them, F S15 is the calculated intermediate value obtained after using a depth-separable convolution with a convolution kernel of 5×5; F S13 is the calculated intermediate value obtained after using a depthwise separable convolution with a convolution kernel of 3×3; σ(·) represents the GELU activation function; DSC 5 (·) indicates a depth-wise separable convolution with a convolution kernel of 5×5, DSC 3 (·) indicates that the convolution is a 3×3 depth-separable convolution, Conv 1 (·) represents 1×1 convolution; F up Represents the output of the upper ladder convolution chain; Step S2-1-3: The lower staircase convolution chain uses 1×1 convolution, 3×3 depth-separable convolution and 5×5 depth-separable convolution in sequence to focus on the overall structural information, and concatenates the output results of the upper staircase convolution chain and the lower staircase convolution chain to obtain image features containing local details and overall information; expressed as: Among them, F S21 is the calculated intermediate value obtained after using 1×1 convolution; F S23 is the calculated intermediate value obtained after using a depth-separable convolution with a convolution kernel of 3×3; F down Represents the output result of the lower staircase convolution chain; Step S2-1-4: Combine the image features containing local details and overall information with the original input features F coarse Add together, and the calculation formula is: F PRFFM =Concat(F up ,F down )+F coarse ; Concat(·) represents the concatenation operation based on the channel dimension.
6. The lightweight mine image super-resolution reconstruction method based on progressive receptive field according to claim 2, It is characterized in that In step S3, the multi-dimensional latent feature fusion module MLFFB fully fuses the latent features of different dimensions in multiple progressive receptive field distillation blocks and combines them with the shallow features. The specific method is: Step S3-1: Fuse the features of corresponding dimensions in different progressive receptive field distillation blocks, expressed as: in, represents the x-dimensional latent features F from the first to the fifth x ; and Represents the y-dimensional potential features F in multiple y and z-dimensional latent features F z ; Step S3-2: Concatenation, using 1×1 convolution and adaptive pooling layers in sequence to integrate feature information of different dimensions and reduce the spatial dimension of the feature map; Step S3-3: Use a set of depth-wise separable convolutions with kernel size 3×3 and Softmax activation function to dynamically assign weights to the two sets of features and combine them with Add together to achieve full integration of multi-dimensional potential features; expressed as: Among them, σ(·) represents the GELU activation function, DSC 3 (·) is a depth-wise separable convolution with a kernel size of 3×3. and is a set of weights generated by the Softmax activation function; H gp represents the pooling operation; F gp Represents the result obtained after the pooling operation; Step S3-4: The fused result F MLFFB With shallow features F 0 After addition, the data are input into the reconstruction module for image reconstruction.
Citation Information
Patent Citations
Mine image super-resolution reconstruction method and system based on hierarchical feature fusion
CN117114994A
Mine image super-resolution reconstruction system and method based on overall attention
CN117173024A
Image super-resolution enlargement model and method
WO2023040108A1
AU2020100200A4
Cited By
Remote sensing image arbitrary scale super-resolution method based on dynamic scale frequency domain convolution
CN119559052A
Lightweight image super-resolution reconstruction system and method for underground mobile equipment
CN119671855A
Image super-resolution reconstruction method and system in deep-ground catastrophe environment fuzzy field of view
CN120198287A
Image super-resolution reconstruction method and system under blurred field of view in deep earth disaster environment
CN120198287B
Mine coal face low-quality blurred image super-resolution reconstruction method and system
CN120672575A
