A mine tunneling working face low-quality image super-resolution reconstruction method and system

Abstract

A mine tunneling working face low-quality image super-resolution reconstruction method and system, the method comprises the following steps: adding MAB and EFN modules in the FMB of the MetaFormer style structure, fusing the MHFE module and the MlSKA module in the MAB, using the MHFE module to perform up-sampling and down-sampling on the output features to simulate a multi-scale high-pass filter, using the MlSKA module to learn the non-local structure information of the output features in S1, and capturing long-distance dependence. The system comprises a shallow feature extraction module, a deep feature extraction module and a reconstruction module. The present application can simplify the calculation, and can realize the efficient combination of global and local information, and at the same time improve the recovery ability of high-frequency details.

Description

TECHNICAL FIELD

[0001] The application relates to a low-quality image super-resolution reconstruction method and system for a mine tunneling working face, and belongs to the technical field of image super-resolution reconstruction. BACKGROUND

[0002] Under the complex environment of a mine, especially on a coal mine tunneling working face, image acquisition and processing face many problems such as complex lighting conditions, uneven light intensity, image blurring caused by a large amount of dust generated by tunneling operations, image noise caused by electronic device interference, and limited computing capacity of hardware devices. In mine tunneling operations, the collected images may be blurred and severely disturbed by noise due to low illumination, strong dynamic changes, dust, smoke and other factors, and it is difficult to provide clear detailed information. This poses a great challenge to the reliability and accuracy of mine automation monitoring systems, personnel positioning, equipment fault detection and environmental risk warning systems. Therefore, it is of great significance and necessity to study the super-resolution reconstruction technology of low-quality images for improving the efficiency and safety of mine tunneling working faces.

[0003] Image super-resolution reconstruction is a technology aimed at reconstructing low-resolution degraded images into high-resolution clear images. At present, the image super-resolution reconstruction methods mainly include interpolation-based methods, reconstruction-based methods and learning-based methods. Although the super-resolution model based on deep learning has made significant progress, it still faces challenges such as high computational complexity, limited deployment, difficulty in balancing global and local information, and difficulty in restoring high-frequency information in practical applications. SUMMARY

[0004] The application aims to provide a low-quality image super-resolution reconstruction method and system for a mine tunneling working face, which can simplify the calculation and realize efficient combination of global and local information while improving the recovery ability of high-frequency details.

[0005] In order to achieve the above-mentioned purpose, the application provides a low-quality image super-resolution reconstruction method for a mine tunneling working face, which comprises the following steps:

[0006] S1, the input low-resolution image is first subjected to shallow feature extraction and feature space transformation, and then deep feature extraction. The deep feature extraction module is composed of a BRB module stack, and the BRB module is composed of an FMB module stack.

[0007] S2, a multi-scale attention module MAB and an efficient feedforward network EFN module are added in the feature fusion module FMB of the MetaFormer style structure; after the features are processed in the FMB, they are transmitted to the MAB for learning of global structure and local details, then after residual mapping, the shallow features are subjected to layer normalization again and transmitted to the EFN module for optimization of non-linear expression capability, and then after residual mapping, the features are output;

[0008] S3, in the MAB, a multi-scale high-frequency gain MHFE module and a multi-scale large kernel separable attention MLSKA module are fused, and the features processed and output by the MHFE module and the MLSKA module are fused in the channel; the EFN module refines important features;

[0009] S4, the output features in S1 are up-sampled and down-sampled using the MHFE module to simulate a multi-scale high-pass filter, and the extracted high-frequency information of different scales is further gained;

[0010] S5, the non-local structure information of the output features in S1 is learned using the MLSKA module to capture long-distance dependencies.

[0011] S6, the reconstruction module receives the features input by the deep feature extraction module and converts them into a final reconstructed image for output.

[0012] Further, the input low-resolution image in S1 is first subjected to a shallow feature extraction module and then a deep feature extraction module, and the feature extraction process is described as follows:

[0013] S1.1, shallow features are extracted from the input low-resolution image:

[0014] F i =Conv i+1 (I k );

[0015] wherein Conv i represents a 3x3 standard convolution operation, I i+1 is the input low-resolution image, and F 1×1 represents the extracted shallow feature map.

[0016] S1.2, deep features are extracted from the features output by the shallow feature extraction module:

[0017] The deep feature extraction module adopts a modular design in cascade, and the output of each module is transmitted to the next module for step-by-step optimization. The process of deep feature extraction is described as follows:

[0018] F 1×1 =BRB 1×1(F i ), i ∈ [1, N];

[0019] where F i+1 is the output of the i+1th basic residual block (BRB) and N is the number of deep modules. By fusing the output features of all BRB modules, the final deep feature representation is obtained:

[0020]

[0021] where F k is the output feature of the kth BRB module, Conc represents feature concatenation, and Elemwise represents element-wise addition.

[0022] Each BRB module further extracts the correlation between features through the combination of residual connection and convolution operation. The formula is described as follows:

[0023]

[0024] where F i is the output of the i-th basic residual block (BRB) as the input of the i+1th BRB, F i+1 is the output of the i+1th FMB, and φ is the GELU activation function.

[0025] Further, the MAB module and the EFN module are embedded in the MetaFormer style structure in S2 to extract deep features, and the extraction process is represented as follows:

[0026]

[0027] where x is the input feature of the FMB, x1 is a temporary variable, and x2 is the output variable of the FMB; LN is the LayerNormer layer normalization module, and MAB and EFN represent the MAB module and the EFN module, respectively.

[0028] Further, the formula for channel fusion of the features processed by the MHFE module and the MLSKA module in S3 is:

[0029] Y, Z = Split(Conv 1×1 (X));

[0030] where Y and Z are sub-features obtained by channel splitting of the input MAB module feature X;

[0031] S = MHFE(Y); V = MLSKA(Z);

[0032] Wherein, MHFE and M LSKA represent multi-scale high-frequency gain module, multi-scale large kernel separable attention module respectively;

[0033]

[0034] Wherein, M is the output of the MAB module;

[0035] The description of the important feature refined by the EFN module is:

[0036] E1, E2 = Split (φ (Conv 1×1 (E)));

[0037] E' = Conv 1×1 (Conc (Conv 3×3 (E1), E2))。

[0038] Wherein, E1, E2 are the sub-features after channel segmentation of E, and E' is the output structure of the EFN module.

[0039] Further, the MHFE module used in S4 uses 2, 4, 8 times up and down sampling to simulate high-pass filter to extract multi-scale high-frequency detail information, and then uses 3x3 convolution kernel to further extract features and gain high-frequency details of the high-frequency information, the process is as follows:

[0040] S4.1, using 1x1 point convolution Conv 1×1 to reduce the dimension of the feature map and output Y0:

[0041] Y0 = φ (Conv 1×1 (Y));Wherein, Y is the input feature of the MHFE module;

[0042] Y1 = U8 (D8 (Y0));

[0043] Y2 = U4 (D4 (Y0));

[0044] Y3 = U2 (D2 (Y0));

[0045] Wherein, D i represents i times down sampling, U i represents i times up sampling;

[0046] While up and down sampling the input feature, two 1x1 point convolution Conv 1×1 are used to learn the channel dimension of the feature:

[0047] Y4 = Conv 1×1 (Conv 1×1 (Y0));

[0048] S4.2, extracting low-frequency information Y from the extracted feature map Y i extracting high-frequency information Y' from Y4 i , realizing the elimination of redundant features, and then using Conv 3×3 to perform gain on high-frequency features and realize differential learning of features:

[0049] Y' i = φ(Conv 3×3 (Y i -Y4)), i∈{1,2,3};

[0050]

[0051]

[0052] where σ is a Sigmoid function, is element multiplication, and S is the output of the MHFE module.

[0053] Further, the specific process of S5 is as follows:

[0054] S5.1, dividing the feature map Z of the input MLSKA into three sub-feature maps Z1, Z2, and Z3, and applying a multi-scale large kernel separable convolution to each sub-feature map:

[0055] Z1, Z2, Z3 = Split(Conv 1×1 (Z));

[0056] where Split represents channel splitting;

[0057] LSKA i,j,1 (F) = Conv 1×1 (DWDConv j×1 (DWDConv 1×j (DWConv i×1 (DWConv 1×i (F)))));

[0058] where LSKA i,j,1 (F) is a large kernel separable convolution function, which is composed of: a 1×i size depth convolution DWConv 1×i , an i×1 size depth convolution DWConv i×1 , a 1×j size depth dilated convolution DWConv 1×j , a j×1 size depth dilated convolution DWConv j×1 with a dilated rate of j, a 1×1 size point convolution Conv 1×1 , and F is an intermediate feature variable;

[0059] S5.2, using large kernel separable attention to self-adjust the features:

[0060]

[0061] The large kernel separable convolution is respectively used to simulate the large kernel convolution with the receptive field size of 7, 23 and 35, and the expansion rates of the corresponding dilated convolution are 2, 3 and 4 respectively;

[0062] S5.3, all sub-feature maps are fused:

[0063] Z' = Conc (Z'1, Z'2, Z'3) ;

[0064]

[0065] Where, Conc is channel splicing, and V is the output of MlSKA.

[0066] Further, the reconstruction module of S6 is responsible for converting the deep feature F d into the final super-resolution image I SR , and the specific process is:

[0067] I SR = Conv 3×3 (Shuffle (φ (Conv 3×3 (F d )))

[0068] Where, Shuffle represents a pixel rearrangement operation, F d represents the output of the deep feature extraction layer, and I SR represents the image super-resolution reconstruction result.

[0069] The application also provides a low-quality image super-resolution reconstruction system for a mine tunneling working face, characterized by comprising a shallow feature extraction module, a deep feature extraction module and a reconstruction module.

[0070] The shallow feature extraction module extracts the basic texture information of the input low-resolution image through single-layer convolution operation, and provides initial feature representation for subsequent deep feature extraction.

[0071] The deep feature extraction module comprises a multi-scale attention module, an efficient feedforward network module, a basic residual block, a multi-scale large kernel separable attention module and a multi-scale high-frequency gain module; the deep feature extraction module adopts a modular design in cascade, and the output of each module is transmitted to the next module for step-by-step optimization.

[0072] The reconstruction module is used to convert the deep feature into the final super-resolution image.

[0073] This invention incorporates a multi-scale attention module (MAB) and an efficient feedforward network (EFN) into the feature fusion module (FMB). Within the MAB, a multi-scale high-frequency gain module (MHFE) and a multi-scale large-kernel separable attention module (MLSKA) are integrated. The MHFE module focuses on mining and amplifying high-frequency information at multiple scales. By introducing a differentiated feature learning mechanism, it fully models and suppresses redundancy of features at different scales, compensating for the shortcomings of long-distance dependency modeling in learning local details. Through efficient high-frequency information extraction, the MHFE module significantly enhances the recovery capability of key details such as edges and textures, providing more refined feature support for super-resolution reconstruction. The MLSKA module combines a large-kernel attention mechanism with a multi-scale mechanism, leveraging the advantage of a large receptive field to capture global structural information at different scales and reducing the computational complexity of self-attention in the Transformer model. A lightweight EFN module replaces the traditional multilayer perceptron (MLP), reducing the number of model parameters and computational complexity. This invention achieves efficient modeling and fusion of global and local features while significantly improving the extraction and recovery capability of high-frequency information, resulting in high-performance super-resolution reconstruction. Attached Figure Description

[0074] Figure 1 This is a schematic diagram of the workflow of the method of the present invention;

[0075] Figure 2 This is a schematic diagram of the shallow feature extraction, deep feature extraction, and reconstruction process of the present invention;

[0076] Figure 3 These are image processing comparison diagrams based on embodiments of the present invention. Detailed Implementation

[0077] The invention will now be further described with reference to the accompanying drawings.

[0078] like Figure 1 As shown, a method for super-resolution reconstruction of low-quality images of a mine tunneling face includes the following steps:

[0079] S1. The input low-resolution image first undergoes shallow feature extraction and feature space transformation, followed by deep feature extraction. The deep feature extraction module is mainly composed of stacked BRB modules, which in turn are stacked FMB modules.

[0080] S2, a multi-scale attention module MAB and an efficient feedforward network EFN module are added in the feature fusion module FMB of the MetaFormer style structure; after the features are processed in the FMB and then transported to the MAB for learning of global structure and local details, the shallow layer features are subjected to again layer normalization processing after residual mapping, and then transported to the EFN module for optimization of non-linear expression capability, and then the features are outputted after residual mapping;

[0081] S3, in the MAB, a multi-scale high-frequency gain MHFE module and a multi-scale large kernel separable attention MLSKA module are fused, and the features processed and outputted by the MHFE module and the MLSKA module are fused in the channel; the EFN module refines important features;

[0082] S4, the output features in S1 are up-sampled and down-sampled using the MHFE module to simulate a multi-scale high-pass filter, and the extracted high-frequency information of different scales is further gained;

[0083] S5, the non-local structure information of the output features in S1 is learned using the MLSKA module, and long-distance dependence is captured.

[0084] S6, the reconstruction module receives the features inputted by the deep feature extraction module and converts them into a final reconstructed image for output.

[0085] Embodiment: The network structure of the application includes a shallow feature extraction module, a deep feature extraction module and a reconstruction module, as shown in Figure 2 The input I LR low-resolution image, a 3x3 convolution is used as a shallow feature extraction layer, and the shallow features extracted are first inputted to the deep feature extraction module for further extraction of deep features, and secondly, as residual features, are mapped to the end of the deep feature extraction module as residual mapping to retain low-level low-frequency information;

[0086] In the deep feature extraction layer, a plurality of basic residual blocks BRB modules are stacked, and the channel dimensions of all BRB modules are spliced and then fused and learned in the channel dimension. Finally, the global residual mapping of the shallow layer features is added, which effectively avoids the training problem of gradient disappearance;

[0087] In the BRB module, a plurality of FMB modules are stacked for deep feature extraction, and the output results of all FMB modules are spliced and fused in the channel dimension, and finally the original input features are received as local residual mapping;

[0088] In the FMB module, layer normalization is performed using the LayerNorm layer, then input into the MAB module for global and local information extraction, and then layer normalization is performed using the LayerNorm layer, and then input into the EFN feedforward network module for channel dimension interaction;

[0089] In the MAB module, there are two parallel branches, one branch uses the MHFE module for high-frequency feature extraction, and the other parallel branch uses the MLSKA module for non-local structure information modeling, and finally the features are fused through simple channel dimension splicing and point convolution, so that the module takes into account the advantages of large receptive field and local detail learning. In the MLSKA module, first, the features are expanded in the channel dimension using point convolution, then divided into three sub-feature maps using channel segmentation, each sub-feature is processed using different scale large kernel separable attention, and the blocking effect is suppressed by using a gating aggregation mechanism. Finally, the three processed features are fused to output a multi-scale large kernel separable attention map, which modulates the input features and outputs. In the MHFE module, first, the features are compressed in the channel dimension using point convolution, then the features are extracted in multiple scales using ×2, ×4, ×8 up-sampling, the extracted high-frequency features are individually amplified, and the features of the three branches are fused through a sigmoid function to output high-frequency feature gain weight to realize the high-frequency information self-modulation function of the input features. Finally, a 3x3 convolution is used to restore the dimension and output the result; in the image reconstruction layer, two 3x3 convolutions and pixel rearrangement operations are used to realize the up-sampling of the image.

[0090] The MLSKA and MHFE modules proposed in the application realize efficient modeling and fusion of global and local features, and greatly improve the extraction and recovery ability of high-frequency information. Figure 3 As shown in (a), the PSNR and SSIM indexes of the low-resolution image of the tunneling working face are 35.91 and 0.9810, respectively, Figure 3 As shown in (b), the PSNR and SSIM indexes of the reconstructed image after super-resolution reconstruction using the application are 36.63 and 0.9842, respectively, which can be clearly seen that the indexes are significantly improved, and the image details are clearer.

Claims

1. A method for super-resolution reconstruction of low-quality images of a mine tunneling working face, characterized in that, Comprising the following steps: S1, the input low-resolution image first passes through the extraction of shallow features and the transformation of feature space, and then the extraction of deep features; wherein the deep feature extraction module is composed of basic residual block BRB module stacking, and the BRB module is composed of feature fusion module FMB stacking; S2, the multi-scale attention module MAB and the efficient feedforward network EFN module are added in the feature fusion module FMB of the MetaFormer style structure; after the features are processed in the FMB and then delivered to the MAB for learning of global structure and local details, after residual mapping, the shallow features are subjected to layer normalization processing again and then delivered to the EFN module for optimization of non-linear expression ability, and after residual mapping, the features are outputted; S3, in the MAB, the multi-scale high-frequency gain MHFE module and the multi-scale large kernel separable attention MLSKA module are fused, and the features processed and outputted by the MHFE module and the MLSKA module are fused in channels; the EFN module refines important features; S4, the output features in S1 are up-sampled and down-sampled using the MHFE module to simulate a multi-scale high-pass filter, and the extracted high-frequency information of different scales is further gained; S5, the non-local structure information of the output features in S1 is learned using the MLSKA module, and long-distance dependence is captured; S6, the reconstruction module receives the features inputted by the deep feature extraction module and converts them into a final reconstructed image for output.

2. The mine tunneling face low-quality image super-resolution reconstruction method according to claim 1, characterized in that, The input low-resolution image in S1 first passes through the shallow feature extraction module and then the deep feature extraction module, The feature extraction process is described as follows: S1.1, shallow features are extracted from the input low-resolution image: F s = Conv 3×3 (I LR ); where Conv 3×3 represents a 3x3 standard convolution operation, I LR is the input low-resolution image, F s represents the extracted shallow feature map; S1.2, deep features are extracted from the features outputted by the shallow feature extraction module: The deep feature extraction module adopts a cascaded modular design, and the output of each module is transmitted to the next module for step-by-step optimization. The process of deep feature extraction is described as follows: F i+1 = BRB i+1 (F i ), i e [1, N] where F i+1 is the output of the i+1th basic residual block BRB, N is the number of deep layers, and the final deep feature representation is obtained by fusing the output features of all BRB modules: where Fkis the output feature of the kth BRB module, Conc denotes the feature concatenation, k is the output feature of the kth BRB module, Conc denotes the feature concatenation, denotes the element-wise addition. Each BRB module further extracts the correlation between features through the combination of residual connection and convolution operation, and the formula is described as follows: where F i is the output of the i-th basic residual block BRB and serves as the input of the i+1-th BRB, F i+1 is the output of the i+1-th BRB, and Φ is the GELU activation function.

3. The method according to claim 2, characterized in that, The MAB module and the EFN module are embedded in the MetaFormer style structure in S2 to extract deep features, and the extraction process is represented as follows: Wherein, x is the input feature of FMB, x1 is a temporary variable, and x2 is the output variable of FMB; LN is the LayerNormer layer normalization module, and MAB and EFN represent the MAB module and the EFN module respectively.

4. The mine tunneling face low-quality image super-resolution reconstruction method according to claim 3, characterized in that, The formula for fusing the features processed and outputted by the MHFE module and the MLSKA module in channels in S3 is: Y, Z = Split(Conv 1×1 (X)); Wherein, Y and Z are sub-features obtained by channel segmentation of the input MAB module feature X; Split represents channel segmentation; S=MHFE(Y);V=MLSKA(Z) Wherein, MHFE and MLSKA represent the multi-scale high-frequency gain module and the multi-scale large kernel separable attention module respectively; M = Conv 1×1 (Conc(S, V)) 0 X; Wherein, M is the output of the MAB module; The description of the EFN module refining important features is as follows: E1, E2 = Split(Φ(Conv 1×1 (E))) ; E' = Conv 1×1 (Conc(Conv 3×3 (E1), E2)); Wherein, E1, E2 are respectively the sub-features after channel segmentation of E, and E' is the output structure of the EFN module.

5. The method according to claim 4, characterized in that, In the MHFE module, 2-fold, 4-fold and 8-fold up-sampling and down-sampling are used to simulate high-pass filters to extract multi-scale high-frequency detail information, and then 3x3 convolution kernels are used to further extract features and gain high-frequency details, the process being as follows: S4.1, using 1x1 point convolution Conv 1×1 Dimensionality reduction is performed on the feature map to output Y0: Yo = Φ(Conv 1×1 (Y)); where Y is the input feature of the MHFE module. Y1=U8(D8(Y0)); Y2=U4(D4(Y0)); Y3=U2(D2(Y0)); where D i represents i times down-sampling, U i represents i times up-sampling; At the same time of up-down sampling of input features, two 1x1 point convolution Conv 1×1 Learning of channel dimension of features: Y4 = Conv 1×1 (Conv 1×1 (Y0)) S4.2, extract low-frequency information Y from Y i extract high-frequency information Y' from Y4 i , realize the elimination of redundant features, and then use Conv 3×3 gain high-frequency features and realize the differentiated learning of features: Y ' i = Φ(Conv 3×3 (Y i - Y4)), i e {1,2,3}; where σ is the Sigmoid function, is element-wise multiplication, S is the output of the MHFE module.

6. The method of claim 5, wherein, The specific process of S5 is as follows: S5.1, the feature map Z of the input MLSKA is divided into three sub-feature maps Z1, Z2 and Z3, and a multi-scale large-kernel separable convolution is applied to each sub-feature map: Z1, Z2, Z3 = Split(Conv 1×1 (Z)); Wherein, Split represents channel segmentation; LSKA i,j,1 (F) = Conv 1×1 (DWDConv j×1 (DWDConv 1×j (DWConv i×1 (DWConv 1×i (F))))) where LSKA i,j,1 (F) is a large-kernel separable convolution function, which is composed of: a depthwise convolution DWConv 1×i of size 1 x i, a depthwise convolution DWConv i×1 of size 1 x j, a depthwise dilated convolution DWDConv 1×j of size j x 1, a depthwise dilated convolution DWDConv j×1 of size 1 x 1, and a point convolution Conv 1×1 , and F is an intermediate feature variable. S5.2, a large-kernel separable attention is used for self-adjustment of the features: The large-kernel separable convolution is used to simulate large-kernel convolution with a receptive field size of 7, 23 and 35, respectively, and the expansion rates of the corresponding dilated convolution are 2, 3 and 4, respectively; S5.3, all sub-feature maps are fused: Z' = Conc(Z ' 1, Z ' 2, Z ' 3); Wherein, Conc is channel concatenation, and V is the output of the MLSKA.

7. The method according to claim 6, characterized in that, The reconstruction module of S6 is responsible for converting the deep features F d into the final super-resolution image I SR The specific process is as follows: I SR = Conv 3×3 (Shuffle(Φ(Conv 3×3 (F d )))) wherein Shuffle denotes a pixel rearrangement operation, F d denotes the output of the depth feature extraction layer, I SR denotes the image super-resolution reconstruction result.

8. A system for applying the method of super-resolution reconstruction of low-quality images of a mine tunneling face according to any of claims 1 to 7, characterized in that, The system comprises a shallow feature extraction module, a deep feature extraction module and a reconstruction module; The shallow feature extraction module extracts basic texture information of the input low-resolution image through single-layer convolution operation, and provides initial feature representation for subsequent deep feature extraction; The deep feature extraction module comprises a multi-scale attention module, an efficient feedforward network module, a basic residual block, a multi-scale large-kernel separable attention module and a multi-scale high-frequency gain module; the deep feature extraction module adopts a cascaded modular design, and the output of each module is transmitted to the next module for step-by-step optimization; The reconstruction module is used to convert the deep features into a final super-resolution image.

Images