A method and system for super-resolution reconstruction of low-quality images of mine tunneling faces

Abstract

This invention provides a method and system for super-resolution reconstruction of low-quality images of mine tunneling faces. The method includes incorporating RSAB and SASSB modules into the HG structure. In SASSB, a semantic neighborhood scanning strategy is used to rearrange the feature sequences, and SASSM is used to recover the spatial structure at the hidden state level. In the MSMF module, local memories at each stage are aggregated using LMF, and GMF is used to adaptively integrate hierarchical memories of different depths. The system includes a shallow feature extraction module, a deep feature extraction module, and a reconstruction module. This invention simplifies computation and enables efficient combination of global and local information, while improving the ability to recover spatial structure and utilize hierarchical memory information.

Description

Technical Field

[0001] This invention relates to the field of image super-resolution reconstruction technology, specifically to a method and system for super-resolution reconstruction of low-quality images of mine tunneling faces. Background Technology

[0002] In the complex environment of mines, especially at coal mine tunneling faces, image acquisition and processing face numerous challenges, including complex lighting conditions, uneven light intensity, image blurring due to large amounts of dust generated during tunneling operations, image noise caused by interference from electronic devices, and limited computing power of hardware. During mine tunneling operations, acquired images may be blurred and subject to severe noise interference due to low illumination, strong dynamic changes, dust, and smoke, making it difficult to provide clear details. This poses a significant challenge to the reliability and accuracy of mine automated monitoring systems, personnel positioning, equipment fault detection, and environmental risk early warning systems. Therefore, researching super-resolution reconstruction technology for low-quality images is of great significance and necessity for improving the operational efficiency and safety of mine tunneling faces.

[0003] Image super-resolution reconstruction is a technique aimed at reconstructing low-resolution degraded images into high-resolution, clear images. Currently, image super-resolution reconstruction methods mainly include interpolation-based methods, reconstruction-based methods, and learning-based methods. Although deep learning-based super-resolution models have made significant progress, they still face challenges in practical applications, such as high computational complexity, limited deployment, difficulty in balancing global and local information, destruction of spatial structure, and insufficient utilization of hierarchical memory. Summary of the Invention

[0004] To address the aforementioned technical shortcomings, the present invention aims to provide a method and system for super-resolution reconstruction of low-quality images of mine tunneling faces, which simplifies calculations and enables efficient combination of global and local information, while improving the ability to restore spatial structures and utilize hierarchical memory information.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for super-resolution reconstruction of low-quality images of a mine tunneling face includes the following steps:

[0007] S1. Input the low-resolution image into the shallow feature extraction module, and extract the shallow features through a 3×3 convolutional layer;

[0008] S2. Shallow features are input into the deep feature extraction module, which consists of cascaded Hidden Groups (HGs). Each HG embeds an RSAB module and a SASSB module. The RSAB module performs local content dependency interactions, while the SASSB module uses a semantic neighborhood scanning strategy to rearrange the feature sequence and uses the built-in SASSM to recover the spatial structure at the hidden state level to complete global modeling. Each HG outputs the corresponding hidden state and final features.

[0009] S3. Input the hidden state and final feature output of each HG into the MSMF module. Use LMF to aggregate the hidden state of each module in each HG to form a stage-level memory. Then use GMF to perform channel attention-guided fusion on all stage-level memories and final output features to generate a fused global memory.

[0010] S4. After further extracting features from the fused global memory through a 3×3 convolutional layer, the features are residually connected with the shallow features obtained in step S1 to obtain deep features.

[0011] S5. The deep features are upsampled by pixel rearrangement through the reconstruction module to generate a super-resolution image.

[0012] Preferably, the shallow feature extraction process in step S1 and the deep feature extraction process in step S2 are described by the following formulas:

[0013] S1.1 Shallow feature extraction process:

[0014] in, Represent a Standard convolution operation, For the input low-resolution image, This represents the extracted shallow feature map;

[0015] S1.2 Deep Feature Extraction Process:

[0016] in, It is the first In the cascade group, the first The characteristics of HG output, It is the first In the cascade group, the first The hidden state of each HG output, It is the number of cascade groups. It is the number of HGs within the cascade group; The final deep feature representation is as follows

[0017]

[0018] in, For multi-stage memory fusion module, It is the output characteristic of the last HG in the last cascade group. For the final deep features, This indicates element-wise addition.

[0019] Preferably, in step S2, the process of embedding the RSAB module and the SASSB module into the HG structure for deep feature extraction is represented by the following formula:

[0020]

[0021]

[0022]

[0023]

[0024] in, These are the input features of HG. , , For temporary variables, For HG's output variables; For layer normalization module, For window-based multi-head self-attention, This is a spatial perception state space model. It is a feedforward network.

[0025] Preferably, in step S2, the specific execution process of SASSM in the SASSB module is as follows:

[0026] First, a semantic neighborhood scanning strategy is used to rearrange the feature sequence. Then, a routing network is used to predict the semantic category of each pixel in the input features, generating pixel classification routes. :

[0027]

[0028] Two-dimensional spatial features Remodeling into a one-dimensional sequence Then, according to the route For pixel sequences Sort the pixels so that semantically related pixels are adjacent in the sequence:

[0029]

[0030] Then, state-space modeling is performed using the SASSM module, and selective scanning is performed on the rearranged feature sequence. The state update process is as follows:

[0031]

[0032] in, for The hidden state at all times As input features, The state transition matrix is ​​a discretized representation. The input projection matrix is ​​discretized;

[0033] Through spatial fusion function Restore the spatial structure at the hidden state level and aggregate the hidden states within the neighborhood:

[0034]

[0035] in, To restore the hidden state features after the spatial structure is restored. For the neighborhood range, For position The Neighborhood location indexes This is a state fusion function based on spatial attention;

[0036] The output calculation process is as follows:

[0037]

[0038] in, To output the projection matrix, It is a directly connected matrix. For output features; SASSM ultimately transforms a one-dimensional sequence Reverse sorting and reshaping into two-dimensional spatial features :

[0039]

[0040] Among them, the state fusion function based on spatial attention will hide the one-dimensional hidden state. Reconstructed into a two-dimensional spatial feature map We apply depthwise separable convolutions to capture spatial context, and further adaptively enhance spatial structure information through SA spatial attention:

[0041]

[0042] Generate spatial attention weights:

[0043]

[0044] Based on spatial attention weight enhancement features:

[0045]

[0046] Then the two-dimensional spatial feature map Reshape into a one-dimensional hidden state .

[0047] Preferably, in step S3, the specific execution process of the MSMF module is as follows:

[0048] Stage-level memory is formed by aggregating the hidden states of each module within each HG using LMF:

[0049]

[0050] in, This indicates a splicing operation. Represents a 1×1 convolution. This indicates averaging across the channel dimension;

[0051] Then, GMF is used to integrate all stage-level memories and the final output features, and the memories at each stage level are normalized and subjected to depthwise convolution transformation:

[0052]

[0053]

[0054] Channel attention is applied to each transformed feature, channel attention weights are generated and normalized:

[0055]

[0056]

[0057] The final fusion of global memories is obtained through a weighted sum:

[0058]

[0059] in, This indicates element-wise multiplication. This indicates element-wise addition.

[0060] Preferably, in step S5, the reconstruction module will extract deep features. Converted into the final super-resolution image The specific process is as follows:

[0061]

[0062] in, This indicates a pixel rearrangement operation. This represents the output of the deep feature extraction layer. This represents the result of image super-resolution reconstruction.

[0063] A system for super-resolution reconstruction of low-quality images of a mine tunneling face includes a shallow feature extraction module, a deep feature extraction module, and a reconstruction module;

[0064] The shallow feature extraction module is used to extract shallow texture features from the input low-resolution image through a single-layer 3×3 convolution operation, providing an initial feature representation for subsequent deep feature extraction.

[0065] The deep feature extraction module includes a cascaded HG, an RSAB module and a SASSB module embedded in the HG, an MSMF module, and a SASSM module built into the SASSB module. The deep feature extraction module adopts a cascaded modular design, with the output of each module passed to the next module for progressive feature optimization.

[0066] The reconstruction module is used to perform pixel rearrangement and upsampling on the deep features output by the deep feature extraction module, converting them into the final super-resolution image.

[0067] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0068] This invention incorporates RSAB and SASSB modules into the HG structure. SASSB employs a semantic neighborhood scanning strategy to rearrange feature sequences and uses SASSM to recover spatial structure at the hidden state level. This enhances local spatial coherence while maintaining global modeling capabilities, compensating for the shortcomings of long-distance dependency modeling in learning local details. The MSMF module aggregates local memories at each stage through LMF and adaptively integrates hierarchical memories of different depths using GMF. The GMF mechanism directly determines the importance of memories at each stage through channel attention, fully exploring multi-level information from fine-grained details to semantic structure. A lightweight module design replaces the traditional complex structure, reducing the number of model parameters and computational complexity. This invention achieves efficient modeling and fusion of global and local features, while significantly improving the ability to recover spatial structure and utilize hierarchical memory information, resulting in high-performance super-resolution reconstruction. Attached Figure Description

[0070] Figure 1 This is a schematic diagram of the workflow of the method of the present invention;

[0071] Figure 2 This is the overall network structure diagram of the present invention;

[0072] Figure 3 This is a structural diagram of the present invention. Detailed Implementation

[0074] The invention will now be further described with reference to the accompanying drawings.

[0075] like Figure 1 As shown, a method for super-resolution reconstruction of low-quality images of a mine tunneling face includes the following steps:

[0076] S1. The input low-resolution image first passes through the shallow feature extraction module, and shallow features are extracted through a 3×3 convolutional layer.

[0077] S2. Shallow features enter the deep feature extraction module, which is composed of cascaded HGs. Each HG contains RSAB and SASSB modules. Local content dependency interactions are performed in RSAB. In SASSB, a semantic neighborhood scanning strategy is used to rearrange the feature sequence and global modeling is performed by restoring the spatial structure at the hidden state level through SASSM. Each HG outputs the hidden state and the final features.

[0078] S3. In the MSMF module, the hidden states of each block in each HG are aggregated by LMF to form stage-level memories. Then, GMF is used to guide the fusion of all stage-level memories and the final output features through channel attention to generate a fused global memory.

[0079] S4. After further feature extraction through a 3×3 convolutional layer, the global memory is combined with shallow features and residual connections are made to obtain deep features.

[0080] S5. The reconstruction module performs pixel rearrangement and upsampling on deep features to generate a super-resolution image.

[0081] Furthermore, the low-resolution image input in S1 first passes through a shallow feature extraction module, and then through a deep feature extraction module. The feature extraction process is described as follows:

[0082] S1.1 Extracting shallow features from the input low-resolution image:

[0083]

[0084] in, Represent a Standard convolution operation, For the input low-resolution image, This represents the extracted shallow feature map.

[0085] S1.2, Perform deep feature extraction on the features output by the shallow feature extraction module:

[0086] The deep feature extraction module adopts a cascaded modular design, where the output of each module is passed to the next module for progressive optimization. The deep feature extraction process is described as follows:

[0087]

[0088] in, It is the first In the cascade group, the first The characteristics of HG output, It is the first In the cascade group, the first The hidden state of each HG output, It is the number of cascade groups. This refers to the number of HGs within the cascaded group. By fusing the output features of all HG modules, the final deep feature representation is obtained:

[0089]

[0090] in, For multi-stage memory fusion module, It is the first HG No. The hidden state features of each block It is the output characteristic of the last HG in the last cascade group. This indicates element-wise addition.

[0091] Furthermore, in S2, the RSAB and SASSB modules are embedded into the HG structure to extract deep features. The extraction process is as follows:

[0092]

[0093]

[0094]

[0095]

[0096] in, These are the input features of HG. , , For temporary variables, For HG's output variables; For layer normalization module, For window-based multi-head self-attention, This is a spatial perception state space model. It is a feedforward network.

[0097] Furthermore, the specific process of the SASSB module in S2 is as follows:

[0098] First, a semantic neighborhood scanning strategy is used to rearrange the feature sequence. Then, a routing network is used to predict the semantic category of each pixel in the input features, generating pixel classification routes. :

[0099]

[0100] Two-dimensional spatial features Remodeling into a one-dimensional sequence Then, according to the route For pixel sequences Sort the pixels so that semantically related pixels are adjacent in the sequence:

[0101]

[0102] Then, state-space modeling is performed using SASSM, and selective scanning is conducted on the rearranged feature sequence. The state update process is as follows:

[0103]

[0104] in, for The hidden state at all times As input features, The state transition matrix is ​​a discretized representation. The input projection matrix is ​​discretized;

[0105] Through spatial fusion function Restore the spatial structure at the hidden state level and aggregate the hidden states within the neighborhood:

[0106]

[0107] in, To restore the hidden state features after the spatial structure is restored. For the neighborhood range, For position The Neighborhood location indexes This is a state fusion function based on spatial attention;

[0108] The output calculation process is as follows:

[0109]

[0110] in, To output the projection matrix, It is a directly connected matrix. For output features; SASSM will ultimately convert the one-dimensional sequence Reverse sorting and reshaping into two-dimensional spatial features :

[0111]

[0112] Among them, the state fusion function based on spatial attention will hide the one-dimensional hidden state. Reconstructed into a two-dimensional spatial feature map We apply depthwise separable convolutions to capture spatial context, and further adaptively enhance spatial structure information through SA spatial attention:

[0113]

[0114] Generate spatial attention weights:

[0115]

[0116] Based on spatial attention weight enhancement features:

[0117]

[0118] Then the two-dimensional spatial feature map Reshape into a one-dimensional hidden state .

[0119] Furthermore, the specific process of the MSMF module in S3 is as follows:

[0120] Stage-level memory is formed by aggregating the hidden states of each block within each HG using LMF:

[0121]

[0122] in, For the first HG No. Hidden state feature maps generated for each block This indicates a splicing operation. express convolution, This indicates averaging across the channel dimension;

[0123] Then, GMF is used to integrate all stage-level memories and the final output features, and the memories at each stage level are normalized and subjected to depthwise convolution transformation:

[0124]

[0125]

[0126] Channel attention is applied to each transformed feature, channel attention weights are generated and normalized:

[0127]

[0128]

[0129] The final fusion of global memories is obtained through a weighted sum:

[0130]

[0131] in, This indicates element-wise multiplication. This indicates element-wise addition.

[0132] Furthermore, in S4, the process of fusing global memory and adding the output features of the last HG layer, passing them through a convolutional layer, and then performing residual connections with shallow features to obtain deep features is as follows:

[0133]

[0134] in, In order to integrate global memory, The output characteristics of the last HG in the last cascade group. These are shallow features. This indicates element-wise addition.

[0135] Furthermore, the S5 reconstruction module is responsible for extracting deep features. Converted into the final super-resolution image The specific process is as follows:

[0136]

[0137] in, This indicates a pixel rearrangement operation. This represents the output of the deep feature extraction layer. This represents the result of image super-resolution reconstruction.

[0138] like Figure 3 As shown, a system for super-resolution reconstruction of low-quality images of a mine tunneling face includes a shallow feature extraction module, a deep feature extraction module, and a reconstruction module.

[0139] The shallow feature extraction module extracts basic texture information from the input low-resolution image through single-layer convolution operations, providing initial feature representations for subsequent deep feature extraction;

[0140] The deep feature extraction module includes HG, RSAB, SASSB, MSMF, and SASSM modules; the deep feature extraction module adopts a cascaded modular design, with the output of each module passed to the next module for step-by-step optimization;

[0141] The reconstruction module is used to convert deep features into a final super-resolution image.

[0142] The network structure of this invention includes a shallow feature extraction module, a deep feature extraction module, and a reconstruction module, such as... Figure 2 As shown. Input For low-resolution images, a 3×3 convolution is used as a shallow feature extraction layer. The extracted shallow features are first input into the deep feature extraction module to further extract deep features, and then they are also used as residual features and mapped to the end of the deep feature extraction module to preserve low-frequency information at low levels.

[0143] In the deep feature extraction layer, several HG modules are stacked, and the hidden states of all HG modules are fused using the MSMF module. Finally, the global residual mapping of shallow features is added to the elements, which effectively avoids the gradient vanishing training problem.

[0144] In the HG module, the RSAB module and SASSB module are used for deep feature extraction, and the hidden states of the two modules are fused through LMF. Finally, the original input features are received as local residual mapping.

[0145] In the RSAB module, the LN layer is used for layer normalization, and then the input is fed into the WMSA module for local content dependency interaction. Then the LN layer is used for layer normalization again, and then the input is fed into the FFN module for channel dimension interaction.

[0146] In the SASSB module, the LN layer is used for layer normalization, and then the result is input into the SASSM module for global modeling. This module uses a semantic neighborhood scanning strategy to rearrange the feature sequence and restores the spatial structure at the hidden state level through SASSM. Then, the LN layer is used for layer normalization, and the result is input into the FFN module for channel-dimensional interaction.

[0147] In the SASSM module, the semantic category of each pixel is predicted first through a routing network to generate pixel classification routes. After reshaping the two-dimensional spatial features into a one-dimensional sequence, the pixel sequence is sorted according to the routes so that semantically related pixels are adjacent in the sequence. Then, selective scanning calculation of the state space model is performed, and the hidden state is updated through discretized state transition matrices and input projection matrices. During the scanning process, the spatial structure is restored at the hidden state level through a spatial fusion function. This function reshapes the one-dimensional hidden state into a two-dimensional spatial feature map, applies depthwise separable convolution to capture spatial context, and adaptively enhances spatial structure information through a spatial attention mechanism. Subsequently, the enhanced two-dimensional feature map is reshaped back into the one-dimensional hidden state. Finally, the output features are calculated through the output projection matrix and the direct connection matrix, and the one-dimensional sequence is reverse-sorted and reshaped into two-dimensional spatial features.

[0148] In the MSMF module, the hidden states of each block within each HG are first aggregated by LMF to form stage-level memories. Then, all stage-level memories and the final output features are integrated by GMF. This mechanism normalizes and performs depthwise convolution transformation on each stage-level memory. Then, channel attention is applied to generate channel attention weights. The importance of each stage memory is adaptively determined by normalizing all channel attention weights. Finally, the fused global memory is obtained by weighted summation.

[0149] The image reconstruction layer incorporates two 3×3 convolutions and pixel rearrangement operations to achieve image upsampling.

[0150] The SASSM and MSMF modules proposed in this invention achieve efficient modeling and fusion of global and local features, while greatly improving the ability to recover spatial structures and utilize hierarchical memory information.

Claims

1. A method for super-resolution reconstruction of low-quality images of a mine tunneling face, characterized in that, Includes the following steps: S1. Input the low-resolution image into the shallow feature extraction module, and extract the shallow features through a 3×3 convolutional layer; S2. Shallow features are input into the deep feature extraction module, which consists of cascaded Hidden Groups (HGs). Each HG embeds an RSAB module and a SASSB module. The RSAB module performs local content dependency interactions, while the SASSB module uses a semantic neighborhood scanning strategy to rearrange the feature sequence and uses the built-in SASSM to recover the spatial structure at the hidden state level to complete global modeling. Each HG outputs the corresponding hidden state and final features. S3. Input the hidden state and final feature output of each HG into the MSMF module. Use LMF to aggregate the hidden state of each module in each HG to form a stage-level memory. Then use GMF to perform channel attention-guided fusion on all stage-level memories and final output features to generate a fused global memory. S4. After further extracting features from the fused global memory through a 3×3 convolutional layer, the features are residually connected with the shallow features obtained in step S1 to obtain deep features. S5. The deep features are upsampled by pixel rearrangement through the reconstruction module to generate a super-resolution image.

2. The method for super-resolution reconstruction of low-quality images of a mine tunneling face as described in claim 1, characterized in that, The shallow feature extraction process in step S1 and the deep feature extraction process in step S2 are described by the following formulas: S1.1 Shallow feature extraction process: in, Represent a Standard convolution operation, For the input low-resolution image, This represents the extracted shallow feature map; S1.2 Deep Feature Extraction Process: in, It is the first In the cascade group, the first The characteristics of HG output, It is the first In the cascade group, the first The hidden state of each HG output, It is the number of cascade groups. It is the number of HGs within the cascade group; The final deep feature representation is as follows: in, For multi-stage memory fusion module, It is the output characteristic of the last HG in the last cascade group. For the final deep features, This indicates element-wise addition.

3. The method for super-resolution reconstruction of low-quality images of a mine tunneling face as described in claim 1, characterized in that, In step S2, the process of embedding the RSAB module and the SASSB module into the HG structure for deep feature extraction is represented by the following formula: in, These are the input features of HG. , , For temporary variables, For HG's output variables; For layer normalization module, For window-based multi-head self-attention, This is a spatial perception state space model. It is a feedforward network.

4. The method for super-resolution reconstruction of low-quality images of a mine tunneling face as described in claim 3, characterized in that, In step S2, within the SASSB module, the specific execution process of SASSM is as follows: First, a semantic neighborhood scanning strategy is used to rearrange the feature sequence. Then, a routing network is used to predict the semantic category of each pixel in the input features, generating pixel classification routes. : Two-dimensional spatial features Remodeling into a one-dimensional sequence Then, according to the route For pixel sequences Sort the pixels so that semantically related pixels are adjacent in the sequence: Then, state-space modeling is performed using the SASSM module, and selective scanning is performed on the rearranged feature sequence. The state update process is as follows: in, for The hidden state at all times As input features, The state transition matrix is ​​a discretized representation. The input projection matrix is ​​discretized; Through spatial fusion function Restore the spatial structure at the hidden state level and aggregate the hidden states within the neighborhood: in, To restore the hidden state features after the spatial structure is restored. For the neighborhood range, For position The Neighborhood location indexes This is a state fusion function based on spatial attention; The output calculation process is as follows: in, To output the projection matrix, It is a directly connected matrix. For output features; SASSM ultimately transforms a one-dimensional sequence Reverse sorting and reshaping into two-dimensional spatial features : Among them, the state fusion function based on spatial attention will hide the one-dimensional hidden state. Reconstructed into a two-dimensional spatial feature map We apply depthwise separable convolutions to capture spatial context, and further adaptively enhance spatial structure information through SA spatial attention: Generate spatial attention weights: Based on spatial attention weight enhancement features: Then the two-dimensional spatial feature map Reshape into a one-dimensional hidden state .

5. The method for super-resolution reconstruction of low-quality images of a mine tunneling face as described in claim 1, characterized in that, In step S3, the specific execution process of the MSMF module is as follows: Stage-level memory is formed by aggregating the hidden states of each module within each HG using LMF: in, This indicates a splicing operation. Represents a 1×1 convolution. This indicates averaging across the channel dimension; Then, GMF is used to integrate all stage-level memories and the final output features, and the memories at each stage level are normalized and subjected to depthwise convolution transformation: Channel attention is applied to each transformed feature, channel attention weights are generated and normalized: The final fusion of global memories is obtained through a weighted sum: in, This indicates element-wise multiplication. This indicates element-wise addition.

6. The method for super-resolution reconstruction of low-quality images of a mine tunneling face as described in claim 1, characterized in that, In step S5, the reconstruction module will extract deep features. Converted into the final super-resolution image The specific process is as follows: in, This indicates a pixel rearrangement operation. This represents the output of the deep feature extraction layer. This represents the result of image super-resolution reconstruction.

7. A system for super-resolution reconstruction of low-quality images of a mine tunneling face, used to perform the method of any one of claims 1 to 6, characterized in that, It includes a shallow feature extraction module, a deep feature extraction module, and a reconstruction module; The shallow feature extraction module is used to extract shallow texture features from the input low-resolution image through a single-layer 3×3 convolution operation, providing an initial feature representation for subsequent deep feature extraction. The deep feature extraction module includes a cascaded HG, an RSAB module and a SASSB module embedded in the HG, an MSMF module, and a SASSM module built into the SASSB module. The deep feature extraction module adopts a cascaded modular design, with the output of each module passed to the next module for progressive feature optimization. The reconstruction module is used to perform pixel rearrangement and upsampling on the deep features output by the deep feature extraction module, converting them into the final super-resolution image.

Images