Mine coal face low-quality fuzzy image super-resolution reconstruction method and system
By employing a multidimensional collaborative covariant feature processing architecture and lightweight convolution operations, the problem of blurred and low-resolution images of coal mining faces is solved, achieving efficient image sharpness enhancement and model lightweighting, making it suitable for resource-constrained mining environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNIV OF MINING & TECH
- Filing Date
- 2025-05-30
- Publication Date
- 2026-05-29
AI Technical Summary
Existing video surveillance images of coal mining faces suffer from problems such as blurry images, low resolution, and lack of detail, resulting in poor image analysis and intelligent recognition performance. Furthermore, existing super-resolution reconstruction methods have high computational costs, making them difficult to apply in real time in resource-constrained mining environments.
A multidimensional collaborative covariant feature processing architecture is adopted. Through multi-receptive field feature extraction and collaborative covariant learning units, combined with lightweight convolution operations, efficient image feature fusion and reconstruction are achieved. This includes three groups of grouped convolutions with different kernel sizes and multi-level lightweight residual attention blocks, which dynamically adjust the feature fusion weights to reduce redundant computation.
While reducing model complexity and the number of parameters, it significantly improves image clarity and reconstruction results, adapts to the computational resource constraints of the mining environment, and enhances image quality and system practicality.
Smart Images
Figure CN120672575B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for super-resolution reconstruction of low-quality blurred images of coal mining faces. Background Technology
[0002] With the continuous maturation of deep learning technology, its application in image processing has deepened, greatly promoting the intelligentization process of many industries. In the coal mining sector, the coal face, as the core working space for direct coal mining underground, encompasses the entire production process from coal seam breaking, coal body cutting, coal transportation to discharge. Video surveillance images from the coal face not only reflect the real-time safety status of the mine's working environment, such as key indicators like gas concentration and support structure stability, but also monitor equipment operating status and maintenance needs, while effectively supervising the behavior and operating procedures of workers. This image data plays a crucial role in improving coal mine safety management, preventing accidents, optimizing production processes, and increasing production efficiency. However, due to the complex underground environment and limited camera equipment, video surveillance images from coal mining faces often suffer from quality problems such as blurry images, low resolution, and missing details, seriously affecting subsequent image analysis and intelligent recognition. Therefore, how to use efficient image processing technologies, especially deep learning-based super-resolution reconstruction methods, to align, enhance, and restore details of low-quality video images has become an important research direction for improving the performance of intelligent monitoring systems in coal mines and ensuring safe production.
[0003] To address this need, mine image super-resolution reconstruction technology has emerged. The core of this technology is to generate high-resolution images from low-resolution images using deep learning algorithms, thereby effectively improving image clarity and allowing for better display of details. Compared to traditional image reconstruction methods, deep learning-based super-resolution reconstruction technology can significantly improve image clarity at a lower cost without hardware upgrades, providing strong support for mine environmental monitoring and production management. This technology reduces equipment replacement costs and improves image quality even with limited storage capacity, providing reliable visual information support for mine safety supervision, equipment maintenance, and mineral resource exploration.
[0004] Currently, super-resolution reconstruction methods for mine images mainly include four steps: first, preprocessing the low-resolution image; second, constructing a deep learning network model, including feature extraction and reconstruction modules; third, training and optimizing the model's parameters; and finally, generating a high-resolution image using the trained model. In existing technologies, super-resolution reconstruction networks often extract richer image features through complex modules. While this improves reconstruction results, it also significantly increases the computational cost of the model, making it difficult to implement real-time applications in resource-constrained scenarios, especially in complex environments such as mines. Secondly, in the feature fusion process, existing methods often neglect the synergistic covariation relationships between multi-dimensional high- and low-frequency information, processing each dimension of features independently. This approach discards the holistic and systematic nature of features at each stage, making it difficult for the model to fully explore the correlations and complementarities between features, thus affecting the effectiveness and efficiency of image reconstruction. Summary of the Invention
[0005] The purpose of this invention is to provide a method and system for super-resolution reconstruction of low-quality blurred images of coal mining faces, which can improve image clarity while maintaining the lightweight model structure, thus meeting the dual requirements of computing resources and processing effect in the mining environment.
[0006] To achieve the above objectives, the present invention provides a method for super-resolution reconstruction of low-quality blurred images of coal mining faces, comprising the following steps:
[0007] S1. Input a low-resolution mine image and extract the multi-receptive field feature information of the image;
[0008] S2. For this multi-receptive field feature, high-frequency feature fusion processing is performed by using multi-dimensional collaborative covariant feature weighted adaptive fusion residual group to achieve lightweight weighted adaptive collaborative feature fusion.
[0009] S3. Perform multi-receptive field feature fusion and upsampling processing on the fusion features to achieve high-resolution mine image reconstruction, and finally output the mine image super-resolution reconstruction result.
[0010] As a further aspect of the present invention: S1 employs three groups of convolutional calculations with kernel sizes of 1×1, 3×3, and 5×5 respectively. The low-frequency features of the multi-receptive field obtained from different convolutional calculations are then spliced along the feature dimension of the tensor to obtain the multi-dimensional low-frequency feature information of the image.
[0011] As a further aspect of the present invention: the multidimensional collaborative covariant feature weighted adaptive fusion residual group in S2 is composed of 4 collaborative covariant learning (CCL) units and 4 multi-level lightweight residual attention (MLRA) blocks; the CCL unit calculates the attention of multi-receptive field features at different scales, and uses the attention to achieve response adjustment to different receptive field features, thereby achieving collaborative covariant processing of multi-receptive field features under attention adjustment at different scales; the MLRA block realizes high-frequency information calculation of multi-receptive field features through alternating calculation of lightweight secondary residual features and multi-receptive field residual attention features.
[0012] As a further aspect of the present invention: using F CCL(i) (·) represents the computation of the i-th covariant learning unit, F MLRA(i) (·) represents the computation of the i-th multi-level lightweight residual attention block. The computation process of the lightweight weighted adaptive collaborative feature fusion result X1 is as follows:
[0013] 1) First, the low-frequency features X0 from the multiple receptive fields are input into the first covariant learning unit F. CCL(1) (·) Calculate and obtain the covariant learning features X ccl(1) Then the input is fed into the first multi-level lightweight residual attention block F. MLRA(1) (·) Calculate and obtain the multi-level lightweight residual attention X mlra(1) The calculation formula is as follows:
[0014] X ccl(1) =F CCL(1) (X0)
[0015] X mlra(1) =F MLRA(1) (X ccl(1) )
[0016] 2) Next, the calculation formulas for the second, third, and fourth multi-level lightweight residual attention blocks are as follows:
[0017] X mlra(i) =F MLRA(i) (X mlra(i-1) )
[0018] Here, i = 2, 3, 4, which allows us to obtain multi-level lightweight residual attention features X with different dimensions. mlra(2) X mlra(3) and X mlra(4) As the number of levels increases, multi-level lightweight residual attention features are more inclined towards attention representation of deep semantics;
[0019] 3) The calculation formulas for the second, third, and fourth covariant learning units are shown below:
[0020] X ccl2 =F CCL(2) (F concat (X mlra(1) ,X mlra(2) ))
[0021] X ccl(3) =F CCL(3) (F concat (X ccl(2) ,X mlra(3) ))
[0022] X ccl(4) =F CCL(4) (F concat (X ccl(3) ,X mlra(4) ))
[0023] 4) Finally, the collaborative covariance learning features at different levels are fused to obtain the lightweight weighted adaptive collaborative feature fusion result. The calculation formula is as follows:
[0024] X1=λ1X ccl(1) +λ2f sigmoid (X ccl(2) )·X ccl(3) +λ3X ccl(4)
[0025] Among them, f sigmoid This represents the calculation of the sigmoid nonlinear transformation, where λ1, λ2, and λ3 represent the adaptive parameters, and here the shallow X... ccl(1) and deep X ccl(4) The traditional feature addition fusion method is used, while the intermediate layer X ccl(2) and X ccl(3) This is achieved by multiplying the responses to unidirectional attention reinforcement;
[0026] In the multidimensional covariant feature weighted adaptive fusion residual group, the CCL unit and MLRA block at the same level co-infer image features and finally perform specific fusion of covariant learning features to achieve lightweight weighted adaptive covariant feature fusion.
[0027] As a further aspect of the present invention: the collaborative covariance learning unit performs large-scale group feature extraction and small-scale feature extraction on the input multi-dimensional feature information to obtain multi-dimensional features under large-scale perspective representation and small-scale perspective representation, and performs adaptive weighted fusion on the extracted features of different scales to obtain collaborative covariance learning features of multi-dimensional features.
[0028] As a further aspect of the present invention: the multi-level lightweight residual attention block is composed of 4 sets of lightweight secondary residual units (LSR) and multi-scale receptive field residual attention units (MRA) alternately, wherein each level of LSR unit and MRA unit constitutes one level of lightweight residual attention block;
[0029] In each lightweight residual attention block, the input features are first extracted by the LSR unit using high-frequency information features represented by the quadratic residual. Then, the MRA unit is used to calculate the residual attention features from multiple receptive field perspectives, thereby adjusting the response intensity of the features of interest from different receptive field perspectives. Finally, the residual attention features are superimposed and fused into the input features.
[0030] To achieve the above objectives, the present invention also provides a super-resolution reconstruction system for low-quality blurred images of coal mining faces, comprising:
[0031] The mine image acquisition module collects raw image data from different times, perspectives, and sensors in a mine scene.
[0032] The preprocessing module has its input end connected to the output end of the mine image acquisition module, and performs preprocessing operations on the images acquired by the mine image acquisition module.
[0033] The feature extraction and encoding module connects its input to the output of the preprocessing module to perform structured encoding of image information and extract multi-scale feature information of the image using a multi-branch, multi-scale convolutional neural network.
[0034] The multidimensional covariance modeling module connects its input to the output of the feature extraction and encoding module, fusing image features from different dimensions and scales to model the covariance relationships between different images.
[0035] The super-resolution reconstruction module connects its input to the output of the multidimensional covariance modeling module. Based on the feature fusion and model processing of the multidimensional covariance modeling module, it outputs a reconstructed high-resolution mine image.
[0036] The reconstruction image evaluation and feedback module connects its input to the output of the super-resolution reconstruction module. It evaluates the quality of the high-resolution mine image output by the super-resolution reconstruction module and uses it for subsequent model optimization or reconstruction feedback.
[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0038] First, efficient feature processing: By developing a multi-dimensional collaborative covariation feature processing architecture, this invention captures the collaborative covariation relationship between multi-dimensional high- and low-frequency features, dynamically adjusts feature fusion weights, avoids redundant calculations, and achieves more accurate and efficient feature processing.
[0039] Second, lightweight model design: Optimized feature processing structure and lightweight convolution operations are adopted to reduce redundant calculations and significantly reduce the number and complexity of model parameters, making the system more suitable for resource-constrained equipment in mining environments while maintaining high reconstruction accuracy.
[0040] Third, adapting to special scenario requirements: In response to the problems of harsh lighting conditions and limitations of acquisition equipment in special environments such as mines, this invention has made special improvements to the feature extraction module. By focusing on key features that contribute to the reconstruction effect through feature co-variation learning units, the image clarity and practicality are effectively improved, meeting the actual needs in complex and special scenarios such as mines. Attached Figure Description
[0041] Figure 1 This is a flowchart of the method for super-resolution reconstruction of low-quality blurred images of coal mining faces according to the present invention.
[0042] Figure 2 This is a block diagram of the multidimensional collaborative covariance feature weighted adaptive fusion residual group of the present invention.
[0043] Figure 3 This is a schematic diagram of the collaborative common variable learning unit of the present invention.
[0044] Figure 4 This is a schematic diagram of the multi-level lightweight residual attention block of the present invention.
[0045] Figure 5 This is a schematic diagram of the lightweight secondary residual unit of the present invention.
[0046] Figure 6 This is a schematic diagram of the multi-receptive field residual attention unit of the present invention.
[0047] Figure 7 This invention is a convolution f SC_1×1 (·) and f SC_1×1 A schematic diagram of the activated pixels (·).
[0048] Figure 8 This is a schematic diagram of the input image and its super-resolution image result of the present invention.
[0049] Figure 9 This invention is a super-resolution reconstruction system for low-quality blurred images of coal mining faces. Detailed Implementation
[0050] The present invention will be further illustrated by the following examples.
[0051] like Figure 1 As shown, the method for super-resolution reconstruction of low-quality blurred images of coal mining faces includes the following steps:
[0052] S1. Input a low-resolution mine image and extract the multi-receptive field feature information of the image;
[0053] S2. For this multi-receptive field feature, high-frequency feature fusion processing is performed by using multi-dimensional collaborative covariant feature weighted adaptive fusion residual group to achieve lightweight weighted adaptive collaborative feature fusion.
[0054] S3. Perform multi-receptive field feature fusion and upsampling processing on the fusion features to achieve high-resolution mine image reconstruction, and finally output the mine image super-resolution reconstruction result.
[0055] Furthermore, S1 employs three groups of convolutions with kernel sizes of 1×1, 3×3, and 5×5. Different kernel sizes result in different receptive fields for the calculated features, while grouped convolutions reduce computational complexity. The grouped convolution function directly calls the Conv2D module in the deep learning framework. The kernel sizes for different convolutions are set to 1, 3, and 5, respectively, and the corresponding output features are set to 64, 32, and 16. The number of convolution groups is set to 4 for all groups. Their calculations are performed using symbols...
[0056] Number f GC_1×1 (·), f GC_3×3 (·) and f GC_5×5 (·) indicates that the low-frequency features obtained from different convolutions are then concatenated along the feature dimensions of the tensor to obtain the multi-dimensional low-frequency feature information of the image. The input low-resolution image is X. LR The formula for calculating the low-frequency feature X0 of the multi-receptive field of an image is as follows:
[0057] X0 = F concat (f GC_1×1 (X LR ),f GC_3×3 (X LR ),f GC_5×5 (X LR ))
[0058] In the formula, F concat This indicates a feature concatenation operation along the tensor feature dimension.
[0059] Furthermore, such as Figure 2As shown, the multidimensional collaborative covariant feature weighted adaptive fusion residual group in S2 consists of four collaborative covariant learning (CCL) units and four multi-level lightweight residual attention (MLRA) blocks. The CCL unit calculates the attention of multi-receptive field features at different scales and uses this attention to adjust the response to different receptive field features, thereby achieving collaborative covariant processing of multi-receptive field features under attention adjustment at different scales. The MLRA block realizes the high-frequency information calculation of multi-receptive field features through the alternating calculation of lightweight quadratic residual features and multi-receptive field residual attention features.
[0060] Furthermore, with F CCL(i) (·) represents the computation of the i-th covariant learning unit, F MLRA(i) (·) represents the computation of the i-th multi-level lightweight residual attention block. The computation process of the lightweight weighted adaptive collaborative feature fusion result X1 is as follows:
[0061] 1) First, the low-frequency features X0 from the multiple receptive fields are input into the first covariant learning unit F. CCL(1) (·) Calculate and obtain the covariant learning features X ccl(1) Then the input is fed into the first multi-level lightweight residual attention block F. MLRA(1) (·) Calculate and obtain the multi-level lightweight residual attention X mlra(1) The calculation formula is as follows:
[0062] X ccl(1) =F CCL(1) (X0)
[0063] X mlra(1) =F MLRA(1) (X ccl(1) )
[0064] 2) Next, the calculation formulas for the second, third, and fourth multi-level lightweight residual attention blocks are as follows:
[0065] X mlra(i) =F MLRA(i) (X mlra(i-1) )
[0066] Here, i = 2, 3, 4, which allows us to obtain multi-level lightweight residual attention features X with different dimensions. mlra(2) X mlra(3) and X mlra(4) As the number of levels increases, multi-level lightweight residual attention features are more inclined towards attention representation of deep semantics;
[0067] 3) The calculation formulas for the second, third, and fourth covariant learning units are shown below:
[0068] X ccl2 =F CCL(2) (F concat (X mlra(1) ,X mlra(2) ))
[0069] X ccl(3) =F CCL(3) (F concat (X ccl(2) ,X mlra(3) ))
[0070] X ccl(4) =F CCL(4) (F concat (X ccl(3) ,X mlra(4) ))
[0071] 4) Finally, the collaborative covariance learning features at different levels are fused to obtain the lightweight weighted adaptive collaborative feature fusion result. The calculation formula is as follows:
[0072] X1=λ1X ccl(1) +λ2f sigmoid (X ccl(2) )·X ccl(3) +λ3X ccl(4)
[0073] Among them, f sigmoid This represents the calculation of the sigmoid nonlinear transformation, where λ1, λ2, and λ3 represent the adaptive parameters, and here the shallow X... ccl(1) and deep X ccl(4) The traditional feature addition fusion method is used, while the intermediate layer X ccl(2) and X ccl(3) This is achieved by multiplying the responses to unidirectional attention;
[0074] In the multidimensional covariant feature weighted adaptive fusion residual group, the CCL unit and MLRA block at the same level co-infer image features and finally perform specific fusion of covariant learning features to achieve lightweight weighted adaptive covariant feature fusion.
[0075] Furthermore, the co-variant learning unit performs large-scale grouping feature extraction and small-scale feature extraction on the input multi-dimensional feature information to obtain multi-dimensional features under large-scale and small-scale perspective representations. It then adaptively weights and fuses the extracted features at different scales to obtain co-variant learning features of the multi-dimensional features, such as... Figure 3 As shown. Assume the input multidimensional features are... Where s represents a feature size of b×c×h×w, and n represents a multidimensional feature containing n features. Covariant learning unit F CCL(i) The calculation process of (·) can be broken down into the following formula:
[0076]
[0077] Among them, F LK (·) and F SK (·) represent large-scale grouped feature extraction and small-scale feature extraction, respectively. Large-scale grouped feature extraction uses grouped convolutions with a kernel size of 5×5 to achieve this function while maintaining a small computational cost. Small-scale feature extraction uses convolutions with a kernel size of 1×1 to achieve this function in a lightweight manner. μ1 and μ2 represent their adaptive weighting coefficients, which are represented using learnable hyperparameters, thereby automatically calculating the optimal weighting coefficients during model training.
[0078] Furthermore, the multi-level lightweight residual attention block is composed of four sets of lightweight secondary residual units (LSRs) and multi-scale receptive field residual attention units (MRAs) alternatingly. Each level of LSR and MRA units constitutes one level of lightweight residual attention block, meaning the multi-level lightweight residual attention block contains four levels of lightweight residual attention blocks, such as... Figure 4 As shown, in each lightweight residual attention block, the input features are first extracted by the LSR unit using quadratic residual representation. Then, the MRA unit calculates residual attention features from multiple receptive field perspectives, thereby adjusting the response intensity of the features of interest from different receptive field perspectives. Finally, the residual attention features are superimposed and fused into the input features. Compared with traditional residual computation, LSR and MRA use more small convolutions and grouped convolutions to keep the modules more lightweight, and introduce quadratic residuals and multi-receptive field residual attention to perform more detailed representation and response adjustment of high-frequency information in the features, enhancing the model's feature representation capability.
[0079] Taking the first-level lightweight residual attention block as an example, the input feature is X ccl(1) Then the output feature X LM(1) This can be expressed using the following formula:
[0080] X LM(1) =α1F MRA(1) (F LSR(1) (X ccl(1) ))+X ccl(1)
[0081] Among them, FMRA(1) express Figure 4 Calculation of the first-level MRA unit, F LSR(1) express Figure 4 The calculation of the first-level LSR unit, where α1 represents the learnable weights for adjusting the intensity of the multi-receptive-field residual attention feature response by the first-level lightweight residual attention block. Similarly, Figure 4 The output feature X of the j-th (j = 2, 3 and 4) level lightweight residual attention block LM(j) The calculation method can be expressed by the following formula:
[0082] X LM(j) =α j F MRA(j) (F LSR(j) (X LM(j-1) ))+X LM(j-1)
[0083] =α j F MRA(j) (X LSR(j) )+X LM(j-1)
[0084] =α j X MRA(j) +X LM(j-1)
[0085] Among them, F MRA(j) express Figure 4 The calculation of the j-th level MRA element, F LSR(j) express Figure 4 Calculation of the j-th level LSR element, X LM(4) It is also the output feature of multi-level lightweight residual attention blocks, X LSR(j) and X MRA(j) Let α represent the output characteristics of the j-th stage LSR unit and MRA unit, respectively. j The learnable weights represent the intensity of the multi-receptive field residual attention feature response modulated by the j-th level lightweight residual attention block.
[0086] The calculation process for lightweight quadratic residual elements is as follows: Figure 5 As shown. Unlike traditional residual units, lightweight quadratic residual units further process the primary residual into a secondary residual and construct a secondary attention residual. Taking the first-level lightweight quadratic residual (LSR) unit as an example, the input feature is X. ccl(1) Then the first-order residual X r1 and the second residual X r2 The calculation is as follows:
[0087] X t1 =f 3×3 (f 1×1 (X ccl(1) ))
[0088] X t2 =f 3×3 (X t1 )
[0089] X t3 =f 3×3 (X t2 )
[0090] X r1 =f 1×1 (f GELU (X t3 ))
[0091] X r2 =f 1×1 (f sigmoid (f 1×1 (f 1×1 (F concat (X t1 ,X t2 ))))·f 1×1 (F concat (X t2 ,X t3 )))
[0092] Among them, f GELU and f sigmoid Let f be the GELU and Sigmoid activation functions, respectively. 1×1 and f 3×3 These represent convolution calculations with kernels of 1×1 and 3×3, respectively. t1 X t2 and X t3 They represent Figure 5 The first, second, and third f in the first residual 3×3 The output feature values. Then, the input feature X of the first-level LSR unit. ccl(1) With the first residual X r1 and the second residual X r2 Fusion yields the output characteristic X of the first-stage LSR unit. LSR(1) The calculation formula is as follows:
[0093] X LSR(1) =f 3×3 (X ccl(1) +X r1 )+X r2
[0094] Similarly, X can be calculated LSR(j) j = 2, 3 and 4.
[0095] The computational flow of a multi-receptive-field residual attention (MRA) unit is as follows: Figure 6As shown. This unit uses the output of the LSR unit at the same level as the input features, respectively from the first level where f SC_1×1 (·) represents a convolution with a specific stride and a kernel size of 1×1, where the activated pixels are convolved with a regular convolution with a kernel size of 1×1. 1×1 The differences between (·) are as follows: Figure 7 As shown, the darker areas represent pixels that are in the convolution activation state during convolution calculation, f SC_1×1 (·) has fewer and sparser active pixels, thus reducing feature computation and enhancing focus on features of interest. GC_3×3 (·) and f GC_1×1 (·) represents grouped convolution calculations with kernel sizes of 3×3 and 1×1, respectively. Grouped convolution calculations further reduce the computational cost, and calculations with different kernel sizes also obtain features under multiple receptive fields. up (·) indicates a Pixeshuffle upsampling layer. Taking the first-level MRA unit as an example, the input feature is X. LSR(1) Multi-sensory field residual characteristics X mres The calculation formula is:
[0096] X sc =f 1×1 (f SC_1×1 (f 1×1 (X LSR(1) )))
[0097] X mres =f up (f GC_3×3 (f GC_1×1 (X sc )+f GC_3×3 (X sc ))+X sc )
[0098] Next, the residual characteristics of multiple receptive fields X mres With the leftmost branch f GC_3×3 The computational features of (·) are fused, and the multi-receptive-field residual attention features X are obtained through the following calculation. MRA(1) The calculation formula is as follows:
[0099] X MRA(1) =X LSR(1) ·f sigmoid (f 1×1 (X mres ))
[0100] Similarly, X can be calculated. MRA(j) j is 2, 3, or 4.
[0101] The output feature X1 after lightweight weighted adaptive collaborative feature fusion is used to reconstruct a high-resolution mine image to obtain a super-resolution image X. SR The calculation process is as follows:
[0102] X SR =f up (f GC_5×5 (X1)+f GC_3×3 (X1))
[0103] Low-resolution images and their corresponding super-resolution images are as follows: Figure 8 As shown.
[0104] like Figure 9 As shown, a super-resolution reconstruction system for low-quality blurred images of coal mining faces includes:
[0105] The mine image acquisition module collects raw image data from different times, perspectives, and sensors in a mine scene.
[0106] The preprocessing module has its input end connected to the output end of the mine image acquisition module, and performs preprocessing operations on the images acquired by the mine image acquisition module.
[0107] The feature extraction and encoding module connects its input to the output of the preprocessing module to perform structured encoding of image information and extract multi-scale feature information of the image using a multi-branch, multi-scale convolutional neural network.
[0108] The multidimensional covariance modeling module connects its input to the output of the feature extraction and encoding module, fusing image features from different dimensions and scales to model the covariance relationships between different images.
[0109] The super-resolution reconstruction module connects its input to the output of the multidimensional covariance modeling module. Based on the feature fusion and model processing of the multidimensional covariance modeling module, it outputs a reconstructed high-resolution mine image.
[0110] The reconstruction image evaluation and feedback module connects its input to the output of the super-resolution reconstruction module. It evaluates the quality of the high-resolution mine image output by the super-resolution reconstruction module and uses it for subsequent model optimization or reconstruction feedback.
Claims
1. A method for super-resolution reconstruction of low-quality blurred images of coal mining faces, characterized in that, Includes the following steps: S1. Input a low-resolution mine image and extract the multi-receptive field feature information of the image; Three groups of convolutions with kernel sizes of 1×1, 3×3 and 5×5 are used for calculation. The low-frequency features of the multi-receptive field obtained by different convolution calculations are then spliced along the feature dimension of the tensor to obtain the multi-dimensional low-frequency feature information of the image. S2. For this multi-receptive field feature, the residual feature is fused by weighted adaptive fusion of multi-dimensional collaborative covariance features to recover the high-frequency information of the image and realize lightweight weighted adaptive collaborative feature fusion. The multidimensional covariant feature weighted adaptive fusion residual group consists of four collaborative covariant learning (CCL) units and four multi-level lightweight residual attention (MLRA) blocks. The CCL unit calculates the attention of multi-receptive field features at different scales and uses this attention to adjust the response to different receptive field features, thereby achieving covariant processing of multi-receptive field features under attention adjustment at different scales. The MLRA block realizes high-frequency information calculation of multi-receptive field features through the alternating calculation of lightweight quadratic residual features and multi-receptive field residual attention features. by Let the computation of the i-th covariant learning unit be and let the input multidimensional features be . , where s represents the feature size is 'n' indicates that the multidimensional features contain a total of n features, and the covariant learning unit is... The calculation process can be broken down into the following formula: ; F concat () indicates a feature concatenation operation along the feature dimension of the tensor; in, and These represent large-scale group feature extraction and small-scale feature extraction, respectively. Large-scale group feature extraction is achieved by using group convolution with a kernel size of 5×5 while maintaining a small computational load. Small-scale feature extraction is achieved by using convolution with a kernel size of 1×1 to achieve the small-scale feature extraction in a lightweight manner. and These represent their adaptive weighting coefficients, which are expressed using learnable hyperparameters, thus automatically calculating the optimal weighting coefficients during model training. The computation of the i-th multi-level lightweight residual attention block is represented by the following: The multi-level lightweight residual attention block is composed of 4 sets of lightweight secondary residual units (LSRs) and multi-scale receptive field residual attention units (MRAs) alternatingly, where each level of LSR and MRA units constitutes one level of lightweight residual attention block; the output features of the j-th level lightweight residual attention block are shown below. The calculation method can be expressed by the following formula: ； in, This represents the calculation of the j-th level MRA element. This represents the calculation of the j-th level LSR element. and Let α represent the output characteristics of the j-th stage LSR unit and MRA unit, respectively. j The learnable weights represent the intensity of the modulated multi-receptive field residual attention feature response of the j-th level lightweight residual attention block; Lightweight weighted adaptive collaborative feature fusion results The calculation process is as follows: 1) First, the low-frequency characteristics of multiple receptive fields... Input the first covariant learning unit Calculate and obtain covariant learning features Then the input is fed into the first multi-level lightweight residual attention block. Calculate and obtain multi-level lightweight residual attention The calculation formula is as follows: ； ； 2) Next, the calculation formulas for the second, third, and fourth multi-level lightweight residual attention blocks are as follows: ； Among them, here This allows us to obtain multi-level lightweight residual attention features in different dimensions. , and As the number of levels increases, multi-level lightweight residual attention features are more inclined towards attention representation of deep semantics; 3) The calculation formulas for the second, third, and fourth covariant learning units are shown below: ； ； ； 4) Finally, the collaborative covariance learning features at different levels are fused to obtain a lightweight weighted adaptive collaborative feature fusion result. The calculation formula is as follows: ； in, This represents the calculation of the sigmoid nonlinear transformation. , and These represent the adaptive parameters, here the shallow layer. and deep The traditional feature addition fusion method is used, while the intermediate layer... and This is achieved by multiplying the responses to unidirectional attention reinforcement; In the multidimensional covariant feature weighted adaptive fusion residual group, the CCL unit and MLRA block at the same level co-infer image features and finally perform specific fusion of covariant learning features to achieve lightweight weighted adaptive covariant feature fusion. S3. Perform multi-receptive field feature fusion and upsampling processing on the fusion features to achieve high-resolution mine image reconstruction, and finally output the mine image super-resolution reconstruction result.
2. The method for super-resolution reconstruction of low-quality blurred images of coal mining faces according to claim 1, characterized in that, The covariant learning unit performs large-scale grouping feature extraction and small-scale feature extraction on the input multidimensional feature information to obtain multidimensional features under large-scale perspective representation and small-scale perspective representation, and performs adaptive weighted fusion of the extracted features at different scales to obtain covariant learning features of multidimensional features.
3. The method for super-resolution reconstruction of low-quality blurred images of coal mining faces according to claim 1, characterized in that, In each lightweight residual attention block, the input features are first extracted by the LSR unit using high-frequency information features represented by the quadratic residual. Then, the MRA unit is used to calculate the residual attention features from multiple receptive field perspectives, thereby adjusting the response intensity of the features of interest from different receptive field perspectives. Finally, the residual attention features are superimposed and fused into the input features.
4. A super-resolution reconstruction system for low-quality blurred images of coal mining faces, characterized in that, The method for super-resolution reconstruction of low-quality blurred images of coal mining faces according to any one of claims 1-3 includes: The mine image acquisition module collects raw image data from different times, perspectives, and sensors in a mine scene. The preprocessing module has its input end connected to the output end of the mine image acquisition module, and performs preprocessing operations on the images acquired by the mine image acquisition module. The feature extraction and encoding module connects its input to the output of the preprocessing module to perform structured encoding of image information and extract multi-scale feature information of the image using a multi-branch, multi-scale convolutional neural network. The multidimensional covariance modeling module connects its input to the output of the feature extraction and encoding module, fusing image features from different dimensions and scales to model the covariance relationships between different images. The super-resolution reconstruction module connects its input to the output of the multidimensional covariance modeling module. Based on the feature fusion and model processing of the multidimensional covariance modeling module, it outputs a reconstructed high-resolution mine image. The reconstruction image evaluation and feedback module connects its input to the output of the super-resolution reconstruction module. It evaluates the quality of the high-resolution mine image output by the super-resolution reconstruction module and uses it for subsequent model optimization or reconstruction feedback.