Mine coal face low-quality blurred image super-resolution reconstruction method and system

Abstract

The invention discloses a super-resolution reconstruction method for a low-quality blurred image of a mine coal face. The super-resolution reconstruction method comprises the following steps: S1, inputting a low-resolution mine image and extracting multi-receptive-field feature information of the image; s2, performing high-frequency feature fusion processing on the multi-receptive-field features by using a multi-dimensional collaborative covariant feature weighted adaptive fusion residual group to realize lightweight weighted adaptive collaborative feature fusion; and S3, performing multi-receptive-field feature fusion and up-sampling processing on the fusion features to realize high-resolution mine image reconstruction, and finally outputting a mine image super-resolution reconstruction result. Meanwhile, the invention discloses a reconstruction system which comprises a mine image acquisition module, a preprocessing module, a feature extraction and coding module, a multi-dimensional collaborative covariant modeling module, a super-resolution reconstruction module and a reconstructed image evaluation and feedback module. According to the method, the image definition can be improved, meanwhile, the light weight of the model structure is kept, and the dual requirements for computing resources and the processing effect in the mine environment are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and system for super-resolution reconstruction of low-quality blurred images of a coal mining face in a mine. Background Art

[0002] As deep learning technology continues to mature, its application in image processing has deepened, significantly driving the intelligentization of many industries. In coal mining, the coal face, the core working space for direct underground coal mining, encompasses the entire production process, from seam collapse and coal cutting to coal transportation and discharge. Video surveillance images from the coal face not only provide real-time insights into the safety status of the mine's operating environment, such as key indicators like gas concentration and support structure stability, but also monitor equipment operating status and maintenance needs, while effectively monitoring worker behavior and operational standards. This image data plays a crucial role in improving coal mine safety management, preventing accidents, optimizing production processes, and increasing production efficiency. However, due to the complex underground environment and limited camera equipment, video surveillance images from coal mine faces often suffer from quality issues such as blur, low resolution, and loss of detail, severely impacting subsequent image analysis and intelligent recognition. Therefore, using efficient image processing techniques, particularly deep learning-based super-resolution reconstruction methods, to align, enhance, and restore detail from low-quality video images has become a key research direction for improving the performance of coal mine intelligent monitoring systems and ensuring safe mine production.

[0003] To address this need, mine image super-resolution reconstruction technology has emerged. This technology, based on deep learning algorithms, generates high-resolution images from low-resolution images, effectively improving image clarity and enabling better visualization of detailed information. Compared to traditional image reconstruction methods, deep learning-based super-resolution reconstruction significantly improves image clarity at a lower cost and without requiring hardware upgrades, providing strong support for mine environmental monitoring and production management. This technology reduces equipment replacement costs while improving image quality even with limited storage capacity, providing reliable visual information support for mine safety supervision, equipment maintenance, and mine resource exploration.

[0004] Currently, the super-resolution reconstruction method for mine images mainly includes four steps: first, pre-processing the low-resolution image; second, building a deep learning network model, including a feature extraction module and a reconstruction module; third, model training and parameter optimization; finally, using the trained model to generate high-resolution images. In the existing technology, super-resolution reconstruction networks often use complex modules to extract richer image features. While this method improves the reconstruction effect, it also significantly increases the computational overhead of the model, making it difficult for the model to be applied in real time in resource-constrained scenarios, especially in complex environments such as mines. Secondly, in the feature fusion process, existing methods often ignore the collaborative covariation relationship between multi-dimensional high- and low-frequency information, and process the features of each dimension independently. This processing method abandons the integrity and systematicity of the features at each stage, making it difficult for the model to fully explore the correlation and complementarity between features, affecting the effect and efficiency of image reconstruction. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for super-resolution reconstruction of low-quality blurred images of coal mining faces in mines, which can improve the clarity of images while keeping the model structure lightweight, and meet the dual requirements of computing resources and processing effects in a mine environment.

[0006] To achieve the above object, the present invention provides a method for super-resolution reconstruction of low-quality blurred images of a coal mining face in a mine, comprising the following steps:

[0007] S1, input low-resolution mine image and extract multi-receptive field feature information of the image;

[0008] S2. For the multi-receptive field features, a multi-dimensional collaborative covariant feature weighted adaptive fusion residual group is used to perform high-frequency feature fusion processing to achieve lightweight weighted adaptive collaborative feature fusion;

[0009] S3. Perform multi-receptive field feature fusion and upsampling processing on the fused features to achieve high-resolution mine image reconstruction, and finally output the mine image super-resolution reconstruction result.

[0010] As a further solution of the present invention: S1 adopts three groups of grouped convolution calculations with kernel sizes of 1×1, 3×3 and 5×5 respectively. The multi-receptive field low-frequency features obtained by different convolution calculations are then feature spliced ​​along the feature dimension of the tensor to obtain multi-dimensional low-frequency feature information of the image.

[0011] As a further solution of the present invention: the multi-dimensional collaborative covariant feature weighted adaptive fusion residual group in S2 is composed of 4 collaborative covariant learning units (CCL) and 4 multi-level lightweight residual attention blocks (MLRA); the CCL unit calculates the attention of multi-receptive field features of different scales, and uses the attention to adjust the response of different receptive field features, thereby achieving collaborative covariation processing of multi-receptive field features under attention adjustment of different scales; the MLRA block realizes the high-frequency information calculation of multi-receptive field features through alternating calculation of lightweight secondary residual features and multi-receptive field residual attention features.

[0012] As a further solution of the present invention: CCL(i) (·) represents the calculation of the i-th collaborative covariation learning unit, F MLRA(i) (·) represents the calculation of the i-th multi-level lightweight residual attention block, and the calculation process of the lightweight weighted adaptive collaborative feature fusion result X1 is as follows:

[0013] 1) First, the multi-receptive field low-frequency feature X0 is input into the first collaborative covariation learning unit F CCL(1) (·) Calculate and obtain the collaborative covariation learning feature X ccl(1) , then input to the first multi-level lightweight residual attention block F MLRA(1) (·) Calculate the multi-level lightweight residual attention X mlra(1) , the calculation formula is as follows:

[0014] X ccl(1) =F CCL(1) (X0)

[0015] X mlra(1) =F MLRA(1) (X ccl(1) )

[0016] 2) Next, the calculation formulas for the second, third, and fourth multi-level lightweight residual attention blocks are as follows:

[0017] X mlra(i) =F MLRA(i) (X mlra(i-1) )

[0018] Here, i = 2, 3, 4, and we can obtain multi-level lightweight residual attention features X of different dimensions. mlra(2) 、X mlra(3) and X mlra(4) ,With the increase of the level, the multi-level lightweight residual attention feature is more inclined to the attention representation of deep semantics;

[0019] 3) The calculation formulas for the second, third, and fourth synergistic covariation learning units are as follows:

[0020] X ccl2 =F CCL(2) (F concat (X mlra(1) ,X mlra(2) ))

[0021] X ccl(3) =F CCL(3) (F concat (X ccl(2) ,X mlra(3) ))

[0022] X ccl(4) =F CCL(4) (F concat (X ccl(3) ,X mlra(4) ))

[0023] 4) Finally, the collaborative covariation learning features at different levels are fused to obtain the lightweight weighted adaptive collaborative feature fusion result. The calculation formula is as follows:

[0024] X1=λ1X ccl(1) +λ2f sigmoid (X ccl(2) )·X ccl(3) +λ3X ccl(4)

[0025] Among them, f sigmoid Represents the sigmoid nonlinear transformation calculation, λ1, λ2 and λ3 represent the adaptive parameters, where the shallow X ccl(1) and deep X ccl(4) The traditional feature addition fusion method is used, and the middle layer X ccl(2) and X ccl(3) The regulation of same-direction attention reinforcement response is achieved through multiplication;

[0026] In the multi-dimensional collaborative co-varying feature weighted adaptive fusion residual group, the CCL unit and MLRA block at the same level collaboratively infer image features and finally perform specific fusion of collaborative co-varying learning features to achieve lightweight weighted adaptive collaborative feature fusion.

[0027] As a further solution of the present invention: the collaborative covariation learning unit performs large-scale grouping feature extraction and small-scale feature extraction on the input multidimensional feature information to obtain multidimensional features under large-scale perspective representation and small-scale perspective representation, and adaptively weighted fusion of the extracted features of different scales to obtain collaborative covariation learning features of the multidimensional features.

[0028] As a further solution of the present invention: the multi-level lightweight residual attention block is composed of four groups of lightweight secondary residual units (LSR) and multi-scale receptive field residual attention units (MRA) alternatingly, where each level of LSR units and MRA units constitutes one level of lightweight residual attention block;

[0029] In each level of lightweight residual attention block, the input features are first extracted by the LSR unit using the quadratic residual representation of high-frequency information features, and then the MRA unit is used to calculate the residual attention features from multiple receptive field perspectives, thereby adjusting the response strength of the features of interest under different receptive field perspectives. Finally, the residual attention features are superimposed and fused into the input features.

[0030] To achieve the above-mentioned object, the present invention further provides a super-resolution reconstruction system for low-quality blurred images of a coal mining face in a mine, comprising:

[0031] Mine image acquisition module, which collects raw image data of mine scenes at different times, viewing angles, and sensors;

[0032] A preprocessing module, wherein the input end of the preprocessing module is connected to the output end of the mine image acquisition module, and performs a preprocessing operation on the image acquired by the mine image acquisition module;

[0033] The feature extraction and encoding module has its input connected to the output of the preprocessing module, performs structured encoding on the image information, and uses a multi-branch, multi-scale convolutional neural network to extract multi-scale feature information of the image;

[0034] A multi-dimensional collaborative co-variation modeling module, whose input is connected to the output of the feature extraction and encoding module, integrates image features from different dimensions and multiple scales, and models the co-variation relationship between different images;

[0035] A super-resolution reconstruction module, wherein the input end of the super-resolution reconstruction module is connected to the output end of the multi-dimensional collaborative co-variation modeling module, and the multi-dimensional collaborative co-variation modeling module is used to fuse features and process the model to output a reconstructed high-resolution mine image;

[0036] The reconstructed image evaluation and feedback module has its input connected to the output of the super-resolution reconstruction module, performs quality evaluation on the high-resolution mine image output by the super-resolution reconstruction module, and uses it for subsequent model optimization or reconstruction feedback.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] First, efficient feature processing: By developing a multi-dimensional collaborative co-variation feature processing architecture, the present invention captures the collaborative co-variation relationship between multi-dimensional high- and low-frequency features, dynamically adjusts feature fusion weights, avoids redundant calculations, and achieves more accurate and efficient feature processing.

[0039] Second, lightweight model design: The use of optimized feature processing structure and lightweight convolution operations reduces redundant calculations and significantly reduces the number and complexity of model parameters, making the system more suitable for resource-constrained equipment in mining environments while maintaining high reconstruction accuracy.

[0040] Third, adapt to the needs of special scenarios: In response to the harsh lighting conditions and limitations of acquisition equipment in special environments such as mines, the present invention has made special improvements to the feature extraction module. Through the feature collaborative covariation learning unit, it focuses on key features that contribute to the reconstruction effect, effectively improving image clarity and practicality, and meeting the actual needs in complex special scenarios such as mines. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of the method for super-resolution reconstruction of low-quality blurred images of a coal mining face in a mine according to the present invention.

[0042] Figure 2 This is a block diagram of the weighted adaptive fusion residual group of multi-dimensional collaborative co-varying features of the present invention.

[0043] Figure 3 It is a schematic diagram of the collaborative variable learning unit of the present invention.

[0044] Figure 4 Schematic diagram of the multi-level lightweight residual attention block of the present invention.

[0045] Figure 5 Schematic diagram of the lightweight quadratic residual unit of the present invention.

[0046] Figure 6 Schematic diagram of the multi-receptive field residual attention unit of the present invention.

[0047] Figure 7 is the convolution f of the present invention SC_1×1 (·) and f SC_1×1 Schematic diagram of activated pixels of (·).

[0048] Figure 8 Schematic diagram of the input image and its super-resolution image result according to the present invention.

[0049] Figure 9 The invention relates to a super-resolution reconstruction system for low-quality fuzzy images of a coal mining face in a mine. DETAILED DESCRIPTION

[0050] The present invention will be further described below by way of examples.

[0051] like Figure 1 As shown in FIG, a super-resolution reconstruction method for a low-quality blurred image of a coal mining face in a mine comprises the following steps:

[0052] S1, input low-resolution mine image and extract multi-receptive field feature information of the image;

[0053] S2. For the multi-receptive field features, a multi-dimensional collaborative covariant feature weighted adaptive fusion residual group is used to perform high-frequency feature fusion processing to achieve lightweight weighted adaptive collaborative feature fusion;

[0054] S3. Perform multi-receptive field feature fusion and upsampling processing on the fused features to achieve high-resolution mine image reconstruction, and finally output the mine image super-resolution reconstruction result.

[0055] Furthermore, S1 uses three groups of grouped convolution calculations with kernel sizes of 1×1, 3×3, and 5×5. Different convolution kernel sizes result in different feature receptive fields, and convolution group calculation reduces the amount of convolution calculation. Here, the grouped convolution calculation function directly calls the Conv2D module in the deep learning framework. The convolution kernel size parameters corresponding to different convolutions are set to 1, 3, and 5, respectively. The corresponding output features are set to 64, 32, and 16, respectively. The number of convolution groups is set to 4, and their calculations are respectively performed using the symbol

[0056] No.f GC_1×1 (·),f GC_3×3 (·) and f GC_5×5 (·) indicates that the multi-receptive field low-frequency features obtained by different convolution calculations are then spliced ​​along the feature dimension of the tensor to obtain the multi-dimensional low-frequency feature information of the image. The input low-resolution image is X LR , then the calculation formula of the image multi-receptive field low-frequency feature X0 is as follows:

[0057] X0=F concat (f GC_1×1 (X LR ),f GC_3×3 (X LR ),f GC_5×5 (X LR ))

[0058] Where, F concat Represents feature concatenation along the feature dimension of the tensor.

[0059] Further, such as Figure 2As shown in the figure, the multi-dimensional collaborative covariant feature weighted adaptive fusion residual group in S2 is composed of four collaborative covariant learning units (CCL) and four multi-level lightweight residual attention blocks (MLRA); the CCL unit calculates the attention of multi-receptive field features of different scales, and uses this attention to adjust the response of different receptive field features, thereby achieving collaborative covariation processing of multi-receptive field features under attention adjustment of different scales; the MLRA block realizes the high-frequency information calculation of multi-receptive field features by alternating calculation of lightweight quadratic residual features and multi-receptive field residual attention features.

[0060] Furthermore, F CCL(i) (·) represents the calculation of the i-th collaborative covariation learning unit, F MLRA(i) (·) represents the calculation of the i-th multi-level lightweight residual attention block, and the calculation process of the lightweight weighted adaptive collaborative feature fusion result X1 is as follows:

[0061] 1) First, the multi-receptive field low-frequency feature X0 is input into the first collaborative covariation learning unit F CCL(1) (·) Calculate and obtain the collaborative covariation learning feature X ccl(1) , then input to the first multi-level lightweight residual attention block F MLRA(1) (·) Calculate the multi-level lightweight residual attention X mlra(1) , the calculation formula is as follows:

[0062] X ccl(1) =F CCL(1) (X0)

[0063] X mlra(1) =F MLRA(1) (X ccl(1) )

[0064] 2) Next, the calculation formulas for the second, third, and fourth multi-level lightweight residual attention blocks are as follows:

[0065] X mlra(i) =F MLRA(i) (X mlra(i-1) )

[0066] Here, i = 2, 3, 4, and we can obtain multi-level lightweight residual attention features X of different dimensions. mlra(2) 、X mlra(3) and X mlra(4) ,With the increase of the level, the multi-level lightweight residual attention feature is more inclined to the attention representation of deep semantics;

[0067] 3) The calculation formulas for the second, third, and fourth synergistic covariation learning units are as follows:

[0068] X ccl2 =F CCL(2) (F concat (X mlra(1) ,X mlra(2) ))

[0069] X ccl(3) =F CCL(3) (F concat (X ccl(2) ,X mlra(3) ))

[0070] X ccl(4) =F CCL(4) (F concat (X ccl(3) ,X mlra(4) ))

[0071] 4) Finally, the collaborative covariation learning features at different levels are fused to obtain the lightweight weighted adaptive collaborative feature fusion result. The calculation formula is as follows:

[0072] X1=λ1X ccl(1) +λ2f sigmoid (X ccl(2) )·X ccl(3) +λ3X ccl(4)

[0073] Among them, f sigmoid Represents the sigmoid nonlinear transformation calculation, λ1, λ2 and λ3 represent the adaptive parameters, where the shallow X ccl(1) and deep X ccl(4) The traditional feature addition fusion method is used, and the middle layer X ccl(2) and X ccl(3) The regulation of same-direction attention reinforcement response is achieved through multiplication;

[0074] In the multi-dimensional collaborative co-varying feature weighted adaptive fusion residual group, the CCL unit and MLRA block at the same level collaboratively infer image features and finally perform specific fusion of collaborative co-varying learning features to achieve lightweight weighted adaptive collaborative feature fusion.

[0075] Furthermore, the collaborative covariation learning unit performs large-scale grouping feature extraction and small-scale feature extraction on the input multi-dimensional feature information to obtain multi-dimensional features under large-scale perspective representation and small-scale perspective representation, and performs adaptive weighted fusion on the extracted features of different scales to obtain collaborative covariation learning features of multi-dimensional features, such as Figure 3 As shown. Assume that the input multidimensional features are Where s represents the feature size of b×c×h×w, and n represents the multi-dimensional feature contains n features. CCL(i) The calculation process of (·) can be decomposed into the following formula:

[0076]

[0077] Among them, F LK (·) and F SK (·) denotes large-scale group feature extraction and small-scale feature extraction, respectively. Large-scale group feature extraction uses group convolution with a kernel size of 5×5 to achieve this function while keeping the computational complexity low. Small-scale feature extraction uses convolution with a kernel size of 1×1 to achieve lightweight implementation. μ1 and μ2 denote their adaptive weighting coefficients, respectively. These coefficients are expressed as learnable hyperparameters, allowing automatic calculation of the optimal weighting coefficients during model training.

[0078] Furthermore, the multi-level lightweight residual attention block is composed of four groups of lightweight secondary residual units (LSR) and multi-scale receptive field residual attention units (MRA) alternately, where each level of LSR units and MRA units constitutes a level of lightweight residual attention block, that is, the multi-level lightweight residual attention block contains four levels of lightweight residual attention blocks, such as Figure 4 As shown in the figure. In each level of lightweight residual attention block, the input features first pass through the LSR unit to extract high-frequency information features represented by quadratic residuals. The MRA unit then calculates residual attention features from multiple receptive field perspectives to adjust the response strength of the features of interest under different receptive field perspectives. Finally, the residual attention features are superimposed and fused into the input features. Compared with traditional residual calculation, LSR and MRA use a lightweight approach of more small convolutions and grouped convolutions to keep the module more lightweight. They also introduce quadratic residuals and multi-receptive field residual attention to provide more detailed representation and response adjustment of high-frequency information in the features, enhancing the model's feature representation capabilities.

[0079] Taking the first-level lightweight residual attention block as an example, the input feature is X ccl(1) , then the output feature X LM(1) It can be expressed using the following formula:

[0080] X LM(1) =α1F MRA(1) (F LSR(1) (X ccl(1) ))+X ccl(1)

[0081] Among them, FMRA(1) express Figure 4 Calculation of the first level MRA unit, F LSR(1) express Figure 4 The calculation of the first-level LSR unit in , α1 represents the learnable weight of the first-level lightweight residual attention block to adjust the response strength of the multi-receptive field residual attention feature. Similarly, Figure 4 The output feature X of the j-th (j=2, 3 and 4) level lightweight residual attention block LM(j) The calculation method can be expressed by the following formula:

[0082] X LM(j) =α j F MRA(j) (F LSR(j) (X LM(j-1) ))+X LM(j-1)

[0083] =α j F MRA(j) (X LSR(j) )+X LM(j-1)

[0084] =α j X MRA(j) +X LM(j-1)

[0085] Among them, F MRA(j) express Figure 4 The calculation of the j-th level MRA unit, F LSR(j) express Figure 4 The calculation of the j-th level LSR unit, X LM(4) It is also the output feature of the multi-level lightweight residual attention block, X LSR(j) and X MRA(j) Represent the output features of the j-th level LSR unit and MRA unit, α j Represents the learnable weight of the j-th level lightweight residual attention block to adjust the response strength of the multi-receptive field residual attention feature.

[0086] The calculation process of lightweight quadratic residual unit is as follows Figure 5 As shown. Different from the traditional residual unit, the lightweight quadratic residual unit further enters the quadratic residual on the basis of the primary residual and constructs the secondary attention residual. Taking the first-level lightweight quadratic residual (LSR) unit as an example, the input feature is X ccl(1) , then the first residual X r1 and the quadratic residual X r2 The calculation of is as follows:

[0087] X t1 =f 3×3 (f 1×1 (X ccl(1) ))

[0088] X t2 =f 3×3 (X t1 )

[0089] X t3 =f 3×3 (X t2 )

[0090] X r1 =f 1×1 (f GELU (X t3 ))

[0091] X r2 =f 1×1 (f sigmoid (f 1×1 (f 1×1 (F concat (X t1 ,X t2 ))))·f 1×1 (F concat (X t2 ,X t3 )))

[0092] Among them, f GELU and f sigmoid are GELU and Sigmoid activation functions respectively, f 1×1 and f 3×3 Respectively represent the convolution calculation of kernel 1×1 and 3×3, X t1 , X t2 and X t3 Respectively Figure 5 The first, second and third f in the first residual 3×3 Then, the input feature X of the first-level LSR unit is ccl(1) and the first-order residual X r1 and the quadratic residual X r2 The fusion obtains the output feature X of the first-level LSR unit LSR(1) , the calculation formula is as follows:

[0093] X LSR(1) =f 3×3 (X ccl(1) +X r1 )+X r2

[0094] Similarly, we can calculate X LSR(j) , j = 2, 3 and 4.

[0095] The calculation process of the multi-receptive field residual attention (MRA) unit is as follows Figure 6As shown. This unit takes the output of the LSR unit at the same level as the input feature, and respectively from the first level, f SC_1×1 (·) represents a specific stride convolution with a kernel size of 1×1, whose activated pixels are convolved with a normal kernel size of 1×1 f 1×1 The difference between (·) is as follows Figure 7 As shown, the dark part represents the pixels in the convolution activation state during convolution calculation, f SC_1×1 (·) There are fewer and sparser activated pixels, which reduces the feature computation and enhances the focus on interesting features. GC_3×3 (·) and f GC_1×1 (·) represents the grouped convolution calculation with kernel size of 3×3 and 1×1 respectively. The grouped convolution calculation further reduces the amount of calculation. The calculation of different kernel sizes also obtains features under multiple receptive fields. up (·) represents the Pixeshuffle upsampling layer. Taking the first-level MRA unit as an example, the input feature is X LSR(1) , multi-receptive field residual feature X mres The calculation formula is:

[0096] X sc =f 1×1 (f SC_1×1 (f 1×1 (X LSR(1) )))

[0097] X mres =f up (f GC_3×3 (f GC_1×1 (X sc )+f GC_3×3 (X sc ))+X sc )

[0098] Next, the multi-receptive field residual feature X mres With the leftmost branch f GC_3×3 The computational features of (·) are fused, and the multi-receptive field residual attention feature X is obtained through the following calculation MRA(1) , the calculation formula is as follows:

[0099] X MRA(1) =X LSR(1) ·f sigmoid (f 1×1 (X mres ))

[0100] Similarly, X can be calculated MRA(j) , j is 2, 3 and 4.

[0101] For the output feature X1 after lightweight weighted adaptive collaborative feature fusion, high-resolution mine image reconstruction is performed to obtain super-resolution image X SR , the calculation process is as follows:

[0102] X SR =f up (f GC_5×5 (X1)+f GC_3×3 (X1))

[0103] The low-resolution image and its corresponding super-resolution image results are as follows Figure 8 shown.

[0104] like Figure 9 As shown in the figure, the super-resolution reconstruction system for low-quality blurred images of coal mining working faces in mines includes:

[0105] Mine image acquisition module, which collects raw image data of mine scenes at different times, viewing angles, and sensors;

[0106] A preprocessing module, wherein the input end of the preprocessing module is connected to the output end of the mine image acquisition module, and performs a preprocessing operation on the image acquired by the mine image acquisition module;

[0107] The feature extraction and encoding module has its input connected to the output of the preprocessing module, performs structured encoding on the image information, and uses a multi-branch, multi-scale convolutional neural network to extract multi-scale feature information of the image;

[0108] A multi-dimensional collaborative co-variation modeling module, whose input is connected to the output of the feature extraction and encoding module, integrates image features from different dimensions and multiple scales, and models the co-variation relationship between different images;

[0109] A super-resolution reconstruction module, wherein the input end of the super-resolution reconstruction module is connected to the output end of the multi-dimensional collaborative co-variation modeling module, and the multi-dimensional collaborative co-variation modeling module is used to fuse features and process the model to output a reconstructed high-resolution mine image;

[0110] The reconstructed image evaluation and feedback module has its input connected to the output of the super-resolution reconstruction module, performs quality evaluation on the high-resolution mine image output by the super-resolution reconstruction module, and uses it for subsequent model optimization or reconstruction feedback.

Claims

1. A super-resolution reconstruction method for low-quality blurred images of coal mining working faces in mines, characterized by: The following steps are involved: S1, input low-resolution mine image and extract multi-receptive field feature information of the image; S2. For the multi-receptive field features, a multi-dimensional collaborative covariant feature weighted adaptive fusion residual group is used to perform high-frequency feature fusion processing to achieve lightweight weighted adaptive collaborative feature fusion; S3. Perform multi-receptive field feature fusion and upsampling processing on the fused features to achieve high-resolution mine image reconstruction, and finally output the mine image super-resolution reconstruction result.

2. The method for super-resolution reconstruction of low-quality blurred images of a coal mining face in a mine according to claim 1, characterized in that: S1 uses three groups of grouped convolution calculations with kernel sizes of 1×1, 3×3, and 5×5 respectively. The multi-receptive field low-frequency features obtained by different convolution calculations are then spliced ​​along the feature dimension of the tensor to obtain multi-dimensional low-frequency feature information of the image.

3. The method for super-resolution reconstruction of low-quality blurred images of a coal mining face in a mine according to claim 1, characterized in that: The multi-dimensional collaborative covariant feature weighted adaptive fusion residual group in S2 is composed of four collaborative covariant learning units (CCL) and four multi-level lightweight residual attention blocks (MLRA); the CCL unit calculates the attention of multi-receptive field features of different scales, and uses this attention to adjust the response of different receptive field features, thereby achieving collaborative covariation processing of multi-receptive field features under attention adjustment of different scales; the MLRA block realizes the high-frequency information calculation of multi-receptive field features by alternating calculation of lightweight quadratic residual features and multi-receptive field residual attention features.

4. The method for super-resolution reconstruction of low-quality blurred images of a coal mining face in a mine according to claim 3, characterized in that: F CCL(i) (·) represents the calculation of the i-th collaborative covariation learning unit, F MLRA(i) (·) represents the calculation of the i-th multi-level lightweight residual attention block, and the calculation process of the lightweight weighted adaptive collaborative feature fusion result X1 is as follows: 1) First, the multi-receptive field low-frequency feature X0 is input into the first collaborative covariation learning unit F CCL(1) (·) Calculate and obtain the collaborative covariation learning feature X ccl(1) , then input to the first multi-level lightweight residual attention block F MLRA(1) (·) Calculate the multi-level lightweight residual attention X mlra(1) , the calculation formula is as follows: X ccl(1) =F CCL(1) (X0) X mlra(1) =F MLRA(1) (X ccl(1) ) 2) Next, the calculation formulas for the second, third, and fourth multi-level lightweight residual attention blocks are as follows: X mlra(i) =F MLRA(i) (X mlra(i-1) ) Here, i = 2, 3, 4, and we can obtain multi-level lightweight residual attention features X of different dimensions. mlra(2) 、X mlra(3) and X mlra(4) ,With the increase of the level, the multi-level lightweight residual attention feature is more inclined to the attention representation of deep semantics; 3) The calculation formulas for the second, third, and fourth synergistic covariation learning units are as follows: X ccl2 =F CCL(2) (F concat (X mlra(1) ,X mlra(2) )) X ccl(3) =F CCL(3) (F concat (X ccl(2) ,X mlra(3) )) X ccl(4) =F CCL(4) (F concat (X ccl(3) ,X mlra(4) )) 4) Finally, the collaborative covariation learning features at different levels are fused to obtain the lightweight weighted adaptive collaborative feature fusion result. The calculation formula is as follows: X1=λ1X ccl(1) +λ2f sigmoid (X ccl(2) )·X ccl(3) +λ3X ccl(4) Among them, f sigmoid Represents the sigmoid nonlinear transformation calculation, λ1, λ2 and λ3 represent the adaptive parameters, where the shallow X ccl(1) and deep X ccl(4) The traditional feature addition fusion method is used, and the middle layer X ccl(2) and X ccl(3) The regulation of same-direction attention reinforcement response is achieved through multiplication; In the multi-dimensional collaborative co-varying feature weighted adaptive fusion residual group, the CCL unit and MLRA block at the same level collaboratively infer image features and finally perform specific fusion of collaborative co-varying learning features to achieve lightweight weighted adaptive collaborative feature fusion.

5. The method for super-resolution reconstruction of low-quality blurred images of a coal mining face in a mine according to claim 4, characterized in that: The collaborative covariation learning unit performs large-scale grouping feature extraction and small-scale feature extraction on the input multi-dimensional feature information to obtain multi-dimensional features under large-scale perspective representation and small-scale perspective representation, and performs adaptive weighted fusion on the extracted features of different scales to obtain collaborative covariation learning features of multi-dimensional features.

6. The method for super-resolution reconstruction of low-quality blurred images of a coal mining face in a mine according to claim 4, characterized in that: The multi-level lightweight residual attention block is composed of four groups of lightweight secondary residual units (LSR) and multi-scale receptive field residual attention units (MRA) alternatingly, where each level of LSR units and MRA units constitutes a level of lightweight residual attention block; In each level of lightweight residual attention block, the input features are first extracted by the LSR unit using the quadratic residual representation of high-frequency information features, and then the MRA unit is used to calculate the residual attention features from multiple receptive field perspectives, thereby adjusting the response strength of the features of interest under different receptive field perspectives. Finally, the residual attention features are superimposed and fused into the input features.

7. A super-resolution reconstruction system for low-quality blurred images of coal mining faces in mines, characterized by: include: Mine image acquisition module, which collects raw image data of mine scenes at different times, viewing angles, and sensors; A preprocessing module, wherein the input end of the preprocessing module is connected to the output end of the mine image acquisition module, and performs a preprocessing operation on the image acquired by the mine image acquisition module; The feature extraction and encoding module has its input connected to the output of the preprocessing module, performs structured encoding on the image information, and uses a multi-branch, multi-scale convolutional neural network to extract multi-scale feature information of the image; A multi-dimensional collaborative co-variation modeling module, whose input is connected to the output of the feature extraction and encoding module, integrates image features from different dimensions and multiple scales, and models the co-variation relationship between different images; A super-resolution reconstruction module, wherein the input end of the super-resolution reconstruction module is connected to the output end of the multi-dimensional collaborative co-variation modeling module, and the multi-dimensional collaborative co-variation modeling module is used to fuse features and process the model to output a reconstructed high-resolution mine image; The reconstructed image evaluation and feedback module has its input connected to the output of the super-resolution reconstruction module, performs quality evaluation on the high-resolution mine image output by the super-resolution reconstruction module, and uses it for subsequent model optimization or reconstruction feedback.

Images