Mine environment super-resolution image reconstruction method based on vibration sensing for mine monitoring

Abstract

The application relates to the technical field of image enhancement, in particular to a mine environment super-resolution image reconstruction method based on vibration induction for mine monitoring, which comprises collecting vibration data and image data and performing pretreatment; the processed vibration data sequence is input into a double-layer LSTM network, vibration data is analyzed by using the LSTM, vibration characteristics at the next moment are predicted, and dynamic compensation parameters are generated; the processed image is input into a convolutional neural network constructed by using a blueprint separable convolution, a residual attention module and a coordinate attention mechanism, and multi-scale feature extraction is performed; time alignment of the vibration data and the image data is performed, corresponding compensation parameters are generated, and meanwhile, spatial alignment and fusion of the two are performed to obtain fusion features; based on the fusion features, super-resolution reconstruction is performed on the image to generate a high-resolution image. Through technical fusion innovation, the recognition quality of image quality in a complex vibration scene is significantly enhanced.

Description

Technical Field

[0001] This application relates to the field of image enhancement technology, and in particular to a method for super-resolution image reconstruction of mine environment based on vibration sensing for mine monitoring. Background Technology

[0002] The dynamic and complex nature of the mining environment exposes monitoring image acquisition to multiple sources of interference: mechanical vibration and geological vibration combine to create multidirectional unsteady disturbances, causing continuous displacement of monitoring equipment, resulting in motion blur and misalignment of features between adjacent frames. Simultaneously, the low illumination and high dust levels underground further exacerbate image noise and detail loss. Existing static super-resolution methods (such as SRCNN) rely on fixed fuzzy model assumptions and cannot adapt to the dynamic changes in vibration scenarios. Traditional video super-resolution techniques suffer from motion estimation errors due to the nonlinear characteristics of vibration direction and amplitude, and their high computational complexity for multi-frame processing makes them unsuitable for the real-time requirements of underground edge devices.

[0003] Existing image reconstruction techniques mostly focus on single-modal data optimization: static super-resolution only utilizes single-frame spatial features, ignoring the dynamic correlation between vibration characteristics and blur patterns; while video super-resolution introduces inter-frame temporal information, it relies on the linear optical flow assumption, making it difficult to model discontinuous motion trajectories caused by multi-source vibrations in mines. Furthermore, traditional methods generally employ fixed-parameter blur correction, failing to dynamically adjust the reconstruction strategy based on external vibration sensing data (direction, amplitude, frequency), thus limiting the ability to recover high-frequency details in complex vibration scenarios.

[0004] Although deep learning-based super-resolution algorithms (such as convolutional neural networks) have improved reconstruction results through feature extraction, their computational framework still has significant shortcomings: existing network models have not designed a multimodal fusion mechanism for vibration data, and image features and vibration parameters are difficult to combine effectively due to differences in representation scale; at the same time, traditional sliding window convolution operations have low efficiency in utilizing the spatiotemporal information of reference frames, and do not dynamically optimize the convolution kernel weights according to vibration characteristics, resulting in high resource occupancy of edge devices and insufficient real-time processing performance, which restricts the improvement of the reliability of mine safety monitoring systems. Summary of the Invention

[0005] The technical problem this invention aims to solve is the poor ability to recover high-frequency details in images under complex vibration scenarios.

[0006] Therefore, this invention provides a method for super-resolution image reconstruction of mine environment based on vibration sensing for mine monitoring.

[0007] The technical solution adopted by this invention to solve its technical problem is:

[0008] A vibration-sensing-based super-resolution image reconstruction method for mine environment monitoring includes,

[0009] S1: Collect vibration data and image data, and preprocess the collected vibration data and image data;

[0010] S2, the preprocessed vibration data sequence is input into a two-layer LSTM network, and the LSTM is used to analyze the vibration data and predict the vibration characteristics at the next moment.

[0011] S3 inputs the preprocessed image into a convolutional neural network constructed using blueprint separable convolution, along with a residual attention module and a coordinate attention mechanism, to perform multi-scale feature extraction;

[0012] S4 utilizes the vibration parameters predicted by LSTM to generate dynamic compensation parameters, which are used to adjust the convolution kernel parameters of the super-resolution network.

[0013] S5, perform temporal alignment of vibration data and image data, generate corresponding compensation parameters, and simultaneously perform spatial alignment and fusion of the two to obtain fused features;

[0014] S6, based on fusion features, adopts a residual network structure, achieves image magnification through multi-layer convolution and upsampling modules, and uses residual learning to reduce information loss, performs super-resolution reconstruction of the image, and generates a high-resolution image.

[0015] Furthermore, in step two, the specific steps for analyzing the vibration data using LSTM include:

[0016] S2.1, Pack vibration data into batches;

[0017] S2.2, batch i The k-th feature X ik (θ k ,α k A k ,ω k Forward propagation is performed;

[0018] S2.3, Based on the threshold value obtained during the forward propagation, the calculation unit outputs y. k With long-term memory;

[0019] S2.4, output y to the unit k Perform fully connected output;

[0020] S2.5, perform steps S2.2 to S2.4 on all data in the batch, calculate the error, calculate the partial derivative of the error with respect to the fully connected weights and unit outputs, and obtain the output gradient step. output and the input gradient δ0;

[0021] S2.7, gradient descent updates the weight matrix and backpropagates the input gradient.

[0022] Furthermore, in step three, the blueprint separation convolution first weights and combines the input in the depth direction, and then performs multi-layer depth convolution on multiple channels simultaneously. The residual attention module first adds the input and the output after multiple convolutional layers to achieve residual learning, and then sends the added output features to the coordinate attention module. Finally, the output features after the coordinate attention module are added to the original input to obtain the final output features of the lightweight residual attention module.

[0023] Furthermore, step three specifically includes the following steps:

[0024] S3.1, using pooling kernels of size (H, 1) and (1, W) to perform pooling operations in both horizontal and vertical directions to obtain a set of perceptual feature maps with sizes of C×H×1 and C×1×W, respectively;

[0025] S3.2, fuse and stitch the perceptual feature maps from the two directions mentioned above to obtain: X′=δ(f 1×1 ([z h ,z w ])), Where δ(·) represents the h_swish activation function; [z h ,z w ] represents the fusion operation of the c-th channel output with height h and width w; f 1×1 This represents a 1×1 convolution; This represents the intermediate feature mapping for encoding spatial information, where r represents the downsampling step size;

[0026] S3.3, split F' into two independent tensors X' h and X' w The feature map X' is modified by passing it through a 1×1 convolutional layer. h and X' w The number of channels is obtained as: g h =σ(f 1×1 (X' h )), g w =σ(f 1×1 (X' w )), Where σ(·) represents the sigmoid activation function; g h and g w As attention weights used to adjust the module's focus on the input image, the final output of the coordinate attention module can be expressed as: Y CA =X c(i,j)×g h ×g w .

[0027] Furthermore, in step four, the vibration prediction parameters obtained after the hidden state output by the LSTM network is mapped by the fully connected layer contain multiple dimensions, which are used to guide the dynamic compensation operation in different regions of the image, thereby achieving an organic combination of local and global compensation.

[0028] Furthermore, in step four, the process of generating the dynamic fuzzy compensation matrix can be expressed as: K comp =g(d t+1 ,m t+1 ), where the g(.) function is based on (d t+1 ,m t+1 Adjust the direction and intensity of the convolution kernel to adapt to the current environmental vibration state.

[0029] Furthermore, in step five, the dynamic compensation parameter is K, and its calculation formula is: Where, θ t Let θ be the direction of vibration at the current moment. t+1 σ represents the vibration direction predicted by LSTM at the next moment, and σ is the fuzzy adjustment parameter. The fusion of vibration data and image data adopts a weighted fusion method, and the fusion weight of vibration features and image features is calculated through a self-attention mechanism.

[0030] Furthermore, in step five, the fused feature F fused Represented as: F fused =α vib ·F vib +α img ·F img , of which F vib The vibration compensation features are obtained by mapping the LSTM output through a fully connected layer, and the weights satisfy α. vib +α img =1.

[0031] A computer device, comprising:

[0032] processor;

[0033] Memory, used to store executable instructions;

[0034] The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the random walk-based underground image detail enhancement algorithm as described above.

[0035] A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the random walk-based image detail enhancement algorithm for underground mining as described above.

[0036] The beneficial effects of this invention are:

[0037] This application utilizes vibration data and employs low-pass and Kalman filtering for noise reduction. Subsequently, a dual-layer LSTM network is used to perform temporal modeling of the vibration data, predicting vibration parameters for the next moment and generating dynamic compensation parameters. Simultaneously, a feature extraction module extracts multi-scale features of the image through blueprint separable convolution and coordinate attention mechanisms. Combined with inter-frame motion compensation information, a feature alignment module performs nonlinear transformations on the features of the target and reference frames to achieve precise alignment. Next, a fusion module combines vibration compensation features with image features through adaptive weight calculation, generating a fused high-dimensional feature representation. Finally, a super-resolution reconstruction module, based on residual learning and coordinate attention mechanisms, upsamples and enhances the fused features to restore the high-resolution image. Compared to existing methods, this invention not only effectively compensates for blurring caused by vibration but also improves the detail restoration capability of super-resolution reconstruction through precise alignment and multi-scale fusion. Furthermore, the lightweight network structure reduces computational complexity and improves inference efficiency, making it widely applicable in real-time demanding applications such as mine monitoring, UAV imaging, and robot navigation. Attached Figure Description

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] Figure 1 This is a flowchart illustrating the implementation of the vibration-sensing-based super-resolution image reconstruction method for mine environment monitoring in this invention.

[0040] Figure 2 This is a structural block diagram of the vibration-sensing-based super-resolution image reconstruction method for mine environment monitoring in this invention.

[0041] Figure 3 This is a schematic diagram of the blueprint separable convolution module in this invention.

[0042] Figure 4 This is a comparison of the detailed effects of using the method of this invention and the blueprint separable convolution method to recognize different images. Detailed Implementation

[0043] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0044] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, features defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0045] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0046] A vibration-sensing-based super-resolution image reconstruction method for mine environment monitoring.

[0047] S1, Data Acquisition and Preprocessing

[0048] S1.1 In the mine monitoring environment, vibration data, including vibration direction, acceleration, amplitude and frequency, is collected through an IMU sensor array; image data is collected through a camera.

[0049] It should be noted that in the IMU sensor array, the range of the triaxial accelerometer is ±10g and the sampling frequency is not less than 50Hz, the range of the triaxial gyroscope is ±250° / s and the sampling frequency is not less than 50Hz, and the range of the seismic wave sensor is ±1000μm / s and the sampling frequency is not less than 10Hz.

[0050] S1.2 Preprocesses the collected vibration data. First, a low-pass filter is used to remove high-frequency noise, then a Kalman filter is used to smooth the data, and finally, the mean and standard deviation are used for standardization.

[0051] S1.3, preprocessing of low-resolution images acquired from mine monitoring. First, optical flow is used to achieve inter-frame motion alignment. Then, non-local mean filtering is used for initial denoising to suppress global noise and repetitive interference. Subsequently, bilateral filtering is used to further enhance image edges and texture details, preserving important structural information while denoising, thus obtaining a clear preprocessed image. Finally, adaptive histogram equalization is used to enhance image contrast, resulting in the preprocessed image.

[0052] S2, the preprocessed vibration data sequence is input into a two-layer LSTM network, and the LSTM is used to analyze the vibration data and predict the vibration characteristics at the next moment.

[0053] S2.1, Package the vibration data, integrate the four dimensions of vibration direction θ, acceleration α, amplitude A, and frequency ω, and divide the data into batches, with the size of each batch ranging from (min, max):

[0054]

[0055] Where n represents the batch size, i represents the batch number, and u and r are the distribution mean and root mean square, respectively.

[0056] S2.2, batch i The k-th feature X ik (θ k ,α k A k ,ω k Forward propagation:

[0057]

[0058] Among them, forgetgate k This represents the forget gate, or input gate. k Indicates input gate, output gate k Indicates the output gate. This indicates that each gate is connected to input X. k The weight, This indicates that each gate is connected to the short-term memory h. k-1 The weights, (b f ,b g ,b j ,b o ) represents the bias term for each layer.

[0059] S2.3, Based on the threshold value obtained in step S2.2, calculate the output and long-term memory:

[0060]

[0061] Among them, c k ,c k-1 Representing long-term memory, y k Indicates the unit output, h k It represents short-term memory.

[0062] S2.4, y of S2.3 k Perform a fully connected output:

[0063] y k+1 =W0·y k

[0064] Where W0 represents the fully connected weights, y k+1 (θ) represents the prediction of the next time step. k+1 ,α k+1 A k+1 ,ω k+1 ).

[0065] S2.5, Perform steps S2.2 to S2.4 on all data in each batch to calculate the error:

[0066]

[0067] Where MSE(.) represents the squared error, X i(k+1) Let W0 represent the (k+1)th feature in the i-th batch of data, and perform error pairing on W0, yi. k The partial derivative is used to obtain the output gradient step. output And the input gradient δ0.

[0068] S2.6, Gradient descent updates the weight matrix and backpropagates the input gradient:

[0069] (w x ,w h b) new =Adam((w x ,w h ,b),backward(δ0))

[0070] in, This represents the updated weight matrix.

[0071] S3 takes the preprocessed image as input and uses a convolutional neural network constructed with blueprint separable convolution, along with a residual attention module and a coordinate attention mechanism, to perform multi-scale feature extraction.

[0072] Blueprint-based separable convolution is an alternative method to standard convolution. Its principle is to first weight and combine features along the depth direction, and then perform depthwise convolution. This method introduces a cross-channel bridging mechanism, allowing simultaneous processing of features from multiple channels, improving feature expressiveness and thus model accuracy. The residual attention module first adds the input and the output from two convolutional layers to achieve residual learning. Then, the added output features are fed into the coordinate attention module. Finally, the output features from the coordinate attention module are added to the initial input to obtain the final output features of the lightweight residual attention module. The specific calculation formula is as follows:

[0073] F mid =F n-1 +f BSConv (ReLu(f BSConv (F n-1 )))

[0074] F n =F n-1 +H CA (F mid )

[0075] Where Fi (i = 1, ..., n) is the input of the i-th residual attention module; Fmid is the output after adding the input and the output of two convolutional layers; fBSConv is the blueprint separation convolution operation; HCA is the coordinate attention mechanism for feature extraction; and ReLU is the ReLU activation function.

[0076] Coordinate attention mechanisms add extra coordinate information to neural networks, enabling the model to better understand the spatial relationships between pixels. It typically adds a coordinate attention module after the original convolutional layers. This module weights features based on pixel location information, allowing the model to focus more on certain important features, thereby improving reconstruction quality. The main steps are as follows:

[0077] S3.1, we use pooling kernels of size (H, 1) and (1, W) to perform pooling operations along the horizontal and vertical directions, obtaining a set of perceptual feature maps with sizes C×H×1 and C×1×W, respectively. Therefore, the output of the c-th channel with height h and the output of the c-th channel with width w are:

[0078]

[0079] S3.2, we fuse and stitch together the perceptual feature maps from the two directions mentioned above to obtain:

[0080]

[0081] In this expression, δ(·) represents the h_swish activation function; [z h ,z w ] indicates a fusion operation; f 1×1 This represents a 1×1 convolution; This represents the intermediate feature mapping for encoding spatial information, and r represents the downsampling step size.

[0082] S3.3, we split F' into two independent tensors X' h and X' w The feature map X' is modified by passing it through a 1×1 convolutional layer. h and X' w The number of channels, therefore, we get:

[0083]

[0084] In this expression, σ(·) represents the sigmoid activation function; g h and g w As attention weights used to adjust the module's focus on the input image, the final output of the coordinate attention module can be expressed as:

[0085] Y CA =X c (i,j)×g h ×g w

[0086] S4 utilizes the vibration parameters predicted by LSTM to generate dynamic compensation parameters, which are used to adjust the convolution kernel parameters of the super-resolution network. The vibration prediction parameters obtained after mapping the hidden state output by the LSTM network through a fully connected layer contain multiple dimensions, which are used to guide the dynamic compensation operations in different regions of the image, achieving an organic combination of local and global compensation.

[0087] The vibration parameters P at the next moment are obtained by forward propagation through an LSTM network. t+1 as follows:

[0088] P t+1 ={a t+1 ,d t+1 ,f t+1 ,m t+1}

[0089] Vibration parameters contain global and local image blur correction information. The dynamic compensation parameters are not only used to correct image blur but also to guide the parameter settings of the upsampling module during image reconstruction, adapting to the dynamic changes in image features under different vibration environments. The process of generating the dynamic blur compensation matrix can be represented as follows:

[0090] K comp =g(dt+1 ,m t+1 )

[0091] Among them, the g(.) function is based on (d t+1 ,m t+1 Adjust the direction and intensity of the convolution kernel to adapt to the current environmental vibration state.

[0092] S5 performs time alignment between vibration data and image data to generate corresponding dynamic compensation parameters, and simultaneously performs spatial alignment and fusion of the two to obtain fused features.

[0093] The dynamic compensation parameter is K, and its calculation formula is:

[0094]

[0095] Where, θ t Let θ be the direction of vibration at the current moment. t+1 Let σ be the vibration direction predicted by LSTM at the next moment, and σ be the fuzzy adjustment parameter. This formula is used to determine the directionality of fuzzy compensation in the image. The fusion step of the vibration data and image data adopts a weighted fusion method, which calculates the fusion weight of vibration features and image features through a self-attention mechanism. The fused feature F fused Given by the following formula: F fused =α vib ·F vib +α img ·F img , of which F vib The vibration compensation features are obtained by mapping the LSTM output through a fully connected layer, and the weights satisfy α. vib +α img =1.

[0096] S6, based on fusion features, employs multi-layer convolution, upsampling, and residual learning modules to perform super-resolution reconstruction of images, generating high-resolution images. The super-resolution image reconstruction step uses a residual network structure, achieving image magnification through multi-layer convolution and upsampling modules, and utilizing residual learning to reduce information loss, thereby improving the image detail restoration effect.

[0097] Example 1

[0098] Step one involves installing multimodal sensors, including triaxial accelerometers, gyroscopes, and seismic wave sensors, around key mine equipment (such as mine car tracks and drilling rigs) to capture environmental vibration characteristics. The sensor deployment points should be 0.5 meters away from the cameras to ensure synchronized data acquisition and reduce viewing angle bias.

[0099] The sensor collects vibration data at a sampling frequency of 50Hz and obtains the following characteristics: acceleration a t : Reflects vibration intensity; direction dt : The principal direction of vibration, calculated by fusing data from the gyroscope; frequency f t Record the vibration frequency range and calculate using FFT; amplitude m t The magnitude of the vibration is quantified and obtained by calculating the signal peak value.

[0100] The collected data needs to be noise-reduced and normalized to improve prediction accuracy.

[0101] Noise reduction: High-frequency noise is eliminated using a low-pass filter (Butterworth filter).

[0102]

[0103] Among them, w i is the filter weight, and x(t) is the input signal.

[0104] Normalization: Performing Min-Max normalization on the data

[0105]

[0106] Mapping data to the range [0,1] reduces the influence of dimensions.

[0107] Step 2, Time Series Forecasting (LSTM)

[0108] Input data: The vibration characteristic data from the most recent 20 frames are used to construct a time series input.

[0109]

[0110] Network structure: An LSTM network with 2 hidden layers and 128 units per layer is used to output the vibration prediction characteristics for the next time step.

[0111]

[0112] The 2-layer LSTM network structure is as follows:

[0113] Input layer: 20×4 dimensions (20 time steps, 4 features)

[0114] Hidden layers: 2 LSTM layers, 128 units per layer

[0115] Output layer: Predicts vibration characteristics at the next moment.

[0116] Training process: The model was trained using 10,000 mine vibration data points. The optimizer used was Adam, the loss function was mean squared error (MSE), and the learning rate was 0.001.

[0117]

[0118] Step 3, Prediction Results

[0119] The trained LSTM model predicts vibration data for use in the dynamic compensation module. For example, if the prediction shows a vibration direction of 45° and an amplitude of 0.8, the fuzzy compensation parameters will be adjusted accordingly.

[0120] Step 4: Generation of fuzzy compensation parameters

[0121] Based on the predicted vibration characteristics (direction and amplitude), a dynamic fuzzy compensation matrix K is generated. comp :

[0122]

[0123] The function g adjusts the direction and intensity of the convolution kernel, for example:

[0124] when The convolution kernel direction is offset by 45°;

[0125] when Increase the weights to enhance the compensation effect.

[0126] Example of dynamic adjustment:

[0127] Assuming the current vibration direction D = 45° and the amplitude M = 0.7, the compensation matrix is:

[0128]

[0129] This matrix is ​​used to adjust the direction of image blur and to dynamically compensate for areas of high vibration.

[0130] Step 5, Super-resolution Reconstruction Module

[0131] Blueprint separable convolutional design: The convolutional structure is divided into two stages: local convolution and global convolution. Local convolution is used for detailed feature extraction and adopts a 3×3 convolution kernel. Global convolution integrates vibration compensation information and full-image features through 5×5 convolution.

[0132] Dynamic fusion: Vibration compensation parameter K comp The weights of the convolution kernels are dynamically adjusted to enable the model to adapt to vibration ambiguity characteristics.

[0133] F'=Conv 5×5 (Conv 3×3 (I)·K comp )

[0134] Among them, K comp It is the aforementioned fuzzy compensation matrix.

[0135] Network structure:

[0136] Input: Low-resolution image ILR and fuzzy compensation matrix K comp

[0137] Blueprint-based convolution technique

[0138] Introducing residual modules and coordinate attention mechanisms

[0139] Output: High-resolution image I HR

[0140] Network flow:

[0141] I HR =Reconstruct(I LR ,K comp )

[0142] Evaluation indicators

[0143] Super-resolution performance was evaluated using PSNR and SSIM:

[0144]

[0145] in:

[0146] PSNR (Peak Signal-to-Noise Ratio): A measure of image quality

[0147] SSIM (Structural Similarity): Evaluates the structural consistency between the reconstructed image and the original image.

[0148] Step 6, Multimodal Fusion Module

[0149] 1. Module Functions

[0150] By integrating vibration sensing data and image features, the reconstruction effect is improved. The fusion weight of vibration features and image features is calculated through a self-attention mechanism.

[0151] 2. Weighted fusion design

[0152] The vibration characteristic branch provides fuzzy correction parameters;

[0153] Image feature branches extract spatial texture information;

[0154] Through the weighted fusion formula:

[0155] F final =αF image +βF vibration

[0156] Where α and β are fusion weight parameters.

[0157] 3. Integration Process

[0158] The LSTM time prediction method described above is used to fuse and match the moment when vibration is about to occur with the image, thereby enriching the image features by aligning the corresponding timestamps.

[0159] The fused features are the original features extracted through preprocessing, which are then appropriately fused and weighted.

[0160] Example 2

[0161] This embodiment uses a vibration-sensing-based super-resolution image reconstruction method for mine environment for mine monitoring to conduct mine monitoring scenario testing in a mine operation area.

[0162] In a certain mine operating area, the track vibration frequency ranges from 10-50Hz, and the operation of the drilling rig causes unstable vibration direction. Monitoring cameras equipped with vibration sensors are installed in the mine operating area to capture low-resolution images.

[0163] The vibration sensor collects environmental vibration characteristics in real time; the vibration characteristics at the next moment are predicted by LSTM, and a dynamic fuzzy compensation matrix is ​​generated; the vibration features are fused with the preprocessed features by the fusion module; the low-resolution image is input into the super-resolution reconstruction module, and the super-resolution reconstruction is completed by combining the compensation information.

[0164] The experimental results are shown in Table 1:

[0165] Table 1. Comparison of experimental data between the method used in this invention and the blueprint-separable convolution method.

[0166]

[0167] The original image PSNR value is 22.5. This invention was compared with Blueprint Separable Convolution (BSC). The comparison data shows that this invention has significant advantages in image quality: the PSNR reaches 30.5dB, higher than BSC's 28.7dB, indicating its greater effectiveness in enhancing pixel-domain details; the SSIM value is 0.92, surpassing BSC's 0.89, confirming its superior protection of image structural integrity in vibration scenarios. Although the computational complexity (1.2G FLOPS) is slightly higher than BSC (0.8G FLOPS), relying on LSTM vibration information prediction and adaptive compensation mechanisms, this invention achieves breakthroughs in vibration compensation and inter-frame alignment capabilities, making it adaptable to complex vibration environments such as mine monitoring and UAV imaging; in contrast, BSC lacks a vibration compensation mechanism and is only suitable for ordinary super-resolution tasks.

[0168] To more intuitively illustrate the technical effects of this invention, we compared the enhancement results of different algorithms through experiments, such as... Figure 4 As shown, Figure 4Images a, b, and c in the image are three actual images of the underground mine environment. In image a, the red box marks the image of the mine wall, and in the smaller images b and c, the red boxes mark two building structures, respectively. Figure 4 The small images a1, b1, and c1 in the figure are magnified views of the images within the red boxes of small images a, b, and c, respectively. The small images a1, b1, and c1 are schematic diagrams of the results of processing the images within the red boxes of small images a, b, and c using blueprint separable convolution. The small images a1, b1, and c1 are schematic diagrams of the results of processing the images within the red boxes of small images a, b, and c using the algorithm of this application.

[0169] Observing the processing results of blueprint-separable convolution reveals significant shortcomings in image detail restoration: when dealing with detailed areas such as wall textures and building structures, the processed image exhibits a blurry texture, failing to accurately restore original detail features. For example, details such as wall perforations and building lines suffer significant loss, resulting in poor visual quality. In contrast, the method of this invention demonstrates a clear advantage in detail processing. For complex textured areas, this invention can accurately capture and enhance details, clearly restoring features such as wall perforations and building lines, resulting in rich and natural image details. Whether it's the complex texture of a mine wall or the fine details of a building structure, the method of this invention maintains detail integrity during the enhancement process, achieving a visual effect significantly superior to the blueprint-separable convolution method, fully demonstrating its technical superiority in image detail restoration and enhancement.

[0170] In summary, the experimental results demonstrate that the proposed algorithm has certain applicability in complex underground mining environments. This invention, through technological integration and innovation, significantly enhances adaptability to complex vibration scenarios while improving image quality. Compared to blueprint-separable convolution methods, it exhibits more comprehensive technological competitiveness, providing more efficient and reliable technical support for scenarios with stringent vibration compensation requirements, such as mine monitoring and robot vision.

[0171] Example 3

[0172] This application provides a computer device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement a vibration-sensing-based super-resolution image reconstruction method for mine monitoring, as provided in the above method embodiments.

[0173] Example 4

[0174] This application embodiment also provides a computer-readable storage medium, which can be disposed in a server to store at least one instruction or at least one program segment related to implementing a vibration-sensing-based super-resolution image reconstruction method for mine monitoring provided in the method embodiment. The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the vibration-sensing-based super-resolution image reconstruction method for mine monitoring provided in the above-described method embodiment. Optionally, in this embodiment, the storage medium can be located in at least one of multiple network servers in a computer network. Optionally, in this embodiment, the storage medium can include, but is not limited to, various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0175] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined by the scope of the claims.

Claims

1. A method for super-resolution image reconstruction of mine environment based on vibration sensing for mine monitoring, characterized in that, include, S1: Collect vibration data and image data, and preprocess the collected vibration data and image data; S2, the preprocessed vibration data sequence is input into a two-layer LSTM network, and the LSTM is used to analyze the vibration data and predict the vibration characteristics at the next moment. S3 inputs the preprocessed image into a convolutional neural network constructed using blueprint separable convolution, along with a residual attention module and a coordinate attention mechanism, to perform multi-scale feature extraction; S4 uses the vibration parameters predicted by LSTM to generate a dynamic fuzzy compensation matrix, which is used to adjust the convolution kernel parameters of the super-resolution network. In step S4, the vibration prediction parameters obtained after the hidden state output by the LSTM network is mapped by the fully connected layer contain multiple dimensions, which are used to guide the dynamic compensation operation in different regions of the image, thereby achieving an organic combination of local and global compensation. The vibration parameters for the next moment are obtained by forward propagation using an LSTM network. ,in, For the acceleration in the next moment, The direction of vibration at the next moment. For the frequency of the next moment, The amplitude at the next moment; the process of generating a dynamic fuzzy compensation matrix through dynamic compensation operation can be represented as: ,in, Function based on The direction and intensity of the convolution kernel are adjusted to adapt to the current environmental vibration state; the dynamic fuzzy compensation matrix is ​​the vibration compensation parameter. S5, perform temporal alignment of vibration data and image data, generate corresponding dynamic compensation parameters, and simultaneously perform spatial alignment and fusion of the two to obtain fused features; In step S5, the dynamic compensation parameter is K, and its calculation formula is: ,in, The direction of vibration at the current moment. The direction of vibration at the next moment is predicted by LSTM. The parameters are adjusted for fuzziness; the fusion of vibration data and image data adopts a weighted fusion method, and the fusion weight of vibration features and image features is calculated through a self-attention mechanism; S6, based on fusion features, adopts a residual network structure, achieves image magnification through multi-layer convolution and upsampling modules, and uses residual learning to reduce information loss, performs super-resolution reconstruction of images, and generates high-resolution images.

2. The method for super-resolution image reconstruction of mine environment based on vibration sensing for mine monitoring according to claim 1, characterized in that, In step S2, the specific steps for analyzing the vibration data using LSTM include: S2.1, Packaging vibration data is batch One batch; S2.2, will batch i The kth feature Perform forward propagation, The vibration direction characteristics of the k-th group, For the acceleration characteristics of the k-th group, The amplitude characteristics of the k-th group are... The frequency characteristics of the k-th group; S2.3, Based on the threshold values ​​obtained during the forward propagation, the calculation unit outputs... With long-term memory; S2.4, for unit output Perform fully connected output; S2.5, Perform steps S2.2~S2.4 on all data in the batch to calculate the error. error ,calculate error The output gradient is obtained by taking the partial derivatives of the fully connected weights and the unit outputs. and input gradient ; S2.7, gradient descent updates the weight matrix and backpropagates the input gradient.

3. The method for super-resolution image reconstruction of mine environment based on vibration sensing for mine monitoring according to claim 1, characterized in that, In step S3, the blueprint separation convolution first weights and combines the input in the depth direction, and then performs multi-layer depth convolution on multiple channels simultaneously. The residual attention module first adds the input and the output after multiple convolutional layers to achieve residual learning, and then sends the added output features to the coordinate attention module. Finally, the output features after the coordinate attention module are added to the original input to obtain the final output features of the lightweight residual attention module.

4. The method for super-resolution image reconstruction of mine environment based on vibration sensing for mine monitoring according to claim 3, characterized in that, Step S3 specifically includes the following steps: S3.1, Use size and The pooling kernel performs pooling operations along both horizontal and vertical directions to obtain a set of perceptual feature maps with dimensions of [sizes not specified]. and ; S3.2, fuse and stitch the perceptual feature maps from the two directions mentioned above to obtain: ,in, express h_switsh Activation function; [] indicates the fusion operation of the output of the c-th channel with height h and the output of the c-th channel with width w; This represents a 1×1 convolution; This represents the intermediate feature mapping for encoding spatial information, where r represents the downsampling step size; S3.3, will Divided into two independent tensors and The feature maps are then modified by 1×1 convolutional layers. and The number of channels is obtained as follows: ,in, This represents the sigmoid activation function; and As attention weights used to adjust the module's focus on the input image, the final output of the coordinate attention module can be expressed as: .

5. The method for super-resolution image reconstruction of mine environment based on vibration sensing for mine monitoring according to claim 1, characterized in that, In step S5, the fusion features Represented as: ,in, The vibration compensation features are obtained by mapping the LSTM output through a fully connected layer, and the weights satisfy... .

6. A computer device, characterized in that, include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the vibration-sensing-based super-resolution image reconstruction method for mine monitoring as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the vibration-sensing-based super-resolution image reconstruction method for mine environment monitoring as described in any one of claims 1 to 5.

Images