Coal mining subsidence early warning system based on image processing

Abstract

The invention discloses a coal mining subsidence early warning system based on image processing, and relates to the technical field of mine environment monitoring, and the system comprises the steps: collecting image data of a coal mining area, carrying out the preprocessing of the image data, and generating standardized image data; performing continuous image frame processing on the standardized image data through an optical flow analysis method, calculating earth surface displacement and deformation information, and generating subsidence dynamic change data; combining the subsidence dynamic change data with the standardized image data, and inputting the combined data into an adaptive convolutional neural network to identify a subsidence area; according to the subsidence change trend and the coal mining progress, subsidence risks are evaluated, and real-time early warning signals are generated. According to the invention, by analyzing the motion trails of the pixel points in the image, the dynamic change of the coal mining area can be captured in real time, the sign of subsidence can be identified in time, early warning can be carried out in advance, and the timeliness and accuracy of monitoring are obviously improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mine environment monitoring, and particularly to an early warning system for coal mining subsidence based on image processing. Background Art

[0002] In recent years, with the continuous expansion of coal mining scale, the problem of coal mine subsidence has become an important factor restricting safe production. Traditional methods for monitoring coal mining subsidence mainly rely on geological exploration and manual observation. These methods not only have low efficiency, but also have problems such as large errors and long periods. With the continuous progress of image processing technology, monitoring systems based on image processing have gradually attracted attention and have been applied in coal mining subsidence monitoring. By obtaining detailed information on the coal mining area through high-definition image data and combining advanced image analysis techniques, the accuracy and timeliness of subsidence monitoring can be improved to a certain extent. However, existing technologies mostly focus on the analysis of static images and lack real-time monitoring and early warning mechanisms for dynamic changes.

[0003] In the existing technologies, common problems in image processing applications include: slow data collection and processing speeds, making it difficult to handle real-time monitoring of large-scale and complex scenarios; in addition, dynamic change monitoring methods such as optical flow analysis are less applied, and it is not possible to effectively respond in a timely manner to subsidence changes during the coal mining process. These deficiencies limit the effectiveness of existing systems in practical applications and result in the timeliness and accuracy of subsidence monitoring being unable to meet the requirements of coal mine safety management. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an early warning system for coal mining subsidence based on image processing to solve the problem of real-time early warning of dynamic changes during the process of coal mining subsidence monitoring.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: The present invention provides an early warning system for coal mining subsidence based on image processing, which includes a preprocessing module that collects image data of the coal mining area and preprocesses the image data to generate standardized image data; An optical flow analysis module that performs continuous image frame processing on the standardized image data by an optical flow analysis method, calculates surface displacement and deformation information, and generates subsidence dynamic change data; A subsidence recognition module that combines the subsidence dynamic change data with the standardized image data and inputs them into an adaptive convolutional neural network to identify the subsidence area; A spatio-temporal prediction module that, based on the subsidence area and combined with the temporal information of the image data, uses a spatio-temporal convolutional neural network to predict the subsidence change trend; The early warning module evaluates the subsidence risk and generates real-time early warning signals according to the subsidence change trend and the coal mining progress.

[0007] As a preferred solution of the early warning system for coal mining subsidence based on image processing according to the present invention, wherein: the image data includes a sequence of surface images, optical flow field data, texture feature data, and edge detection data.

[0008] As a preferred solution of the early warning system for coal mining subsidence based on image processing according to the present invention, wherein: the preprocessing of the image data to generate standardized image data is specifically carried out as follows. Apply a Gaussian kernel to denoise the image data. Use an image registration method to spatially align the denoised image data and perform temporal synchronization through the timestamps of the image data. Scale the temporally synchronized image data to a unified size and perform pixel value normalization to form standardized image data.

[0009] As a preferred solution of the early warning system for coal mining subsidence based on image processing according to the present invention, wherein: the continuous image frames of the standardized image data are processed by an optical flow analysis method to calculate the surface displacement and deformation information and generate subsidence dynamic change data, and the specific steps are as follows. Extract adjacent image frame pairs from the standardized image data in the order of timestamps. Based on the adjacent image frame pairs, use sparse optical flow to calculate the motion information, and then refine it through dense optical flow to generate an optical flow field. Apply median filtering and regional consistency constraints to correct the noise of the optical flow field and output the optical flow vectors. Analyze the optical flow vectors and calculate the displacement magnitude of each pixel in the surface image sequence to form a displacement field. Based on the displacement field, calculate the gradient amplitude through the gradient components to identify the deformation area. Perform second-order partial derivative calculation on the deformation area to construct a Hessian matrix. Analyze the curvature change of the area through the Hessian matrix to refine the subsidence boundary. Integrate the displacement field, the deformation area, and the refined subsidence boundary, and generate subsidence dynamic change data in combination with the timestamp order of the image data.

[0010] As a preferred solution of the early warning system for coal mining subsidence based on image processing according to the present invention, wherein: based on the displacement field, calculating the gradient amplitude through the gradient components to identify the deformation area, the specific steps are as follows. Use the Sobel operator to calculate the gradient of the displacement field and extract the horizontal gradient component of the displacement field. The gradient component perpendicular to the displacement field ; Use the Sobel convolution kernel to filter the horizontal gradient component and the vertical gradient component respectively; Calculate the gradient magnitude of each pixel in the displacement field according to the filtered gradient components. The expression is: ; Wherein, represents the gradient magnitude of the pixel point , represents the modulus of the two-dimensional gradient vector; Set the deformation detection threshold ; Compare the gradient magnitude with the deformation detection threshold to generate the pixel value of the binary image . The expression is: ; Wherein, when , it means that when the gradient magnitude is greater than the deformation detection threshold , the pixel point belongs to the deformation area and is assigned a value of ; Otherwise, that is, when the gradient magnitude is less than or equal to the deformation detection threshold , the pixel point does not belong to the deformation area and is assigned a value of .

[0011] As a preferred solution of the early warning system for coal mining subsidence based on image processing according to the present invention, wherein: the combination of subsidence dynamic change data and standardized image data is input into the adaptive convolutional neural network to identify the subsidence area. The specific steps are as follows Fuse the subsidence dynamic change data and the standardized image data through a weighted attention mechanism to form multi-modal feature data; Perform standardization processing on the multi-modal feature data, and construct a feature tensor through the standardized multi-modal feature data; Input the feature tensor into the dual-branch structure; For the global feature branch, generate a global deformation feature vector based on the global convolution kernel through the dynamic convolution layer ; For the local feature branch, adopt a local receptive field to dynamically adjust the convolution kernel and output a local deformation feature vector of the boundary details and curvature changes ; Through the feature diversity constraint function , complement the features learned by the global branch and the local branch and output a multi-modal feature map. The expression is: ; Among them, represents the total number of pixel points in the standardized image data, represents the index variable of pixel points in the standardized image data, represents the th global feature of the pixel point, represents the th local feature of the pixel point, represents the similarity degree after alignment of the th global feature vector and local feature vector of the pixel point in the same dimensional space, represents the square of the Frobenius norm, identifier, represents the local identifier; Introduce cross-branch interaction, use the cross-branch attention mechanism to establish interdependent relationships between different branches of the multi-modal feature map ; On the basis of cross-branch interaction, perform adaptive fusion to generate a feature fusion representation, and the expression is: ; Among them, represents the activation function, represents the probability that the pixel point belongs to the subsidence area; Input the feature fusion representation into the adaptive convolution layer, and generate a probability map of the subsidence area through adaptive convolution operation; Set the pixel determination threshold ; If at this time, the pixel point is determined as a non-subsidence area; If at this time, the pixel point is determined as a subsidence area.

[0012] As a preferred solution of the early warning system for coal mining subsidence based on image processing described in the present invention, among them: the introduction of cross-branch interaction, using the cross-branch attention mechanism to establish interdependent relationships between different branches of the multi-modal feature map , the specific steps are as follows, Perform standardization processing on the multi-modal feature map; Divide the standardized multi-modal feature map into multiple branches, and each branch represents different types of input features; Process the input features of each branch through the convolution layer and pooling layer, extract the corresponding local features and global features, and generate the feature representation of each branch; Model the relationships between different branches through a fully connected layer and output an initial weight matrix; Calculate the cross-branch attention distribution using the initialized weight matrix and generate a normalized attention matrix through the Softmax function; Weight the feature maps of each branch according to the attention matrix and extract the mutual dependence features between each branch and its adjacent branches.

[0013] As a preferred solution of the coal mining subsidence warning system based on image processing according to the present invention, wherein: based on the subsidence area, combined with the temporal information of the image data, a spatio-temporal convolutional neural network is used to predict the subsidence change trend. The specific steps are as follows. Perform temporal synchronization on the collected image data to generate temporal image data; Segment the temporal image data into multiple data segments by the sliding window method; Perform convolution in the spatial dimension to extract local features and extract the local features of each data segment; Perform convolution operations in the time dimension to learn the time change features in the data segments and capture the dynamic evolution of the subsidence; In the convolutional layer, dynamically adjust the size and shape of the convolutional kernel according to the dynamic evolution of the subsidence; Integrate the spatial features and time features through a spatio-temporal convolutional network to construct a spatio-temporal feature vector; Based on the spatio-temporal feature vector, predict the subsidence change trend through a deep learning algorithm.

[0014] As a preferred solution of the coal mining subsidence warning system based on image processing according to the present invention, wherein: the step of dynamically adjusting the size and shape of the convolutional kernel according to the dynamic evolution of the subsidence in the convolutional layer is as follows. Use a standard convolutional kernel to perform preliminary subsidence area feature extraction to obtain spatial features; Fuse the spatial features and displacement field data through feature splicing to generate temporal features; Based on the temporal features and displacement field data, calculate the subsidence change rate of the subsidence area, and identify sudden subsidence areas, stable subsidence areas, and large-depth subsidence areas; Dynamically adjust the size of the convolutional kernel according to the change rate; For sudden subsidence areas, use a small convolutional kernel to focus on local changes; For stable subsidence areas, use a large convolutional kernel to capture the subsidence trend; For large-depth subsidence areas, the shape of the convolutional kernel is adaptively adjusted according to the subsidence depth.

[0015] As a preferred solution of the early warning system for coal mining subsidence based on image processing according to the present invention, wherein: evaluating the subsidence risk and generating a real-time warning signal according to the subsidence change trend and the coal mining progress, the specific steps are as follows, Predict the subsidence dynamic data of the subsidence area based on optical flow analysis and spatio-temporal convolutional neural network; Obtain the mining progress data from the coal mining progress; Align the timestamps of the subsidence dynamic data and the mining progress data, and resample them to the same time interval by using the time window sliding method; Extract the subsidence rate from the aligned subsidence dynamic data, and establish the correlation between subsidence and mining in combination with the mining progress data; Input the correlation between subsidence and mining into the time series analysis model to train the subsidence change trend prediction model; Define the subsidence risk index based on the relationship between the predicted subsidence change trend and the mining progress; Dynamically adjust the subsidence risk index according to the phased changes in coal mining output by the subsidence change trend prediction model, and generate a warning signal in real time.

[0016] The beneficial effects of the present invention are as follows: through the preprocessing and standardization of image data, the comparability and analysis accuracy of the data can be improved. The preprocessing module effectively converts the collected image data into a standardized format suitable for analysis, providing an accurate basis for subsequent optical flow analysis. The optical flow analysis module can analyze the movement trajectories of pixel points in the image, capture the dynamic changes in the coal mining area in real time, identify the signs of subsidence in a timely manner, and give early warnings, significantly improving the timeliness and accuracy of monitoring. In addition, the real-time and high-precision analysis capabilities of the early warning system can better support coal mine managers to make scientific decisions in the face of subsidence risks, reduce the possibility of accidents, ensure the safety of the coal mining process, and improve the economic benefits and safety guarantee level of coal mine production. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic diagram of the early warning system for coal mining subsidence based on image processing in Embodiment 1.

[0019] Figure 2 It is a flowchart of the method for generating standardized image data in Embodiment 1. Detailed Embodiment

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0023] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides an early warning system for coal mining subsidence based on image processing, comprising the following steps: The preprocessing module collects image data of the coal mining area, preprocesses the image data, and generates standardized image data.

[0024] Furthermore, the image data includes surface image sequences, optical flow data, texture feature data, and edge detection data; Apply Gaussian kernel to denoise image data; Specifically, a two-dimensional Gaussian filter is used to smooth the image data, and the high-frequency noise is effectively removed through the convolution operation while retaining the edge features; the standard deviation and window size of the Gaussian kernel are dynamically adjusted according to the image resolution to achieve the best balance between denoising and detail retention.

[0025] The image registration method is used to spatially align the denoised image data, and the time series is synchronized through the timestamp of the image data; Specifically, an image registration method based on feature point matching is adopted to perform spatial alignment on the denoised image data. By detecting feature points in the images (such as SIFT or ORB features), extracting the feature point descriptors of each image, and using a matching algorithm (such as KNN matching) to find the corresponding relationships of the feature points in adjacent image frames, the RANSAC algorithm is then used to remove the mismatched points, calculate the homography matrix between the images, and perform geometric transformation on the images according to the homography matrix to achieve spatial alignment. After completing the spatial alignment, the adjacent image frames are sorted according to the timestamp information of each image to ensure that the image data is arranged in order in terms of time, thereby completing the temporal synchronization and providing a continuous time and space basis for subsequent data processing.

[0026] The image data after temporal synchronization is scaled to a unified size, and at the same time, pixel value normalization is performed to form standardized image data.

[0027] The optical flow analysis module processes consecutive image frames of the standardized image data through an optical flow analysis method, calculates the surface displacement and deformation information, and generates subsidence dynamic change data.

[0028] Furthermore, adjacent image frame pairs are extracted from the standardized image data in timestamp order; Based on the adjacent image frame pairs, sparse optical flow is used to calculate the motion information, and then the dense optical flow is refined to generate an optical flow field for capturing pixel-level surface motion; Median filtering and regional consistency constraints are applied to correct the noise of the optical flow field, and optical flow vectors are output; The optical flow vectors are analyzed to calculate the displacement magnitudes of each pixel in the surface image sequence, forming a displacement field; Based on the displacement field, the gradient magnitude is calculated through the gradient components to identify the deformation region; Specifically, the Sobel operator is used to calculate the gradient of the displacement field, and the horizontal gradient component of the displacement field is extracted and the vertical gradient component of the displacement field , and the expressions are: ; ; where represents the displacement value of each pixel point in the surface image sequence, represents the rate of change of the surface displacement in the horizontal direction, represents the rate of change of the surface displacement in the vertical direction, represents the horizontal coordinate of the pixel point in the image data, represents the vertical coordinate of the pixel point in the image data, represents the partial derivative; Using the Sobel convolution kernel, filter the gradient components in the horizontal and vertical directions respectively; Calculate the gradient magnitude of each pixel in the displacement field based on the filtered gradient components. The expression is: ; where, represents the gradient magnitude of the pixel point , represents the modulus of the two-dimensional gradient vector; Preferably, the magnitude of the gradient reflects the intensity of surface movement in the displacement field, which helps to locate possible deformation areas.

[0029] Set the deformation detection threshold ; Specifically, after calculating the gradient magnitude, form a gradient magnitude set for all pixel points and perform statistical analysis on this set to calculate the global mean and standard deviation . The mean of the gradient magnitude represents the global average level, and the standard deviation represents the degree of dispersion of the gradient magnitude. Based on these statistical results, set the deformation detection threshold as the weighted sum of the mean of the gradient magnitude and several times the standard deviation. The expression is: ; where, is the weight factor, usually selected according to the actual detection requirements, and the value range is generally between 1.5 and 3.0 to balance the sensitivity and false detection rate of deformation area detection; It should be noted that after setting, apply the deformation detection threshold to the binarization process of the gradient magnitude, compare whether the gradient magnitude of each pixel point exceeds the deformation detection threshold one by one, and generate a binary image, where the part with pixel value is marked as the deformation area, and the part with pixel value represents the non-deformation area. Through this dynamic adjustment method, the deformation detection threshold can adapt to the gradient distribution characteristics of different displacement fields, ensuring the accuracy and robustness of deformation area detection.

[0030] Compare the gradient magnitude with the deformation detection threshold to generate the pixel value of the binary image. The expression is: ; where, when , it means that when the gradient magnitude is greater than the deformation detection threshold , the pixel point belongs to the deformation area and is assigned a value of ; Otherwise, i.e., the gradient magnitude is less than or equal to the deformation detection threshold , the pixel does not belong to the deformation region and is assigned .

[0031] Calculate the second-order partial derivatives of the deformation region to construct the Hessian matrix; Analyze the curvature change of the region through the Hessian matrix to refine the subsidence boundary; Specifically, by calculating the eigenvalues of the Hessian matrix, judge the curvature characteristics of the pixel points, and identify the boundary feature regions. Screen out the pixel points with significant eigenvalues, mark them as boundary pixel points, and optimize the continuity of the boundary by combining the gradient magnitude data. Perform interpolation smoothing on the preliminarily identified boundary pixels, and use the cubic spline interpolation method to eliminate the jagged effect while retaining the true curvature information. Finally, generate refined subsidence boundary data, including the precise positions and curvature change information of the boundary pixels, providing high-precision basic data for subsequent subsidence dynamic analysis.

[0032] Integrate the displacement field, deformation region, and refined subsidence boundary, and generate subsidence dynamic change data in combination with the time series.

[0033] The subsidence recognition module combines the subsidence dynamic change data with the standardized image data and inputs them into the adaptive convolutional neural network to recognize the subsidence region.

[0034] Furthermore, the subsidence dynamic change data and the standardized image data are fused through a weighted attention mechanism to form multi-modal feature data; Perform standardization processing on the multi-modal feature data, and construct a feature tensor through the standardized multi-modal feature data; Preferably, standardization is to eliminate the scale difference of the data, so that each feature is learned at the same scale, which helps the network to converge and improve performance.

[0035] It should be noted that the construction of the feature tensor is achieved by fusing the standardized modal data of each modality to form a unified tensor form. First, according to the data dimensions of each modality, select an appropriate way to splice the data. For example, splice the image data and the corresponding feature data of each modality along the channel dimension (i.e., the feature dimension) to form a high-dimensional feature vector. Then, in order to ensure that the feature tensor can be input into the deep learning network, it is necessary to perform dimension expansion on the spliced data to make it meet the input requirements of the convolutional neural network (CNN) or the spatio-temporal convolutional neural network (ST-CNN).

[0036] Specifically, for the image data and feature data at each time step, they are arranged along the time dimension to form a tensor with strong temporal characteristics, so as to reflect the temporal variation characteristics of the data. Then, batch normalization is applied to further standardize the feature tensor, reduce the problem of gradient vanishing or gradient explosion during training, and accelerate the convergence speed of the network. Finally, the standardized multi-modal feature data is constructed into a feature tensor with temporal and spatial information, which can be used as the input of the subsequent adaptive convolutional neural network (CNN) or spatio-temporal convolutional neural network (ST-CNN) model for the identification and prediction tasks of the subsidence area.

[0037] Input the feature tensor into the dual-branch structure; It should be noted that inputting the feature tensor into the dual-branch structure aims to parallelly process different feature information through two independent network branches, so as to fully explore the multi-dimensional information of the data and enhance the identification and prediction capabilities of the subsidence area.

[0038] Preferably, in this dual-branch structure, one branch focuses on extracting static image information from spatial features, such as surface texture and edge features, and the other branch focuses on capturing dynamic temporal information, such as the motion changes and subsidence evolution between image frames. Through this parallel processing, not only can the fusion effect of multi-modal features of image data be improved, but also the accurate identification and prediction capabilities of the model for the subsidence area and deformation trend can be enhanced. Compared with the traditional single-branch structure, this solution avoids information loss and single dependence on feature information. Existing technologies mostly rely on the processing of single spatial features or temporal features and are difficult to take into account the spatio-temporal information of both at the same time, while the dual-branch structure can efficiently integrate these two types of information and improve the processing efficiency and accuracy.

[0039] For the global feature branch, through the dynamic convolutional layer, a global deformation feature vector is generated based on the global convolutional kernel ; Specifically, the standardized image data and the results of optical flow analysis (including displacement field, deformation area, and subsidence boundary) are used as inputs and fed into the dynamic convolutional layer. The dynamic convolutional layer performs convolution operations on the input data by adaptively adjusting the size, shape, and weight of the convolutional kernel to extract global deformation features. The convolutional kernel is adjusted in real time according to the deformation pattern of the image to capture the subsidence changes of the ground surface at different temporal and spatial scales. After the convolution operation, a feature map is generated, and through the global average pooling operation, the feature map is compressed into a global deformation feature vector with a fixed dimension . Among them, the global deformation feature vector contains the overall information of the ground surface deformation and provides global feature data for the subsequent identification of the subsidence area and trend prediction.

[0040] For the local feature branch, a convolutional kernel is dynamically adjusted using a local receptive field to output a local deformation feature vector of boundary details and curvature changes. ; It should be noted that the refined subsidence boundary region is input into the local feature branch. The local feature branch will dynamically adjust the size and shape of the convolutional kernel according to the boundary details and curvature change regions using the local receptive field (LocalReceptiveField) to adapt to different deformation regions.

[0041] Specifically, within each boundary detail region, a dynamic convolutional kernel adjustment mechanism is adopted. First, according to the curvature information and deformation amplitude of the local region, by designing the scale change of the convolutional kernel, the convolutional kernel can accurately identify the boundary details. Then, based on the convolutional kernel adjusted by this local receptive field, a convolutional operation is performed to extract the deformation features within each local region, including the minute changes in boundary details and curvature changes. At this time, the size and shape of the convolutional kernel will adaptively change according to the features of each local region in the image to better capture the boundary details and changes. Finally, the feature map obtained through the convolutional operation will be further compressed into a local deformation feature vector. . Among them, the local deformation feature vector contains the change information of boundary details and the features of curvature changes within the local region, and will play an important role in subsequent subsidence region identification and trend prediction.

[0042] Through the feature diversity constraint function , the features of global branch learning and local branch learning are complemented and a multi-modal feature map is output. The expression is: ; Among them, represents the total number of pixel points in the normalized image data, represents the index variable of the pixel points in the normalized image data, represents the global feature of the th pixel point, represents the local feature of the th pixel point, represents the similarity degree after alignment of the global feature vector and the local feature vector of the th pixel point in the same dimensional space, represents the square of the Frobenius norm, identifies, represents the local identification; Cross-branch interaction is introduced, and a cross-branch attention mechanism is used to establish interdependent relationships between different branches of the multi-modal feature map. ; Specifically, perform normalization processing on the multi-modal feature map; It should be noted that the normalization processing includes image preprocessing, denoising processing, image registration, temporal synchronization, pixel normalization, and normalization operations.

[0043] Divide the normalized multi-modal feature map into multiple branches, and each branch represents different types of input features; Process the input features of each branch through convolutional layers and pooling layers to extract corresponding local features and global features, and generate feature representations for each branch; Model the relationships between different branches through fully connected layers and output an initial weight matrix; Calculate the cross-branch attention allocation using the initial weight matrix, and generate a normalized attention matrix through the Softmax function; Weight the feature maps of each branch according to the attention matrix, and extract the mutual dependence features between each branch and its adjacent branches.

[0044] Specifically, based on the attention matrix obtained in the previous step, it represents the mutual dependence relationships between the branches. This matrix is generated through fully connected layers and normalized by Softmax to ensure that the attention weights of each branch are appropriate and the sum is 1. Next, apply this attention matrix to the feature maps of each branch for weighting operations. For each branch feature map, we use the corresponding elements of the attention matrix to weight the relationships with other branches.

[0045] For example, assume there is an attention matrix that represents the dependence relationships between the branches. For example, the elements in the matrix represent the attention weights from one branch to another. Each feature map is weighted and fused with the feature maps of other branches through this attention weight matrix. Specifically, the feature map of a certain branch is adjusted according to its correlation with other branches through the weights of the attention matrix. If the weight of a certain branch is large, it means that it has a stronger influence on other branches, and vice versa.

[0046] Furthermore, the weighted feature map will be obtained by weighted summing the feature maps of all other branches according to the corresponding weights. In this way, the feature map of the current branch not only depends on its own features but also can comprehensively fuse with the features of its adjacent branches. This weighted fusion process ensures that each branch can absorb the information of other branches associated with it, thereby enhancing the expression ability of the overall features. Finally, the weighted feature map will extract the mutual dependence relationships between each branch and its adjacent branches, providing more accurate and comprehensive support for the perception of the subsidence area and its changes.

[0047] On the basis of cross-branch interaction, perform adaptive fusion to generate a feature fusion representation, and the expression is: ; Among them, represents the activation function, represents the probability of a pixel belonging to the subsidence area; Input the feature fusion representation into the adaptive convolution layer, and generate a probability map of the subsidence area through adaptive convolution operation; Set the pixel determination threshold ; Specifically, extract the displacement value of each pixel from the normalized image data, where the displacement value represents the offset of the pixel during the surface deformation process. The optical flow vector obtained through the optical flow analysis method reflects the pixel displacement between adjacent image frames. Next, calculate the displacement magnitude of each pixel, which can be obtained by analyzing the optical flow vector, that is, calculate the displacement amplitude of each pixel. Then, according to the distribution of the displacement amplitudes, statistically calculate the displacement magnitudes of all pixels in the entire image dataset, and calculate the mean and standard deviation of this distribution. Based on these statistical results, set the pixel determination threshold .

[0048] It should be noted that the pixel determination threshold is generally selected as the mean plus several times the standard deviation, used to distinguish pixels within the normal range and pixels with obvious displacements. Usually, the mean plus 2 to 3 times the standard deviation can be selected as the pixel determination threshold .

[0049] If , the pixel is determined to be a non-subsidence area; If , the pixel is determined to be a subsidence area.

[0050] The spatio-temporal prediction module, based on the subsidence area, combines the temporal information of the image data, and uses a spatio-temporal convolutional neural network to predict the subsidence change trend.

[0051] Furthermore, synchronize the time sequence of the collected image data to ensure that each frame of the image is aligned with its corresponding timestamp, thereby generating time-sequence image data; Divide the time-sequence image data into multiple data segments by the sliding window method; Among them, each segment contains several consecutive frames, used to capture the local spatio-temporal features of the subsidence change.

[0052] Perform convolution in the spatial dimension to extract local features (such as displacement, deformation, etc.), and extract the local features of each data segment; It should be noted that by learning the spatial patterns in the image through the convolutional layer, the local change features of the subsidence area can be captured.

[0053] Perform a convolution operation in the time dimension to learn the time-varying features in the data segments, capture the dynamic evolution of subsidence, and enable the spatio-temporal convolutional neural network to identify the trends and patterns of subsidence over time; Introduce an adaptive mechanism in the convolutional layer to dynamically adjust the size and shape of the convolutional kernel according to the dynamic evolution of subsidence; Specifically, use a standard convolutional kernel to perform preliminary feature extraction of the subsidence area to obtain spatial features; Fuse the spatial features and displacement field data through feature concatenation to generate temporal features; Based on the temporal features and displacement field data, calculate the subsidence change rate of the subsidence area, and identify sudden subsidence areas, stable subsidence areas, and large-depth subsidence areas; It should be noted that the ground displacement information of consecutive time frames is extracted from the temporal feature data and displacement field data. Among them, the temporal feature data and displacement field data include the displacement amounts (in the X, Y, and Z directions) of each time frame and the displacement rate of each pixel point. By calculating the position differences between consecutive frames, the displacement rate of each subsidence area can be obtained. The specific steps are as follows: Extract any two consecutive time frames from the temporal data and calculate the displacement vector of each pixel point; then, use the differences of these displacement vectors to calculate the displacement rate of the subsidence area, that is, the subsidence change rate of each subsidence area is obtained by dividing the displacement amount by the time difference. Next, by calculating statistics such as the mean and variance of the displacement rates of each pixel point within each subsidence area, an overall change rate index of the subsidence area is generated. This process can reveal the dynamic evolution of the subsidence area in different time periods, provide the subsidence rate information of different subsidence areas, and thus provide a basis for subsequent subsidence area identification.

[0054] Furthermore, based on the calculated subsidence change rate, first cluster the regions to identify different rate regions. By setting a threshold for the change rate, the subsidence areas are divided into different groups: areas with a higher rate are regarded as sudden subsidence areas; areas with a slow and stable rate change are stable subsidence areas; and areas with a larger subsidence depth need to be further identified based on the depth values of the displacement field. The specific steps are as follows: According to the subsidence change rate, set multiple rate intervals. For example, areas with a rate higher than a certain threshold are classified as sudden subsidence areas; areas with a rate lower than another threshold and a slow change are classified as stable subsidence areas. For large-depth subsidence areas, combining the displacement depth information in the displacement field data, select areas greater than a certain depth threshold as large-depth subsidence areas. In this way, the subsidence areas can be automatically divided into three categories: sudden subsidence, stable subsidence, and large-depth subsidence, providing a basis for subsequent dynamic prediction and risk assessment.

[0055] Dynamically adjust the size of the convolutional kernel according to the change rate; for sudden subsidence areas, use a small convolutional kernel to focus on local changes; Specifically, by comparing the sequential data before and after, the areas with a higher rate of change are identified, and these areas are the sudden subsidence areas. The characteristics of the sudden subsidence areas are manifested as drastic changes in a short period of time, usually accompanied by local drastic displacements. For these areas, in order to more accurately capture the subtle local changes, it is necessary to use a smaller convolutional kernel for processing. In the specific operation, first, according to the calculated rate of change, the areas with a rate of change higher than the set threshold are selected as the sudden subsidence areas. For the sudden subsidence areas, a small convolutional kernel of 3x3 or 5x5 is used for feature extraction. The small convolutional kernel can focus on local changes and reduce the smoothing effect of the large-scale convolutional kernel, thus effectively capturing the local deformation details of the sudden subsidence. In addition, by dynamically adjusting the stride of the convolutional kernel, the receptive field of the small convolutional kernel in the local area is further enhanced, and the sensitivity to the sudden subsidence changes is improved.

[0056] For the stable subsidence areas, a large convolutional kernel is used to capture the subsidence trend; Specifically, the areas with a slow or near-zero rate of change obtained by calculation can be determined as the stable subsidence areas. It should be noted that the subsidence trend shows a relatively stable change, usually accompanied by a gradual settlement over a long period of time. In order to accurately capture the subsidence trend, it is necessary to use a large convolutional kernel for feature extraction. In the specific operation, first, the subsidence areas with a rate of change lower than the set threshold are screened out as the stable subsidence areas. Then, a larger convolutional kernel such as 7x7 or 9x9 is used to perform convolutional processing on the subsidence areas. The large convolutional kernel can effectively capture the subsidence change trend over a long period of time, reduce noise interference, and smooth the change characteristics of the stable subsidence areas, so as to better capture the laws and trends of long-term subsidence. In addition, using a larger convolutional kernel can better understand the overall deformation characteristics of the area rather than local short-term changes.

[0057] For the large-depth subsidence areas, the shape of the convolutional kernel is adaptively adjusted according to the subsidence depth.

[0058] Specifically, according to the subsidence depth, the degree of deformation of the area can be further analyzed through the displacement field data. For the subsidence areas with significant depth, the shape of the convolutional kernel needs to be adaptively adjusted. This is because the subsidence areas with a larger depth are often accompanied by complex deformation patterns, such as irregular subsidence or drastic local morphological changes. In order to adapt to this complex deformation, the shape of the convolutional kernel will be dynamically adjusted according to the deformation characteristics of the subsidence area. For example, for the areas showing significant depth changes, the shape of the convolutional kernel may need to be adjusted from the standard square (such as 3x3 or 5x5) to an asymmetric or rectangular convolutional kernel to better cover different directions of the deformed area. This adaptive adjustment process is based on the subsidence depth, and by calculating the displacement gradient or local curvature within the area, the adjustment direction and size of the convolutional kernel are determined, so as to better capture the complex morphology of these deep subsidence areas.

[0059] Integrate spatial features and temporal features through a spatio-temporal convolutional network to construct a spatio-temporal feature vector; Based on the spatio-temporal feature vector, predict the subsidence change trend through a deep learning algorithm.

[0060] Specifically, after the spatio-temporal feature vector is standardized, it is ensured that the eigenvalue of each dimension has a similar distribution, so as to eliminate the dimensional difference between features. The standardized spatio-temporal feature vector is passed as input into the deep neural network. The first layer of the deep neural network model is usually a fully connected layer, and the features are mapped through a non-linear activation function (such as ReLU) to enhance the learning ability of the deep neural network model for complex non-linear relationships. Next, the deep neural network model gradually processes the spatio-temporal features through hierarchical structures such as multi-layer perceptron (MLP), convolutional neural network (CNN), or recurrent neural network (RNN). In the time series modeling part, if a recurrent neural network (such as LSTM or GRU) is used, the deep neural network model will analyze the dynamic features of the subsidence area over time in a recursive manner, capture the long-term time dependence, and identify the patterns and trends of subsidence changes.

[0061] Preferably, through the combination of feedforward and recursive operations, the deep neural network model can not only capture the local features of subsidence from a spatial perspective, but also grasp its evolution process from a temporal perspective. After multiple layers of feature transformation and time series modeling, the final output layer of the deep neural network model will generate the prediction results of the subsidence change trend.

[0062] An early warning module, which evaluates the subsidence risk and generates a real-time warning signal according to the subsidence change trend and the coal mining progress.

[0063] Furthermore, based on optical flow analysis and spatio-temporal convolutional neural network, predict the subsidence dynamic data of the subsidence area; Obtain the mining progress data from the coal mining progress; Specifically, it is necessary to extract data related to mining operations from the coal mine dispatching platform or operation management platform, including information such as the location of the working face, the mining depth, the mining rate, and the change of the mining area. These information are usually recorded in the form of timestamps, so it is necessary to ensure the timeliness and integrity of the data. Next, by formatting and standardizing these mining progress data, it is ensured that various data have consistent units and measurement standards, and they are converted into a time series format suitable for subsequent analysis to form mining data.

[0064] Align the timestamps of the subsidence dynamic data and the mining progress data, and resample them into the same time interval by using the time window sliding method; Extract the subsidence rate from the aligned subsidence dynamic data, and establish the correlation between subsidence and mining by combining the mining progress data; Preferably, by establishing the correlation between subsidence and mining, the internal connection between mining activities and subsidence changes can be clearly revealed, providing a quantitative way to understand the correlation between the occurrence of subsidence and the mining progress. Specifically, by extracting the subsidence rate and combining it with the mining progress data, the impact of different stages of the mining process on surface subsidence can be captured more accurately, thereby providing a reliable data basis for subsequent subsidence trend prediction and risk assessment.

[0065] Input the correlation between subsidence and mining into the time series analysis model to train a subsidence change trend prediction model; Define the subsidence risk index based on the relationship between the predicted subsidence change trend and the mining progress; It should be noted that defining the subsidence risk index requires analyzing features such as the subsidence rate and the amplitude of displacement changes, and combining different stages of coal mining (such as mining depth, working face location, etc.) to determine the factors affecting subsidence, and defining the subsidence risk index according to these factors. Specifically, by synchronously comparing the historical change trend of subsidence with the mining progress, the degree of influence of different mining stages on subsidence changes is identified, and then a risk threshold is set. For example, when the displacement rate exceeds the preset range or the subsidence area continues to expand, the risk index increases. Finally, by dynamically adjusting the risk threshold, the risk assessment can reflect the actual progress of coal mining and the subsidence change trend in real time, so as to achieve more accurate risk warning.

[0066] Dynamically adjust the subsidence risk index according to the phased changes in coal mining output by the subsidence change trend prediction model, and generate warning signals in real time.

[0067] In summary, through the preprocessing and standardization of image data, the present invention can improve the comparability and analysis accuracy of data. The preprocessing module effectively converts the collected image data into a standardized format suitable for analysis, providing an accurate basis for subsequent optical flow analysis. The optical flow analysis module can analyze the movement trajectories of pixel points in the image, capture the dynamic changes in the coal mining area in real time, identify the signs of subsidence in a timely manner, and give early warnings, significantly improving the timeliness and accuracy of monitoring. In addition, the real-time and high-precision analysis capabilities of the warning system can better support coal mine managers to make scientific decisions in the face of subsidence risks, reduce the possibility of accidents, ensure the safety of the coal mining process, and improve the economic benefits and safety guarantee level of coal mine production.

[0068] Embodiment 2, referring to Table 1, is the second embodiment of the present invention. To further verify the technical solution of the present invention, experimental simulation data of the warning system for coal mining subsidence based on image processing are given.

[0069] A certain coal mining area was selected as the test site. This mining area is located in a typical coal mining region, and there is an obvious subsidence risk on the ground surface. The test involves four main steps: image data collection, optical flow analysis, subsidence area identification, subsidence trend prediction, and real-time warning. All test steps are combined with the technical solution of the present invention and compared with the existing related technologies.

[0070] First, the high-resolution drone shooting technology is used to collect high-frequency ground surface image data in the coal mining area. The collected image data includes ground surface image sequences, optical flow field data, texture feature data, and edge detection data. After the image data is collected, Gaussian kernel is used to denoise the image data to ensure the removal of noise interference during the shooting process. Next, the image registration method is applied to spatially align the denoised image data, and the time series synchronization is performed through the timestamps of the image data. Finally, all the time series synchronized image data are scaled to a unified size, and the pixel values are normalized to form standardized image data.

[0071] The standardized image data is processed by consecutive image frames through the optical flow analysis method. Specifically, first, adjacent image frame pairs are extracted from the standardized image data in the order of timestamps, and the motion information is calculated based on the sparse optical flow. Then, the dense optical flow is used to refine and generate the optical flow field. On this basis, median filtering and regional consistency constraints are used to correct the noise of the optical flow field, and the optical flow vectors are output. Further, the optical flow vectors are analyzed, the displacement magnitudes of each pixel in the ground surface image sequence are calculated to generate a displacement field, and the gradient magnitude is calculated through the gradient components to identify the deformation area.

[0072] In the subsidence area identification module, the subsidence dynamic change data is combined with the standardized image data, and the subsidence area is identified through an adaptive convolutional neural network. The weighted attention mechanism is used to fuse the subsidence dynamic change data with the image data to generate multi-modal feature data, and the feature tensor is constructed through normalization processing. Then, the feature tensor is input into the dual-branch convolutional neural network to extract global and local features respectively, and the subsidence area is identified by combining the multi-modal features. The spatio-temporal prediction module based on the spatio-temporal convolutional neural network combines the time series information to predict the subsidence change trend and generates a real-time warning signal.

[0073] The warning module evaluates the subsidence risk according to the subsidence change trend and the coal mining progress. Specifically, the dynamic data of the subsidence area predicted by combining the optical flow analysis and the spatio-temporal convolutional neural network is aligned with the coal mining progress data in terms of timestamps, and the time series analysis model is used to predict the subsidence change trend. Through the correlation analysis of the subsidence rate and the mining progress, a warning signal is generated in real time.

[0074] Specifically, as shown in Table 1 below: Table 1 Comparison Table of Experimental Data

[0075] As can be seen from the table, the technical solution of the present invention shows obvious advantages in many aspects, and the specific analysis is as follows: Image data acquisition frequency: The system of the present invention collects image data through drones at a high frequency, and the acquisition frequency reaches 50 times per day, which is significantly higher than the data acquisition frequencies of the prior art based on geological survey methods (5 times per day) and radar interferometry (3 times per day). This advantage ensures that during the coal mining process, the changing data of the ground surface can be obtained in real time and accurately, enhancing the response speed and accuracy of the subsidence warning system.

[0076] Subsidence recognition accuracy: In terms of subsidence area recognition, the present invention adopts an adaptive convolutional neural network, combined with multi-modal information of image data (such as optical flow field, texture features, etc.), and the accuracy of subsidence recognition reaches 95%. In contrast, the recognition accuracy of the prior art based on geological survey methods is only 70%, and the accuracy based on radar interferometry is also only 80%. This difference shows the high efficiency and reliability of the present invention in subsidence area recognition.

[0077] Subsidence change prediction accuracy: The present invention predicts the subsidence change trend through a spatio-temporal convolutional neural network, and the prediction accuracy reaches ±2 days, which has a significant advantage compared with ±7 days of the prior art based on geological survey methods. This enables the system to predict the dynamic evolution of subsidence more accurately in advance, providing a more precise risk assessment for coal mining.

[0078] Real-time warning generation time: The system of the present invention can generate real-time warning signals within 5 seconds, while the prior art generally takes 30 seconds (based on geological survey methods) to 25 seconds (based on surface settlement monitoring instruments) to generate warning signals. By optimizing the algorithm and hardware configuration, the present invention significantly improves the real-time performance, and can provide timely warnings during the rapidly changing coal mining process, reducing the subsidence risk.

[0079] System deployment cost: Although the system deployment cost of the present invention is 300,000 yuan, it is more cost-effective compared with 500,000 yuan and 600,000 yuan based on radar interferometry and geological survey methods, and is also lower than 800,000 yuan based on surface settlement monitoring instruments. Therefore, the present invention not only has advantages in technology, but also has obvious competitiveness in economy, and is suitable for large-scale popularization and application.

[0080] It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and not to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solution of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solution of the present invention, and it should be covered by the scope of the claims of the present invention.

Claims

1. An early warning system for coal mining subsidence based on image processing, characterized in that: include, A preprocessing module collects image data of the coal mining area, preprocesses the image data, and generates standardized image data; The optical flow analysis module processes the standardized image data in continuous frames through the optical flow analysis method, calculates the surface displacement and deformation information, and generates the dynamic change data of the subsidence; The subsidence identification module combines the subsidence dynamic change data with the standardized image data and inputs them into the adaptive convolutional neural network to identify the subsidence area; The spatiotemporal prediction module uses a spatiotemporal convolutional neural network to predict the subsidence trend based on the subsidence area and the time series information of the image data; The early warning module assesses the subsidence risk and generates real-time early warning signals based on the subsidence trend and coal mining progress.

2. The coal mining subsidence early warning system based on image processing as claimed in claim 1, characterized in that: The image data includes a ground surface image sequence, optical flow field data, texture feature data and edge detection data.

3. The coal mining subsidence early warning system based on image processing as claimed in claim 2, characterized in that: The image data is preprocessed to generate standardized image data, and the specific steps are as follows: Apply Gaussian kernel to denoise image data; The image registration method is used to spatially align the denoised image data, and the time series is synchronized through the timestamp of the image data; The time-synchronized image data is scaled to a uniform size and pixel values ​​are normalized to form standardized image data.

4. The coal mining subsidence early warning system based on image processing as claimed in claim 3, characterized in that: The optical flow analysis method is used to process the standardized image data in continuous image frames, calculate the surface displacement and deformation information, and generate the subsidence dynamic change data. The specific steps are as follows: extracting adjacent image frame pairs in time stamp order from the normalized image data; Based on adjacent image frame pairs, the motion information is calculated using sparse optical flow, and then the optical flow field is generated through dense optical flow refinement; Apply median filtering and regional consistency constraints to correct the noise of the optical flow field and output the optical flow vector; Analyze the optical flow vector, calculate the displacement of each pixel in the surface image sequence, and form a displacement field; Based on the displacement field, the gradient amplitude is calculated through the gradient components to identify the deformation area; Calculate the second-order partial derivatives of the deformation area and construct the Hessian matrix; The curvature change of the region is analyzed by the Hessian matrix to refine the subsidence boundary; The displacement field, deformation area and refined subsidence boundary are integrated and combined with the time stamp sequence of image data to generate subsidence dynamic change data.

5. The coal mining subsidence early warning system based on image processing as claimed in claim 4, characterized in that: Based on the displacement field, the gradient amplitude is calculated through the gradient component to identify the deformation area. The specific steps are as follows: Use the Sobel operator to calculate the gradient of the displacement field and extract the horizontal gradient component of the displacement field The gradient component perpendicular to the displacement field ; The Sobel convolution kernel is used to filter the horizontal gradient component and the vertical gradient component respectively; The gradient amplitude of each pixel in the displacement field is calculated based on the gradient component after filtering. The expression is: ； in, Represents pixel The gradient amplitude of Represents the modulus of the two-dimensional gradient vector; Set the deformation detection threshold ; The gradient amplitude and the deformation detection threshold Compare and generate pixel values ​​of binary image , the expression is: ； Among them, when When , it means that when the gradient amplitude is greater than the deformation detection threshold , pixel Belongs to the deformation area and is assigned ; Otherwise, the gradient amplitude is less than or equal to the deformation detection threshold , the pixel does not belong to the deformation area, and is assigned .

6. The coal mining subsidence early warning system based on image processing as claimed in claim 5, characterized in that: The subsidence dynamic change data is combined with the standardized image data and input into the adaptive convolutional neural network to identify the subsidence area. The specific steps are as follows: The subsidence dynamic change data and the standardized image data are fused through the weighted attention mechanism to form multimodal feature data; Performing standardization on the multimodal feature data, and constructing a feature tensor through the standardized multimodal feature data; Input the feature tensor into the dual-branch structure; For the global feature branch, a global deformation feature vector is generated based on the global convolution kernel through a dynamic convolution layer. ; For the local feature branch, the local receptive field is used to dynamically adjust the convolution kernel to output the local deformation feature vector of boundary details and curvature changes. ; By constraining the function through feature diversity , the features of global branch learning and local branch learning are complemented and a multimodal feature map is output. The expression is: ； in, Represents the total number of pixels in the standardized image data. The index variable representing the pixel point in the standardized image data, Indicates The global features of pixels, Indicates The local features of pixels, It indicates the The similarity between the global feature vector and the local feature vector of each pixel after alignment in the same dimensional space. represents the square of the Frobenius norm, Represents a global identifier. Indicates a local identifier; Introducing cross-branch interactions and using cross-branch attention mechanisms to establish interdependencies between different branches of multimodal feature maps ; Based on the cross-branch interaction, adaptive fusion is performed to generate feature fusion representation, which is expressed as: ； in, represents the activation function, Represents pixel Probability of belonging to a subsidence area; The feature fusion representation is input into the adaptive convolution layer, and the probability map of the sinking area is generated through the adaptive convolution operation; Set pixel determination threshold ; like When , the pixel is judged as a non-sinking area; like , the pixel is judged as a sinking area.

7. The coal mining subsidence early warning system based on image processing as claimed in claim 6, characterized in that: The cross-branch interaction is introduced and the cross-branch attention mechanism is used to establish mutual dependencies between different branches of the multimodal feature map. , the specific steps are as follows, Normalize the multimodal feature maps; The standardized multimodal feature map is divided into multiple branches, each branch represents a different type of input features; The input features of each branch are processed through convolutional layers and pooling layers, the corresponding local features and global features are extracted, and the feature representation of each branch is generated; The relationship between different branches is modeled through the fully connected layer, and the initialization weight matrix is ​​output; Use the initialized weight matrix to calculate the attention allocation across branches, and generate a normalized attention matrix through the Softmax function; The feature maps of each branch are weighted according to the attention matrix to extract the interdependent features of each branch and its adjacent branches.

8. The coal mining subsidence early warning system based on image processing as claimed in claim 7, characterized in that: Based on the subsidence area and combined with the time series information of the image data, the spatiotemporal convolutional neural network is used to predict the subsidence change trend. The specific steps are as follows: Performing time-series synchronization on the collected image data to generate time-series image data; The time series image data is divided into a plurality of data segments by a sliding window method; Perform convolution in the spatial dimension to extract local features and extract local features of each data segment; Perform convolution operations on the time dimension to learn the time-varying features in the data segments and capture the dynamic evolution of the subsidence; In the convolution layer, the size and shape of the convolution kernel are dynamically adjusted according to the dynamic evolution of the sink; The spatial and temporal features are integrated through the spatiotemporal convolutional network to construct the spatiotemporal feature vector; Based on the spatiotemporal feature vectors, the subsidence trend is predicted by deep learning algorithm.

9. The coal mining subsidence early warning system based on image processing as claimed in claim 8, characterized in that: In the convolution layer, the size and shape of the convolution kernel are dynamically adjusted according to the dynamic evolution of the sink. The specific steps are as follows: Use standard convolution kernel to perform preliminary feature extraction of subsidence area and obtain spatial features; The spatial features and displacement field data are fused through feature splicing to generate time series features; Based on the time series characteristics and displacement field data, the subsidence change rate of the subsidence area is calculated, and the sudden subsidence area, stable subsidence area and deep subsidence area are identified; Dynamically adjust the size of the convolution kernel according to the rate of change; For sudden subsidence areas, a small convolution kernel is used to focus on local changes; For stable subsidence areas, a large convolution kernel is used to capture the subsidence trend; For deep subsidence areas, the shape of the convolution kernel is adaptively adjusted according to the subsidence depth.

10. The coal mining subsidence early warning system based on image processing as claimed in claim 9, characterized in that: According to the subsidence change trend and coal mining progress, the subsidence risk is assessed and a real-time early warning signal is generated. The specific steps are as follows: Based on optical flow analysis and spatiotemporal convolutional neural network, the subsidence dynamic data of the subsidence area is predicted; Obtain mining progress data from coal mining progress; The subsidence dynamic data and mining progress data were time stamp aligned and resampled to the same time interval using the time window sliding method; Extract the subsidence rate from the aligned subsidence dynamic data and establish the correlation between subsidence and mining by combining it with the mining progress data; The correlation between subsidence and mining is input into the time series analysis model to train a subsidence trend prediction model; Define subsidence risk indicators based on the relationship between the predicted subsidence trend and mining progress; According to the phased changes in coal mining output by the subsidence change trend prediction model, the subsidence risk index is dynamically adjusted and early warning signals are generated in real time.

Images