Multi-element monitoring and early warning device for mining area

Abstract

The invention relates to the technical field of mining area safety protection, in particular to a mining area multi-element monitoring and early warning device which comprises a space-based monitoring module, an air-based monitoring module, a foundation monitoring module, a data processing module, a change detection module, an early warning module, a big data management module, a user interaction module, a communication module and a power module. According to the invention, a sky-ground integrated monitoring system is constructed, satellite remote sensing, unmanned aerial vehicle aerial photography and ground measurement technologies are combined, mining area global coverage and key area refined monitoring are realized, field workload can be greatly reduced, the monitoring period is shortened, the patrol frequency is improved, the management cost is effectively reduced, and the monitoring efficiency is improved through automatic and intelligent monitoring means. The human input of manual inspection and data processing is greatly reduced, and the project cycle is shortened.

Description

Technical Field

[0001] The present invention relates to the technical field of mine safety protection, and in particular to a mine multi-element monitoring and early warning device. Background Art

[0002] The existing mining area is relatively vast and has huge resource reserves. The resource mining cycle will be a relatively long process. Long-term resource mining will inevitably cause varying degrees of surface subsidence, which will affect the topography, landforms and ecological environment in and around the mining area. At the same time, there may also be certain non-compliant and illegal land occupation behaviors on the surface of the mining area, which will have an adverse impact on the normal and orderly development and safe production of underground coal resources. Therefore, it is necessary to carry out inspections and monitoring of the background landforms and ecological environment changes in the mining area in advance to ensure the development order of the mining area and the rights and interests of Xinjie Energy Company.

[0003] At present, Xinjie Energy Company mainly adopts manual on-site inspection to carry out surface feature inspection and geological environment monitoring in mining areas. This method is labor-intensive, inefficient, and blind, and it is difficult to guarantee the inspection accuracy and precise prevention and control of geological disasters. The effect of manual inspection is not satisfactory. In recent years, satellite remote sensing, UAV aerial survey, big data, cloud computing and other intelligent means have begun to be gradually applied in the field of inspection and monitoring. The monitoring model based on the integration of sky and ground can greatly improve the efficiency and quality of periodic surface feature inspection and ecological environment monitoring in mining areas. In order to effectively safeguard the rights and interests of Xinjie Energy Company, in response to the problems faced in mining area inspections and the needs of geological disaster prevention and control and mining area ecological environment monitoring, it is urgent to carry out research on multi-source data fusion, automatic extraction and recognition of remote sensing image information, ecological environment monitoring and evaluation, and big data steward system construction to provide support for land acquisition compensation, resettlement, industrial square design, geological disaster prevention and control, and ecological restoration in mining area development.

[0004] Therefore, in response to the above problems, the present invention proposes a multi-dimensional monitoring and early warning device for mining areas. By constructing an integrated "sky-ground" monitoring system for mining areas, periodic ground inspections of mining areas based on remote sensing big data are carried out, and an integrated monitoring big data steward system for mining areas is developed to achieve high-frequency, intelligent, automatic and precise inspections of ground objects in mining areas. Summary of the Invention

[0005] In order to overcome the problems of high labor intensity and low efficiency in the current status of mining area ground feature inspection and ecological environment monitoring, the present invention proposes a mining area multi-element monitoring and early warning device.

[0006] The technical solution of the present invention is: a multi-dimensional monitoring and early warning device for a mining area, comprising: Space-based monitoring module, used to obtain high-resolution optical image data of the entire mining area or well field through satellite remote sensing; Air-based monitoring module, used to obtain centimeter-level optical image data of key areas within the mining area through drone aerial photography; The ground-based monitoring module is used to conduct on-site verification and precise mapping of changing ground features identified by satellite remote sensing or drone aerial photography using ground-based measurement equipment; The data processing module is used to screen, match and fuse the multi-source data obtained by the space-based, air-based and ground-based monitoring modules; The change detection module is used to intelligently identify ground feature information and detect changes in the fused image data based on deep learning algorithms; The early warning module is used to generate early warning information based on the change detection results and push it to users through a visual interface or mobile terminal; Big data management module, used to store and manage multi-source monitoring data, change detection results and warning information, and provide data query and analysis functions; User interaction module, used to provide a graphical operation interface to support users to view and operate monitoring data, test results and warning information; Communication module, used to realize data transmission between modules inside the device and with external devices; The power module is used to provide power support for each module of the device.

[0007] Preferably, the space-based monitoring module supports the acquisition of satellite image data with a spatial resolution of 0.5 meters to 15 meters, with a revisit period of 10 days to 1 quarter. It can flexibly configure data acquisition plans according to the monitoring needs of different areas of the mining area, give priority to the use of domestic commercial satellite constellation data such as Gaojing, Beijing No. 3, and Jilin No. 1, and increase data redundancy through a multi-satellite source combination plan.

[0008] Preferably, the air-based monitoring module is equipped with a surveying optical camera or a lidar, which can take aerial photos of key change areas identified by satellite remote sensing with centimeter-level accuracy to obtain the location, area, geometry, height, volume and construction progress of the ground objects.

[0009] Preferably, the data processing module adopts a method that combines wavelet transform, local algorithm and principal component analysis. First, wavelet transform is used to perform multi-scale frequency division processing on multi-source images, and then the detail information is enhanced through the local variance algorithm. It is combined with principal component analysis to improve the spectral distortion phenomenon, and finally a high-resolution multispectral fusion image is generated.

[0010] Preferably, the change detection module is based on the U-Net deep learning model, which supports pixel-level classification and change detection of five types of land features: buildings, earth piles, water bodies, roads and woodlands. The model encoder uses convolution and pooling layers to extract multi-level features, and the decoder restores spatial details through deconvolution and jump connections, and finally outputs the category and boundary information of the land feature change map.

[0011] Preferably, the backbone model of the change detection module is ResNet 18, which is superior to ResNet 34 in terms of average Kappa, average pixel accuracy and average frequency-weighted intersection-over-union (IoU), and can ensure an accuracy rate of not less than 0.7 and a recall rate of not less than 0.9 when the area of the change patch is greater than 200 pixels.

[0012] Preferably, the early warning module supports classification by land feature change type and early warning level, and intuitively displays the spatial distribution of change spots through map positioning. Users can view the image comparison, attribute information and verification status of the spots. The system automatically pushes the spots that need on-site verification to the mobile inspection APP, realizing rapid discovery, precise positioning and closed-loop management of change information. The early warning levels are divided into high, medium and low.

[0013] Preferably, the big data management module adopts the PostgreSQL database and PostGIS spatial extension module to support distributed storage and efficient retrieval of massive multi-source monitoring data, realizes unified data management through multi-level indexing and standardized interfaces, and provides data governance, intelligent analysis and visualization display functions.

[0014] Preferably, the user interaction module includes a PC platform based on WebGIS and a mobile inspection APP. The PC platform provides three functional modules: remote sensing land feature monitoring, ecological environment monitoring and smart inspection, and supports image display, change detection, statistical analysis and task management. The mobile APP supports on-site map verification, attribute entry, photo uploading and navigation positioning, realizing intelligent inspection operations that collaborate with internal and external industries.

[0015] Preferably, the device also includes a technical training module to carry out special training on system operation, data analysis and on-site review for mining staff. The training content includes the functional use of the big data butler system, the principles of change detection algorithms, the operation of the mobile inspection APP and the specifications of ground feature verification, to ensure that users can skillfully use the system.

[0016] Beneficial effects of the present invention: 1. The present invention achieves full coverage of the mining area and refined monitoring of key areas by constructing an integrated "sky-ground" monitoring system, combining satellite remote sensing, drone aerial photography and ground measurement technology. Compared with traditional manual inspection methods, it can greatly reduce field workload, shorten the monitoring cycle, increase the inspection frequency, ensure timely discovery and disposal of land changes, and effectively reduce management costs. Through automated and intelligent monitoring methods, it greatly reduces the manpower input of manual inspections and data processing, shortens the project cycle, and at the same time, through early detection and intervention of violations, avoids land acquisition compensation disputes and economic losses caused by rush construction, rush planting and other behaviors, saves a lot of management costs for mining companies, and improves overall operational efficiency.

[0017] 2. The change detection algorithm based on the U-Net deep learning model, combined with the optimized ResNet 18 backbone network, performs well in pixel-level classification tasks, with an average Kappa value of 0.748 and an average pixel accuracy of 0.954. It can accurately identify changes in five types of land features, including buildings, earth piles, and water bodies, thereby significantly reducing manual intervention and lowering the rates of false detection and missed detection, meeting the high-frequency, high-precision inspection needs of mining areas.

[0018] 3. Through multi-source remote sensing data fusion technology that combines wavelet transform, local algorithm and principal component analysis, we can effectively integrate image data of different resolutions, temporal phases and spectral characteristics to generate high-quality fused images, enhance the expression of ground object details and spectral information, provide a more reliable data basis for subsequent change detection, and improve the accuracy and consistency of the overall monitoring results.

[0019] 4. Through the combination of the WebGIS platform and the mobile inspection APP, seamless connection between PC and mobile terminals is achieved. Users can view monitoring data, perform change analysis and manage inspection tasks anytime and anywhere. It also supports on-site photography, attribute entry and navigation positioning, optimizes the internal and external collaborative process, lowers the threshold for system use, and improves the overall efficiency of mine inspection work. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Shown is a schematic diagram of the system architecture of the present invention; Figure 2 What is shown is a schematic diagram of the overall technical process of the present invention; Figure 3 Shown is a schematic diagram of the sample construction process of the present invention; Figure 4 Shown is a schematic diagram of the deep learning U-Net network framework of the present invention; Figure 5 What is shown is a schematic diagram of the workflow of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0022] See also Figure 1 The present invention provides an embodiment: a multi-dimensional monitoring and early warning device for a mining area, comprising: Space-based monitoring module, used to obtain high-resolution optical image data of the entire mining area or well field through satellite remote sensing; Air-based monitoring module, used to obtain centimeter-level optical image data of key areas within the mining area through drone aerial photography; The ground-based monitoring module is used to conduct on-site verification and precise mapping of changing ground features identified by satellite remote sensing or drone aerial photography using ground-based measurement equipment; The data processing module is used to screen, match and fuse the multi-source data obtained by the space-based, air-based and ground-based monitoring modules; The change detection module is used to intelligently identify ground object information and detect changes in the fused image data based on deep learning algorithms; The early warning module is used to generate early warning information based on the change detection results and push it to users through a visual interface or mobile terminal; Big data management module, used to store and manage multi-source monitoring data, change detection results and early warning information, and provide data query and analysis functions; User interaction module, used to provide a graphical operation interface to support users to view and operate monitoring data, test results and warning information; Communication module, used to realize data transmission between modules inside the device and with external devices; The power module is used to provide power support for each module of the device.

[0023] Furthermore, the design of the multi-element detection and monitoring system for mining areas is explained: In response to the requirements of different mining areas for data types and spatiotemporal resolution of images, this study constructs an integrated sky-ground monitoring system for mining areas that combines "satellite remote sensing (space-based) + UAV aerial photography (air-based) + ground observation (ground-based)". Satellite remote sensing (space-based) mainly serves the ground feature inspection of the entire mining area or the well field. Sub-meter-level high-resolution optical satellite remote sensing images are used for ground feature inspection; UAV aerial photography (air-based) mainly serves the on-site surveying and mapping of ground feature inspections in key and detailed observation areas within the mining area. Through aerial photography of centimeter-level optical images, high-resolution auxiliary verification data is provided for the satellite remote sensing ground feature inspection results; ground observation (ground-based) mainly serves the on-site review and evidence collection of ground feature inspections in the mining area. Through on-site measurements, on-site review and evidence collection are provided for the satellite remote sensing ground feature change inspection results.

[0024] First, for satellite remote sensing monitoring star source screening: According to the requirements of satellite remote sensing image spatial resolution and acquisition cycle for land object inspection and ecological environment monitoring in different zones of the mining area, the existing available satellite sources are screened, and a list of commonly used satellites that meet the data requirements of 0.5m spatial resolution / 10-day revisit cycle, 1m spatial resolution / 1-month revisit cycle, and 15m spatial resolution / 1-quarter revisit cycle are counted. The satellite source combination scheme is analyzed and designed, and the data redundancy is moderately increased by optimizing the combination scheme to avoid the risk of meteorological influence on the revisit data of a single satellite source, and to ensure the requirements of land object inspection for remote sensing data acquisition cycle. In the present invention, the remote sensing images with 0.5m spatial resolution / 10-day revisit cycle are preferably selected from domestic commercial satellite constellation data such as Gaojing, Beijing-3, and Jilin-1; the remote sensing images with 1m spatial resolution / 1-month revisit cycle are preferably selected from domestic commercial satellite constellation data such as Gaofen series and Beijing-2.

[0025] Second, for drone aerial monitoring: Aerial drone monitoring of mining areas primarily utilizes on-site aerial surveys using industrial-grade mapping drones. To monitor features such as buildings in mining areas, satellite remote sensing identifies areas of surface change. Using drone payloads like optical cameras and lidar, these key areas of change are then verified and precisely mapped. This information, along with imagery, captures attributes such as location, area, shape, height, volume, and construction progress.

[0026] Third, precise ground measurement: In response to the inspection needs of buildings and other land features in mining areas, after satellite remote sensing identifies areas of land feature changes on the mining area's surface, mobile surveying equipment is used to conduct manual on-site review and verification and precise mapping of restricted areas for drone aerial surveys, and to obtain attributes and image information such as location, area, shape, height, volume, and construction progress of the changed land features.

[0027] Furthermore, the functional design of the integrated mining area monitoring big data steward system development is explained: First, enter your account number, password, and verification code on the login interface to log in. After successful login, you will enter the main interface of the system, which is divided into three functional modules: remote sensing land feature monitoring module, ecological environment monitoring module, and smart inspection module.

[0028] Explanation of remote sensing ground feature monitoring: Monitoring overview: Statistics include the total number of monitoring times and the number of problem spots.

[0029] Remote sensing image list: Unified management of remote sensing image data of each well and each period. Data can be selected for display separately, and the remote sensing image list can be filtered by data acquisition time.

[0030] Dynamic monitoring of land feature changes: By comparing two phases of image data, corresponding land feature change information can be extracted and monitoring results can be viewed by well.

[0031] Data information: Displays a list of current change spots. Spots can be filtered by village, change type, and warning status. Spot locations can be located and their attribute information can be displayed.

[0032] Spot information: supports adding spots; modifying spot attribute information and range; deleting spots; exporting spot data, etc.

[0033] Map spot type statistics: statistics on the number of different types of maps in the current period.

[0034] Explanation of ecological environment monitoring: Ecological environment data screening: You can set the start and end time or enter keywords for screening.

[0035] Ecological and environmental data display: Access mining area vegetation, soil, terrain conditions, and synthetic image data layers, which can be displayed separately.

[0036] Explanation of smart inspection: Verification of ground feature inspection spots: After selecting the task scope and monitoring time, the data of the spots to be verified will be displayed, and the data can be filtered by spot type and verification status.

[0037] Spot verification: Click to select spot data in the data list, the spot position will be located on the map, and the spot attribute information will be displayed. The previous and next image comparison function can be used to view the previous and next image to determine the changes in the spot. In the spot attribute information pop-up window, the spot verification status can be changed (verified / need on-site verification). Spots that require on-site verification will be pushed to the mobile terminal for on-site verification.

[0038] Mission status: can display the verification status of each mission area.

[0039] Description of the Smart Inspection APP: Task list: Inspectors can select the corresponding inspection task in the task list and view the number of unchecked spots and checked spots in the task.

[0040] Task details: You can view the verified / unverified spots within the task and locate the spots.

[0041] Image patch verification: You can view the basic information of the image patch, upload on-site evidence photos, and enter voice notes / text notes on this page.

[0042] Add new spots: Add new spots by inputting the type of spots and drawing the range of spots.

[0043] Further, the key technologies of the present invention are described as follows: During the system design process, a multi-layer architecture model was adopted, and component technology was used to achieve the reusability of basic modules, realize the flexibility, openness and scalability of the platform. The platform design adopts the currently popular Internet development microservice technology architecture thinking and cloud computing service concept. Each function independently calls the database and follows the Restful API interface. Each application function can be agilely developed and iterated. The platform development mainly adopts the most popular Spring framework. The advantage of this framework is that it can provide flexible and convenient support for the application layer. The system deployment adopts Docker-based containerization technology to facilitate cross-platform and virtualization.

[0044] The database of the present invention is described as follows: The system database uses PostgreSQL, the most high-performance database software in the field of open source spatial information software. The spatial object extension module PostGIS built on it makes it a truly large-scale spatial database. PostGIS adds the ability to store and manage spatial data to the object-relational database PostgreSQL. The biggest feature of PostGIS is that it complies with and implements some OpenGIS specifications, making it the most famous open source GIS database.

[0045] The vector data publishing technology of the present invention is described as follows: Vector data publishing technology is a method of sharing vector datasets with users. Vector data refers to geographic data composed of geometric elements such as points, lines, and surfaces, such as maps, road networks, and boundaries.

[0046] The tile map service technology of the present invention is described as follows: Tile map service is a way of publishing map data by dividing map data into multiple small blocks (tiles) and organizing them according to a certain hierarchical structure. Through tile map service, users can load and display map data in units of tiles, achieving fast map browsing and interaction.

[0047] The basic principle of tile map service is to divide map data into a series of tiles, each of which has its own coordinates and unique identifier. Usually, map data is divided into different levels, and the number of tiles in each level is four times that of the previous level. This hierarchical structure allows users to choose to load tiles of different levels as needed to achieve operations such as zooming and panning the map.

[0048] The advantages of tile map services include: Fast loading speed: The tile map service divides map data into small blocks and loads them on demand, providing fast map loading and rendering effects.

[0049] Compressed storage: Tile map services use tile segmentation, which can greatly reduce the storage space of map data and improve data transmission efficiency.

[0050] Offline use: Once the map data is divided into tiles and stored on the server, users can download the required tile data to their local computer and use the map data offline.

[0051] Flexibility and customizability: Tile map services can be customized according to user needs, including selecting different map styles, adding custom layers and markers, etc.

[0052] Wide application: Tile map services are widely used in online map services, geographic information systems, mobile device applications, and web map applications.

[0053] Common tile map services include Google Maps, OpenStreetMap, and Mapbox. These services provide APIs and tools that allow developers to easily integrate and use tile map data to build various map applications and services.

[0054] Furthermore, the high concurrency demands of numerous users are taken into consideration during the system design process. By optimizing the system architecture design, the system's concurrent processing capabilities are improved to ensure smooth system operation.

[0055] High-concurrency architecture technology refers to improving the system's concurrent processing capabilities and performance stability by optimizing the system architecture and related technical means when facing a large number of concurrent requests.

[0056] Furthermore, the screening of multi-source remote sensing data of the present invention is described as follows: In response to the requirements of mining area ground object inspection for remote sensing image data, by analyzing factors such as satellite revisit cycle, transit time, image resolution, availability, data price, image quality, etc., the alternative satellite source pairs are further screened to meet the mining area data application needs. At the same time, considering that a single satellite source is easily affected by factors such as weather when revisiting the mining area, the matching of multi-source remote sensing data with the project is analyzed, and automatic algorithms for screening and combining data from different satellite sources are studied to screen out images that meet the cloud content, number of bands, and ground object information required by the project, so that the project has a certain degree of remote sensing information redundancy and complementarity within a monitoring cycle to ensure the data collection cycle.

[0057] Furthermore, the matching of multi-source remote sensing data of the present invention is described as follows: Based on the template matching framework, this study adopts a multimodal remote sensing image matching method to match homonymous points based on the structural features between images. The geometric structure feature descriptor constructed by the phase congruence model with illumination and contrast invariance is used to achieve automatic registration of multi-source remote sensing images. The idea is as follows: a sliding window of size i×j is selected based on the difference between the reference image and the image to be corrected; the phase congruence intensity value and direction of each pixel in the window are calculated; the window is divided into several blocks (tentatively, a block contains m×m cells), where each block contains several cell units (tentatively, a cell unit contains n×n pixels), where i, j, m, and n are the parameters to be studied; the phase congruence direction histogram of the block and cell is calculated and normalized to eliminate the influence of illumination changes; the gradient direction histogram vectors within all blocks are collected together to form a phase histogram feature vector describing the entire image; and indicators such as the sum of squared grayscale differences (SSD) and normalized correlation coefficient (NCC) are used as matching similarity measures to identify homonymous points.

[0058] Furthermore, the fusion of multi-source remote sensing data of the present invention is described as follows: Multi-source remote sensing data is mainly reflected in the characteristics of multi-phase, multi-spectral, multi-sensor, multi-platform and multi-resolution. The differences between different star source sensors and imaging platforms cause the existence of "five differences" (one or more of the imaging characteristics, geometric differences, scale differences, perspective differences, and dimensional differences) and "three differences" (different environments, different weather, different weather conditions, etc.) between multimodal images. According to the fusion purpose and fusion level, a suitable fusion algorithm is used to synthesize the aforementioned remote sensing image data that have completed spatial matching (or the extracted image features or attribute descriptions of pattern recognition), and then generate high-resolution multispectral image data, so that the detailed information and spectral information of the mining area targets can be more richly expressed. This study uses the method of combining "wavelet analysis + local algorithm + principal component analysis" to carry out the fusion of multi-source and multi-modal remote sensing images.

[0059] Further, the wavelet transform of the present invention is described: By utilizing the multi-scale frequency division characteristics of wavelet transform, the signal is gradually refined at multiple scales through scaling and translation operations, so that high-frequency information is subdivided in time and low-frequency information is subdivided in frequency. While decomposing the signal into independent parts between space and time, the information contained in the original signal is not lost. In addition, a suitable orthogonal basis can be found to achieve zero-redundant signal decomposition. Then, according to the characteristics of different frequency information, operations are performed separately, so that the fusion of images can be well achieved.

[0060] Further, the local algorithm of the present invention is described: The local variance and local difference weighted methods are adopted. The local variance algorithm can better reflect the detailed information of the image, and the local difference weighted operation can better retain the low-frequency information of the image. Combining the two algorithms can improve the disadvantage of information loss when decomposing at different scales in wavelet transform and obtain more effective fusion results.

[0061] The application process of the local variance algorithm is as follows: first, determine a moving operation window of appropriate size and calculate the variance value of the window. Then, calculate the variance value of each window in the high-frequency information image in order from left to right and from top to bottom. Determine the lowest threshold through calculation, and then compare the variance values of the corresponding windows of the two images. According to the selection rules, the parameter with the larger value is taken as the central pixel value of the window corresponding to the fused high-frequency component. Calculate in sequence until the entire image is calculated, and finally obtain the new high-frequency component.

[0062] Furthermore, the image fusion algorithm combined with principal component analysis (PCA) is explained: Principal component analysis (PCA) has a good effect on preserving spectral information and can improve the spectral distortion phenomenon of wavelet transform. Combining the principles and advantages of local algorithms, this paper adopts an image fusion algorithm based on the combination of improved wavelet transform and PCA transform based on local algorithms. This new fusion algorithm fully combines the advantages of the three methods to obtain high-quality fused images. The processing steps are as follows: Data preprocessing: Linear interpolation is used to resample the multispectral images to make the pixel sizes of the images to be fused consistent. On this basis, the mining area images are cropped to obtain the original images to be fused. Perform PCA transformation on the cropped multispectral image and take its first three components; then perform histogram matching on the first principal component and the panchromatic image; The matched first principal component and the full-color band are respectively subjected to wavelet transformation to obtain their respective high-frequency and low-frequency information; the high-frequency information is fused using the local variance fusion rule to obtain new high-frequency information, while the low-frequency information is fused using the local difference weighted fusion rule to obtain new low-frequency information after fusion; The new high and low frequency information is transformed by inverse wavelet to obtain the new first component after fusion, and then the new first principal component is transformed with the original second and third components by inverse PCA to obtain the final fusion result image.

[0063] See also Figure 2 , further, the overall technical process of the present invention is described: The overall technical process mainly consists of four key steps: establishment of interpretation signs, creation of sample data sets, model selection and training, and comparison and verification of information extraction results. First, the interpretation signs of the study area are established, and the newly added buildings, earth piles, rivers, lakes (reservoirs), roads, woodlands and other map areas in the mining area are interpreted to create a training sample data set; secondly, the U-Net deep learning model and the pyramid scene interpretation network model are applied to remote sensing images to compare the model accuracy; finally, the trained model is used to compare and verify the information extraction results of the mining area.

[0064] Furthermore, in order to subsequently create a reliable and stable sample dataset, it is first necessary to clarify the interpretation signs. First, after identifying the changed area through visual interpretation, different types of land features are manually marked, that is, different types of land feature changes are marked with different colors. The specific label category comparison is shown in Table 1.

[0065] Table 1 Label comparison table

[0066] See also Figure 3 Furthermore, in response to the high-frequency inspection needs of mining areas, a four-channel (red light band, green light band, blue light band, and near-infrared band) remote sensing dataset was constructed for the changing features in the mining area in central Inner Mongolia based on the Gaojing-1 remote sensing data with a resolution of 0.5m.

[0067] To ensure the reliability and comprehensiveness of the sample, new features such as buildings, earth piles, water bodies, roads, and woodlands were selected from different areas within the study area for sample production. Furthermore, during the sample dataset production process, the accuracy and balance of labels should be ensured to avoid noise labels and sample imbalance. Furthermore, since visual interpretation is used to distinguish different features based on eight factors, including color, texture, size, shape, shadow, pattern, location, and surrounding systems, high-quality image data with uniform color and no clouds or fog should be selected as the base map before labeling. During the labeling process, in addition to accurately and completely marking the type and boundaries of the features, special attention should be paid to removing easily confused noise samples. After data cleaning, a total of 2052 512*512 pixel four-channel slices were generated, which meets the resolution requirements for monitoring changing features in the mining area's area of interest.

[0068] The labels were then double-checked to ensure accuracy. We ultimately obtained 2,053 sets of 512x512 slice experimental data, which were divided into training, validation, and test sets in a ratio of 8:1:1.

[0069] Further explanation of model selection and training: Convolutional neural networks (CNNs), a widely used model in the field of deep learning, have the advantage of directly processing multidimensional image data without the need for complex preprocessing, thus eliminating the more complex image feature extraction and data reconstruction processes. In the task of image patch recognition in mines, recognition is mainly based on the spectral information of remote sensing images, and the U-Net model is selected for pixel-level classification training.

[0070] See also Figure 4 The U-Net model consists of two parts: an encoder and a decoder. In the encoding stage, the model uses two 3×3 convolution kernels for feature extraction, then applies the ReLU activation function, and uses a 2×2 maximum pooling layer to reduce the feature dimension and expand the receptive field. After each sampling, the image size is halved, and the feature dimension is doubled. This hierarchical structure enables the model to effectively extract high-level features of the image and filter out irrelevant information. In the decoding stage, the model upsamples through the deconvolution layer, and then uses two 3×3 convolution kernels to gradually restore the image details, and finally restores the feature map to the size of the original input image. After each sampling, the image size is doubled, and the feature dimension is halved. In order to restore the image details more accurately, the encoder and decoder reuse low-level feature information through jump connections.

[0071] After selecting the U-Net model, its backbone model, that is, the combination of convolutional layers and pooling layers, is also required. In this study, ResNet was selected as the backbone model. ResNet aims to improve the limitations of traditional overlay networks and optimize network performance by introducing residual structures, solving the degradation problem in deep networks. ResNet approximates the residual function by superimposing nonlinear layers and uses identical shortcut connections to achieve identical mapping. Such connections neither introduce additional parameters nor increase the complexity of calculations. The two network scales commonly used by ResNet are mainly used, namely ResNet 18 and ResNet 34.

[0072] After determining parameters such as the data training set, model, and backbone model, model training began. To ensure effective training, the maximum number of epochs was set to 20. After completing these parameter configurations, a mining area within the study region was selected as the target for building a training dataset for extracting ground feature information from areas of interest. Information extraction experiments were conducted using both the U-Net model and the Pyramid scene. Based on these experimental results, the model and parameter combination with the best performance were selected for use in extracting ground feature information from mining areas across the entire study area.

[0073] Further, the accuracy evaluation of the present invention is described: When dealing with binary classification problems, each sample in the dataset is originally labeled as positive or negative. The classifier's task is to classify each sample as positive or negative, that is, to produce a classification result. In order to evaluate the accuracy of the classification, a set of symbols is used: In data classification, TP (true positive) represents the set of samples that are marked as positive in the original dataset and are also correctly judged as positive during the classification process; FN (false negative) represents the set of samples that are marked as positive in the original dataset but are judged as negative after classification; FP (false positive) represents the set of samples that are marked as negative in the original dataset but are judged as positive after classification; TN (true negative) represents the set of samples that are marked as negative in the original dataset and are also judged as negative after classification. As shown in the table below, T and F respectively indicate whether the classification result is correct (true or false); P and N respectively indicate whether the classification result is positive or negative, as shown in Table 2.

[0074] Table 2 Representation method of classification results

[0075] Furthermore, the Kappa statistic is a measure of the consistency between a classification model's predictions and actual observations. It takes into account the random accuracy of the classification and compares the model's accuracy to random accuracy, providing a more comprehensive assessment. Average Kappa is the average of the Kappa coefficients for all categories.

[0076] ; in, It is the sum of the number of correctly classified samples in each category divided by the total number of samples (i.e., overall classification accuracy). Assume that the number of true samples in each category is a1, a2, ..., ac; and the number of predicted samples in each category is b1, b2, ..., bc; the total number of samples is n, then: ; The calculated results of the Kappa coefficient range from -1 to 1, but usually, the Kappa value is between 0 and 1. This range can be divided into five intervals, each representing a different level of consistency, as shown in Table 3: Table 3 Kappa coefficient grading standard table

[0077] Furthermore, accuracy refers to the proportion of samples that the model correctly predicts as positive among all samples predicted as positive, and average accuracy is the average of the accuracy of all categories.

[0078] ; Furthermore, the recall rate refers to the ratio of the number of samples successfully predicted by the model as positive to the total number of actual positive samples, and the average recall rate is the average of the recall rates of all categories.

[0079] ; The F1 score is the harmonic mean of precision and recall, which is used to comprehensively evaluate the performance of the classification model. The average F1 score is the average of the F1 scores of all categories.

[0080] ; Precision is the accuracy, also known as the precision rate, which is the number of samples predicted as positive in the study area. It represents the proportion of samples predicted correctly in the predicted classification results. The formula is as follows: ; Pixel accuracy refers to the ratio of correctly classified pixels to the total number of pixels in the classification results. This parameter measures the proportion of correctly classified pixels among all pixels and is a relatively simple and intuitive evaluation indicator. The average pixel accuracy is the average value of the accuracy of each type of pixels.

[0081] Average classification accuracy calculates the average accuracy of all classes. It is calculated by averaging the accuracy of each class.

[0082] Intersection over Union (IoU) is a measure of overlap between two classes. In the context of image segmentation, IoU calculates the area of intersection between the predicted segmentation mask and the ground truth mask divided by the area of union. The mean intersection over union (MIoU) is the average IoU over all classes.

[0083] ; ; Among them, K represents the category, i represents the true value, and j represents the predicted value. It means that i is predicted to be j, which is FN; It means that j is predicted to be i, which is FP; Indicates that the prediction is correct, which is TP.

[0084] Frequency-weighted IoU assigns a weight to each class based on its frequency in the dataset, which helps give more weight to classes that appear more frequently. Average frequency-weighted IoU is the weighted average IoU of all classes.

[0085] ; Furthermore, this study mainly used two deep residual network models, ResNet 18 and ResNet 34, in the selection of backbone models. The eight evaluation parameters in the accuracy evaluation section were used to comprehensively measure the accuracy and effectiveness of the ResNet 18 and ResNet 34 models as shown in the following table. The performance of the two models on image classification tasks was compared. Table 4 shows the performance of the two models under different evaluation parameters.

[0086] Table 4 Performance evaluation of the two models on image classification tasks

[0087] See also Figure 5 , further, the workflow of the present invention is described: The space-based monitoring module is used to obtain satellite remote sensing image data with a resolution of 0.5-15 meters for the entire mining area. The collection plan is configured according to the monitoring needs of different sub-areas of the mining area. High-resolution satellite data is used in key areas. At the same time, the air-based monitoring module is used to carry out drone aerial photography of the identified key change areas to obtain optical images or lidar data with centimeter-level accuracy. For areas that cannot be covered by satellites and drones, the ground-based monitoring module is used to perform precise ground measurements, forming an integrated "sky-ground" multi-source data collection network.

[0088] The collected original satellite and UAV images are subjected to radiation correction, geometric correction and image enhancement processing to eliminate the influence of sensor distortion, atmospheric scattering and other factors. At the same time, the GPS / Beidou positioning system is used to unify the coordinates of the ground measurement data to ensure that all data have a consistent spatial reference benchmark, laying the foundation for subsequent multi-source data fusion.

[0089] Wavelet transform is used to perform multi-scale decomposition of multi-source images, extract information features of different frequency bands, combine with local variance algorithm to enhance image details, and then use principal component analysis to improve spectral fidelity. Finally, a fused image with both high spatial resolution and high spectral characteristics is generated, effectively improving the recognition of ground features.

[0090] Based on historical images and field survey data, professionals manually annotated areas of typical land feature changes and established a sample database containing five types of land features: buildings, earth piles, water bodies, roads, and woodlands. The samples were stored in the form of 512×512 pixel slices and divided into training set, validation set, and test set in a ratio of 8:1:1, providing sufficient training data for the deep learning model.

[0091] The U-Net network architecture is adopted with ResNet18 as the backbone network. Supervised learning is performed on the sample database. Feature extraction and reconstruction are achieved through the encoder-decoder structure and skip connections. The cross entropy loss function and Adam optimizer are used for parameter optimization. After multiple iterative training, a change detection model with stable performance is obtained.

[0092] The trained model is applied to the newly acquired fused image to automatically identify and classify ground feature change patches, and output detection results containing attributes such as change type, location coordinates, and area size. At the same time, the detection results are post-processed to eliminate false detections caused by factors such as cloud occlusion and shadows, ensuring the reliability of the detection results.

[0093] Based on the type and severity of changes in the detection results, the system automatically generates graded warning information. Major high-risk changes (such as large-scale illegal construction) immediately trigger a red warning, while general changes generate a yellow warning. The warning location is marked on the map through a visual interface, and the verification task is pushed to the mobile terminals of relevant personnel.

[0094] Inspection personnel receive early warning information through mobile APP, navigate to the changed area for on-site verification, use the APP to take photos of the scene, record the details of the changes, and transmit the verification results back to the system in real time. For confirmed illegal changes, the system automatically generates disposal suggestions and notifies relevant departments.

[0095] The confirmed change information will be updated to the mining area feature database as the baseline data for the next monitoring period. At the same time, the full process data of this monitoring (original images, processing results, verification records, etc.) will be archived and stored to form a complete monitoring file to provide data support for long-term change analysis of the mining area.

[0096] Regularly sample and evaluate the monitoring results, analyze cases of false detection and missed detection, and continuously optimize algorithm performance by adding new samples and adjusting model parameters. At the same time, improve the system interface and function design based on user feedback to enhance the practicality and usability of the system.

Claims

1. The multi-dimensional monitoring and early warning device for mining areas is characterized by: Includes: Space-based monitoring module, used to obtain high-resolution optical image data of the entire mining area or well field through satellite remote sensing; Air-based monitoring module, used to obtain centimeter-level optical image data of key areas within the mining area through drone aerial photography; The ground-based monitoring module is used to conduct on-site verification and precise mapping of changing ground features identified by satellite remote sensing or drone aerial photography using ground-based measurement equipment; The data processing module is used to screen, match and fuse the multi-source data obtained by the space-based, air-based and ground-based monitoring modules; The change detection module is used to intelligently identify ground feature information and detect changes in the fused image data based on deep learning algorithms; The early warning module is used to generate early warning information based on the change detection results and push it to users through a visual interface or mobile terminal; Big data management module, used to store and manage multi-source monitoring data, change detection results and warning information, and provide data query and analysis functions; User interaction module, used to provide a graphical operation interface to support users to view and operate monitoring data, test results and warning information; Communication module, used to realize data transmission between modules inside the device and with external devices; The power module is used to provide power support for each module of the device.

2. The multi-element monitoring and early warning device for mining areas according to claim 1, characterized in that: The space-based monitoring module supports the acquisition of satellite image data with a spatial resolution of 0.5 meters to 15 meters, with a revisit period of 10 days to 1 quarter. It can flexibly configure data acquisition plans according to the monitoring needs of different areas of the mining area, give priority to the use of domestic commercial satellite constellation data such as Gaojing, Beijing-3, and Jilin-1, and increase data redundancy through a multi-satellite source combination plan.

3. The multi-element monitoring and early warning device for mining areas according to claim 1, characterized in that: The air-based monitoring module is equipped with a surveying optical camera or a lidar, which can take aerial photos of key change areas identified by satellite remote sensing with centimeter-level accuracy to obtain the location, area, geometry, height, volume and construction progress of the ground objects.

4. The multi-element monitoring and early warning device for mining areas according to claim 1, characterized in that: The data processing module adopts a method that combines wavelet transform, local algorithm and principal component analysis. First, wavelet transform is used to perform multi-scale frequency division processing on multi-source images, then the detail information is enhanced through the local variance algorithm, and the spectral distortion phenomenon is improved by combining principal component analysis, and finally a high-resolution multispectral fusion image is generated.

5. The multi-element monitoring and early warning device for mining areas according to claim 1, characterized in that: The change detection module is based on the U-Net deep learning model and supports pixel-level classification and change detection of five types of land features: buildings, earth piles, water bodies, roads, and woodlands. The model encoder uses convolution and pooling layers to extract multi-level features, and the decoder restores spatial details through deconvolution and skip connections, ultimately outputting the category and boundary information of the land feature change map.

6. The multi-element monitoring and early warning device for mining areas according to claim 1, characterized in that: The backbone model of the change detection module preferably uses ResNet 18, which is superior to ResNet 34 in terms of average Kappa, average pixel accuracy, and average frequency-weighted intersection-over-union (IoU). It can ensure an accuracy rate of no less than 0.7 and a recall rate of no less than 0.9 when the area of the change patch is greater than 200 pixels.

7. The multi-element monitoring and early warning device for mining areas according to claim 1, characterized in that: The early warning module supports classification by land feature change type and warning level, and intuitively displays the spatial distribution of change spots through map positioning. Users can view the image comparison, attribute information and verification status of the spots. The system automatically pushes the spots that need on-site verification to the mobile inspection APP, realizing the rapid discovery, precise positioning and closed-loop management of change information. The warning levels are divided into high, medium and low.

8. The multi-element monitoring and early warning device for mining areas according to claim 1, characterized in that: The big data management module uses the PostgreSQL database and PostGIS spatial extension module to support distributed storage and efficient retrieval of massive multi-source monitoring data, realizes unified data management through multi-level indexing and standardized interfaces, and provides data governance, intelligent analysis and visualization display functions.

9. The multi-element monitoring and early warning device for mining areas according to claim 1, characterized in that: The user interaction module includes a PC platform based on WebGIS and a mobile inspection APP. The PC platform provides three functional modules: remote sensing land feature monitoring, ecological environment monitoring and smart inspection. It supports image display, change detection, statistical analysis and task management. The mobile APP supports on-site map verification, attribute entry, photo upload and navigation positioning, realizing intelligent inspection operations that collaborate with internal and external industries.

10. The multi-element monitoring and early warning device for mining areas according to claim 1, characterized in that: The device also includes a technical training module, which provides special training for mining staff on system operation, data analysis and on-site review. The training content includes the functional use of the big data butler system, the principles of change detection algorithms, the operation of the mobile inspection app and the specifications of ground feature verification, to ensure that users can use the system proficiently.

Images