A mine supervision method, device and readable storage medium

Abstract

This invention provides a mine monitoring method, apparatus, and readable storage medium. The method includes: acquiring video data and ground acoustic data of the surrounding environment of the mine through monitoring cameras and ground acoustic monitoring sensors deployed in the mine's surrounding environment; fusing the video data and ground acoustic data to form a multimodal feature vector; inputting the multimodal feature vector into a trained AI analysis model, using the trained AI analysis model to perform real-time analysis of the video feature vector and ground acoustic feature vector, comprehensively judging whether geological hazards exist, and obtaining a geological hazard identification result; and based on the geological hazard identification result, performing audible and visual alarms, and / or emergency broadcasts, and / or emergency response. This method, apparatus, and readable storage medium can solve the problem that existing mine monitoring methods mainly rely on specialized monitoring personnel to monitor on-site safety production, which has the problem of being unable to detect hidden and sudden safety threats in a timely and effective manner.

Description

Technical Field

[0001] This invention relates to the field of safety supervision technology, and in particular to a mine supervision method, apparatus and readable storage medium. Background Technology

[0002] With the advancement of technology, video surveillance systems and ground acoustic monitoring systems have become indispensable parts of industrial production and public safety, especially in high-risk industrial environments such as mines and chemical plants, where their role is particularly important. Traditional video surveillance systems mainly rely on manual monitoring and post-event analysis, which suffers from problems such as response delays and high rates of false alarms and missed alarms, making it difficult to meet the needs of real-time safety supervision. Ground acoustic monitoring systems are also widely used in open-pit mines, but their utilization rate and effectiveness need to be improved. Especially in emergency situations, untimely or inaccurate information transmission often leads to delayed rescue responses, increasing the severity of accidents and casualties. Mining enterprises face numerous production safety hazards, and the safety of workers and various vehicles and equipment is difficult to guarantee. Mining enterprises need to implement monitoring and broadcasting of the mining area to ensure the safety of workers.

[0003] In the existing technology, the main supervision methods of enterprises such as open-pit mines are relatively simple, which consist of video surveillance systems, large display screens, and monitoring software clients. Dedicated monitoring personnel are responsible for monitoring the on-site safety production situation. However, this method has the problem of not being able to detect hidden and sudden safety threats in a timely and effective manner. Even if abnormal situations occur, they are often not detected by monitoring personnel in the first place, resulting in missing the best intervention opportunity and increasing the probability of safety accidents and the severity of their consequences. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the above-mentioned shortcomings of the prior art by providing a mine supervision method, device and readable storage medium, so as to solve the problem that the existing mine supervision methods are mainly responsible for monitoring the on-site safety production situation by specialized monitoring personnel, which has the problem that they cannot detect hidden and sudden safety threats in a timely and effective manner.

[0005] In a first aspect, the present invention provides a mine supervision method, the method comprising:

[0006] Video and acoustic data of the environment surrounding the mine are acquired by deploying surveillance cameras and ground acoustic sensors in the surrounding environment.

[0007] The video data and ground acoustic data are fused to form a multimodal feature vector, which includes a video feature vector and a ground acoustic feature vector.

[0008] The multimodal feature vectors are input into a trained artificial intelligence (AI) analysis model. The trained AI analysis model is used to perform real-time analysis on the video feature vectors and ground sound feature vectors to comprehensively determine whether there is a geological disaster and obtain the geological disaster identification result.

[0009] Based on the identification results of the geological hazards, sound and light alarms, and / or emergency broadcasts, and / or emergency response measures will be implemented.

[0010] Furthermore, the surveillance cameras are deployed at multiple video monitoring points in the environment surrounding the mine to monitor and collect video data in the environment surrounding the mine.

[0011] The ground acoustic monitoring sensors are deployed at multiple ground acoustic monitoring points in the environment surrounding the mine to monitor and collect ground acoustic data in the environment surrounding the mine.

[0012] Furthermore, the process of fusing the video data and ground acoustic data to form a multimodal feature vector specifically includes:

[0013] The video data and ground sound data are time-aligned to ensure that the timestamps of the video frames and the ground sound data are synchronized.

[0014] Key features are extracted from the time-aligned video data and ground acoustic data respectively, and the extracted key features are mapped to the same feature space and then concatenated to form the multimodal feature vector.

[0015] Furthermore, the geological hazards include landslides, collapses, abnormal rock strata activity, and earthquakes. The trained AI analysis model is used to perform real-time analysis of the video feature vectors and ground acoustic feature vectors to comprehensively determine whether a geological hazard exists, thus obtaining the geological hazard identification result. Specifically, this includes:

[0016] The trained AI analysis model is used to analyze the video feature vector in real time to obtain the first detection result of whether there are signs of geological disasters. The signs of geological disasters include signs of landslides, signs of collapse, signs of abnormal activity of rock strata, and signs of earthquakes.

[0017] The trained AI analysis model is used to analyze the signal features in the ground acoustic feature vector in real time to obtain a second detection result of whether there is an acoustic signal generated by geological disaster.

[0018] Based on the combined results of the first and second detections, a geological hazard is identified, and the geological hazard identification result is obtained.

[0019] Furthermore, the video feature vector includes occlusion regions, and before fusing the video data and ground acoustic data to form a multimodal feature vector, the method further includes:

[0020] Obtain the continuously recorded videos required for landslide monitoring from the video data;

[0021] From the continuously recorded video, an image is extracted at regular intervals as a reference frame, and the next image is used as the target frame to form an image pair;

[0022] The reference frame and target frame in each image pair are respectively input into the trained occlusion detection model to predict the occlusion region.

[0023] Furthermore, the trained AI analysis model is used to perform real-time analysis on the video feature vectors to obtain a first detection result indicating whether there are signs of landslides, specifically including:

[0024] The optical flow algorithm in the trained AI analysis model is used to estimate the optical flow field between the two images in each image pair that needs to be monitored for landslides in the video feature vector, and discontinuous optical flow vectors are identified; wherein, in the process of estimating the optical flow field, the influence of the occlusion area on the optical flow estimation is considered, and the contribution of the occlusion area is ignored when constructing the cost volume;

[0025] The region corresponding to the identified discontinuous optical flow vector is compared with the predicted occlusion region in the corresponding image. If the optical flow discontinuity of a certain region matches the occlusion region in the corresponding image, then the certain region is determined to be an occlusion region; otherwise, the certain region is determined to be an unocclusion region.

[0026] For image pairs with unoccluded areas, the optical flow field estimation is gradually refined through convolutional and deconvolutional layers in a deep neural network, and motion segmentation techniques are used to separate different motion regions to distinguish between landslide areas and moving targets.

[0027] The optical flow field velocity and direction in the landslide area within the target time period are calculated and compared with the statistical characteristics of the optical flow field in the same time period in historical data. If both the optical flow field velocity and direction exceed the preset threshold, it is determined that there are signs of a landslide.

[0028] Furthermore, the step of issuing audible and visual alarms, and / or emergency broadcasts, and / or taking emergency measures based on the identification results of the geological disaster specifically includes:

[0029] If the geological hazard identification result indicates the existence of a certain type of geological hazard, the mine risk level corresponding to that type of geological hazard is obtained;

[0030] Based on the mine risk level, sound and light alarms, and / or emergency broadcasts, and / or emergency responses are implemented, wherein the emergency responses include at least one of the following: sending evacuation instructions, remotely controlling machinery to stop operation, cutting off power, and activating emergency rescue equipment.

[0031] Secondly, the present invention provides a mine monitoring device, the device comprising:

[0032] The data acquisition module is used to acquire video data and ground sound data of the environment surrounding the mine through surveillance cameras and ground sound monitoring sensors deployed in the environment surrounding the mine.

[0033] A data fusion module, connected to the data acquisition module, is used to fuse the video data and ground acoustic data to form a multimodal feature vector, wherein the multimodal feature vector includes a video feature vector and a ground acoustic feature vector;

[0034] A geological disaster identification module, connected to the data fusion module, is used to input the multimodal feature vector into a trained artificial intelligence (AI) analysis model, and use the trained AI analysis model to perform real-time analysis on the video feature vector and ground sound feature vector to comprehensively determine whether a geological disaster exists, thereby obtaining the geological disaster identification result.

[0035] An emergency management module, connected to the geological disaster identification module, is used to perform audible and visual alarms, and / or emergency broadcasts, and / or emergency response based on the identification results of the geological disasters.

[0036] Thirdly, the present invention provides a mine monitoring device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to implement the mine monitoring method described in the first aspect above.

[0037] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the mine supervision method described in the first aspect.

[0038] The data acquisition module is used to acquire video data and ground sound data of the environment surrounding the mine through surveillance cameras and ground sound monitoring sensors deployed in the environment surrounding the mine.

[0039] A data fusion module, connected to the data acquisition module, is used to fuse the video data and ground acoustic data to form a multimodal feature vector, wherein the multimodal feature vector includes a video feature vector and a ground acoustic feature vector;

[0040] A geological disaster identification module, connected to the data fusion module, is used to input the multimodal feature vector into a trained artificial intelligence (AI) analysis model, and use the trained AI analysis model to perform real-time analysis on the video feature vector and ground sound feature vector to comprehensively determine whether a geological disaster exists, thereby obtaining the geological disaster identification result.

[0041] An emergency management module, connected to the geological disaster identification module, is used to perform audible and visual alarms, and / or emergency broadcasts, and / or emergency response based on the identification results of the geological disasters.

[0042] This invention provides a mine monitoring method, apparatus, and readable storage medium. First, video and ground acoustic data of the surrounding environment are acquired using surveillance cameras and ground acoustic sensors deployed around the mine. Then, the video and ground acoustic data are fused to form a multimodal feature vector, which includes video and ground acoustic feature vectors. Next, the multimodal feature vector is input into a trained AI analysis model, which performs real-time analysis on the video and ground acoustic feature vectors to comprehensively determine the presence of geological hazards, thus obtaining a geological hazard identification result. Finally, based on the geological hazard identification result, audible and visual alarms, and / or emergency broadcasts, and / or emergency response are implemented. This invention revolutionizes mine monitoring technology by combining AI video analysis and AI ground acoustic monitoring to achieve comprehensive intelligent monitoring of the mine's operating environment. Specifically, this invention uses deployed surveillance cameras and ground acoustic sensors to collect and fuse video and ground acoustic data in real time to form a multimodal feature vector. Subsequently, a trained AI analysis model is used for precise real-time analysis, significantly improving the accuracy and timeliness of geological hazard identification. Once any anomaly is detected, the system will immediately trigger audible and visual alarms, emergency broadcasts, and emergency response measures, buying valuable time for staff to react. This enables intelligent early warning and rapid response to geological disasters, reducing casualties and significantly improving the safety of mining operations. It also addresses the problem that existing mine supervision methods, which rely primarily on dedicated monitoring personnel to oversee on-site safety, often fail to detect hidden and sudden safety threats in a timely and effective manner. Attached Figure Description

[0043] Figure 1 This is a flowchart of a mine supervision method according to Embodiment 1 of the present invention;

[0044] Figure 2 This is a schematic diagram of the structure of a mine monitoring system according to an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram of the structure of a mine monitoring device according to Embodiment 2 of the present invention;

[0046] Figure 4 This is a schematic diagram of a mine monitoring device according to Embodiment 3 of the present invention. Detailed Implementation

[0047] To enable those skilled in the art to better understand the technical solution of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0048] It is understood that the specific embodiments and accompanying drawings described herein are merely for explaining the invention and are not intended to limit the invention.

[0049] It is understood that, without conflict, the various embodiments and features in the embodiments of the present invention can be combined with each other.

[0050] It is understood that, for ease of description, only the parts related to the present invention are shown in the accompanying drawings, while the parts unrelated to the present invention are not shown in the drawings.

[0051] It is understood that each unit or module involved in the embodiments of the present invention may correspond to only one entity structure, or may be composed of multiple entity structures, or multiple units or modules may be integrated into one entity structure.

[0052] It is understood that the terms "first," "second," etc., in the embodiments of the present invention are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0053] It is understood that, without conflict, the functions and steps marked in the flowcharts and block diagrams of this invention may occur in a different order than that marked in the accompanying drawings.

[0054] It is understood that the flowcharts and block diagrams of this invention illustrate the possible architecture, functions, and operations of systems, apparatuses, devices, and methods according to various embodiments of this invention. Each block in the flowchart or block diagram may represent a unit, module, program segment, or code, containing executable instructions for implementing the specified function. Furthermore, each block or combination of blocks in the block diagram and flowchart can be implemented using a hardware-based system to achieve the specified function, or using a combination of hardware and computer instructions.

[0055] It is understood that the units and modules involved in the embodiments of the present invention can be implemented by software or by hardware. For example, the units and modules can be located in a processor.

[0056] Example 1:

[0057] This embodiment provides a method for mine supervision, such as Figure 1As shown, the method includes:

[0058] Step S101: Acquire video data and ground sound data of the environment surrounding the mine through surveillance cameras and ground sound monitoring sensors deployed in the environment surrounding the mine.

[0059] In this embodiment, the surveillance cameras are deployed at multiple video monitoring points in the surrounding environment of the mine to monitor and collect video data of the surrounding environment. These monitoring points include mine slopes, open-pit environments, and mine tunnels, enabling comprehensive monitoring of the surrounding environment. The ground acoustic monitoring sensors are deployed at multiple ground acoustic monitoring points in the surrounding environment to monitor and collect ground acoustic data. These ground acoustic monitoring points can be locations within the mine prone to geological disasters, allowing for the immediate detection of emitted sound waves in the event of collapses, landslides, or other geological hazards.

[0060] It should be noted that the environment surrounding a mine includes the mine itself as well as the surrounding areas outside the mine (such as surface industrial plants).

[0061] Step S102: The video data and ground acoustic data are fused to form a multimodal feature vector, which includes a video feature vector and a ground acoustic feature vector.

[0062] In this embodiment, considering that relying solely on video or ground acoustic data for prediction in geological disaster identification scenarios often has limitations and is prone to misjudgment or omission, a more comprehensive and multi-dimensional feature vector is formed by fusing these two types of data to achieve complementary and coordinated analysis of video and ground acoustic data. Since the multi-modal feature vector combines the advantages of both video and ground acoustic data, it can identify geological disasters more quickly and accurately. For example, when signs of landslides or collapses are captured in video footage, analyzing the frequency and intensity of crustal vibrations in the ground acoustic data can quickly determine the severity and development trend of the disaster.

[0063] Optionally, fusing the video data and ground acoustic data to form a multimodal feature vector specifically includes:

[0064] The video data and ground sound data are time-aligned to ensure that the timestamps of the video frames and the ground sound data are synchronized.

[0065] Key features are extracted from the time-aligned video data and ground acoustic data respectively, and the extracted key features are mapped to the same feature space and then concatenated to form the multimodal feature vector.

[0066] In this embodiment, the timestamps of video data and ground acoustic data are synchronized to associate data at the same point in time. By recording the precise timestamps of each segment of video data and ground acoustic data, the correspondence between the two is ensured. Then, key features of each are extracted (such as object categories and location information in the video, and frequency distribution and energy spectrum in the ground acoustic data). After feature extraction, these features are mapped to the same feature space and fused to form a multimodal feature vector.

[0067] Step S103: Input the multimodal feature vector into the trained AI (Artificial Intelligence) analysis model, and use the trained AI analysis model to perform real-time analysis on the video feature vector and ground sound feature vector to comprehensively determine whether there is a geological disaster and obtain the geological disaster identification result.

[0068] In this embodiment, in addition to identifying geological disasters, the AI ​​analysis model can also automatically identify and mark violations such as people not wearing safety helmets and vehicles entering illegally.

[0069] Optionally, the geological hazards include landslides, collapses, abnormal rock strata activity, and earthquakes. The step of using the trained AI analysis model to perform real-time analysis of the video feature vectors and ground sound feature vectors to comprehensively determine whether a geological hazard exists, and obtaining the geological hazard identification result, specifically includes:

[0070] The trained AI analysis model is used to analyze the video feature vector in real time to obtain the first detection result of whether there are signs of geological disasters. The signs of geological disasters include signs of landslides, signs of collapses, signs of abnormal activity of rock strata, and signs of earthquakes.

[0071] The trained AI analysis model is used to analyze the signal features in the ground acoustic feature vector in real time to obtain a second detection result of whether there is an acoustic signal generated by geological disaster.

[0072] Based on the combined results of the first and second detections, a geological hazard is identified, and the geological hazard identification result is obtained.

[0073] In this embodiment, the first detection result includes the presence or absence of signs of a certain type of geological hazard, or it may also include signs of illegal activities. When the first detection result indicates the presence of signs of a certain type of geological hazard, and the second detection result indicates the presence of acoustic signals generated by that type of geological hazard, it can be determined that that type of geological hazard exists. For example, when the first detection result indicates the presence of signs of an earthquake, and the second detection result indicates the presence of acoustic signals generated by an earthquake, it can be determined that an earthquake exists.

[0074] Optionally, the video feature vector includes occlusion regions, and before fusing the video data and ground acoustic data to form a multimodal feature vector, the method further includes:

[0075] Obtain the continuously recorded videos required for landslide monitoring from the video data;

[0076] From the continuously recorded video, an image is extracted at regular intervals as a reference frame, and the next image is used as the target frame to form an image pair.

[0077] The reference frame and target frame in each image pair are respectively input into the trained occlusion detection model to predict the occlusion region.

[0078] In this embodiment, an image is extracted every certain number of frames from the continuously recorded video that needs to be monitored for landslides, and the next image is taken as the target frame. The image pairs are formed and saved separately. For example, the reference frame is saved as frame_t.png and the target frame is saved as frame_t+1.png. The images are converted to a uniform RGB format and resized (e.g., 256x256 pixels). Then, the trained occlusion detection model is used to predict the occlusion area.

[0079] Optionally, the trained AI analysis model is used to perform real-time analysis on the video feature vector to obtain a first detection result indicating whether there are signs of landslide, specifically including:

[0080] The optical flow algorithm in the trained AI analysis model is used to estimate the optical flow field between the two images in each image pair that needs to be monitored for landslides in the video feature vector, and discontinuous optical flow vectors are identified; wherein, in the process of estimating the optical flow field, the influence of the occlusion area on the optical flow estimation is considered, and the contribution of the occlusion area is ignored when constructing the cost volume;

[0081] The region corresponding to the identified discontinuous optical flow vector is compared with the predicted occlusion region in the corresponding image. If the optical flow discontinuity of a certain region matches the occlusion region in the corresponding image, then the certain region is determined to be an occlusion region; otherwise, the certain region is determined to be an unocclusion region.

[0082] For image pairs with unoccluded areas, the optical flow field estimation is gradually refined through convolutional and deconvolutional layers in a deep neural network, and motion segmentation techniques are used to separate different motion regions to distinguish between landslide areas and moving targets.

[0083] The optical flow field velocity and direction in the landslide area within the target time period are calculated and compared with the statistical characteristics of the optical flow field in the same time period in historical data. If both the optical flow field velocity and direction exceed the preset threshold, it is determined that there are signs of a landslide.

[0084] In this embodiment, considering that occlusion regions can lead to inaccurate optical flow estimation, when utilizing the optical flow algorithm in the trained AI analysis model, we need to estimate the optical flow field between landslide monitoring image pairs in the video feature vector and consider the impact of occlusion regions on optical flow estimation. When constructing the cost volume (a data structure used to measure the matching degree of candidate optical flow vectors), if pixels from occluded regions are directly included, the matching cost of these regions may be misleading because pixels from occluded regions may not have clear corresponding points in subsequent frames. Therefore, when constructing the cost volume, we ignore the contribution of occluded regions to eliminate their influence. The optical flow algorithm is based on a deep neural network, and the moving target is a movable target other than the landslide area, such as large machinery or other movable objects. The preset threshold can be set based on historical data, the characteristics of the landslide area, etc.

[0085] Step S104: Based on the identification results of the geological disaster, issue audible and visual alarms, and / or issue emergency broadcasts, and / or issue emergency responses.

[0086] Specifically, if the geological hazard identification result indicates the existence of a certain type of geological hazard, the mine risk level corresponding to the certain type of geological hazard is obtained; based on the mine risk level, audible and visual alarms, and / or emergency broadcasts, and / or emergency response are carried out, wherein the emergency response includes at least one of the following: sending evacuation instructions, remotely controlling machinery to stop operation, cutting off power, and activating emergency rescue equipment.

[0087] In this embodiment, different types of geological disasters have corresponding mine risk levels. For example, landslides may be classified as general risk levels, while earthquakes may be classified as major risk levels. Different mine risk levels can correspond to different emergency measures, which may be one or more of the following: audible and visual alarms, emergency broadcasts, and emergency response.

[0088] It should be noted that existing video analytics technologies only support the analysis of violations by personnel and vehicles, lacking dynamic monitoring and analysis capabilities for disaster scenarios such as landslides and collapses in open-pit mines, and thus failing to detect safety threats in a timely and effective manner. This invention provides a mine supervision method that, with simple operation, can effectively monitor the surrounding environment of the mining area. Furthermore, it can trigger alarms in dangerous situations, alerting workers and increasing their response time, thereby reducing casualties and improving the safety factor of the mining area.

[0089] In one specific embodiment, a mine supervision method and system are provided. This system acquires on-site environmental conditions in open-pit mines and surface industrial plants by deploying video surveillance and ground acoustic monitoring sensors. The received information is comprehensively processed, and AI analysis algorithms are used to determine whether any abnormalities have occurred. When the video analysis module detects geological hazards such as landslides, or the ground acoustic monitoring module captures abnormal activity in nearby rock strata, the system comprehensively analyzes and automatically triggers an early warning. Based on the results of the AI ​​comprehensive analysis, corresponding measures are taken, such as alarm notifications, evacuation instructions, and remote control of machinery shutdown. Specifically, the AI ​​analysis algorithm can perform real-time analysis of the video stream, automatically identifying and marking abnormal behaviors and phenomena such as personnel not wearing safety helmets, vehicles entering illegally, and landslide precursors. Simultaneously, through the deployed ground acoustic sensor array, it captures sound wave signals generated by seismic waves or other geological activities, enabling early warning of unstable states in the mine and phenomena such as rock strata movement. The system performs data fusion analysis on the received information, using AI analysis algorithms to determine whether any abnormalities have occurred. Once a potential risk is detected, the system automatically triggers an early warning, ensuring that mine management personnel can take timely preventative measures to avoid accidents.

[0090] like Figure 2 As shown, the mine monitoring system includes a video surveillance module, a ground sound monitoring module, an AI analysis module, and an emergency response module. The video surveillance module monitors the environment near the mine area; the ground sound monitoring module monitors ground sounds within the mine area; the AI ​​analysis module uses AI algorithms to identify video and ground sound data, thereby making judgments on safety warnings; and the emergency response module, based on the identification results, manages and handles situations through audible and visual alarms and emergency broadcasts.

[0091] This system integrates AI video analytics and ground acoustic monitoring to achieve comprehensive monitoring of the mining environment. It improves the video data processing and analysis, ground acoustic monitoring signal source processing, and integrated processing processes in open-pit mine disaster analysis and early warning technology. Furthermore, it proposes a closed-loop response method from monitoring and early warning to emergency response. Through improvements to the entire process from analysis to early warning to response, the system enhances the effectiveness and comprehensiveness of open-pit mine disaster analysis and early warning. Specific improvements include:

[0092] 1. Improve disaster analysis and early warning technologies for open-pit mine scenarios.

[0093] 1) Regarding improvements in video data processing and analysis, current video analysis technologies only support the analysis of violations by personnel, vehicles, etc. They lack dynamic monitoring and analysis capabilities for disaster scenarios such as landslides and collapses in open-pit mines. This system, targeting open-pit mine scenarios, achieves dynamic environmental understanding through video environment change detection technology, thereby improving disaster analysis and early warning technologies.

[0094] In open-pit mine environments, the presence of large machinery can obscure landslide areas, preventing optical flow algorithms from accurately estimating the motion of these areas. Conventional optical flow estimation typically estimates the optical flow field between two frames without considering the impact of occlusion, potentially leading to incorrect estimations and weakened dynamic environment understanding. This invention addresses open-pit mine scenarios by improving conventional video data processing, enhancing the ability to handle complex backgrounds and non-rigid motion, identifying occluded areas, and eliminating their influence during optical flow estimation, thereby improving dynamic environment understanding.

[0095] In practical applications like open-pit mine environments, images are extracted from continuously recorded video at regular intervals as reference frames and the next image as the target frame, forming image pairs that are saved separately. For example, the reference frame is saved as frame_t.png and the target frame as frame_t+1.png. These are converted to a uniform RGB format and resized (e.g., 256x256 pixels). A fixed-size region is randomly cropped from the original images (reference and target frames), and the images are randomly rotated and translated to simulate the observation effects from different perspectives (different perspectives help to more accurately identify discontinuities). Open-source optical flow calculation libraries (such as the Farneback method in OpenCV or DeepFlow) are used to calculate the optical flow field between the two images. The optical flow vectors are analyzed to identify discontinuous optical flow vectors, which are then compared with actual image occlusion conditions to verify occlusion and non-occlusion.

[0096] Each pixel is classified as "occluded" or "unoccluded" (using a cross-entropy loss function, etc.). The difference between the predicted occluded region and the actual occluded region is measured by analyzing the optical flow vector. Backpropagation is then performed to update the network weights to minimize the loss function. This process is repeated for cyclic training (e.g., 50 times), with each cyclic training using all data in the training set until the network converges or reaches the predetermined number of training epochs. The trained occlusion detection model is then used to predict occlusion regions in the input image. A deep learning-based optical flow algorithm is used to estimate the optical flow field between each pair of adjacent frames. During optical flow estimation, the contribution of occluded regions is ignored when constructing the cost volume. The cost values ​​in the cost volume are aggregated to obtain the optimal optical flow vector for each pixel. Subsequently, the gradient of the optical flow vector is calculated. Regions with large gradient magnitudes are likely occlusion boundaries. An occlusion mask is generated to identify the occluded regions, thus eliminating their influence.

[0097] The optical flow field estimation is progressively refined using convolutional and deconvolutional layers in the network. Subsequently, the estimated optical flow field is smoothed and filtered to reduce noise. Motion segmentation techniques are used to separate different motion regions to distinguish between landslide areas and moving targets. Abnormal movement in landslide areas is detected based on changes in the optical flow field. This can be achieved by comparing the velocity and direction of the optical flow field with the statistical characteristics of historical data. If a significant increase in the velocity of the landslide area is detected, an anomaly is considered to have occurred.

[0098] By incorporating an improved occlusion detection mechanism, complex backgrounds and non-rigid motion can be handled more effectively, improving the accuracy of optical flow estimation. In practical applications, appropriate adjustments and optimizations can be made based on the specific characteristics of the mining environment.

[0099] 2) Improve the comprehensive analysis process by synchronizing video data and ground acoustic data in time, extracting and combining their respective key features (such as object categories and location information in the video and frequency distribution and energy spectrum in the ground acoustic data), and forming a multimodal feature vector by fusion. Use gradient boosting trees to fuse these features, and combine anomaly detection and risk assessment technologies to achieve intelligent early warning of potential safety risks.

[0100] First, ensure the synchronization of the timestamps of the video frames and the ground acoustic data. Through feature extraction and mapping, map the two types of features into the same feature space. By splicing, a new multimodal feature vector is formed. By comprehensively analyzing the video analysis results and the ground acoustic monitoring data, the mine risk level in the open-pit mine is assessed, and an early warning mechanism is triggered when a safety hazard is detected. The mine risk level can be customized according to the actual production needs of the enterprise, such as general risk level, major risk level, and serious risk level.

[0101] Feature fusion process: After the data is collected and transmitted to the transmission center, the timestamps of the video data and the ground acoustic data are synchronized, and the data at the same time point are associated. By recording the precise timestamps of each segment of video data and ground acoustic data, the correspondence between the two is ensured. Then, the location information of disasters (such as landslide signs) is extracted through image segmentation and target detection. The detected landslide signs (i.e., landslide signs detected by optical flow estimation) and location information are converted into feature vectors. The frequency distribution and energy spectrum of the ground acoustic signal are calculated by Fourier transform and converted into feature vectors. The video features and ground acoustic features are mapped to the same feature space. The feature vectors of different types are vertically concatenated together in sequence to form a multimodal feature vector (modal feature vector = [video feature vector; ground acoustic feature vector]) and the fusion and analysis are completed.

[0102] The fused multimodal feature vectors are used as input features, and anomalies marked in historical data are used as labels to form a training dataset. Multiple weak classifiers are gradually built through multiple iterations, and these weak classifiers are weighted and combined to form a strong classifier. Parameters such as the model learning rate and maximum tree depth are adjusted using methods such as cross-validation. During model evaluation, independent test datasets are used to assess the model's performance, ensuring good generalization ability. Thresholds are set for video analysis results and ground acoustic monitoring data; when a feature exceeds the threshold, an early warning is triggered. For example, when video analysis detects signs of landslides on a mine slope, ground acoustic monitoring data is combined to determine if there are signs of geological activity. Once analysis confirms a landslide, earthquake, or other disaster, an early warning is immediately generated, and the results are fed back to the emergency response system.

[0103] It should be noted that, taking into account both video analysis results and ground acoustic monitoring results, for example, if a landslide is detected but is extremely minor and does not pose a risk level requiring the evacuation of personnel from the site, the ground acoustic monitoring results can be used to help determine the severity of the landslide. If the ground acoustic monitoring results indicate a higher risk, then an early warning will be issued.

[0104] Through these improvements, the monitoring system integrating AI video analytics and ground acoustic monitoring can more efficiently monitor the safety status of open-pit mines and take timely action when potential risks arise, minimizing the occurrence of accidents. Compared with traditional technologies, these improvements enhance the comprehensiveness, accuracy, and real-time nature of detection, providing intelligent decision support.

[0105] 2. Innovate a closed-loop response system from monitoring and early warning to emergency response.

[0106] When the intelligent early warning system triggers an alert, the emergency response module takes corresponding measures based on the fused data analysis results. For example, by analyzing historical data of ground acoustic signals under normal conditions, a baseline model of the normal state is established. The real-time monitored ground acoustic signals are compared with the baseline model. If the signals deviate significantly from the normal state in terms of frequency distribution, energy distribution, or statistical characteristics, an anomaly is considered to exist. When the video analysis module detects a gathering of people in a certain area, and the ground acoustic monitoring module simultaneously captures abnormal activity in the nearby rock strata, the system automatically activates the emergency plan, guiding on-site personnel to quickly evacuate to a safe area through alarm notifications, evacuation instructions, and other means. In addition, the emergency response can also link with the remote control system of equipment, immediately stopping the operation of large machinery after a disaster is detected, reducing losses to machinery, equipment, and personnel, and conducting on-site safety assessments and follow-up handling through the emergency response team, thereby minimizing the losses caused by the accident. Through a closed-loop integrated monitoring and early warning system to emergency response, comprehensive safety assurance can be provided for open-pit mines, helping to improve the overall safety management level of the mine.

[0107] Based on this system, the corresponding mine supervision method includes the following steps:

[0108] S1: By installing a group of surveillance cameras at multiple monitoring points in the environment surrounding the mine, the images of the environment surrounding the mine are monitored and collected. The surveillance camera group is distributed and fixed at multiple monitoring points in the mine, including mine slopes, open-pit environments, mine tunnels, etc., thereby achieving comprehensive monitoring of the environment surrounding the mine.

[0109] S2: By installing ground sound monitoring sensor groups at multiple monitoring points in the environment surrounding the mine, ground sound data in the environment surrounding the mine is monitored and collected. Among them, the ground sound monitoring sensor group is used to monitor sound waves in the mine ground. In the event of a collapse or landslide in the mine, the emitted sound wave data can be sensed immediately.

[0110] S3: The AI ​​analysis module performs AI analysis and comprehensive judgment on image and ground sound data. It uses algorithms to analyze video streams in real time, automatically identify and mark abnormal behaviors and phenomena such as personnel not wearing safety helmets, vehicles entering illegally, and landslide precursors. At the same time, it analyzes and captures sound wave signals generated by seismic waves or other geological activities, and provides early warnings of unstable states in mines, rock strata movement, and other phenomena. The results of the comprehensive analysis are fed back to the emergency response module.

[0111] S4: If the system detects an anomaly in the comprehensive analysis of video recognition and ground sound recognition, it will activate the prompting unit to remind staff to verify. If the system determines that the current disaster risk coefficient is high after comprehensive analysis, it will activate the triggering unit to open the warning module and issue an alarm through the lights and broadcasts in the warning module.

[0112] It should be noted that the improvements of the mine supervision method provided by this invention over the prior art are mainly reflected in the following aspects:

[0113] a) This invention improves the handling of occlusion in open-pit mine video analysis to achieve dynamic environment understanding. In open-pit mine environments, the presence of large machinery may obscure landslide areas, causing optical flow algorithms to fail to accurately estimate the movement of these areas. Conventional optical flow estimation typically estimates the optical flow field between two frames without considering the impact of occlusion, potentially leading to incorrect estimations and weak dynamic environment understanding. This invention improves the conventional video data processing for open-pit mine scenarios by using a trained occlusion detection model to predict occlusion areas in the input image. A deep learning-based optical flow algorithm estimates the optical flow field between each pair of adjacent frames. During optical flow estimation, the contribution of occlusion areas is ignored when constructing the cost volume, thus eliminating the influence of occluded areas. The optical flow field estimation is progressively refined through convolutional and deconvolutional layers in the network. Subsequently, the estimated optical flow field is smoothed and filtered to reduce noise. Motion segmentation techniques are used to separate different motion regions to distinguish between landslide areas and moving targets. Detecting abnormal movement in landslide areas based on changes in the optical flow field involves comparing the velocity and direction of the optical flow field with the statistical characteristics of historical data. A significant increase in the velocity of the landslide area is considered an anomaly. Improvements by incorporating an occlusion detection mechanism can more effectively handle complex backgrounds and non-rigid motion, enhancing the accuracy of optical flow estimation. In practical applications, appropriate adjustments and optimizations can be made based on the specific characteristics of the mining environment.

[0114] (b) Improved video and ground acoustic data fusion analysis process: By integrating video surveillance data and ground acoustic monitoring data across modalities, an AI analysis model is constructed and trained on a large amount of data. AI algorithms enable complementary and linked analysis of video images and ground acoustic monitoring data, achieving the identification and prediction of violations, risks, and disasters. The system performs real-time analysis of the acquired video streams, identifying and tracking people, equipment, and abnormal behaviors within the mining area by analyzing the frequency, amplitude, and other characteristics of earthquake and landslide signs and ground acoustic signals. Simultaneously, the ground acoustic monitoring system continuously listens for underground micro-seismic activity, blasting vibrations, or other ground acoustic signals, analyzing changes in geological structures or potential hazards, achieving comprehensive and high-precision monitoring of complex environmental anomalies. Cross-modal data fusion analysis solves the limitations and poor effectiveness of traditional single monitoring methods, improving the system's perception capabilities and early warning accuracy.

[0115] c) Improvements to the automated emergency response mechanism for open-pit mines: The system continuously scans various indicators in the environment. When anomalies are detected in video and ground acoustic data through data analysis models, a trigger unit is activated to control the emergency response module. If anomalies are detected in both video and ground acoustic analysis, both audible and visual alarms and emergency response are triggered simultaneously. If only video analysis identifies violations or anomalies involving personnel or vehicles, staff are prompted to take action, allowing for confirmation and resolution. If the issue is not resolved within a specified time, the alarm module is automatically triggered, increasing its priority. When only anomalies are detected in video or ground acoustic analysis, a choice must be made between confirming resolution and triggering the alarm module, avoiding situations where there is no feedback. Automatic system triggering, rather than waiting for manual intervention, significantly enhances the accuracy of identification and processing efficiency, shortens the time from monitoring to response, strengthens the ability to respond to emergencies, solves the problems of poor monitoring effectiveness and slow response speed in traditional methods, improves risk control capabilities, and reduces enterprise losses.

[0116] It should be noted that the mine supervision method provided by this invention improves the technology for monitoring open-pit mines, achieving comprehensive intelligent supervision of the mining environment through AI video analysis and AI ground acoustic monitoring. The system can monitor and analyze video images and underground acoustic signals in real time, promptly identify potential safety hazards, and immediately issue audible and visual alarms and broadcast warnings and handling instructions through the broadcast system once an anomaly is detected. This eliminates the need for cumbersome manual operations, effectively improving the safety management level and emergency response efficiency of mine operations.

[0117] The mine supervision method provided in this invention first acquires video and ground sound data of the surrounding environment through monitoring cameras and ground sound monitoring sensors deployed in the mine's vicinity. Then, the video and ground sound data are fused to form a multimodal feature vector, which includes video and ground sound feature vectors. Next, the multimodal feature vector is input into a trained AI analysis model, which performs real-time analysis on the video and ground sound feature vectors to comprehensively determine the presence of geological hazards, thus obtaining a geological hazard identification result. Finally, based on the geological hazard identification result, audible and visual alarms, and / or emergency broadcasts, and / or emergency response are implemented. This invention revolutionizes mine supervision technology by combining AI video analysis and AI ground sound monitoring to achieve comprehensive intelligent supervision of the mine's operating environment. Specifically, this invention uses deployed monitoring cameras and ground sound sensors to collect and fuse video and ground sound data in real time to form a multimodal feature vector. Subsequently, a trained AI analysis model is used for precise real-time analysis, significantly improving the accuracy and timeliness of geological hazard identification. Once any anomaly is detected, the system will immediately trigger audible and visual alarms, emergency broadcasts, and emergency response measures, buying valuable time for staff to react. This enables intelligent early warning and rapid response to geological disasters, reducing casualties and significantly improving the safety of mining operations. It also addresses the problem that existing mine supervision methods, which rely primarily on dedicated monitoring personnel to oversee on-site safety, often fail to detect hidden and sudden safety threats in a timely and effective manner.

[0118] Example 2:

[0119] like Figure 3 As shown, this embodiment provides a mine monitoring device for executing the above-described mine monitoring method, including:

[0120] Data acquisition module 11 is used to acquire video data and ground sound data of the environment surrounding the mine through monitoring cameras and ground sound monitoring sensors deployed in the environment surrounding the mine.

[0121] The data fusion module 12, connected to the data acquisition module 11, is used to fuse the video data and ground acoustic data to form a multimodal feature vector, wherein the multimodal feature vector includes a video feature vector and a ground acoustic feature vector;

[0122] The geological disaster identification module 13 is connected to the data fusion module 12 and is used to input the multimodal feature vector into the trained artificial intelligence (AI) analysis model. The trained AI analysis model is used to perform real-time analysis on the video feature vector and the ground sound feature vector to comprehensively determine whether a geological disaster exists and obtain the geological disaster identification result.

[0123] The emergency management module 14 is connected to the geological disaster identification module 13 and is used to perform audible and visual alarms, and / or emergency broadcasts, and / or emergency response based on the identification results of the geological disasters.

[0124] Optionally, the surveillance cameras are deployed at multiple video monitoring points in the environment surrounding the mine to monitor and collect video data in the environment surrounding the mine.

[0125] The ground acoustic monitoring sensors are deployed at multiple ground acoustic monitoring points in the environment surrounding the mine to monitor and collect ground acoustic data in the environment surrounding the mine.

[0126] Optionally, the data fusion module 12 includes:

[0127] The time synchronization unit is used to time-align the video data and the ground sound data to ensure that the timestamps of the video frames and the ground sound data are synchronized.

[0128] The feature mapping unit is used to extract key features from the time-aligned video data and ground acoustic data respectively, and then concatenate the extracted key features to the same feature space to form the multimodal feature vector.

[0129] Optionally, the geological hazards include landslides, collapses, abnormal rock strata activity, and earthquakes, and the geological hazard identification module 13 includes:

[0130] The video analysis unit is used to perform real-time analysis of the video feature vector using the trained AI analysis model to obtain a first detection result of whether there are signs of geological disasters, including signs of landslides, signs of collapses, signs of abnormal activity of rock strata, and signs of earthquakes.

[0131] The ground acoustic analysis unit is used to perform real-time analysis of the signal features in the ground acoustic feature vector using the trained AI analysis model to obtain a second detection result of whether there is a sound wave signal generated by geological disasters.

[0132] The result fusion unit is used to comprehensively determine whether a geological disaster exists based on the first detection result and the second detection result, and obtain the geological disaster identification result.

[0133] Optionally, the video feature vector includes an occlusion region, and the device further includes:

[0134] The video acquisition module is used to acquire continuously recorded videos that need to be monitored for landslides from the video data;

[0135] The image pair forming module is used to extract an image as a reference frame from the continuously recorded video at regular intervals, and use the next image as the target frame to form an image pair;

[0136] The occlusion region prediction module is used to input the reference frame and target frame in each image pair into the trained occlusion detection model to predict the occlusion region.

[0137] Optionally, the video analytics unit includes:

[0138] An optical flow estimation unit is used to estimate the optical flow field between two images in each image pair that needs to be monitored for landslides in the video feature vector using the optical flow algorithm in the trained AI analysis model, and to identify discontinuous optical flow vectors; wherein, in the process of estimating the optical flow field, the influence of the occlusion region on the optical flow estimation is considered, and the contribution of the occlusion region is ignored when constructing the cost volume;

[0139] The region matching unit is used to compare the region corresponding to the identified discontinuous optical flow vector with the predicted occlusion region in the corresponding image. If the optical flow discontinuity of a certain region matches the occlusion region in the corresponding image, then the certain region is determined to be an occlusion region; otherwise, the certain region is determined to be an unocclusion region.

[0140] The region segmentation unit is used to refine the optical flow field estimation step by step through convolutional and deconvolutional layers in a deep neural network for image pairs with non-occluded areas, and uses motion segmentation technology to separate different motion regions to distinguish landslide areas from moving targets.

[0141] The landslide indication judgment unit is used to calculate the optical flow field velocity and direction of the landslide area within the target time period, and compare them with the optical flow field statistical characteristics of the same time period in historical data. If both the optical flow field velocity and direction exceed the preset threshold, it is judged that there is a landslide indication.

[0142] Optionally, the emergency management module 14 includes:

[0143] The mine risk level determination unit is used to obtain the mine risk level corresponding to the geological hazard if the identification result of the geological hazard indicates the existence of a certain type of geological hazard;

[0144] An emergency response determination unit is used to issue audible and visual alarms and / or emergency broadcasts and / or emergency responses based on the mine risk level, wherein the emergency responses include at least one of the following: sending evacuation instructions, remotely controlling machinery to stop operation, cutting off power, and activating emergency rescue equipment.

[0145] Example 3:

[0146] refer toFigure 4 This embodiment provides a mine monitoring device, including a memory 21 and a processor 22. The memory 21 stores a computer program, and the processor 22 is configured to run the computer program to execute the mine monitoring method in Embodiment 1.

[0147] The memory 21 is connected to the processor 22. The memory 21 can be a flash memory, a read-only memory or other memory, and the processor 22 can be a central processing unit or a microcontroller.

[0148] Example 4:

[0149] This embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the mine supervision method in Embodiment 1 above.

[0150] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules, or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other memory technologies, CD-ROM (Compact Disc Read-Only Memory), DVD or other optical disc storage, cartridges, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer.

[0151] In summary, the mine monitoring method, apparatus, and readable storage medium provided in this invention first acquire video and ground acoustic data of the surrounding environment of the mine through monitoring cameras and ground acoustic monitoring sensors deployed in the mine's surrounding environment; then, the video and ground acoustic data are fused to form a multimodal feature vector, which includes video feature vectors and ground acoustic feature vectors; next, the multimodal feature vector is input into a trained artificial intelligence (AI) analysis model, which uses the trained AI analysis model to perform real-time analysis of the video and ground acoustic feature vectors, comprehensively judging whether geological hazards exist, and obtaining geological hazard identification results; finally, based on the geological hazard identification results, audible and visual alarms, and / or emergency broadcasts, and / or emergency response are implemented. This invention innovates mine area monitoring technology by combining AI video analysis and AI ground acoustic monitoring to achieve comprehensive intelligent monitoring of the mine's operating environment. Specifically, this invention uses deployed monitoring cameras and ground acoustic sensors to collect and fuse video and ground acoustic data in real time to form a multimodal feature vector. Subsequently, the trained AI analysis model was used for precise real-time analysis, significantly improving the accuracy and timeliness of geological disaster identification. Once any anomaly is detected, the system immediately triggers audible and visual alarms, emergency broadcasts, and emergency response measures, buying valuable time for staff to react. This enables intelligent early warning and rapid response to geological disasters, reducing casualties and greatly improving the safety of mining operations. It also addresses the problem that existing mine supervision methods, which rely primarily on dedicated monitoring personnel to oversee on-site safety, often fail to detect hidden and sudden safety threats in a timely and effective manner.

[0152] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A method for mine supervision, characterized in that, The method includes: Video and acoustic data of the environment surrounding the mine are acquired by deploying surveillance cameras and ground acoustic sensors in the surrounding environment. The video data and ground acoustic data are fused to form a multimodal feature vector, which includes a video feature vector and a ground acoustic feature vector. The multimodal feature vectors are input into a trained artificial intelligence (AI) analysis model. The trained AI analysis model is used to perform real-time analysis on the video feature vectors and ground sound feature vectors to comprehensively determine whether there are geological hazards and obtain the geological hazard identification results. Based on the identification results of the geological hazards, sound and light alarms, and / or emergency broadcasts, and / or emergency response measures will be implemented.

2. The method according to claim 1, characterized in that, The surveillance cameras are deployed at multiple video monitoring points in the environment surrounding the mine to monitor and collect video data in the environment surrounding the mine. The ground acoustic monitoring sensors are deployed at multiple ground acoustic monitoring points in the environment surrounding the mine to monitor and collect ground acoustic data in the environment surrounding the mine.

3. The method according to claim 1, characterized in that, The process of fusing the video data and ground acoustic data to form a multimodal feature vector specifically includes: The video data and ground sound data are time-aligned to ensure that the timestamps of the video frames and the ground sound data are synchronized. Key features are extracted from the time-aligned video data and ground acoustic data respectively, and the extracted key features are mapped to the same feature space and then concatenated to form the multimodal feature vector.

4. The method according to claim 2, characterized in that, The geological hazards include landslides, collapses, abnormal rock strata activity, and earthquakes. The trained AI analysis model is used to perform real-time analysis of the video feature vectors and ground acoustic feature vectors to comprehensively determine whether a geological hazard exists, thus obtaining the geological hazard identification result, specifically including: The trained AI analysis model is used to analyze the video feature vector in real time to obtain the first detection result of whether there are signs of geological disasters. The signs of geological disasters include signs of landslides, signs of collapses, signs of abnormal activity of rock strata, and signs of earthquakes. The trained AI analysis model is used to analyze the signal features in the ground acoustic feature vector in real time to obtain a second detection result of whether there is an acoustic signal generated by geological disaster. Based on the combined results of the first and second detections, a geological hazard is identified, and the geological hazard identification result is obtained.

5. The method according to claim 4, characterized in that, The video feature vector includes occlusion regions. Before fusing the video data and ground acoustic data to form a multimodal feature vector, the method further includes: Obtain the continuously recorded videos required for landslide monitoring from the video data; From the continuously recorded video, an image is extracted at regular intervals as a reference frame, and the next image is used as the target frame to form an image pair. The reference frame and target frame in each image pair are respectively input into the trained occlusion detection model to predict the occlusion region.

6. The method according to claim 5, characterized in that, The trained AI analysis model is used to perform real-time analysis on the video feature vector to obtain a first detection result indicating whether there are signs of landslide, specifically including: The optical flow algorithm in the trained AI analysis model is used to estimate the optical flow field between the two images in each image pair that needs to be monitored for landslides in the video feature vector, and discontinuous optical flow vectors are identified; wherein, in the process of estimating the optical flow field, the influence of the occlusion area on the optical flow estimation is considered, and the contribution of the occlusion area is ignored when constructing the cost volume; The region corresponding to the identified discontinuous optical flow vector is compared with the predicted occlusion region in the corresponding image. If the optical flow discontinuity of a certain region matches the occlusion region in the corresponding image, then the certain region is determined to be an occlusion region; otherwise, the certain region is determined to be an unocclusion region. For image pairs with unoccluded areas, the optical flow field estimation is gradually refined through convolutional and deconvolutional layers in a deep neural network, and motion segmentation techniques are used to separate different motion regions to distinguish between landslide areas and moving targets. The optical flow field velocity and direction in the landslide area within the target time period are calculated and compared with the statistical characteristics of the optical flow field in the same time period in historical data. If both the optical flow field velocity and direction exceed the preset threshold, it is determined that there are signs of a landslide.

7. The method according to claim 1, characterized in that, The provision of audible and visual alarms, and / or emergency broadcasts, and / or emergency response based on the identification results of the geological disasters specifically includes: If the geological hazard identification result indicates the existence of a certain type of geological hazard, the mine risk level corresponding to that type of geological hazard is obtained; Based on the mine risk level, sound and light alarms, and / or emergency broadcasts, and / or emergency responses are implemented, wherein the emergency responses include at least one of the following: sending evacuation instructions, remotely controlling machinery to stop operation, cutting off power, and activating emergency rescue equipment.

8. A mine monitoring device, characterized in that, The device includes: The data acquisition module is used to acquire video data and ground sound data of the environment surrounding the mine through surveillance cameras and ground sound monitoring sensors deployed in the environment surrounding the mine. A data fusion module, connected to the data acquisition module, is used to fuse the video data and ground acoustic data to form a multimodal feature vector, wherein the multimodal feature vector includes a video feature vector and a ground acoustic feature vector; A geological disaster identification module, connected to the data fusion module, is used to input the multimodal feature vector into a trained artificial intelligence (AI) analysis model, and use the trained AI analysis model to perform real-time analysis on the video feature vector and ground sound feature vector to comprehensively determine whether a geological disaster exists, thereby obtaining the geological disaster identification result. An emergency management module, connected to the geological disaster identification module, is used to perform audible and visual alarms, and / or emergency broadcasts, and / or emergency response based on the identification results of the geological disasters.

9. A mine monitoring device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to implement the mine supervision method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the mine supervision method as described in any one of claims 1-7.

Images