Non-coal mine safety production intelligent video analysis and early warning system based on deep learning

Abstract

The invention belongs to the technical field of non-coal mine safety production supervision, and provides a non-coal mine safety production intelligent video analysis and early warning system based on deep learning, comprising five modules: a video stream generation module accesses an underground camera real-time video stream, and pushes the video stream to an AI intelligent analysis engine after denoising, frame synchronization and image enhancement preprocessing; the AI intelligent analysis engine processes the video frames in parallel based on a multi-task deep learning model, identifies safety production violation behaviors and outputs labels and confidence coefficients; after receiving and analyzing, the early warning management module generates a structured early warning event record; the data pushing module pushes the early warning event record to the supervision client; and the business closed-loop processing module is used for visualizing the early warning position on the GIS map, receiving and disposing the feedback state, and associatively storing the early warning record, the feedback state and the timestamp to the database. According to the invention, real-time video analysis and intelligent early warning of a non-coal mine underground complex environment can be realized through cooperation of multiple modules.

Description

Technical Field

[0001] This invention relates to the field of safety production supervision technology in non-coal mines, specifically to intelligent video analysis and early warning applications that integrate computer vision, deep learning, edge-cloud collaborative computing, geographic information systems (GIS) and network communication technologies. It is particularly suitable for complex working environments in non-coal mines with low light, high dust, and dense personnel, and can realize real-time identification, automatic early warning, location visualization, and closed-loop management of early warning-handling-feedback-storage of underground safety production violations. Background Technology

[0002] Non-coal mines (such as metal and non-metal mines) are important sources of raw materials for industrial production. However, their underground working environment is characterized by low lighting, high dust levels, confined spaces, and dynamically changing production layouts, resulting in an extremely high risk of safety accidents (such as collapses, mechanical injuries, and accidents caused by personnel violations). Therefore, underground safety supervision is a core requirement of industry management. However, existing non-coal mine safety supervision technologies have many shortcomings, making it difficult to meet the requirements of real-time, accurate, and comprehensive supervision. Specific problems are as follows: First, traditional supervision methods are inefficient, risky, and lack real-time performance. This is reflected in the following aspects: Traditional non-coal mine underground supervision mainly relies on manual inspections and fixed video recordings. The coverage of manual inspections is limited, making it impossible to achieve all-weather, blind-spot-free monitoring of all underground work areas (especially high-risk key areas); the harsh underground environment (such as dust and toxic and harmful gases) poses a threat to the personal safety of inspection personnel, and the frequency and efficiency of inspections are limited by labor costs, making it difficult to cover high-frequency areas; the identification of violations of safety production relies on the subjective experience of inspection personnel, which is prone to missed or false detections due to fatigue and judgment bias. Moreover, after a violation is discovered manually, it must be reported manually, resulting in a delayed early warning response and an inability to prevent accidents from starting.

[0003] Secondly, existing video surveillance systems lack intelligent analysis capabilities. This is reflected in the following aspects: some video surveillance equipment deployed in non-coal mines still only functions as video acquisition, local / cloud storage, and post-event review. It cannot proactively identify violations in the video (such as personnel not wearing safety helmets or illegally crossing electronic fences). It requires supervisory personnel to monitor the video stream 24 hours a day, which is labor-intensive and prone to blind spots. The video images are not optimized for the low-light and dusty environment underground, resulting in high image noise and low clarity. Even with manual review, it is difficult to accurately identify details of violations, further reducing the effectiveness of supervision.

[0004] Third, general-purpose deep learning models have poor adaptability and are difficult to cope with complex underground scenarios. This is reflected in the following ways: Although deep learning technologies (such as object detection and behavior recognition) are used in safety supervision in industries such as transportation and construction, their adaptability to non-coal mine underground scenarios is seriously insufficient. General-purpose models are trained based on data from conventional scenarios (such as urban roads and indoor office areas) and do not cover special scenarios in underground environments such as low light, high dust, and dense equipment. This leads to a significant drop in the accuracy of the model in underground environments (such as misidentifying dust shadows as personnel or missing the behavior of not wearing a safety helmet in low light). At the same time, the network bandwidth in underground is limited. If all video streams are uploaded to the cloud for analysis, it is easy to cause network congestion and analysis delay (usually >10 seconds), which cannot meet the needs of real-time early warning. If only edge node analysis is relied upon, the limited computing power of edge devices cannot run complex models, resulting in insufficient multi-task recognition capabilities.

[0005] Fourth, the existing system is fragmented, lacking business closure and flexible adaptability. This is reflected in the following aspects: some non-coal mines have attempted to introduce simple early warning systems, but these systems suffer from functional fragmentation and poor adaptability. They often use fixed coordinates to set up electronic fences, which cannot adapt to dynamic changes in the mine's production layout (such as tunnel face advancement and equipment relocation). Adjustments require redeployment of hardware / software, making the operation complex. They can only generate early warning information and cannot be combined with GIS maps to visualize the location of violations, making it difficult for supervisors to quickly locate them. Furthermore, they do not track the follow-up handling of early warning events (such as whether rectification is carried out and the rectification time), failing to form a complete regulatory link of early warning-handling-feedback-storage. At the same time, when multiple edge nodes interact with cloud data, there is a lack of security protection mechanisms (such as unencrypted transmission), which can easily lead to the leakage of mine production data. Edge node models cannot be updated efficiently, and the recognition accuracy decreases after long-term use, requiring manual redeployment and resulting in high maintenance costs.

[0006] Fifth, the limitations of single-behavior recognition cannot cover multiple types of violation scenarios. This is reflected in the fact that non-coal mine underground violations are diverse (such as equipment operation violations, personnel leaving their posts, overcrowding in confined spaces, not wearing safety helmets, illegal intrusion, etc.), but existing models are mostly designed for single violations and lack the ability to process multiple tasks in parallel. Multiple independent models need to be deployed to identify them separately, resulting in poor system compatibility and high resource consumption (such as doubling memory usage and increasing computation time), making it difficult to adapt to the complex supervision needs underground.

[0007] In summary, there is an urgent need for an intelligent video analysis and early warning technology that integrates scenario-based deep learning models, edge-cloud collaboration, flexible configuration, and business closed loop to improve the intelligence and precision of safety production supervision in non-coal mines. Summary of the Invention

[0008] This invention provides a deep learning-based intelligent video analysis and early warning system for safety production in non-coal mines. It aims to address existing problems in underground safety production supervision in non-coal mines, such as high reliance on manual labor, insufficient intelligent analysis capabilities, poor model-scenario adaptability, fragmented system functions, and incomplete coverage of multiple violation scenarios. This system achieves real-time, accurate identification, intelligent early warning, and full-process supervision of various types of violations in the complex underground environment of non-coal mines.

[0009] This invention provides a deep learning-based intelligent video analysis and early warning system for safety production in non-coal mines, comprising: The video stream generation module is configured to receive real-time video streams from multiple network cameras deployed underground in non-coal mines in accordance with the GB / T 28181 protocol, preprocess the video streams, and push the preprocessed video frames to the AI ​​intelligent analysis engine module through a high-speed data transmission channel; the preprocessing includes video denoising, frame synchronization, and image enhancement. The AI ​​intelligent analysis engine module is configured to communicate with the video stream generation module and, based on a multi-task deep learning model optimized by transfer learning and pre-trained, performs parallel processing on the pre-processed video frames to identify and output labels and confidence levels of safety production violations in real time. The early warning management module is configured to communicate with the AI ​​intelligent analysis engine module, receive and analyze the tags and confidence levels of safety production violations, and generate structured early warning event records when the confidence level of a safety production violation exceeds a preset threshold. The data push module is configured to communicate with the early warning management module, encapsulate early warning event records into JSON format data packets, and push them to one or more monitoring clients via a RESTful API built on the HTTP protocol; The business closed-loop processing module is configured to interact with the data push module and the regulatory client. It is used to visualize the location of the early warning event on the GIS map interface, receive the handling feedback status from the regulatory client, and associate and store the early warning event record, handling feedback status and handling timestamp in the database.

[0010] Furthermore, the multi-task deep learning model includes a YOLOv8 or Faster R-CNN backbone network for object detection and a 3D convolutional neural network (3D CNN) or long short-term memory neural network (LSTM) for behavior recognition. The multi-task deep learning model is trained using multi-scale data augmentation on proprietary datasets of underground low-light, dusty, and densely populated environments.

[0011] Furthermore, violations of safety regulations include at least one of the following: improper operation of mining equipment, failure to wear safety helmets, unauthorized intrusion into key areas defined by electronic fences, staff leaving their posts, and the number of personnel exceeding the safety threshold in confined space operation areas.

[0012] Furthermore, the method for identifying key areas defined by an unauthorized intrusion into an electronic fence includes the following steps: Pre-set the electronic fence boundary in the video frame of the pre-processed video frame; Human detection and tracking using a multi-task deep learning model; The relative positional relationship between the centroid of the human target and the boundary of the electronic fence is calculated. Based on the relative positional relationship, it is determined whether the centroid of the human target has illegally intruded into the boundary of the electronic fence. If it has illegally intruded into the boundary of the electronic fence, an early warning is triggered, and the result of the person illegally intruding into the key area defined by the electronic fence is obtained.

[0013] Furthermore, the default method for defining the electronic fence boundary is as follows: a polygonal area is manually drawn through the GIS map interface or automatically generated by importing CAD drawings of the mining area. When a change in the production layout of the mining area is detected, the electronic fence boundary can be manually adjusted through the GIS map interface, and the adjusted parameters are synchronized in real time to the detection algorithm of the multi-task deep learning model.

[0014] Furthermore, the warning event record should include at least the event type, the time of occurrence, the associated camera number, a screenshot of the violating target, and related short video clips.

[0015] Furthermore, the video stream generation module includes an edge-cloud collaborative processing unit, which is configured with at least one edge computing node and a cloud platform; The edge-cloud collaborative processing unit is configured as follows: By deploying edge computing nodes in non-coal mines underground or near-mine areas and connecting them to network cameras, a lightweight deep learning model is run to perform preliminary analysis of the video stream. When signs of safety violations or simple safety violations are identified, the first analysis data is uploaded to the cloud platform. The first analysis data includes the associated camera number, screenshots of the violating target, associated short video clips, and analysis results. The preliminary analysis includes: real-time detection and tracking of personnel and vehicle targets in the video stream, and determining whether the detected and tracked personnel and vehicle targets show signs of safety violations or simple safety violations. The cloud platform, which is connected to one or more edge computing nodes, performs deep and broad analysis on the first analysis data based on a configured first deep learning model to obtain confirmation results and statistical results of safety production violations. The broad analysis involves cross-regional and cross-time period behavior pattern mining and statistical analysis. Based on the confirmation results and statistical results of safety production violations, the lightweight deep learning model running within the edge computing nodes is incrementally updated or optimized.

[0016] Furthermore, the incremental update or optimization method is as follows: A federated learning framework is adopted to dynamically fuse the parameters of the first deep learning model in the cloud platform with the parameters of the lightweight deep learning model on the edge computing nodes through secure multi-party computation (MPC) to ensure data privacy protection. Specifically, a distributed training framework is built based on the FedProx algorithm. The cloud platform initializes the parameters of the first deep learning model. Each edge computing node trains the lightweight deep learning model based on its local dataset. The lightweight deep learning model parameters are encrypted and uploaded to the cloud platform through the secure multi-party computation (MPC) protocol. The cloud platform performs weighted aggregation on the parameters of the first deep learning model and the parameters of the lightweight deep learning model to generate an updated first deep learning model. Using a knowledge distillation technique based on KL divergence, the logits distribution of the updated first deep learning model on the cloud platform is transferred to the lightweight deep learning model. Specifically, the updated first deep learning model is used as the teacher model, and the lightweight deep learning model is used as the student model. A softening distillation strategy with a temperature coefficient T=10 is employed to transfer the logits distribution knowledge from the teacher model to the student model. The distillation loss function is: , in, This represents the total loss value, used to measure the overall difference between the model's predictions and the actual situation; This represents cross-entropy loss, used to measure the difference between the actual value and the predicted result; The true label represents the actual numerical value of the sample. This represents the model's predicted value, which is the model's prediction result for the sample; KL divergence loss is used to measure the difference between the target probability distribution and the source probability distribution; Represents the target probability distribution. 0.3 represents the source probability distribution; 0.3 represents the weight of cross-entropy loss in the total loss value, and 0.7 represents the weight of KL divergence loss in the total loss value. The cloud platform distributes the distilled and optimized student model parameters to each edge computing node through incremental transmission. After loading the student model parameters, the edge computing nodes complete the lightweight deep learning model update, with an update cycle of ≤24 hours.

[0017] Furthermore, the judgment of signs of safety production violations includes: detecting personnel moving rapidly into key areas defined by electronic fences, a sudden increase in the number of personnel and vehicle targets in the work area, or detecting personnel not wearing safety helmets. The confidence level of these signs is within a preset confidence threshold range. The confidence threshold range is defined as the probability interval between the explicit violation judgment threshold and the full compliance threshold learned by the lightweight deep learning model through a large number of labeled samples during the training process. The confidence threshold range is determined by confusion matrix analysis during the federated learning training phase. When the recall rate is ≥90% and the false positive rate is ≤5%, the confidence threshold range is set to 0.6 to 0.8.

[0018] Furthermore, the video stream generation module also includes a dynamic task allocation unit, which is configured as follows: Monitor the computing load data of the cloud platform and the real-time bandwidth data of the network; When the computing load on the cloud platform exceeds 80% or the real-time bandwidth of the network is less than 5Mbps, the granularity of the analysis task will be adjusted from multi-frame sequence analysis to single-frame analysis, and local caching of edge computing nodes will be enabled. The granularity of the analysis task is determined by the number of image frames in a single video analysis task, the coverage of the target detection area, and the dimensional level of feature extraction. When the network's real-time bandwidth data reaches the set congestion threshold or the computing load data reaches the set high load threshold, a target instruction is generated and sent to the edge computing node. The target instruction is used to instruct the edge computing node to perform more comprehensive local video analysis and processing, and the edge computing node generates the final warning information based on its local analysis and processing results.

[0019] Compared with existing technologies, this invention has the following advantages and beneficial effects: First, it breaks through the limitations of traditional manual supervision models, achieving a closed-loop intelligent supervision process through multi-module collaborative work, reducing reliance on manual inspections and improving supervision efficiency and accuracy. Second, it adopts a multi-task deep learning model optimized by transfer learning, combined with training on a proprietary underground dataset, enhancing the model's adaptability to complex environments such as low light and high dust levels, and improving the accuracy of identifying multiple violation scenarios. Third, the system has a high degree of functional integration, covering complete links such as video access, intelligent analysis, early warning generation, data push, and business closed-loop processing, avoiding functional fragmentation and realizing integrated supervision of non-coal mine safety production. Fourth, it supports the identification of multiple types of violations, including violations such as improper equipment operation, failure to wear safety helmets, and illegal intrusion, expanding the scope of supervision and meeting the diverse safety management needs of non-coal mines. Fifth, the edge-cloud collaborative processing mechanism and dynamic task allocation strategy effectively balance computing resources and network bandwidth, ensuring that the system can still operate efficiently and stably in complex environments, and improving real-time analysis and early warning capabilities.

[0020] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0021] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 A schematic diagram of the structure of a deep learning-based intelligent video analysis and early warning system for safety production in non-coal mines. Figure 2 A schematic diagram illustrating the steps involved in identifying key areas for unauthorized intrusion into an electronic fence. Figure 3 This is a diagram illustrating the content recorded for an early warning event. Detailed Implementation

[0023] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0024] This invention provides a deep learning-based intelligent video analysis and early warning system for safety production in non-coal mines, such as... Figure 1 As shown, it includes: The video stream generation module is configured to receive real-time video streams from multiple network cameras deployed underground in non-coal mines in accordance with the GB / T 28181 protocol, preprocess the video streams, and push the preprocessed video frames to the AI ​​intelligent analysis engine module through a high-speed data transmission channel; the preprocessing includes video denoising, frame synchronization, and image enhancement. The AI ​​intelligent analysis engine module is configured to communicate with the video stream generation module and, based on a multi-task deep learning model optimized by transfer learning and pre-trained, performs parallel processing on the pre-processed video frames to identify and output labels and confidence levels of safety production violations in real time. The early warning management module is configured to communicate with the AI ​​intelligent analysis engine module, receive and analyze the tags and confidence levels of safety production violations, and generate structured early warning event records when the confidence level of a safety production violation exceeds a preset threshold. The data push module is configured to communicate with the early warning management module, encapsulate early warning event records into JSON format data packets, and push them to one or more monitoring clients via a RESTful API built on the HTTP protocol; The business closed-loop processing module is configured to interact with the data push module and the regulatory client. It is used to visualize the location of the early warning event on the GIS map interface, receive the handling feedback status from the regulatory client, and associate and store the early warning event record, handling feedback status and handling timestamp in the database.

[0025] The working principle of the above technical solution is as follows: In order to realize a deep learning-based intelligent video analysis and early warning system for safe production in non-coal mines, this invention achieves real-time monitoring and management of violations of safety regulations in underground non-coal mines through fully automated processing of video acquisition, intelligent analysis, early warning push, and closed-loop handling; a video stream generation module is proposed, which acquires real-time video from underground cameras through a protocol conforming to the national standard (GB / T 28181), and after preprocessing such as noise reduction, synchronization, and enhancement, it is transmitted to the AI ​​analysis module through a high-speed channel; the GB / T 28181 protocol is the national standard "Technical Requirements for Information Transmission, Exchange, and Control of Networked Security Video Surveillance Systems", which specifies the network communication protocol for video surveillance equipment, supports cross-vendor and cross-platform video stream access and control, and can uniformly access real-time underground video streams from network cameras of brands such as Hikvision and Dahua through this protocol; This paper proposes an AI intelligent analysis engine module that utilizes a deep learning model optimized by transfer learning to process video frames in parallel, identify violations in real time, and output labels and confidence scores. Transfer learning optimization is a machine learning technique that transfers knowledge from pre-trained models (such as models trained on public image datasets) to new tasks (such as non-coal mine violation identification). By fine-tuning to adapt to the target scenario, it reduces dependence on small sample data and improves model training efficiency and accuracy. For example, a YOLO model pre-trained on ImageNet can be transferred to the non-coal mine violation identification task, and the model parameters can be fine-tuned using a small amount of labeled data from underground mines to quickly achieve high-precision identification. The multi-task deep learning model is a neural network model that can process multiple related tasks simultaneously. By sharing feature extraction layers, it reduces computational costs and improves the effect of multi-task collaborative learning. An early warning management module is proposed to assess the confidence level of violations output by the AI ​​model. If the confidence level exceeds a threshold, a structured early warning record is generated. The confidence level is the reliability assessment value of the AI ​​model on the recognition result. The higher the value, the stronger the certainty that the model judges the behavior to be a violation. The structured early warning event record includes standardized data records with fixed fields (such as event ID, time, location, violation type, confidence level, etc.) to facilitate storage, transmission and parsing. A data push module is proposed, which encapsulates early warning event records into JSON format and pushes them to the monitoring client via a RESTful API based on the HTTP protocol. The RESTful API is an interface specification designed based on the HTTP protocol, which locates resources through URLs and uses methods such as GET / POST to achieve data interaction. A closed-loop management module is proposed, which receives regulatory and handling feedback at the GIS map visualization early warning location, and associates and stores the early warning, handling status, and timestamp to the database to form a complete management closed loop. GIS map visualization refers to the use of Geographic Information System (GIS) technology to mark the geographical location of early warning events on an electronic map, intuitively displaying the spatial distribution. Structured data is data with a fixed format and fields, such as JSON, XML, or database table structure, which is easy for computers to parse and process. Fields such as violation type, confidence level, and location in the early warning event record are all preset structures and can be directly stored in a MySQL database. In specific applications, such as video monitoring of underground tunneling faces in non-coal mines, the real-time video stream from the tunneling face camera is first accessed via the GB / T 28181 protocol. Preprocessing removes image noise caused by underground dust, enhancing image clarity in low-light environments. Then, based on a multi-task model optimized through transfer learning, the video frames are analyzed in parallel to identify "personnel not wearing safety helmets" (confidence 0.95) and "illegally stacked materials" (confidence 0.88). Since both confidence levels exceed the threshold of 0.8, the system generates a structured warning record containing time (10:20), location (K3 area of ​​the tunneling face), and type. The JSON-formatted warning is then pushed to the ground monitoring center client via a RESTful API. The monitoring center clicks on the warning location on the GIS map, dispatches personnel to handle the situation on-site, and reports "the behavior of not wearing safety helmets has been corrected." The system automatically stores the warning record, handling status, and timestamp (10:35) in the database.

[0026] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the entire process of video acquisition, intelligent analysis, early warning push and closed-loop handling is automated, which changes the situation of traditional manual monitoring that relies on manual inspection and has a slow response. It realizes the immediate detection and rapid intervention of violations of safety production regulations, and can improve the intelligence level and real-time performance of safety production supervision in non-coal mines.

[0027] In one embodiment, the multi-task deep learning model includes a YOLOv8 or Faster R-CNN backbone network for object detection, a 3D convolutional neural network (3D CNN) or a long short-term memory neural network (LSTM) for behavior recognition, and the multi-task deep learning model is trained using multi-scale data augmentation on a proprietary dataset of underground low-light, dusty, and densely populated environments.

[0028] The working principle of the above technical solution is as follows: The multi-task deep learning model integrates network structures with different functions to achieve multiple tasks. It uses YOLOv8 or Faster R-CNN backbone networks for object detection. These two networks perform well in the field of object detection and can accurately identify specific target objects in images. The multi-task deep learning model refers to a deep learning model that can process multiple different tasks simultaneously. In this scenario, it performs both object detection and behavior recognition. Object detection aims to identify the category of target objects in images or videos and determine their location, which is then marked in the form of bounding boxes. YOLOv8 is a fast and accurate object detection algorithm that can predict the category and location of all targets in an image in one forward propagation. Faster R-CNN is a region-based convolutional neural network object detection algorithm that generates candidate regions that may contain targets through a Region Proposal Network (RPN), and then classifies and refines the locations of these regions. Action recognition is achieved by using 3D convolutional neural networks (3D CNNs) or long short-term memory neural networks (LSTMs). Action recognition is the process of identifying the actions of a subject in a video. 3D convolutional neural networks (3D CNNs) add a temporal dimension to the traditional 2D convolutional neural networks, enabling them to process both spatial and temporal information simultaneously. Long short-term memory neural networks (LSTMs) are a special type of recurrent neural network (RNN) that can solve the gradient vanishing and gradient exploding problems in RNNs, effectively handle dependencies in long-term sequence data, and are suitable for analyzing information such as actions and behaviors that change over time. The model is trained using proprietary datasets of low-light, dusty, and densely populated underground environments, enabling it to be optimized for these specific environments. Multi-scale data augmentation involves transforming data at different scales, such as scaling and cropping, to increase data diversity and improve the model's adaptability to targets and scenes at different scales. This allows the model to better detect and recognize targets in complex real-world environments. Multi-scale data augmentation processes raw data at different scales to enhance the model's ability to detect and recognize targets at different scales. For example, when training a target detection model, images of the same object can be trained at different sizes, enabling the model to better recognize the object at different distances and of different sizes.

[0029] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, the integration advantages of multi-task deep learning models can be fully utilized, and by organically combining target detection and behavior recognition tasks, comprehensive intelligent perception of complex underground scenes in non-coal mines can be achieved.

[0030] In one embodiment, safety violations include at least one of the following: improper operation of mining equipment, failure to wear a safety helmet, unauthorized intrusion into a key area defined by an electronic fence, staff leaving their posts, and the number of personnel in a confined space operation area exceeding a safety threshold.

[0031] The working principle of the above technical solution is as follows: By defining specific situations of safety production violations (such as improper operation of mining equipment, failure to wear safety helmets, unauthorized intrusion into key areas defined by electronic fences, workers leaving their posts, and the number of personnel exceeding the safety threshold in confined space work areas), corresponding monitoring methods and systems are used to identify whether these violations occur, thereby ensuring safe production. Improper operation of mining equipment refers to the behavior of operators who do not operate the mining equipment in accordance with the established operating specifications, procedures, and requirements. This may lead to safety accidents such as equipment damage and personnel injuries. For example, starting the mining equipment without conducting regular inspections as required, or incorrectly adjusting the equipment parameters. Failure to wear a safety helmet refers to workers not wearing safety helmets in work scenarios where wearing a safety helmet is required. An electronic fence is a virtual boundary constructed using electronic technology to delineate specific key areas and prevent unauthorized personnel from entering. Unauthorized intrusion into key areas defined by electronic fences refers to unauthorized personnel. Unauthorized entry into a key area designated by an electronic fence, such as an explosives storage area in a mine, constitutes illegal intrusion. "Staff leaving their post" refers to employees leaving their work station without permission or approval during normal working hours, such as mine staff leaving on duty for personal matters. A confined space refers to a closed or partially enclosed space with narrow entrances and exits, not designed as a fixed workplace, poor natural ventilation, and prone to the accumulation of toxic, harmful, flammable, or explosive substances or insufficient oxygen. The safety threshold is the maximum number of personnel allowed to work in the confined space, determined based on its size, ventilation conditions, and the nature of the work. If the actual number of personnel entering the confined space exceeds this threshold, it is considered exceeding the safety limit. For example, in a small underground pipeline repair confined space, the safe working capacity is assessed to be a maximum of 5 people, but 8 people enter, which is considered exceeding the safety threshold.

[0032] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, it is possible to accurately define and specifically identify various typical violations in the process of safe production in non-coal mines, and provide judgment criteria for forming a comprehensive monitoring system covering key aspects such as equipment operation, personnel protection, area control, on-duty personnel and confined space operations.

[0033] In one embodiment, such as Figure 2 As shown, the method for identifying unauthorized intrusion into key areas defined by an electronic fence includes the following steps: Pre-set the electronic fence boundary in the video frame of the pre-processed video frame; Human detection and tracking using a multi-task deep learning model; The relative positional relationship between the centroid of the human target and the boundary of the electronic fence is calculated. Based on the relative positional relationship, it is determined whether the centroid of the human target has illegally intruded into the boundary of the electronic fence. If it has illegally intruded into the boundary of the electronic fence, an early warning is triggered, and the result of the person illegally intruding into the key area defined by the electronic fence is obtained.

[0034] The working principle of the above technical solution is as follows: By pre-setting a flexibly adjustable electronic fence boundary in the video frame, a multi-task deep learning model is used to detect and track human targets in real time, calculate the relative positional relationship between the centroid of the human target and the electronic fence boundary, thereby determining whether an illegal intrusion has occurred and triggering an early warning; human tracking refers to associating the identity of detected human targets in consecutive video frames and maintaining continuous tracking of their movement trajectory; the centroid of the human target refers to the geometric center of the bounding rectangle of the human target (usually calculated as the intersection of the diagonals of the rectangle), which is used to represent the position of the human target; the relative positional relationship refers to the spatial positional relationship between the centroid of the human target and the electronic fence boundary (polygon), mainly determining whether the point is inside, outside or on the boundary of the polygon; In practical applications, for example, if a mining area needs to focus on protecting the area surrounding its explosives depot, the human detection and tracking system first checks for the presence of people in each frame of video (human detection). For instance, if worker A and suspicious person B are detected, rectangular frames are drawn around them in the frame. Then, the movement of worker A and suspicious person B is continuously tracked in subsequent video frames (human tracking). The centroid (center point) of each person's rectangular frame is calculated. If worker A's centroid remains within the normal working area outside the electronic fence (polygon), the system determines that there is no intrusion. If suspicious person B attempts to approach the explosives depot, and their centroid moves from outside the electronic fence to inside (crossing the boundary of the electronic fence), based on this relative positional relationship (centroid inside the polygon), it is determined to be an illegal intrusion, and an audible and visual alarm is immediately triggered. At the same time, the location and trajectory of the intruder B are marked on the monitoring screen to remind security personnel to take timely action.

[0035] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, real-time detection and tracking of human targets can be achieved by pre-setting a flexibly adjustable electronic fence boundary and combining it with a multi-task deep learning model. By accurately calculating the relative positional relationship between the centroid of the human target and the electronic fence boundary, intelligent judgment and early warning of illegal intrusion behavior can be achieved.

[0036] In one embodiment, the default method for the electronic fence boundary is: to manually draw a polygonal area through the GIS map interface or to automatically generate it by importing CAD drawings of the mining area. When a change in the production layout of the mining area is detected, the electronic fence boundary can be manually adjusted through the GIS map interface, and the adjusted parameters are synchronized to the detection algorithm of the multi-task deep learning model in real time.

[0037] The working principle of the above technical solutions is as follows: An electronic fence refers to a virtual warning area boundary defined on the screen by software in a video surveillance scenario, used to delineate key or restricted areas; a GIS map interface refers to the visual operation interface of a Geographic Information System (GIS), which can be used to view, edit, and manage geospatial data; CAD drawings refer to engineering drawings generated by Computer-Aided Design (CAD), containing precise geometric shapes and dimensions; human detection refers to the technology of identifying and locating human targets from video frames, typically outputting a rectangular bounding box containing the human body. In practical applications, for example, if a mining area needs to focus on protecting the area surrounding its explosives depot, security personnel can manually draw a polygonal area to surround the explosives depot on the GIS map interface of the monitoring system when setting up the electronic fence boundary. If the mining area is expanded and the location of the explosives depot is slightly adjusted, security personnel can manually drag the vertices of the polygon through the GIS map interface to adjust the boundary of the electronic fence. The adjusted new boundary parameters will be immediately updated in the system's detection algorithm.

[0038] The beneficial effects of the above technical solution are as follows: the solution provided in this embodiment can achieve flexibility and accuracy in setting up electronic fence boundaries. It can not only support the rapid generation of virtual warning areas that conform to the actual layout of the mining area by importing CAD drawings, but also allow convenient adjustment through the GIS map interface when the production layout changes, ensuring that the electronic fence always matches the actual situation of the mining area.

[0039] In one embodiment, such as Figure 3 As shown, the warning event record should include at least the event type, the time of occurrence, the associated camera number, a screenshot of the violating target, and an associated short video clip.

[0040] The working principle of the above technical solution is as follows: By structurally recording key information of early warning events, complete traceability, rapid location, and effective analysis of events are achieved, providing data support for subsequent processing, evidence preservation, and system optimization. Early warning event records refer to structured data sets that collect and store detailed information on safety production violations detected by the system. Event type refers to the nature or category used to define the early warning event, such as area intrusion, left-behind items, or failure to wear a safety helmet. Occurrence time refers to the precise moment when the early warning event is detected and triggered by the system, usually accurate to the second or millisecond level. Associated camera number refers to a unique identifier corresponding to the monitoring camera that generated the early warning event, used to determine the physical location and specific monitoring equipment where the event occurred. Violation target screenshot refers to a static image captured from the monitoring video stream at the time the early warning event occurs, clearly showing the violation target and its violation status. Associated short video clip refers to a dynamic video segment containing the complete process of the event, captured from the video stream of the corresponding camera around the time the early warning event occurred.

[0041] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, the key elements of the early warning event can be systematically integrated and stored in a standardized manner, which not only ensures the comprehensiveness and accuracy of the event information, but also provides a basis for managers to trace the event process and clarify the responsible parties.

[0042] In one embodiment, the video stream generation module includes an edge-cloud collaborative processing unit, which is configured with at least one edge computing node and a cloud platform; The edge-cloud collaborative processing unit is configured as follows: By deploying edge computing nodes in non-coal mines underground or near-mine areas and connecting them to network cameras, a lightweight deep learning model is run to perform preliminary analysis of the video stream. When signs of safety violations or simple safety violations are identified, the first analysis data is uploaded to the cloud platform. The first analysis data includes the associated camera number, screenshots of the violating target, associated short video clips, and analysis results. The preliminary analysis includes: real-time detection and tracking of personnel and vehicle targets in the video stream, and determining whether the detected and tracked personnel and vehicle targets show signs of safety violations or simple safety violations. The cloud platform, which is connected to one or more edge computing nodes, performs deep and broad analysis on the first analysis data based on a configured first deep learning model to obtain confirmation results and statistical results of safety production violations. The broad analysis involves cross-regional and cross-time period behavior pattern mining and statistical analysis. Based on the confirmation results and statistical results of safety production violations, the lightweight deep learning model running within the edge computing nodes is incrementally updated or optimized.

[0043] The working principle of the above technical solution is as follows: intelligent analysis and model optimization of non-coal mine safety production violations are achieved through edge-cloud collaborative processing units. First, edge computing nodes deployed underground or near the mine connect to network cameras and run lightweight deep learning models to perform preliminary analysis of the video stream: real-time detection and tracking of personnel and vehicle targets, and determination of whether there are signs or simple violations of safety production. The localized computing devices deployed underground or near the mine in non-coal mines are directly connected to the network cameras and are responsible for the preliminary processing of low-latency video streams. The lightweight deep learning model is a miniaturized model running on the edge nodes, characterized by high speed and low resource consumption, and is suitable for real-time target detection and simple violation identification.

[0044] When a relevant situation is identified, the edge node uploads the first analysis data, including the camera number, screenshot of the violation target, related short video clips, and analysis results, to the cloud platform. The cloud platform is a remote centralized computing platform with powerful computing capabilities, responsible for deep analysis, cross-regional / cross-time period statistics, and model optimization. Based on the configured first deep learning model, the cloud platform performs deep analysis on the received first analysis data to confirm safety production violations, and conducts broad analysis across regions and time periods to mine behavioral patterns and perform statistics, obtaining confirmation results and statistical results. The first deep learning model is a large-scale and complex model running on the cloud platform with high accuracy, used for deep analysis and behavioral pattern mining of the data uploaded from the edge.

[0045] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, a low-latency and high-precision unified analysis of non-coal mine safety production violations is achieved through an edge-cloud collaborative architecture; localized processing at edge nodes ensures real-time performance and avoids latency issues caused by long-distance data transmission, enabling rapid response to simple violations such as personnel not wearing safety helmets underground and vehicles speeding; the in-depth analysis and model optimization of the cloud platform improve the overall system's recognition accuracy and intelligence level, and through cross-regional and cross-time period data statistics and behavioral pattern mining, high-risk areas and high-incidence periods can be accurately located, providing targeted decision support for safety management.

[0046] In one embodiment, the incremental update or optimization method is as follows: A federated learning framework is adopted to dynamically fuse the parameters of the first deep learning model in the cloud platform with the parameters of the lightweight deep learning model on the edge computing nodes through secure multi-party computation (MPC) to ensure data privacy protection. Specifically, a distributed training framework is built based on the FedProx algorithm. The cloud platform initializes the parameters of the first deep learning model. Each edge computing node trains the lightweight deep learning model based on its local dataset. The lightweight deep learning model parameters are encrypted and uploaded to the cloud platform through the secure multi-party computation (MPC) protocol. The cloud platform performs weighted aggregation on the parameters of the first deep learning model and the parameters of the lightweight deep learning model to generate an updated first deep learning model. Using a knowledge distillation technique based on KL divergence, the logits distribution of the updated first deep learning model on the cloud platform is transferred to the lightweight deep learning model. Specifically, the updated first deep learning model is used as the teacher model, and the lightweight deep learning model is used as the student model. A softening distillation strategy with a temperature coefficient T=10 is employed to transfer the logits distribution knowledge from the teacher model to the student model. The distillation loss function is: , in, This represents the total loss value, used to measure the overall difference between the model's predictions and the actual situation; This represents cross-entropy loss, used to measure the difference between the actual value and the predicted result; The true label represents the actual numerical value of the sample. This represents the model's predicted value, which is the model's prediction result for the sample; KL divergence loss is used to measure the difference between the target probability distribution and the source probability distribution; Represents the target probability distribution. 0.3 represents the source probability distribution; 0.3 represents the weight of cross-entropy loss in the total loss value, and 0.7 represents the weight of KL divergence loss in the total loss value. The cloud platform distributes the distilled and optimized student model parameters to each edge computing node through incremental transmission. After loading the student model parameters, the edge computing nodes complete the lightweight deep learning model update, with an update cycle of ≤24 hours.

[0047] The working principle of the above technical solution is as follows: A federated learning framework and knowledge distillation technology are used to incrementally update or optimize the lightweight model of the edge nodes. Specifically: the cloud initializes the parameters of the first deep learning model; each edge node trains the lightweight model using its local dataset and uploads the model parameters to the cloud via encrypted multi-party computation (MPC) protocol; the cloud uses the FedProx algorithm to weighted aggregate its own model parameters and the parameters uploaded by the edge nodes to generate the updated first deep learning model; the federated learning framework is a distributed machine learning framework that allows each node to train the model locally, uploading only the parameters rather than the original data, and achieving collaborative model optimization through encrypted aggregation, protecting data privacy; secure multi-party computation (MPC) is a cryptographic technique that ensures that multiple participants can collaboratively compute without disclosing their private data; the FedProx algorithm is a distributed training algorithm in federated learning, which alleviates the model convergence problem caused by non-independent identically distributed (Non-IID) data by adding a proximal regularization term to the loss function; Next, using the updated cloud model as the teacher model and the lightweight edge model as the student model, a softening distillation strategy with a temperature coefficient T=10 is employed. The logits distribution knowledge from the teacher model is transferred to the student model according to the total loss function. Finally, the cloud distributes the distilled and optimized student model parameters incrementally to each edge node. The edge nodes load the parameters and complete the update, with an update cycle of ≤24 hours. Knowledge distillation is a technique that transfers knowledge (such as probability distribution) from a complex teacher model to a simple student model, improving performance while maintaining lightweight design. KL divergence (Kullback-Leibler Divergence) is an indicator that measures the difference in output distribution between the teacher and student models; a smaller value indicates a more similar distribution. Logits distribution refers to the original score distribution of the model's output layer before activation, including richer class probability information. In specific applications, such as monitoring video streams captured by network cameras in a non-coal mine underground area, a lightweight model is first run on an edge computing node to detect people (e.g., not wearing safety helmets) and vehicles (e.g., speeding) in the video in real time, tracking the target's movement trajectory. If a person is identified as not wearing a safety helmet (a simple violation), a screenshot of the violation target and a 10-second associated short video clip are immediately captured, and the camera number "CAM-001" is marked. An analysis result of the person not wearing a safety helmet is generated, and this initial analysis data is encrypted and uploaded to the cloud. Then, the cloud platform calls the first deep learning model to perform deep analysis on the uploaded data (e.g., confirming that "not wearing a safety helmet" is a valid violation), while also performing breadth analysis: cross-regional (comparing CAM-001 with data from other underground cameras) reveals that "the proportion of not wearing a safety helmet in the eastern area reaches 30%", and cross-time period (statistics of the past 7 days) reveals that "the violation rate of the middle shift is twice that of the morning shift", generating a confirmation result ("the violation is valid in the CAM-001 area") and a statistical result ("the middle shift in the eastern area has a high incidence of violations"). Then, the cloud platform initializes the first deep learning model. The model parameters are generated by each edge node (such as CAM-001, CAM-002, etc.) training a lightweight model based on local historical violation data. The parameters are then encrypted and uploaded to the cloud platform via MPC. The cloud platform uses the FedProx algorithm to weight and aggregate the parameters (e.g., the weight of the East Zone node is higher) to generate an updated first deep learning model. Then, using the updated first deep learning model as the "teacher" and the edge lightweight model as the "student", a softening distillation strategy with a temperature coefficient of T=10 is used to transfer the logits distribution knowledge of the teacher model to the student model. Next, the cloud platform sends the distillation-optimized student model parameters (only transmitting the differences from the old model) to each edge node. The CAM-001 node loads the parameters and completes the update. The entire process is completed within 24 hours. After the update, the accuracy of this node in identifying complex violation signs such as "helmet obstruction" increases from 70% to 92%. Through daily model updates, edge nodes can identify underground violations in real time and accurately, while the cloud provides decision-making basis for mine safety management through cross-regional / time-period analysis (such as strengthening safety training for the East Zone shift).

[0048] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the combination of federated learning framework and knowledge distillation technology can achieve continuous iterative optimization of lightweight model while protecting the privacy of local data in non-coal mines, and ensure that edge nodes can still continuously improve their ability to identify complex violations in resource-constrained environments.

[0049] In one embodiment, the judgment of signs of safety production violations includes: detecting personnel moving rapidly into key areas defined by electronic fences, a sudden increase in the number of personnel and vehicle targets in the work area, or detecting personnel not wearing safety helmets. The confidence level of these signs is within a preset confidence threshold range. The confidence threshold range is defined as the probability interval between the explicit violation judgment threshold and the full compliance threshold learned by the lightweight deep learning model through a large number of labeled samples during the training process. The confidence threshold range is determined by confusion matrix analysis during the federated learning training phase. When the recall rate is ≥90% and the false positive rate is ≤5%, the confidence threshold range is set to 0.6 to 0.8.

[0050] The working principle of the above technical solution is as follows: a lightweight deep learning model deployed on the edge nodes performs preliminary processing on the real-time video stream. When a person is detected moving rapidly towards the key area defined by the electronic fence, a dynamic tracking mechanism is immediately triggered. The system predicts whether there is a risk of the person crossing the boundary by combining the target's movement trajectory. If the number of people and vehicles in the work area increases sharply within a preset time window, the system will automatically start the density anomaly analysis module. By comparing with historical data from the same period, it will determine whether it is an abnormal clustering phenomenon. When the confidence level of a person not wearing a safety helmet is in the critical range of 0.6 to 0.8, the edge node will encrypt and upload the frame image and related contextual features (such as lighting conditions and degree of occlusion) to the cloud platform. The cloud uses a high-precision model for secondary verification and retrieves the judgment results of similar recent scenes in the area for cross-validation. If the cloud verifies that there is a risk of violation, an early warning message is generated and pushed to the management personnel terminal. If the risk is ruled out after verification, the judgment data is used as a negative sample and fed back to the edge node model through knowledge distillation technology to further optimize the lightweight model's recognition accuracy of the critical state.

[0051] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, it is possible to achieve accurate identification and dynamic early warning of violations of safety production regulations through a multi-dimensional abnormal symptom detection and hierarchical processing mechanism.

[0052] In one embodiment, the video stream generation module further includes a dynamic task allocation unit, which is configured to: Monitor the computing load data of the cloud platform and the real-time bandwidth data of the network; When the computing load on the cloud platform exceeds 80% or the real-time bandwidth of the network is less than 5Mbps, the granularity of the analysis task will be adjusted from multi-frame sequence analysis to single-frame analysis, and local caching of edge computing nodes will be enabled. The granularity of the analysis task is determined by the number of image frames in a single video analysis task, the coverage of the target detection area, and the dimensional level of feature extraction. When the network's real-time bandwidth data reaches the set congestion threshold or the computing load data reaches the set high load threshold, a target instruction is generated and sent to the edge computing node. The target instruction is used to instruct the edge computing node to perform more comprehensive local video analysis and processing, and the edge computing node generates the final warning information based on its local analysis and processing results.

[0053] The working principle of the above technical solution is as follows: The dynamic task allocation unit constructs an adaptive task scheduling mechanism by real-time monitoring of the cloud platform's computing load and network bandwidth. When the cloud computing load exceeds 80% or the network bandwidth is less than 5Mbps, the system first reduces resource consumption by adjusting the granularity of the analysis task, simplifying the analysis task that originally required processing multiple frame sequences into single-frame analysis. At the same time, it reduces the coverage of the target detection area and lowers the dimensional level of feature extraction, thereby reducing the amount of data processed in a single analysis task. Enabling the local caching of edge computing nodes can avoid the transmission of duplicate data and further alleviate network pressure. When network congestion occurs or the computing load reaches a high load threshold, the system will trigger a deeper level of task migration, allowing the edge computing nodes to undertake more comprehensive local video analysis and processing work through target instructions, including complete parsing of the video stream, multi-dimensional feature extraction, and judgment of complex target behavior. Computational load data refers to the computational workload undertaken by the cloud platform during operation, typically expressed as a percentage, reflecting the current utilization of the cloud platform's computing resources. For example, assuming the total computing resources of the cloud platform are equivalent to 100 computing "units," if 80 "units" of resources are being used to process various tasks, the computational load data is 80%. Real-time bandwidth data represents the rate at which the network can transmit data at a given moment, usually measured in Mbps (megabits per second), reflecting the network's real-time data transmission capability. For example, if the network's real-time bandwidth data is 5 Mbps, it means that the network can transmit approximately 5 megabits of data per second; this rate determines factors such as the smoothness of video playback. Analysis task granularity refers to quantitative indicators such as the number of image frames, the coverage of the target detection area, and the dimensionality of feature extraction in a single video analysis task, reflecting the fineness and scale of the analysis task. Local caching refers to the storage area equipped on the edge computing node itself, used for temporary data storage. This allows for rapid data retrieval when needed, reducing data acquisition time and network transmission burden. The congestion threshold is a pre-defined critical value for real-time network bandwidth data. When the real-time network bandwidth data reaches or falls below this value, network congestion is considered to have occurred. For example, assuming a congestion threshold of 3Mbps, when the real-time network bandwidth data drops to 3Mbps or lower, it indicates potential network congestion, and the system needs to take appropriate measures. The high load threshold is a pre-defined critical value for cloud platform computing load data. When the computing load data reaches or exceeds this value, it indicates that the cloud platform is operating under high load. For example, if the high load threshold is set to 90%, when the cloud platform computing load data reaches 90% or higher, it indicates that cloud platform computing resources are strained, requiring adjustments to task allocation strategies. The target instruction is an instruction generated by the dynamic task allocation unit under specific conditions (network congestion or high load) and sent to the edge computing node. It instructs the edge computing node to perform specific and more comprehensive local video analysis processing tasks.

[0054] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, it is possible to achieve collaborative scheduling between cloud and edge computing nodes, and effectively balance system resource consumption and analysis accuracy by dynamically adjusting the granularity of analysis tasks and task allocation strategies.

[0055] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A deep learning-based intelligent video analysis and early warning system for safety production in non-coal mines, characterized in that, The application relates to a safety production monitoring system based on AI intelligent analysis, comprising: a video stream generation module configured to access real-time video streams from a plurality of network cameras deployed in a non-coal mine shaft according to a GB / T 28181 protocol, and to pre-process the video streams, and to push the pre-processed video frames to an AI intelligent analysis engine module through a high-speed data transmission channel; the pre-processing comprises video denoising, frame synchronization and image enhancement; the AI intelligent analysis engine module is configured to be in communication connection with the video stream generation module, and to perform parallel processing on the pre-processed video frames based on a multi-task deep learning model optimized and pre-trained through transfer learning, so as to identify and output labels and confidence levels of safety production violation behaviors in real time; a pre-warning management module is configured to be in communication connection with the AI intelligent analysis engine module, to receive the labels and confidence levels of the safety production violation behaviors and to analyze the same, and to generate a structured pre-warning event record when the confidence level of the safety production violation behaviors exceeds a preset threshold; a data pushing module is configured to be in communication connection with the pre-warning management module, to encapsulate the pre-warning event record into a data packet in JSON format, and to push the data packet to one or more supervision clients through a RESTful API constructed based on an HTTP protocol; a business closed-loop processing module is configured to interact with the data pushing module and the supervision clients, to visualize the position of the pre-warning event on a GIS map interface, to receive a disposal feedback state from the supervision clients, and to store the pre-warning event record, the disposal feedback state and a disposal timestamp in association into a database. 2.The deep learning-based non-coal mine safety production intelligent video analysis and early warning system according to claim 1, characterized in that, The multi-task deep learning model comprises a YOLOv8 or Faster R-CNN backbone network for target detection, a 3D convolutional neural network (3D CNN) or a long short-term memory neural network (LSTM) for behavior recognition, and the multi-task deep learning model is trained through a special data set of a low-illumination, multi-dust and personnel-intensive scene in a mine shaft and by using multi-scale data enhancement. 3.The deep learning based non-coal mine safety production intelligent video analysis and early warning system according to claim 1, characterized in that, The safety production violation behaviors include at least one of a violation operation of mining equipment, non-wearing of a safety helmet, illegal invasion of a key area defined by an electronic fence by a person, off-duty of a worker and exceeding of a safety threshold of a number of persons in a limited space operation area.

4. The deep learning-based non-coal mine safety production intelligent video analysis and early warning system according to claim 3, characterized in that, The identification method of the illegal invasion of the key area defined by the electronic fence by the person comprises the following steps: presetting an electronic fence boundary in a video picture of the pre-processed video frame; performing human body detection and tracking by using the multi-task deep learning model; calculating a relative position relationship between a human body target centroid and the electronic fence boundary, determining whether the human body target centroid illegally invades the electronic fence boundary based on the relative position relationship, and triggering a pre-warning if the human body target centroid illegally invades the electronic fence boundary, and obtaining a result of the illegal invasion of the key area defined by the electronic fence by the person. 5.The deep learning based non-coal mine safety production intelligent video analysis and early warning system according to claim 4, characterized in that, The preset mode of the electronic fence boundary is that a polygon area is manually drawn through a GIS map interface or is automatically generated by importing a mine CAD drawing, when a mine production layout change is detected, the electronic fence boundary is manually adjusted through the GIS map interface, and the adjusted parameters are synchronously transmitted to a detection algorithm of the multi-task deep learning model in real time. 6.The deep learning based non-coal mine safety production intelligent video analysis and early warning system according to claim 1, characterized in that, The pre-warning event record at least includes event type, occurrence time, associated camera number, violation target screenshot and associated short video clip.

7. The deep learning-based non-coal mine safety production intelligent video analysis and early warning system according to claim 6, characterized in that, The video stream generation module comprises an edge-cloud collaborative processing unit, which is configured with at least one edge computing node and one cloud platform; The edge-cloud collaborative processing unit is configured to: Through the edge computing node deployed in the non-coal mine underground or near-well area and connected with the network camera, a lightweight deep learning model is run to preliminarily analyze the video stream, and when a safety production violation behavior sign or a simple safety production violation behavior is identified, first analysis data is uploaded to the cloud platform; The first analysis data includes the associated camera number, the violation target screenshot, the associated short video clip and the analysis result; wherein the preliminary analysis includes: real-time detection and tracking of personnel and vehicle targets in the video stream, and judgment of whether the detected and tracked personnel and vehicle targets exist safety production violation behavior signs or simple safety production violation behaviors; Through the cloud platform in communication connection with one or more edge computing nodes, the first analysis data is deeply analyzed and broadly analyzed based on the configured first deep learning model to obtain the confirmation result of safety production violation behavior and the statistical result of safety production violation behavior; wherein the broad analysis is a cross-regional and cross-period behavior pattern mining and statistical analysis; and according to the confirmation result of safety production violation behavior and the statistical result of safety production violation behavior, the lightweight deep learning model running in the edge computing node is incrementally updated or optimized. 8.The deep learning based non-coal mine safety production intelligent video analysis and early warning system according to claim 7, characterized in that, The incremental updating or optimization mode is: Using a federated learning framework, the dynamic fusion of the first deep learning model parameters in the cloud platform and the lightweight deep learning model parameters of the edge computing node is realized through secure multi-party computation (MPC) to ensure data privacy protection, specifically: a distributed training framework is constructed based on the FedProx algorithm, the cloud platform initializes the first deep learning model parameters, each edge computing node trains the lightweight deep learning model based on the local data set, uploads the lightweight deep learning model parameters to the cloud platform through the secure multi-party computation (MPC) protocol, the cloud platform aggregates the first deep learning model parameters and the lightweight deep learning model parameters by weighting to generate the updated first deep learning model; Using the knowledge distillation technology based on KL divergence, the logits distribution of the updated first deep learning model of the cloud platform is migrated to the lightweight deep learning model, specifically: taking the updated first deep learning model as the teacher model and the lightweight deep learning model as the student model, using a softening distillation strategy with a temperature coefficient T=10 to migrate the logits distribution knowledge of the teacher model to the student model; the distillation loss function is: ， wherein, represents a total loss value, used to measure the overall difference between the model prediction result and the true situation; represents a cross-entropy loss, used to measure the difference between the true value and the prediction result; represents a true label, which is the true value of the sample, represents a prediction value of the model, which is the prediction result of the model on the sample; is a KL divergence loss, used to measure the difference between the target probability distribution and the source probability distribution; represents a target probability distribution, represents a source probability distribution; 0.3 represents the weight of the cross-entropy loss in the total loss value, and 0.7 represents the weight of the KL divergence loss in the total loss value; The cloud platform transmits the student model parameters optimized by distillation to each edge computing node through incremental transmission, and the edge computing node loads the student model parameters to complete the lightweight deep learning model update, and the update period is ≤24h. 9.The deep learning based non-coal mine safety production intelligent video analysis and early warning system according to claim 7, characterized in that, The judgment of the safety production violation behavior signs includes: detecting that the personnel move quickly to the electronic fence defined key area, the number of personnel and vehicle targets in the work area increases sharply, or the confidence degree of detecting that the personnel do not wear safety helmets is in the preset confidence degree critical range; the confidence degree critical range is defined as the probability interval between the clear violation judgment threshold and the complete compliance threshold learned by the lightweight deep learning model in the training process through a large number of labeled samples; the confidence degree critical range is determined through the confusion matrix analysis in the federal learning training stage, when the recall rate is greater than or equal to 90% and the false alarm rate is less than or equal to 5%, the confidence degree critical range is set to 0.6 to 0.

8. 10.The deep learning based non-coal mine safety production intelligent video analysis and early warning system according to claim 7, characterized in that, The video stream generation module further includes a dynamic task allocation unit, which is configured to: monitor the computing load data of the cloud platform and the real-time bandwidth data of the network; when the computing load data of the cloud platform exceeds 80% or the real-time bandwidth data of the network is less than 5Mbps, the analysis task granularity is adjusted from multi-frame sequence analysis to single-frame analysis, and the local cache of the edge computing node is enabled; the analysis task granularity size is the number of image frames in a single video analysis task, the coverage range of target detection area and the dimension level of feature extraction; when the real-time bandwidth data of the network reaches the set congestion condition threshold or the computing load data reaches the set high load condition threshold, a target instruction is generated and sent to the edge computing node, the target instruction is used to instruct the edge computing node to perform more comprehensive local video analysis processing, and the edge computing node generates the final warning information based on the local analysis processing result.

Images