A mine operation violation detection and grading early warning method and system

Abstract

This application relates to the field of mine operation detection and early warning technology, and discloses a method and system for detecting and classifying early warning of violations in mine operations. The method includes: constructing a violation classification system and establishing a mapping between equipment and violation nodes; generating a semantic raw data stream; extracting and aligning features to generate standardized data packets; fusing features to infer violations and analyzing the coupling to generate correlation; calling a framework to calculate a normalized total risk score based on multi-dimensional parameters; classifying and handling violations, correcting weights based on feedback and performing incremental learning. The system includes: a data acquisition and classification module, an edge preprocessing module, a large model detection module, a risk quantification module, an early warning and handling module, and a closed-loop optimization module. This invention uses a lightweight cloud-based large model to perform cross-modal feature fusion of visual, sensor, and positioning multimodal data, and introduces a historical accident case library to analyze the causal coupling strength between different violations, thereby enabling the identification of complex risks that are difficult to detect with a single sensor.

Description

Technical Field

[0001] This invention relates to the field of mine operation detection and early warning technology, specifically to a method and system for detecting and classifying early warning of violations in mine operations. Background Technology

[0002] Mining operations are typically high-risk, with production environments encompassing narrow underground spaces and complex open-pit terrain. They involve multiple interactions between personnel, heavy machinery, and the geological environment. As safety production supervision requirements increase, digital monitoring technology is gradually becoming a core means to replace manual inspections and prevent production accidents. Deploying video surveillance networks and various sensor networks to achieve real-time perception and data collection of the work site has become a common trend in the current intelligent construction of mines.

[0003] Existing mine safety monitoring applications primarily rely on discrete subsystems. Visual analysis systems are typically based on convolutional neural networks, identifying obvious characteristics such as not wearing safety helmets, not wearing reflective clothing, or personnel entering restricted areas. Environmental and equipment monitoring systems collect physical quantities such as gas concentration and equipment temperature in real time. Alarm logic often employs static threshold matching, meaning that when monitored values ​​or image recognition confidence levels exceed preset thresholds, the system automatically triggers an alarm and records the violation information, thereby assisting safety management personnel in post-incident tracing or immediate intervention.

[0004] However, existing mine safety detection and early warning technologies often suffer from multiple risk events caused by the coupling of equipment status and personnel behavior. Current systems lack cross-modal causal analysis, making it difficult to identify hidden, complex risks. Alarm judgments ignore the cumulative effect of risks over time and differences in on-site working conditions. The degree of danger of similar violations is not differentiated between core and auxiliary areas. This crude approach, lacking quantitative grading, easily leads to alarm overload, causing on-site personnel fatigue and desensitization. Furthermore, the dynamic changes in the mine environment, coupled with the fixed parameters of models trained offline using general data, and the lack of feedback-based automatic optimization loops, result in detection accuracy often declining over time, making it difficult to adapt to the continuous evolution of the on-site environment. Therefore, this invention provides a method and system for detecting and grading early warning of violations in mine operations to address the shortcomings of existing technologies. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for detecting and classifying early warning of violations in mining operations. By constructing a standardized violation classification system, utilizing a cloud-edge collaborative architecture to process multimodal data, and combining large-scale model inference with a general risk quantification framework, it enables dynamic detection, risk quantification assessment, and closed-loop parameter optimization of violations in complex mining operation scenarios.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The first aspect of this invention provides a method for detecting and classifying early warning of violations in mining operations, comprising the following steps: S1. Construct a three-level standardized classification system for violations based on personnel, equipment, and environmental attributes, and establish a mapping relationship between on-site data collection equipment and violation points in the three-level standardized classification system for violations; S2. Drive the acquisition device to acquire visual, sensor and positioning data of the mine site, match the corresponding violation classification label in the mapping relationship according to the acquisition device and add it to the acquired data to generate a raw data stream carrying prior violation category semantics; S3. The edge computing node performs visual enhancement, numerical filtering and spatial correlation processing on the original data stream, extracts key features and performs spatiotemporal alignment, and generates standardized multimodal data packets. S4. The cloud server uses a lightweight large model to perform cross-modal feature fusion on the multimodal data packets, infers the set of violation types and judgment confidence of the current time slice, and retrieves the historical accident case database to analyze the causal coupling strength between violations and generate violation correlation degree. S5. Call the general risk quantification framework, substitute the set of violation types, judgment confidence and violation correlation into the risk scoring model, and calculate the weighted sum of the four dimensions of basic judgment, real-time working conditions, historical trends and related risks in combination with real-time working condition parameters to generate a normalized total risk score. S6. Map the total risk score to a preset grading threshold range to determine the risk level, and trigger the corresponding level of on-site audible and visual alarm or equipment cut-off linkage response strategy. Calculate the risk deviation based on the response feedback results, dynamically correct the weight parameters of the risk scoring model, and feed the selected samples back to the lightweight large model for incremental learning.

[0008] Preferably, in step S1, the step of constructing a three-level standardized classification system for violations based on personnel, equipment, and environmental attributes further includes: A three-level multi-branch tree data structure is established. The first-level dimension is defined as personnel violations, equipment violations, and environmental violations, and an independent root index ID is assigned. Under the first-level dimension, a second-level dimension is constructed as an intermediate index layer for statistical analysis. Construct a third-level dimension as the leaf node of a multi-branch tree, corresponding to a specific single violation entity, and provide a preset severity benchmark value for each third-level dimension leaf node; The severity baseline value is set as a dimensionless normalized value and statically written into the database of the three-level standardized classification system for violations.

[0009] Preferably, in step S3, generating standardized multimodal data packets specifically includes: The visual processing unit performs dark channel prior dehazing or histogram equalization on the original video stream and calculates the change amplitude of adjacent frames to extract key frames to generate visual feature vectors. The sensor signal processing unit performs filtering on the time series sensor data and removes outliers to generate sensor feature vectors. The location data processing unit identifies entities by tag ID and maps 3D coordinates to topological nodes of the mine digital map to generate location feature vectors. The visual feature vector, sensor feature vector, and location feature vector are stamped with a unified microsecond-level timestamp using a clock synchronization module, and the violation classification label is written into the header field of the data packet and uploaded to the cloud via an encryption protocol.

[0010] Preferably, in step S4, the step of generating the violation correlation degree further includes: By utilizing the cross-modal attention mechanism within a lightweight large model, visual features are used as query vectors, and sensor features and location features are used as key-value pairs to generate a comprehensive scene representation vector containing explicit visual information and implicit environmental features. Based on the comprehensive scene representation vector, the effective set of violations is identified through reasoning, and results with a confidence level lower than a preset threshold are filtered out. The pre-trained historical accident case library is retrieved, and the co-occurrence frequency and causative mechanism of each violation element in the effective set of violations in historical accidents are analyzed. When the set contains a combination of violations with a strong causal chain or risk superposition effect, the output violation correlation value is greater than the preset high correlation threshold; when the set contains only a single or physically unrelated violation, the output violation correlation value is less than the preset low correlation threshold.

[0011] Preferably, in step S5, the weighted sum of the four dimensions of associated risk specifically includes: The score of the basic judgment item is proportional to the arithmetic mean of the judgment confidence of all violations in the current valid violation set; The real-time operating condition score is determined by the weighted aggregation of personnel risk coefficient, equipment risk coefficient, and environmental risk coefficient. The historical trend score is composed of the frequency coefficient of similar violations and the regional risk accumulation coefficient, which is used to reflect the time accumulation effect of risk. The score for associated risk items is obtained by multiplying the degree of association of violations by the preset weight of associated items, and is used to quantify the nonlinear risk gain brought about by the coupling of multiple factors.

[0012] Preferably, the calculation of the real-time operating condition item score includes: The personnel risk coefficient is calculated by weighting the baseline value of the severity of the violation with the normalized distance of the personnel from the risk source. When the personnel are in the danger zone of the electronic fence, the weight of the distance factor is increased. The equipment risk coefficient is calculated by weighting the severity benchmark value of equipment violations with the equipment importance coefficient, and the importance coefficient of core mining equipment is higher than that of auxiliary transportation equipment. The environmental risk coefficient is calculated by weighting the severity benchmark of the environmental anomaly with the environmental impact range coefficient, which is determined based on the spatial distribution gradient of the sensor data.

[0013] Preferably, in step S6, the triggering of the corresponding level of on-site audible and visual alarm or equipment disconnection linkage handling strategy specifically includes: Risks are categorized into four levels: general, moderate, severe, and extremely severe. For general risks, only a pop-up message is sent and recorded. For moderate risks, a mobile vibration alert is sent and an on-site audible and visual alarm is activated. For general risks, only a pop-up message is sent to the monitoring terminal to provide a notification, and the violation is recorded; For more serious risks, a mobile vibration alert is sent to relevant personnel, simultaneously activating the on-site audible and visual alarms for immediate warning. For serious risks, the alarm area is highlighted on the command center's large screen, and a countdown protection process is initiated, automatically escalating upon timeout. For particularly serious risks, a circuit breaker trip command is sent to the regional power supply system to cut off the power supply, while an evacuation command is played and access to the risk area is locked.

[0014] Preferably, in step S6, the dynamic correction of the weight parameters of the risk scoring model specifically includes: The actual handling feedback results of the early warning event are obtained and mapped to a reference score. The difference between the reference score and the total risk score calculated by the system is used to obtain the risk deviation. If the risk deviation exceeds the preset tolerance threshold, then according to the risk deviation and the preset learning rate parameter, the weight coefficients corresponding to the risk items that caused the deviation are subjected to gradient correction, and the corrected weight coefficients are written into the configuration database to take effect in the next calculation cycle.

[0015] Preferably, in step S6, the step of feeding high-value samples back to the lightweight large model for incremental learning further includes: Samples with absolute risk deviation exceeding the threshold and boundary samples with confidence in the critical interval are selected. The multimodal data of the selected boundary samples and the correct labels after manual correction are combined to form a training dataset. Call the fine-tuning interface of the lightweight large model, and adopt the low-rank adaptation or adapter fine-tuning strategy to update the parameters of the top task adaptation layer while keeping the general feature parameters of the underlying model frozen.

[0016] A second aspect of the present invention provides a mining operation violation detection and graded early warning system, comprising: The data acquisition and classification module is used to drive the acquisition device to obtain data and associate it with classification tags based on a three-level standardized classification system for violations. The edge preprocessing module is used to receive the collected data stream and perform cleaning, feature extraction and standardized encapsulation to generate standardized multimodal data packets; The large model detection module is used to perform feature fusion and inference on multimodal data packets using a lightweight large model, and output the set of violation types, judgment confidence and violation correlation degree; The risk quantification module is used to call a general risk quantification framework and combine real-time operating condition parameters to calculate a normalized total risk score. The early warning and response module is used to determine the risk level based on the overall risk score and trigger coordinated responses. The closed-loop optimization module is used to calculate risk bias, dynamically update the weight parameters of the risk scoring model, and drive the large model detection module to perform incremental learning.

[0017] This invention provides a method and system for detecting and classifying early warnings of violations in mining operations. It has the following beneficial effects: 1. This invention uses a lightweight cloud-based large model to perform cross-modal feature fusion of visual, sensor, and positioning multimodal data, and introduces a historical accident case database to analyze the causal coupling strength between different violations. This enables the identification of complex risks that are difficult to detect with a single sensor. By outputting quantified violation correlation, it accurately assesses the nonlinear risk gain caused by the coupling of multiple factors, solving the problems of data isolation and difficulty in dealing with complex concurrent violation scenarios in traditional monitoring methods, and improving the accuracy of risk identification in complex operating environments.

[0018] 2. This invention constructs a general risk quantification framework that includes four dimensions: basic judgment, real-time operating conditions, historical trends, and associated risks. It transforms heterogeneous detection data into a normalized total risk score. This quantification method comprehensively considers detection confidence, on-site physical environment parameters, and the time accumulation effect of risks, and has carried out a refined classification and assessment of the safety status of mine operations. Based on the calculated risk level, it can accurately trigger differentiated linkage and disposal strategies, ensuring that high-risk events are subject to mandatory intervention while avoiding unnecessary production stoppages caused by low-risk fluctuations.

[0019] 3. This invention establishes a closed-loop optimization mechanism based on actual handling feedback. By calculating the deviation between the system score and the actual risk, it dynamically executes the weight parameter correction in the risk quantification formula and the incremental learning of the large model, so that the system can automatically calibrate the evaluation model according to the dynamic changes in the mining operation environment, and continuously update the safety knowledge base using the selected boundary samples. This solves the problem of the detection accuracy decaying over time due to the fixed parameters and lagging model iteration in traditional industrial monitoring systems, and extends the effective life cycle of the system. Attached Figure Description

[0020] Figure 1 This is a system architecture diagram of the present invention; Figure 2 This is a flowchart of the method steps of the present invention; Figure 3 This is a flowchart of the large-scale model detection and analysis process of the present invention. Detailed Implementation

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] See attached document Figure 1 , Figure 1 This is a module architecture diagram according to an embodiment of the present invention. The present invention provides a mining operation violation detection and graded early warning system. The system is deployed based on a cloud-edge collaborative architecture and includes a data acquisition and classification module, an edge preprocessing module, a large model detection module, a risk quantification module, an early warning and handling module, and a closed-loop optimization module.

[0023] The data acquisition and classification module is used to acquire raw data based on a preset three-level standardized classification system for violations. It uses visual acquisition devices, various types of sensing devices, and positioning devices deployed at the mine site to define mine violations according to the three-level violation dimensions, providing a unique identifier and key attributes for each violation. When the acquisition devices acquire data, they associate the raw data with the corresponding classification tags according to the key attributes of the monitored violation type, thus completing the initial data structuring.

[0024] The edge preprocessing module is deployed on edge computing nodes. To adapt to the special working environment of high humidity, high dust, and explosiveness in underground mines, the edge computing node hardware adopts a mine-grade explosion-proof edge box with an IP68 or higher protection rating and AI inference acceleration capability (preferably 24 TOPS or higher computing power). It is physically deployed within 500 meters of underground mine chambers and open-pit mine working faces to receive raw data with classification labels and perform differentiated cleaning and feature extraction operations. For visual data, it performs enhanced denoising and keyframe extraction; for sensor data, it performs filtering, smoothing, and outlier removal; for location data, it associates personnel or equipment IDs and outputs location features. This edge preprocessing module converts the processed data into a standardized format with timestamps and uploads it to the cloud via a mine-grade intrinsically safe 5G module and an encrypted data bus.

[0025] The large-scale model detection module is deployed on a cloud server cluster. It receives standardized multimodal data uploaded by the edge preprocessing module, loads a lightweight LLaVA large-scale model, and uses an attention mechanism to fuse features from visual images, sensor parameters, and localization trajectories. Based on the fused comprehensive feature vector, this module loads a three-level classification safety knowledge base for inference, outputting a set of violation types, a judgment confidence matrix, and violation correlation. This module has incremental learning capabilities, supporting fine-tuning of the safety knowledge base parameters through input labeled samples.

[0026] The risk quantification module connects to the large model detection module to receive the detection results output by the large model. This module incorporates a general risk quantification framework, using a unified risk scoring formula to transform different types and scenarios of violation risks into normalized risk scores. The calculation process involves a weighted summation of four dimensions: basic judgment items, real-time operating condition items, historical trend items, and related risk items. The weight coefficients for each dimension can be configured according to the mine type or operating scenario.

[0027] The early warning and response module is used to make decisions based on the risk score output by the risk quantification module. This module has preset threshold ranges to classify risks into four levels: general, relatively severe, serious, and extremely serious. When the risk score falls within a specific range, the module triggers the corresponding level of early warning response, coordinating with on-site equipment, communication terminals, and emergency systems to implement response measures. Simultaneously, the module generates and transmits a structured information package containing a complete chain of evidence and response recommendations.

[0028] The closed-loop optimization module provides support for the entire system management process. This module performs periodic batch processing analysis tasks (e.g., every 24 hours or at the end of a work shift). It retrieves all types of violation data and early warning handling records from the database for the previous period, maps the actual handling feedback (manual confirmation level) to a standard reference value, calculates the deviation between this value and the system's total risk score, and sets a deviation threshold of 0.05. Samples with an absolute deviation greater than 0.05 are considered valid for optimization. The module dynamically adjusts the weight parameters in the general risk quantification framework using a preset update formula. Weight adjustment stops when the risk deviation is ≤5% for three consecutive periods to ensure parameter stability. This closed-loop optimization module filters boundary samples with confidence levels in the critical interval [0.45, 0.55] and feeds new violation cases back to the large model detection module. Each iteration has ≥50 samples to drive continuous model iteration.

[0029] See attached document Figure 2 , Figure 2 This is a flowchart illustrating a method for detecting and classifying early warning of violations in mining operations according to an embodiment of the present invention. The present invention provides a method for detecting and classifying early warning of violations in mining operations, comprising the following steps: S1, Construct a three-level standardized classification system for violations and collect multimodal data; The system establishes a three-level standardized classification system for violations that includes three primary dimensions: personnel, equipment, and environment, and presets a severity benchmark value for the end-level violation node; The system activates various collection devices to acquire visual, sensor, and positioning data in the mining operation scenario and automatically associates the data with corresponding classification labels; S2, preprocessing and feature extraction of multimodal data at the edge: The edge computing node receives the collected raw data, performs image enhancement and keyframe extraction on the visual data to output visual features, performs numerical filtering on the sensor data to output sensor features, and performs spatial correlation on the positioning data to output positioning features; The processed multimodal data is uniformly converted into a standard format and uploaded to the cloud. S3 is a violation detection and correlation analysis based on a lightweight large model. The cloud-based large model receives pre-processed multimodal data and generates a comprehensive feature vector through feature fusion. The large model performs inference based on the comprehensive feature vector, identifies the specific set of violation types, calculates the judgment confidence of each violation, and analyzes the correlation strength between different violations to output the violation correlation degree. S4, based on a general framework for calculating the risk of violations; the system calls the general risk quantification framework, and substitutes the confidence level, correlation degree, and collected environmental and equipment parameters output by the large model into the risk scoring formula; by calculating the weighted sum of basic judgment items, real-time operating condition items, historical trend items, and related risk items, a unified risk score representing the comprehensive risk status of the current scenario is obtained. S5 features full-scenario graded early warning judgment and differentiated linkage response; the system compares the calculated risk score with the preset four-level early warning threshold to determine the current risk level; based on the determined level, the system executes the corresponding linkage response strategy, including sending pop-up reminders, on-site sound and light alarms, linkage display on the command screen, or cutting off the power supply of the equipment, and generates a structured information package for storage and transmission. S6, based on actual feedback, dynamically optimizes parameters and performs closed-loop iteration; the system stores and analyzes the data of the entire process of early warning and handling, calculates the deviation between the risk score and the actual situation; based on this deviation, the system dynamically updates the weight parameters in the risk quantification formula, and feeds new cases back to the large model for incremental learning, so as to achieve continuous optimization of detection accuracy and risk assessment model.

[0030] In this embodiment, the specific execution process of step S1, which involves constructing a three-level standardized classification system for traffic violations and collecting multimodal data, is as follows: A three-tiered standardized classification system for violations covering all scenarios in the mining industry was established. This system employs a three-tiered multi-branch tree data structure to logically aggregate and structurally define discrete violations. (System definition complete set) As a set of violation categories, at the first level, mine violations are divided into three main categories: personnel violations, equipment violations, and environmental violations, respectively marked as follows: , and Each primary dimension is assigned an independent root index ID to ensure independent addressing of data streams during subsequent risk attribution.

[0031] Building upon the first-level dimension, the system extends downwards to construct a second-level dimension, further subdividing the nature of violations. For violations committed by personnel... Subordinate nodes for preventing violations Nodes of behavioral violations and qualification violation nodes For equipment violations Nodes with parameters exceeding limits are set below. Status fault node And maintaining non-compliant nodes For environmental violations Gas over-limit node is set below. Dust and vibration abnormal nodes and abnormal nodes in environmental parameters Second-level dimension nodes serve as an intermediate index layer, used for category aggregation in subsequent statistical analysis.

[0032] The system further constructs a third dimension, namely the leaf nodes of a multi-branch tree, corresponding to specific, observable single violations. These violations are categorized by behavior. For example, it includes equipment operated in violation of regulations. Unauthorized entry into dangerous areas For specific sub-items, the system assigns leaf nodes to each third-level dimension. Assign a preset severity baseline value The baseline for this severity It is a dimensionless numerical value, and its range of values ​​satisfies The system presets this value based on mine safety operating procedures and historical accident statistics. For example, it may be set for high-risk behaviors such as "entering dangerous areas". Set a baseline value for its severity. ; Regarding the basic protective measure of "not wearing a helmet" Set a baseline value for its severity. Regarding environmental issues such as "excessive methane concentration"... Set a baseline value for its severity. In this way, the system transforms unstructured legal and regulatory provisions into computer-calculate weight parameters.

[0033] After completing the classification system construction, multimodal data acquisition operations are performed. This involves driving visual acquisition devices, sensing devices, and positioning devices deployed at the mine site to acquire physical information about the work area in parallel. For the visual acquisition devices, the system acquires continuous video stream data, covering the visible light and infrared bands, to capture personnel movements, equipment appearance, and environmental thermal imaging features. For the sensing devices, the system acquires environmental parameters (such as gas concentration and dust concentration) and equipment operating parameters (such as current and vibration frequency) sampled in a time series. For the UWB positioning module, the system acquires the real-time spatial coordinates of the tags. The specific hardware selection and underlying communication protocol implementation for the aforementioned visual acquisition devices, sensing devices, and positioning devices can be referred to existing mine IoT technology standards by those skilled in the art, as these are well-known technologies in the field and will not be elaborated upon here.

[0034] While collecting data, the system performs data-classification tag association operations, maintains a mapping table between devices and monitoring targets, and automatically matches the collected raw data stream with the classification tags in the three-level classification system based on the physical deployment location and monitoring function of the data collection devices. Establish index associations. For example, automatically associate data streams from gas sensors deployed at the tunneling face. Tags are automatically associated with video streams from cameras deployed in high-voltage power distribution rooms. (Entering a dangerous area) and (Equipment parameters exceed limits) label. This association mechanism ensures that the data subsequently input into the large model has clear prior semantic information, realizing structured processing at the data source level.

[0035] In this embodiment, the specific execution process of the preprocessing and feature extraction step of the edge multimodal data in step S2 is as follows: Edge computing nodes receive multimodal raw data streams uploaded by front-end acquisition devices and distribute them to the vision processing unit, sensor signal processing unit, and positioning data processing unit in parallel to perform preprocessing tasks based on data type identifiers. Through edge computing, data noise is cleaned and key features are extracted, reducing the amount of redundant data uploaded to the cloud and improving the data quality for subsequent large model inference.

[0036] The visual processing unit performs image enhancement and keyframe extraction operations on the received raw video stream data. The unit preprocesses the video frames using a dark channel prior dehazing algorithm or histogram equalization algorithm to remove image blur and low-light noise caused by the high-dust environment of underground mines, improving image contrast and clarity. Based on image enhancement, the unit executes a keyframe extraction strategy. It calculates the pixel difference or optical flow change vector between two adjacent frames, marking the current frame as a keyframe and retaining it only if the change exceeds a preset motion threshold; static or repetitive frames with changes below the threshold are discarded. The unit encodes and compresses the extracted keyframes to generate a visual feature vector sequence. This visual feature vector sequence retains effective visual information including personnel movements, equipment status changes, and environmental anomalies, reducing data transmission bandwidth usage.

[0037] The sensor signal processing unit performs numerical cleaning on the received time-series sensor data, such as gas concentration, equipment temperature, and vibration frequency. This unit uses a sliding window filter or a Kalman filter to smooth the original numerical sequence, suppressing random high-frequency noise caused by electromagnetic interference. The unit also executes outlier removal logic, setting upper and lower bound thresholds based on statistical principles to zero outliers that exceed reasonable physical ranges or correct them using linear interpolation. The filtered and cleaned sensor data is then encapsulated into a one-dimensional numerical feature vector. This one-dimensional numerical feature vector accurately reflects the real changing trend of environmental and equipment operating parameters.

[0038] The positioning data processing unit performs spatiotemporal correlation operations on the received UWB positioning tag coordinate data. First, this unit retrieves the database using the tag ID to identify the person or equipment entity bound to the tag. Then, it maps the original 3D coordinates onto the topological nodes of the mine's digital map, calculating the specific area where the entity is located (e.g., tunnel face, transport roadway, rest area). Finally, the positioning data processing unit fuses the entity ID, physical coordinates, and the encoded area to generate a location feature vector containing spatial relationships. After independently processing the data from each modality, the edge computing node performs data standardization and encapsulation. The clock synchronization module inside the node is... , and After adding a uniform microsecond-level timestamp, the node encapsulates these three sets of feature vectors into a standardized multimodal data packet according to a predefined data protocol. This multimodal data packet carries the violation classification label associated in step S1 in its header field. The data is then sent to the cloud server via an edge-side encryption module (such as SSL / TLS). This standardized encapsulation ensures that data of different modalities can be aligned and merged in the same time slice within the large cloud model.

[0039] See attached document Figure 3 In this embodiment, the specific execution process of step S3, which is based on a lightweight large model for violation detection and correlation analysis, is as follows: The cloud-based large-scale model intelligent detection module receives standardized multimodal data packets uploaded from the edge device. These data packets contain timestamp-aligned visual feature vectors. Sensing feature vectors and location feature vectors A pre-trained lightweight large model based on the LLaVA-7B architecture is loaded onto a cloud server cluster. This model undergoes vertical adaptation and supervised fine-tuning (SFT) within the mining safety field. The training dataset contains over 100,000 historical mine accident samples, with a focus on over 30,000 complex violation cases involving multiple coupled factors, enhancing the model's ability to understand complex causal chains. To meet the demands of high-concurrency real-time inference in the cloud, the model undergoes structured channel pruning, compressing the parameter size to 40% of the original model, achieving an inference speed of ≥20fps while maintaining feature extraction accuracy.

[0040] The large model first performs a multimodal feature fusion operation. It utilizes the model's internal cross-modal attention mechanism to fuse visual features. As a query vector, the sensing features with location features As key-value pairs, the correlation weights between different modal data are calculated. Through weighted aggregation, the lightweight LLaVA large model maps heterogeneous physical signals to a unified high-dimensional semantic space, generating a comprehensive scene representation vector. This comprehensive scene representation vector not only contains explicit visual information in the image, but also integrates implicit numerical features of environmental parameters and spatial topological relationships between entities.

[0041] Based on comprehensive scene representation vectors, the lightweight LLaVA large model calls upon the built-in three-level classification safety knowledge base for violation identification and inference. The model outputs a set of violation types within the current time slice. This set of violation types includes classification identifiers from the three-tier standardized violation classification system, in the form of... For each violation identified in the set of violation types Lightweight LLaVA large model synchronously outputs the corresponding decision confidence. The confidence level of this judgment is a numerical probability indicator, and its value range is [value range missing]. This indicates the confidence level of the detection result. The system sets a confidence filtering threshold. If the confidence level of a certain violation The system determined the result to be an invalid misjudgment and directly removed it from the set. Only high-confidence valid violations are retained for subsequent calculations.

[0042] After identifying the valid set of violations, the lightweight LLaVA large-scale model performs violation correlation inference analysis to quantify the causal coupling strength when different types of violations occur simultaneously. The lightweight LLaVA large-scale model retrieves its pre-trained historical accident case library and analyzes the set. The co-occurrence frequency and causative mechanism of each violation element in historical accidents were analyzed, and the normalized violation correlation value was calculated and output. ( If the set contains only a single violation or multiple physically unrelated violations (such as "not wearing a safety helmet" and "water accumulation in a remote pumping station"), the lightweight LLaVA large model will have a lower output. Value (close to 0); if the set contains a combination of violations with a strong causal chain or risk superposition effect (such as "exceeding the gas concentration limit" accompanied by "electrical equipment explosion failure"), the large model outputs a higher value. The value (close to 1) indicates this correlation. This directly reflects the probability tendency of safety accidents caused by the current state of complex violations, providing a dimension to support subsequent risk quantification in addition to the severity of a single violation.

[0043] The large model intelligent detection module has a built-in incremental learning interface. When new violations or special operating scenarios occur at the mine site, the management personnel collect a small amount of on-site data (50-100 records) and annotate it in the system background. The system quickly injects the new scenario features into the model's safety knowledge base without changing the main parameter structure of the lightweight LLaVA large model, through adapter fine-tuning or low-rank adaptation technology.

[0044] In this embodiment, the specific execution process of step S4, the violation risk quantification calculation step based on the general framework, is as follows: The risk quantification module receives the set of violation types output by the large model detection module. Confidence matrix and correlation of violations Simultaneously, it reads the environmental parameters and device status of the current scene, calls the built-in general risk quantification framework, and substitutes the above heterogeneous data into a unified mathematical model to calculate and generate a normalized total risk score. The overall risk score This indicates the overall safety risk status of the mining operation scenario at the current time slice: ; In the formula, This represents the score for the basic judgment items, used to reflect the reliability of the test results; This represents the real-time operating condition score, used to indicate the current level of danger in the physical environment; The score represents the historical trend item, which is used to reflect the cumulative effect of risk over time; This represents the score of the associated risk item, used to quantify the nonlinear risk gain caused by the coupling of multiple factors.

[0045] The system calculates basic judgment items based on the statistical characteristics of the detection results. The arithmetic mean of the confidence scores of all violations in the current valid violation set is calculated, and this arithmetic mean is multiplied by a preset base ratio coefficient. When the confidence scores of the lightweight LLaVA large model for violation identification are generally high, the score of this item increases accordingly, indicating that the system has a strong certainty in judging the existence of current risks, thus occupying a basic share in the total score.

[0046] Calculate real-time operating conditions based on the real-time state of the current physical scene. By employing a weighted aggregation approach, the real-time risk characteristics of personnel, equipment, and environment are integrated, and the calculation structure is as follows: ; In the formula, This is the normalization factor for the operating condition (e.g., 0.4). , , These represent personnel risk coefficient, equipment risk coefficient, and environmental risk coefficient, respectively. These are the scenario-specific weight coefficients for the corresponding dimensions, ensuring that the sum of the weights equals 1. The system dynamically loads these weights into the configuration table based on the type of work scenario; for example, it increases the environmental weight when working on a tunneling face. Increase the weight of the equipment in the electromechanical chamber. .

[0047] For each sub-coefficient in the above formula, the following refined calculation is performed: For personnel risk coefficient The system comprehensively considers two core indicators: personnel qualification level and distance from the risk source. The system reads the electronic tag ID of the personnel involved and searches their special operation qualification database. If unlicensed operation or a qualification level that does not match the current operation (e.g., low-qualified personnel operating high-risk equipment) is detected, it is judged as a high-risk factor. Simultaneously, it combines UWB positioning data to calculate the Euclidean distance between the personnel and the hazard source (e.g., the cutting section of a coal mining machine). If the personnel are within the electronic fence's danger zone, the system automatically increases the risk coefficient based on the reciprocal of the distance. That is, the lower the personnel qualification level and the closer to the risk source, the higher the risk coefficient. The higher the value; For equipment risk coefficient The system quantifies equipment importance based on its most recent maintenance date. First, it categorizes field equipment into core equipment (e.g., fans, elevators) and auxiliary equipment (e.g., belt conveyors) according to the equipment register, assigning higher basic weights to core equipment. Simultaneously, the system retrieves maintenance records from the equipment's lifecycle management system via an interface, calculating the interval between the current time and the most recent maintenance. If this interval exceeds a preset safety inspection cycle, a risk penalty factor that increases over time is introduced. In other words, the more critical the equipment and the longer it has been in an "overdue maintenance" state, the higher the penalty factor. The higher the value; For environmental risk coefficient The system integrates data from multiple sensors, including those for methane concentration, dust density, temperature, and humidity. It collects environmental parameters of the work area in real time and compares and normalizes them against the critical thresholds specified in the "Coal Mine Safety Regulations." For example, when methane concentration approaches the warning value or dust density remains high, the system non-linearly increases the environmental risk value using an exponential function; simultaneously, it combines the spatial distribution gradient of environmental sensors to determine the impact range coefficient of the abnormal environment. That is, the closer the environmental parameters are to or exceed the critical value, the wider the scope of the abnormal impact, and the higher the coefficient. The higher the value.

[0048] Introducing the time dimension to calculate historical trend items The historical trend item consists of two parts: the frequency of similar violations and the cumulative risk level of the area. The system counts the number of times the same type of violation occurs in the area within a short period (e.g., the last 72 hours), compares it with a preset frequency threshold to obtain a frequency coefficient, and simultaneously performs a time-decay weighted summation of all historical risk scores generated in the area within the last 24 hours to obtain a cumulative risk coefficient. The system adds the frequency coefficient and the cumulative risk coefficient and multiplies it by the historical item weight (e.g., 0.2) to reflect the deteriorating safety trend of the area in the current score.

[0049] Calculate the association risk item using the association degree output by the large model. The violation correlation degree output in step S3 Multiplying by the weight of the related item (e.g., 0.15), when multiple strongly correlated violations occur simultaneously (such as gas exceeding limits accompanied by electrical explosion failure), the score of this item increases, thus increasing the overall risk score. It can overcome the limitations of traditional linear superposition and accurately reflect the risks of major accidents that may be caused by the coupling of multiple factors.

[0050] In this embodiment, the specific execution process of step S5, the full-scenario hierarchical early warning determination and differentiated linkage handling step, is as follows: The tiered early warning and response module receives the total risk score output by the general risk quantification module. The system determines risk levels based on preset classification standards. Risk threshold ranges are pre-defined in the system memory, dividing mine safety risks into four progressive levels. The specific mapping rules are as follows: when When this occurs, it is classified as a Level 1 risk, corresponding to a general warning. when At that time, it was determined to be a level 2 risk, corresponding to a relatively severe warning; when At that time, it was determined to be a level three risk, corresponding to a severe warning; when At that time, it was determined to be a Level 4 risk, corresponding to a particularly severe warning. The system performs a scoring assessment every second to ensure that changes in risk status are captured in real time. In such cases, the system treats it as a safety fluctuation and does not trigger an alarm process.

[0051] Based on the determined risk level, the system calls different hardware interfaces and software services to execute differentiated linkage response strategies. This tiered response mechanism aims to balance production efficiency and safety assurance, avoid unnecessary production stoppages caused by low-risk events, and ensure that high-risk events are subject to mandatory intervention.

[0052] For Level 1 risks, a lightweight alert strategy is implemented. The system calls the web server interface to send a pop-up message to the management backend of the monitoring center, displaying only a text description of the violation. The system writes the current violation record into the historical database and sets a 30-day rolling overlay strategy for subsequent routine inspections and statistical analysis, without interfering with on-site operations.

[0053] For Level 2 risks, the system provides two-way warnings both on-site and remotely. In addition to triggering a pop-up window on the web interface, the system sends vibration alerts to the mobile app of the responsible area manager via message push service. The system also sends control commands to the audible and visual alarms in the violation area via an industrial bus (such as Modbus or CAN bus), driving them to flash yellow and sound an alarm for 30 seconds, directly reminding on-site workers to correct the violation.

[0054] For Level 3 risks, comprehensive monitoring and manual intervention requests are implemented. The system links with the command center's LED screen, highlighting the alarm area with a red border on the digital twin interface. It also automatically retrieves real-time video streams from three surrounding cameras and plays them in a pop-up loop. The system starts a software-defined timer guardian process with a 10-minute countdown threshold. If the system does not receive manual confirmation or a deactivation signal from management personnel before the countdown ends, the guardian process automatically upgrades the risk level to Level 4, triggering the next level of handling procedures.

[0055] For Level 4 risks, mandatory physical blocking and emergency broadcasting are implemented. The system sends circuit breaker trip commands to the regional power supply system via the PLC (Programmable Logic Controller) interface, cutting off the power supply to mining equipment and auxiliary transportation equipment, thus eliminating the risk source at the physical level. In parallel, the system sends a high-priority audio stream to the emergency broadcasting system, looping evacuation instructions. The system also sends locking commands to the access control system, closing the electric gates leading to the risk area to prevent unauthorized entry, and sends a rescue request data packet containing location information to the mine rescue center via the API interface.

[0056] While performing the aforementioned actions, the system generates a structured information package containing a complete chain of evidence, which encapsulates a set of violation types. Overall risk score Four sub-parameter values This includes corresponding on-site visual screenshots and sensor data snapshots. The structured information packet is uploaded to an immutable log server or blockchain node via an encrypted transmission protocol to ensure the authenticity and legal validity of the data during the accident tracing process. This structured data record provides standardized input samples for parameter closed-loop optimization in subsequent step S6.

[0057] In this embodiment, the specific execution process of step S6, which involves dynamic parameter optimization and closed-loop iteration based on actual feedback, is as follows: The closed-loop optimization service module periodically accesses the historical disposal record database stored in the system to perform parameter calibration and model update tasks. This is used to solve the technical pain point of traditional industrial control systems having fixed parameters and being unable to adapt to the dynamic changes in the mining environment. The module achieves adaptive convergence of algorithm parameters by establishing a feedback loop.

[0058] The closed-loop optimization service module performs risk deviation calculations and extracts system scores from early warning events. The system retrieves the actual handling feedback results corresponding to the warning event. These feedback results originate from the judgment information entered by safety management personnel on the post-event confirmation interface, or from structured data generated based on subsequent accident investigation reports. The system maps the manually verified actual risk level to a numerical reference score. Calculate the deviation between the reference score and the system score. ,Right now .like If the absolute value exceeds the preset tolerance threshold (e.g., 0.05), the system determines that the warning has a deviation and marks the event as a valid negative sample.

[0059] Based on the calculated deviation value, the system dynamically updates the weight parameters. The system loads the current weight configuration file of the general risk quantification framework, specifying the weight coefficients for the risk items that caused the deviation. (Weights for personnel, equipment, or environment) Perform gradient correction. The correction process follows a preset update formula: ; In the formula, These are the updated weighting coefficients; This represents the weighting coefficient for the current period; The preset learning rate parameter has a value range of (0, 1) and is used to control the step size of parameter adjustment to prevent system parameter oscillation due to single feedback. The risk deviation calculated above; This is the contribution factor of this sub-item in the current event, used to allocate the total deviation to each specific sub-item, through updated weighting coefficients. The data is written to the configuration database and takes effect in the next calculation cycle, thereby enabling the risk quantification model to gradually approach the actual evaluation standards for mining operations.

[0060] While optimizing numerical parameters, the system performs incremental iterations of the large model knowledge base, and the system automatically filters out... High-value samples with large absolute values ​​and confidence levels Boundary samples in the critical interval. By combining the multimodal data (images, sensor values) of these samples with the manually corrected labels to form a new training dataset, the fine-tuning interface of the lightweight large model is called in the background. The low-rank adaptation or adapter fine-tuning technique is adopted. This technique keeps the general feature parameters of the bottom layer of the large model unchanged and only updates the parameters of the top task adaptation layer, injecting the new violation patterns and association logic into the model's safety knowledge base.

[0061] After completing parameter updates and model fine-tuning, the system executes a gray-scale verification process. The closed-loop optimization service module deploys the updated algorithm version in a shadow runtime environment and performs backtesting using historical data streams from the past 24 hours. The system compares the detection rate and false positive rate of the old and new versions. Only when the new version reduces the false positive rate while maintaining the same detection rate will a version switch command be generated, pushing the updated model and parameters to the edge nodes and cloud servers in the production environment, completing the system's closed-loop evolution.

[0062] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting and grading prewarning of mine operation violation, characterized in that, The method comprises the following steps: S1, constructing a three-level standardized violation classification system based on personnel, equipment and environment attributes, and establishing a mapping relationship between a field collection device and a violation node in the three-level standardized violation classification system; S2, driving the collection device to acquire visual, sensing and positioning data of the mine site, matching the corresponding violation classification label in the mapping relationship according to the collection device, and adding it to the collected data to generate an original data stream carrying prior violation category semantics; S3, an edge computing node performs visual enhancement, numerical filtering and spatial correlation processing on the original data stream, extracts key features and performs time and space alignment to generate a standardized multi-modal data packet; S4, a cloud server uses a lightweight large model to perform cross-modal feature fusion on the multi-modal data packet, infers the violation type set and the determination confidence of the current time slice, and retrieves the historical accident case library to analyze the causal coupling strength between violations to generate a violation correlation degree; S5, calling a general risk quantification framework, substituting the violation type set, determination confidence and violation correlation degree into a risk scoring model, combining real-time working condition parameters to calculate the weighted sum of four dimensions of basic judgment, real-time working condition, historical trend and associated risk, and generating a normalized total risk score; S6, mapping the total risk score to a preset grading threshold interval to determine the risk level, triggering the corresponding level of on-site sound and light alarm or device cutting linkage disposal strategy, and calculating the risk deviation based on the disposal feedback result, dynamically correcting the weight parameters of the risk scoring model, and feeding the filtered samples back to the lightweight large model for incremental learning.

2. The method according to claim 1, characterized in that, In step S1, the step of constructing a three-level standardized violation classification system based on personnel, equipment and environment attributes further comprises: establishing a three-level multi-branch tree data structure, defining the first level dimension as personnel violation, equipment violation and environment violation and assigning an independent root index ID, and constructing the second level dimension as a statistical analysis intermediate index layer under the first level dimension; constructing the third level dimension as the leaf node of the multi-branch tree, corresponding to the specific single violation behavior entity, and providing a preset severity benchmark value for each three-level dimension leaf node; The severity benchmark value is set as a dimensionless normalized value and is statically written into the database of the three-level standardized violation classification system.

3. The method of claim 1, wherein the method further comprises: In step S3, the generation of the standardized multi-modal data packet specifically comprises: using a visual processing unit to perform dark channel prior dehazing or histogram equalization on the original video stream, calculating the change amplitude of adjacent frames to extract key frames to generate a visual feature vector, and using a sensor signal processing unit to perform filtering on time series sensor data and remove outliers to generate a sensor feature vector; using a positioning data processing unit to identify entity identity through label ID, mapping three-dimensional coordinates to the topological nodes of the mine digital map to generate a location feature vector; using a clock synchronization module to stamp a uniform microsecond-level timestamp on the visual feature vector, sensor feature vector and location feature vector, and writing the violation classification label into the data packet header field and uploading it to the cloud through an encryption protocol.

4. The method of claim 1, wherein, In step S4, the step of generating the rule violation correlation degree further comprises: Using the cross-modal attention mechanism in the lightweight large model, the visual features are taken as query vectors, and the sensor features and location features are taken as key-value pairs to generate a comprehensive scene representation vector containing explicit visual information and implicit environmental features; Based on the comprehensive scene representation vector, an effective rule violation set is identified by reasoning, and results with a judgment confidence lower than a preset threshold are filtered out, a pre-trained historical accident case library is searched, and the co-occurrence frequency and damage mechanism of each rule violation element in the effective rule violation set in the historical accident are analyzed; When the set contains a rule violation combination with a strong causal chain or a risk superposition effect, a rule violation correlation degree value greater than a preset high correlation threshold is output; when the set contains only a single rule violation or rule violations that are physically unrelated, a rule violation correlation degree value less than a preset low correlation threshold is output.

5. The method of claim 1, wherein the method further comprises: In step S5, the weighted sum of the four dimensions of the associated risk specifically comprises: The basic judgment item score is proportional to the arithmetic mean of the judgment confidence of all rule violations in the current effective rule violation set; The real-time working condition item score is determined by the weighted aggregation result of the personnel risk coefficient, the equipment risk coefficient and the environmental risk coefficient; The historical trend item score is composed of the same type of rule violation frequency coefficient and the regional risk accumulation coefficient, which is used to reflect the time accumulation effect of the risk; The associated risk item score is obtained by multiplying the rule violation correlation degree by a preset associated item weight, which is used to quantify the nonlinear risk gain brought by the coupling of multiple factors.

6. The method of claim 5, wherein the method further comprises: The calculation of the real-time working condition item score comprises: The personnel risk coefficient is calculated by weighting the severity benchmark value of the rule violation and the normalized distance of the personnel from the risk source, and when the personnel are in the dangerous area of the electronic fence, the weight of the distance factor is increased; The equipment risk coefficient is calculated by weighting the severity benchmark value of the equipment rule violation and the equipment importance coefficient, and the importance coefficient of the core mining equipment is higher than that of the auxiliary transportation equipment; The environmental risk coefficient is calculated by weighting the severity benchmark value of the environmental anomaly and the environmental impact range coefficient, and the environmental impact range coefficient is determined according to the spatial distribution gradient of the sensor data.

7. The method of claim 1, wherein the method further comprises: In step S6, the triggering of the corresponding level of on-site sound and light alarm or device cut-off linkage disposal strategy specifically comprises: The risk is divided into four levels: general, heavier, serious and particularly serious; for general risk, only a pop-up message is sent and recorded; for heavier risk, a mobile end vibration reminder is sent and a field sound and light alarm is driven; For serious risk, the alarm area is highlighted on the command center large screen and a countdown daemon process is started, which is automatically upgraded when it times out; for particularly serious risk, a circuit breaker tripping instruction is sent to the regional power supply system to cut off the power supply, and a evacuation instruction is played and the access to the risk area is locked.

8. The method of claim 1, wherein, In step S6, the dynamic correction of the weight parameters of the risk scoring model specifically comprises: The actual disposal feedback result of the early warning event is obtained and mapped as a reference score, the difference between the reference score and the total risk score calculated by the system is calculated to obtain a risk deviation; If the risk deviation exceeds the preset tolerance threshold, a gradient correction is performed on a weight coefficient corresponding to a risk item that generates the deviation according to the risk deviation and a preset learning rate parameter, and the corrected weight coefficient is written into a configuration database to take effect in a next calculation period.

9. The method of claim 1, wherein, In step S6, the step of feeding back the high-value sample to the lightweight large model for incremental learning further includes: Screening samples with risk deviation absolute values exceeding a threshold value and boundary samples with determination confidence in a critical interval, and grouping multi-modal data of the screened boundary samples and artificially corrected correct labels into a training data set; Calling a fine-tuning interface of the lightweight large model, adopting a low-rank adaptation or adapter fine-tuning strategy, and updating top-layer task adaptation layer parameters while keeping bottom-layer general feature parameters frozen.

10. A mine operation violation detection and grading early warning system applied to the mine operation violation detection and grading early warning method of any one of claims 1-9, characterized in that, Comprise: A data acquisition and classification module for acquiring data and associating classification labels based on a three-level standardized violation classification system; An edge preprocessing module for receiving the acquired data stream and performing cleaning, feature extraction and standardized packaging to generate standardized multi-modal data packets; A large model detection module for performing feature fusion and reasoning on the multi-modal data packets using the lightweight large model to output a violation type set, a determination confidence and a violation correlation degree; A risk quantification module for calculating a normalized total risk score in combination with real-time working condition parameters using a general risk quantification framework; An early warning and disposal module for determining a risk level and triggering a linkage disposal according to the total risk score; A closed-loop optimization module for calculating a risk deviation, dynamically updating weight parameters of a risk score model and driving the large model detection module to perform incremental learning.

Images