A safety monitoring and intervention system for electrical work sites based on machine vision

Abstract

The present invention relates to the technical field of electrical operation safety management, and specifically to a machine vision-based electrical operation site safety monitoring and intervention system, comprising a multimodal data acquisition module, a behavior feature extraction module, a risk identification module, an automatic intervention module and a data storage and visualization module, wherein the multimodal data acquisition module acquires multimodal data in real time, the behavior feature extraction module extracts features by deep learning technology, the risk identification module uses a machine learning model to perform a comprehensive analysis on behavior, environment and equipment status, and identifies high-risk behavior, environmental anomalies and equipment failures, the automatic intervention module generates intervention instructions according to risk analysis results, and the data storage and visualization module realizes hierarchical storage and dynamic display of multimodal data and risk results; the present invention can realize real-time monitoring, accurate risk identification and intelligent intervention of electrical operation sites, and improve the efficiency and reliability of operation safety management.

Description

Technical Field

[0001] The present invention relates to the technical field of electrical operation safety management, and in particular to an electrical operation site safety monitoring and intervention system based on machine vision. Background Art

[0002] Electrical work is an important and high-risk link in industrial production, involving high-voltage equipment operation, complex environment monitoring and personnel behavior management. Its safety is crucial to personnel life, equipment assets and production efficiency. With the rapid development of industrial automation and digitalization, the amount and types of data involved in electrical work have increased significantly. The safety management of the work site urgently needs to introduce technologies such as machine vision, artificial intelligence and big data analysis to improve the ability of real-time monitoring and risk prediction.

[0003] Existing electrical operation safety monitoring technology has obvious deficiencies in many aspects. First, the risk identification capability is limited. Most systems rely on only a single data source (such as behavioral data or equipment status data) and lack comprehensive analysis of multimodal data, which may lead to potential risks being missed or misjudged. Secondly, the intervention strategy is not intelligent enough. Existing systems mostly rely on fixed rules or manual intervention and lack dynamic response capabilities based on risk levels, making it impossible to achieve accurate and efficient safety protection in complex scenarios. Summary of the invention

[0004] The invention provides an electrical operation site safety monitoring and intervention system based on machine vision.

[0005] A machine vision-based electrical operation site safety monitoring and intervention system includes a multimodal data acquisition module, a behavior feature extraction module, a risk identification module, an automatic intervention module, and a data storage and visualization module, wherein;

[0006] The multimodal data acquisition module collects multimodal data of the electrical operation site in real time through a machine vision camera and an environmental sensor, wherein the multimodal data includes operator behavior image data, environmental data, and equipment operation status data, wherein;

[0007] The operator behavior image data includes action posture and position trajectory;

[0008] The environmental data include temperature, humidity and electromagnetic radiation intensity;

[0009] The equipment operation status data includes electrical equipment surface temperature data, equipment operation current and voltage data, and equipment vibration frequency and amplitude data;

[0010] The behavior feature extraction module processes the collected operator behavior image data and uses a deep learning model based on a two-dimensional convolutional neural network to extract safety-related behavior features, including posture key points, behavior trajectory and behavior category recognition, wherein;

[0011] The key points of the posture include the distance between the head, limbs, tools and equipment, which are used to determine whether the operation complies with the specification;

[0012] The behavior trajectory analyzes the motion trend in the frame sequence based on the optical flow method to detect potential abnormal actions;

[0013] The behavior category recognition is used to classify the behavior of operators and provide accurate behavior input for the risk identification module;

[0014] The risk identification module conducts a comprehensive analysis of the extracted behavior features, environmental data and equipment operation status data based on the support vector machine model, wherein the behavior features are used to determine whether the operation of the operator is standardized, the environmental data is used to evaluate the safety of the working environment, and the equipment operation status data is used to detect potential failures in the operation of the equipment;

[0015] The automatic intervention module generates intervention instructions according to the analysis results of the risk identification module. The intervention instructions include sound and light warning signals, equipment emergency control and video prompts. The sound and light warning signals are used to remind operators to pay attention to potential risks, the equipment emergency control is used to handle equipment failures, and the video prompts are used to assist managers in making decisions;

[0016] The data storage and visualization module is used to store and visualize the collected multimodal data and risk identification results.

[0017] Optionally, the multimodal data acquisition module includes:

[0018] Behavior image data collection: By deploying high-resolution video surveillance cameras, the operator's action posture and position trajectory are continuously captured at 24 frames per second;

[0019] Environmental data collection: Use high-precision temperature and humidity sensors and electromagnetic radiation sensors to collect environmental temperature, humidity and electromagnetic radiation intensity respectively;

[0020] Equipment operation status data collection: The surface temperature data of the equipment is collected through the surface infrared temperature sensor, the current and voltage data of the equipment operation are obtained using the embedded sensor, and the vibration frequency and amplitude data of the equipment operation are collected using the vibration sensor.

[0021] Optionally, the behavior feature extraction module includes:

[0022] Data preprocessing: normalize the input behavioral image data to remove data deviations caused by illumination and noise, and use bilateral filtering algorithm to perform image denoising;

[0023] Posture key point extraction: A deep learning model based on a two-dimensional convolutional neural network is used to extract key points, including the relative positions of the head, limbs, tools, and equipment;

[0024] Behavior compliance judgment: The relationship between operation distance and angle is calculated based on the key point coordinate data to determine whether the operator's operation complies with safety regulations.

[0025] Optionally, the behavior feature extraction module further includes:

[0026] Behavior trajectory analysis: Analyze the motion trend in the behavior image frame sequence based on the optical flow method and calculate the motion vector field of the operator;

[0027] Combine motion trends and the operator's motion vector field to detect potential abnormal actions, including accidentally touching high-voltage equipment or approaching dangerous areas;

[0028] Behavior category recognition: Use the decision tree classification model to classify the extracted behavior trajectory features. The classification rules of the model are generated through training data;

[0029] Abnormal behavior detection: Dynamically monitor the classification results to determine whether the behavior category is abnormal (such as operations that deviate from safety regulations). When abnormal behavior is detected, a behavior abnormality flag is generated to provide input for the risk identification module.

[0030] Optionally, the risk identification module includes:

[0031] Behavior feature input: Receive the posture key points, behavior trajectory and behavior category recognition results output by the behavior feature extraction module, and standardize the behavior features to ensure data consistency;

[0032] Support vector machine model construction: Use the support vector machine model to classify behavioral features;

[0033] Standardization judgment: Based on the classification results of the support vector machine model, judge whether the operator's operation complies with safety regulations. If the classification result is "non-standard", mark the behavior as high-risk behavior.

[0034] Optionally, the risk identification module evaluates the safety of the operating environment and detects potential failures of equipment operation by comprehensively analyzing the environmental data and equipment operation status data, including:

[0035] Data input: Receive environmental data (including temperature, humidity and electromagnetic radiation intensity) and equipment operation status data (including surface temperature, current and voltage, vibration frequency and amplitude) provided by the multimodal data acquisition module, and perform time series alignment processing to ensure the dynamic consistency of environmental parameters and equipment status;

[0036] Analysis of abnormal environmental status: Calculate the abnormal deviation value of environmental data (temperature, humidity and electromagnetic radiation intensity) item by item, and calculate the abnormality degree of comprehensive environmental data;

[0037] Fault identification: Build a fault identification model based on support vector machine to detect potential faults in equipment operation status data, including abnormal surface temperature, abnormal current and abnormal vibration frequency;

[0038] Comprehensive analysis: The safety score of the overall working environment is calculated by combining the results of the abnormal environmental status analysis and the fault identification results.

[0039] Optionally, the automatic intervention module generates sound and light warning signals and video prompts according to the analysis results of the risk identification module to remind operators and auxiliary managers to make decisions, specifically including:

[0040] Intervention trigger condition determination: Receive the identification results of the risk identification module, including behavioral risk identification, environmental anomaly identification, and equipment failure identification results, establish a risk level mechanism, divide the risk levels into low risk, medium risk, and high risk, and determine the intervention trigger conditions based on the risk level;

[0041] Sound and light warning signal generation: Activate sound and light alarm devices according to medium or high risk trigger conditions;

[0042] Video prompt generation: Combine the operator behavior image data collected by the multimodal data acquisition module with the risk identification results to display high-risk areas through visual annotation;

[0043] Synchronous output of warnings and prompts: Sound and light warning signals are output through on-site equipment to remind operators to pay attention to safety, and video prompts are output through management terminals to assist managers to grasp risk situations in real time and make decisions.

[0044] Optionally, the automatic intervention module generates equipment control instructions and handles equipment failures according to the analysis results of the risk identification module, including:

[0045] Equipment emergency control trigger condition determination: Receive the equipment fault identification results provided by the risk identification module, and set the emergency control trigger conditions based on the type and severity of the equipment fault;

[0046] Emergency control command generation: Generate targeted control commands based on trigger conditions;

[0047] Emergency control execution: The control command is transmitted to the execution terminal through the wireless communication unit, the execution of the control command is monitored in real time, and the execution result is fed back;

[0048] Post-control safety status assessment: Reassess the safety status of on-site equipment and working environment based on the execution results of emergency control.

[0049] Optionally, the data storage and visualization module stores the collected multimodal data and risk identification results, including:

[0050] Data classification and processing: receiving the multimodal data collected by the multimodal data acquisition module and the risk level and classification results identified by the risk identification module, and classifying and storing them;

[0051] Hierarchical storage construction: adopt distributed storage architecture to build three-layer storage areas;

[0052] Data compression and backup: compress the stored data.

[0053] Optionally, the data storage and visualization module visualizes the multimodal data and risk identification results, including:

[0054] Dynamic monitoring interface: Provides a dynamic monitoring interface to dynamically display changes in operator behavior trajectories, environmental data, and equipment operating status data;

[0055] Historical data analysis interface: Provides a historical data analysis interface to provide users with long-term risk analysis and abnormal event statistics by tracing back the behavior trajectory of operators, environmental trends and equipment operating status;

[0056] User interaction function: The user interaction function provides real-time data annotation, customized query and data export functions, and supports the generation of visual reports.

[0057] Beneficial effects of the present invention:

[0058] The present invention uses a multimodal data acquisition module to achieve real-time acquisition of operator behavior, operating environment parameters, and equipment operating status data, and combines the collaborative work of the behavior feature extraction module and the risk identification module to conduct a comprehensive analysis of potential risks at the work site. Specifically, the key points of the operator's posture, behavior trajectory, and category are extracted through a deep learning algorithm, and a comprehensive analysis is performed based on a support vector machine model, which can efficiently identify irregular operations, high-risk environments, and potential equipment failures, greatly improving the accuracy and real-time nature of risk identification, and providing an accurate basis for safety intervention.

[0059] In the present invention, the automatic intervention module combines the risk identification results to generate a multi-level safety intervention strategy, including sound and light warning signals, equipment emergency control and video prompts. The system can remind operators of potential safety hazards through sound and light signals in medium-risk situations, take equipment emergency control to quickly eliminate hazards in high-risk situations, and provide dynamic visual prompts for managers. It not only achieves real-time safety protection for the work site, but also significantly reduces the risk of accidents caused by human operating errors through intelligent intervention means.

[0060] In the present invention, the data storage and visualization module efficiently manages multimodal data and risk identification results through a hierarchical storage architecture, ensuring the compatibility of real-time data processing and historical data analysis. The real-time dynamic monitoring interface intuitively displays the real-time status and high-risk areas of the work site, and the historical data analysis interface supports deep behavioral pattern backtracking, environmental trend analysis, and equipment status review, providing a scientific basis for long-term safety management optimization. In addition, the user interaction function enhances the flexibility of data annotation and report generation, further improving the convenience and operability of the system in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 A schematic diagram of a system flow of an embodiment of the present invention;

[0063] Figure 2 The figure is a schematic diagram of the risk identification module flow in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0065] It should be noted that the references to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0066] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0067] like Figure 1-Figure 2 As shown, a machine vision-based electrical operation site safety monitoring and intervention system includes a multimodal data acquisition module, a behavior feature extraction module, a risk identification module, an automatic intervention module and a data storage and visualization module, wherein;

[0068] The multimodal data acquisition module collects multimodal data of the electrical operation site in real time through machine vision cameras and environmental sensors. The multimodal data includes operator behavior image data, environmental data and equipment operation status data, among which;

[0069] The operator behavior image data includes action posture and position trajectory;

[0070] Environmental data includes temperature, humidity, and electromagnetic radiation intensity;

[0071] Equipment operation status data includes electrical equipment surface temperature data, equipment operation current and voltage data, and equipment vibration frequency and amplitude data;

[0072] The behavior feature extraction module processes the collected operator behavior image data and uses a deep learning model based on a two-dimensional convolutional neural network to extract safety-related behavior features, including posture key points, behavior trajectory, and behavior category recognition, among which;

[0073] Posture key points include the distance between the head, limbs, tools and equipment, which are used to determine whether the operation is in compliance with the specifications;

[0074] Behavior trajectories analyze motion trends in frame sequences based on the optical flow method to detect potential abnormal actions (such as accidentally touching high-voltage equipment or approaching dangerous areas);

[0075] Behavior category recognition is used to classify operator behaviors (such as operation, standby, movement, etc.) to provide accurate behavior input for the risk identification module;

[0076] The risk identification module conducts a comprehensive analysis of the extracted behavioral features, environmental data, and equipment operation status data based on the support vector machine model. The behavioral features are used to determine whether the operator's operation is standardized, the environmental data is used to evaluate the safety of the working environment, and the equipment operation status data is used to detect potential failures in equipment operation.

[0077] The automatic intervention module generates intervention instructions based on the analysis results of the risk identification module. The intervention instructions include sound and light warning signals, equipment emergency control and video prompts. Sound and light warning signals are used to remind operators to pay attention to potential risks, equipment emergency control is used to deal with equipment failures, and video prompts are used to assist managers in making decisions.

[0078] The data storage and visualization module is used to store and visualize the collected multimodal data and risk identification results.

[0079] The multimodal data acquisition module includes:

[0080] Behavior image data collection: By deploying high-resolution video surveillance cameras, the operator's action posture and position trajectory are continuously captured at 24 frames per second;

[0081] Environmental data collection: Use high-precision temperature and humidity sensors and electromagnetic radiation sensors to collect environmental temperature, humidity and electromagnetic radiation intensity respectively;

[0082] The data is collected at a sampling interval of 0.1 seconds and calibrated using multi-sensor fusion technology. The formula is as follows:

[0083] D e =ω1T+ω2H+ω3R;

[0084] Among them, D e is the calibrated environmental data; T is temperature, H is humidity, R is electromagnetic radiation intensity, ω1, ω2, ω3 are weight factors of each parameter;

[0085] Equipment operation status data collection: The surface temperature data of the equipment is collected through the surface infrared temperature sensor, the current and voltage data of the equipment operation are obtained using the embedded sensor, and the vibration frequency and amplitude data of the equipment operation are collected using the vibration sensor. The data sampling frequency is not less than 10Hz;

[0086] The multimodal data acquisition module provides comprehensive and real-time multimodal data support for the behavior feature extraction module, risk identification module and data storage and visualization module by integrating image acquisition equipment and environment and equipment status sensors.

[0087] The behavior feature extraction module includes:

[0088] Data preprocessing: Normalize the input behavioral image data to remove data deviations caused by illumination and noise, and use the bilateral filtering algorithm to perform image denoising, which can be expressed as:

[0089]

[0090] Among them, I ′ (x) is the pixel value after filtering, S is the filtering window, σ s and σ r are the spatial and intensity parameters respectively, W is the normalization coefficient;

[0091] Posture key point extraction: A deep learning model based on a two-dimensional convolutional neural network is used to extract key points, including the relative positions of the head, limbs, tools, and equipment;

[0092] The key point positioning is achieved by regressing the key point coordinates, and the model loss function is expressed as:

[0093]

[0094] Among them, L pose is the loss function, is the key point coordinate predicted by the model, P i is the coordinate of the key points actually marked, and N is the total number of key points;

[0095] Behavior compliance judgment: Combine the key point coordinate data to calculate the relationship between the operation distance and angle to determine whether the operator's operation complies with safety regulations. The calculation formula is expressed as:

[0096]

[0097] Among them, d is the distance between key points, (x1, y1) and (x2, y2) are the coordinates of two key points;

[0098] Through data preprocessing and posture key point extraction, the behavior feature extraction module realizes the standardized analysis of the operator's operating behavior, providing accurate key point input for subsequent behavior trajectory analysis and behavior category identification.

[0099] The behavior feature extraction module also includes:

[0100] Behavior trajectory analysis: Based on the optical flow method, the motion trend in the behavior image frame sequence is analyzed and the motion vector field of the operator is calculated. The optical flow calculation is expressed as:

[0101]

[0102] in, represents the image brightness gradient, V is the motion vector, Indicates the brightness change over time;

[0103] Combine motion trends and the operator's motion vector field to detect potential abnormal actions, including accidentally touching high-voltage equipment or approaching dangerous areas;

[0104] Behavior category recognition: Use the decision tree classification model to classify the extracted behavior trajectory features. The classification rules of the model are generated through training data. The classification process includes the following steps:

[0105] (1) Splitting nodes in the decision tree sequentially according to the specific value range of the behavior characteristics (such as speed, direction, regional location, etc.);

[0106] (2) Each node selects the split feature according to the information gain formula, which is as follows:

[0107]

[0108] Among them, IG(S,A) is the information gain of feature A, H(S) is the entropy of the current set, S v A is a sample subset whose feature A takes value v;

[0109] (3) According to the final classification results of the leaf nodes, the behavior trajectories are divided into categories such as "operation", "standby", and "movement";

[0110] Abnormal behavior detection: Dynamically monitor the classification results to determine whether the behavior category is abnormal (such as operations that deviate from safety regulations). When abnormal behavior is detected, an abnormal behavior flag is generated to provide input for the risk identification module.

[0111] The behavior feature extraction module realizes dynamic monitoring and anomaly detection of operator behavior through behavior trajectory analysis and category recognition based on decision trees, providing accurate and clearly classified behavior data for subsequent modules.

[0112] The risk identification module includes:

[0113] Behavior feature input: Receive the posture key points, behavior trajectory and behavior category recognition results output by the behavior feature extraction module, and standardize the behavior features to ensure data consistency;

[0114] Support vector machine model construction: Use the support vector machine model to classify behavioral features. The model formula is as follows:

[0115]

[0116] Among them, f(x) is the classification result, x is the input feature vector, and x i is the support vector, y i is the behavior category label (standard or irregular), α i is the weight, K(x i ,x) is the kernel function, b is the bias;

[0117] Standardization judgment: Based on the classification results of the support vector machine model, judge whether the operator's operation complies with safety regulations. If the classification result is "non-standard", mark the behavior as high-risk behavior;

[0118] By making normative judgments on the extracted behavioral features, the risk identification module can detect operators' illegal operations in real time and provide accurate risk information for subsequent modules.

[0119] The risk identification module conducts a comprehensive analysis of environmental data and equipment operating status data to assess the safety of the operating environment and detect potential failures in equipment operation, including:

[0120] Data input: Receive environmental data (including temperature, humidity and electromagnetic radiation intensity) and equipment operation status data (including surface temperature, current and voltage, vibration frequency and amplitude) provided by the multimodal data acquisition module, and perform time series alignment processing to ensure the dynamic consistency of environmental parameters and equipment status;

[0121] Analysis of abnormal environmental status: Calculate the abnormal deviation value of environmental data (temperature, humidity and electromagnetic radiation intensity) item by item, and calculate the abnormality degree of comprehensive environmental data;

[0122] The abnormal deviation value is expressed as:

[0123]

[0124] Among them, D i represents the deviation value of environmental data i, x i is the current environment data value, x safe To preset a safety threshold;

[0125] The calculation of the anomaly degree of comprehensive environmental data is expressed as:

[0126]

[0127] Among them, D e is the comprehensive environmental abnormality, n is the number of environmental data items;

[0128] If D e >Δ(abnormality threshold), mark the current environment as abnormal, and provide the abnormality D to the comprehensive analysis step e ;

[0129] Fault identification: A fault identification model based on support vector machine is constructed to detect potential faults in the equipment operation status data, including abnormal surface temperature, abnormal current and abnormal vibration frequency. The model formula is as follows:

[0130]

[0131] Among them, f(z) is the discrimination result, z i is the support vector, z is the input feature vector, including the device surface temperature value, current value and vibration frequency value, y i is the category label (normal / faulty), α i is the support vector weight, K(z i ,z) is the kernel function, b is the bias;

[0132] f(z)>0 indicates that the device is operating normally, and f(z)≤0 indicates that the device has a potential fault;

[0133] Comprehensive analysis: The safety score of the overall working environment is calculated based on the results of the environmental abnormality analysis and the fault identification results. The scoring formula is as follows:

[0134] S=w1D e +w2R;

[0135] Among them, S is the safety score, w1, w2 are weight coefficients, and D e is the abnormality of environmental data, and R is the equipment failure risk score.

[0136] The automatic intervention module generates sound and light warning signals and video prompts based on the analysis results of the risk identification module to remind operators and auxiliary managers to make decisions, including:

[0137] Intervention trigger condition determination: Receive the identification results of the risk identification module, including behavioral risk identification, environmental anomaly identification, and equipment failure identification results, establish a risk level mechanism, divide the risk level into low risk, medium risk, and high risk, and determine the intervention trigger conditions based on the risk level, including:

[0138] Low risk: The working environment or behavior is slightly abnormal and no intervention is required;

[0139] Medium risk: Potential safety hazards are detected, triggering audible and visual warning signals;

[0140] High risk: A clear safety risk is detected, triggering both an audible and visual warning signal and a video prompt;

[0141] Sound and light warning signal generation: Activate sound and light alarm devices according to medium or high risk trigger conditions;

[0142] Video prompt generation: Combine the operator behavior image data collected by the multimodal data acquisition module with the risk identification results to display high-risk areas through visual annotation;

[0143] The following information is dynamically superimposed on the video screen:

[0144] High-risk areas are marked in red;

[0145] The current location of the operator and their behavior risk status;

[0146] Abnormal value prompts for environmental data and equipment operating status;

[0147] Synchronous output of warnings and prompts: Sound and light warning signals are output through on-site equipment to remind operators to pay attention to safety, and video prompts are output through management terminals to assist managers to grasp risk situations in real time and make decisions;

[0148] Through audio and visual warning signals and video prompts, the automatic intervention module can remind operators of potential risks in real time, while providing managers with clear risk visualization information.

[0149] The automatic intervention module generates equipment control instructions and handles equipment failures based on the analysis results of the risk identification module, including:

[0150] Equipment emergency control trigger condition determination: Receive the equipment fault identification results provided by the risk identification module, and set the emergency control trigger conditions based on the type and severity of the equipment fault, including:

[0151] Equipment high temperature failure: triggering equipment cooling measures or shutdown;

[0152] Abnormal equipment current: triggering power-off protection;

[0153] Excessive vibration of equipment: triggering shutdown and isolation measures;

[0154] Emergency control command generation: Generate targeted control commands based on trigger conditions. Command types include:

[0155] Power-off protection: Cut off the power supply of the equipment through the control system to prevent the accident from expanding;

[0156] Device isolation: Activate the device isolation system to disconnect the faulty device from the operating network;

[0157] Operation mode adjustment: reduce the equipment operating load or enable backup equipment;

[0158] Emergency control execution: The control command is transmitted to the execution terminal through the wireless communication unit, the execution of the control command is monitored in real time, and the execution results are fed back, including whether the fault is eliminated, the response time of the control system and the operating status of the execution terminal;

[0159] Post-control safety status assessment: Based on the results of emergency control execution, reassess the safety status of on-site equipment and operating environment. If the fault cannot be eliminated, repeat the control instructions or prompt the management personnel to take manual intervention measures;

[0160] Through equipment emergency control, the automatic intervention module can effectively handle equipment failures, prevent accidents from expanding, and provide immediate fault handling feedback for subsequent risk management.

[0161] The data storage and visualization module stores the collected multimodal data and risk identification results, including:

[0162] Data classification and processing: Receive the multimodal data collected by the multimodal data acquisition module and the risk level and classification results identified by the risk identification module, format the received data, and classify and store them. The classification includes:

[0163] Real-time data: including real-time frames of behavioral image data, environmental data, and the latest values ​​of device status data;

[0164] Historical data: including behavior feature extraction results, risk identification results, and time series of real-time data;

[0165] Abnormal data: including high-risk behaviors, abnormal environmental conditions, and equipment failure records;

[0166] Hierarchical storage construction: Adopting distributed storage architecture, constructing three-tier storage areas, including:

[0167] The first layer: real-time data buffer, used to store data within a short time window in real time;

[0168] The second layer: historical data storage layer, which uses time index and data type index structure to support efficient query;

[0169] The third layer: abnormal data storage layer, providing high-priority access and analysis;

[0170] Data compression and backup: compress the stored data.

[0171] The data storage and visualization module visualizes multimodal data and risk identification results, including:

[0172] Dynamic monitoring interface: Provides a dynamic monitoring interface to dynamically display the behavior trajectory of operators, environmental data, and changes in equipment operating status data, and highlights high-risk areas to reflect the safety status of the work site in real time, helping operators and managers to identify and respond to potential risks in a timely manner;

[0173] Historical data analysis interface: Provides a historical data analysis interface to provide users with long-term risk analysis and abnormal event statistics by tracing back the behavior trajectory of operators, environmental trends and equipment operating status. It supports flexible screening and visual display, and provides in-depth data support for optimizing operation management and safety improvements.

[0174] User interaction function: The user interaction function provides real-time data annotation, customized query and data export functions, providing users with efficient data management methods, and supports the generation of visual reports, which can be exported in the form of PDF or charts to enhance the scientificity and operability of decision support.

[0175] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0176] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A machine vision-based electrical operation site safety monitoring and intervention system, characterized in that: It includes multimodal data acquisition module, behavior feature extraction module, risk identification module, automatic intervention module and data storage and visualization module, among which; The multimodal data acquisition module collects multimodal data of the electrical operation site in real time through a machine vision camera and an environmental sensor, wherein the multimodal data includes operator behavior image data, environmental data, and equipment operation status data, wherein; The operator behavior image data includes action posture and position trajectory; The environmental data include temperature, humidity and electromagnetic radiation intensity; The equipment operation status data includes electrical equipment surface temperature data, equipment operation current and voltage data, and equipment vibration frequency and amplitude data; The behavior feature extraction module processes the collected operator behavior image data and uses a deep learning model based on a two-dimensional convolutional neural network to extract safety-related behavior features, including posture key points, behavior trajectory and behavior category recognition, wherein; The key points of the posture include the distance between the head, limbs, tools and equipment, which are used to determine whether the operation complies with the specification; The behavior trajectory analyzes the motion trend in the frame sequence based on the optical flow method to detect potential abnormal actions; The behavior category identification is used to classify the behavior of operators and provide behavior input for the risk identification module; The risk identification module conducts a comprehensive analysis of the extracted behavior features, environmental data and equipment operation status data based on the support vector machine model, wherein the behavior features are used to determine whether the operation of the operator is standardized, the environmental data is used to evaluate the safety of the working environment, and the equipment operation status data is used to detect potential failures in the operation of the equipment; The automatic intervention module generates intervention instructions according to the analysis results of the risk identification module. The intervention instructions include sound and light warning signals, equipment emergency control and video prompts. The sound and light warning signals are used to remind operators to pay attention to potential risks, the equipment emergency control is used to handle equipment failures, and the video prompts are used to assist managers in making decisions; The data storage and visualization module is used to store and visualize the collected multimodal data and risk identification results.

2. The electrical operation site safety monitoring and intervention system based on machine vision according to claim 1 is characterized in that: The multimodal data acquisition module comprises: Behavioral image data collection: By deploying video surveillance cameras, the operator's action posture and position trajectory are continuously captured; Environmental data collection: Use temperature and humidity sensors and electromagnetic radiation sensors to collect environmental temperature, humidity and electromagnetic radiation intensity respectively; Equipment operation status data collection: The surface temperature data of the equipment is collected through the surface infrared temperature sensor, the current and voltage data of the equipment operation are obtained using the embedded sensor, and the vibration frequency and amplitude data of the equipment operation are collected using the vibration sensor.

3. The electrical operation site safety monitoring and intervention system based on machine vision according to claim 2 is characterized in that: The behavior feature extraction module comprises: Data preprocessing: normalize the input behavioral image data to remove data deviations caused by illumination and noise, and use bilateral filtering algorithm to perform image denoising; Posture key point extraction: A deep learning model based on a two-dimensional convolutional neural network is used to extract key points, including the relative positions of the head, limbs, tools, and equipment; Behavior compliance judgment: The relationship between operation distance and angle is calculated based on the key point coordinate data to determine whether the operator's operation complies with safety regulations.

4. The electrical operation site safety monitoring and intervention system based on machine vision according to claim 3 is characterized in that: The behavior feature extraction module also includes: Behavior trajectory analysis: Analyze the motion trend in the behavior image frame sequence based on the optical flow method and calculate the motion vector field of the operator; Combine motion trends and the operator's motion vector field to detect potential abnormal actions, including accidentally touching high-voltage equipment or approaching dangerous areas; Behavior category recognition: Use the decision tree classification model to classify the extracted behavior trajectory features. The classification rules of the model are generated through training data; Abnormal behavior detection: Dynamically monitor the classification results to determine whether the behavior category is abnormal. When abnormal behavior is detected, a behavior abnormality flag is generated to provide input for the risk identification module.

5. The electrical operation site safety monitoring and intervention system based on machine vision according to claim 4 is characterized in that: The risk identification module includes: Behavior feature input: receiving the posture key points, behavior trajectory and behavior category recognition results output by the behavior feature extraction module, and standardizing the behavior features; Support vector machine model construction: Use the support vector machine model to classify behavioral features; Standardization judgment: Based on the classification results of the support vector machine model, judge whether the operator's operation complies with safety regulations. If the classification result is "non-standard", mark the behavior as high-risk behavior.

6. The electrical operation site safety monitoring and intervention system based on machine vision according to claim 5 is characterized in that: The risk identification module evaluates the safety of the operating environment and detects potential failures of equipment operation by comprehensively analyzing environmental data and equipment operation status data, including: Data input: Receive environmental data and equipment operation status data provided by the multimodal data acquisition module, and perform time series alignment processing; Analysis of abnormal environmental status: Calculate the abnormal deviation value of each environmental data item, and calculate the abnormality degree of the comprehensive environmental data; Fault identification: Build a fault identification model based on support vector machine to detect potential faults in equipment operation status data, including abnormal surface temperature, abnormal current and abnormal vibration frequency; Comprehensive analysis: The safety score of the overall working environment is calculated by combining the results of the abnormal environmental status analysis and the fault identification results.

7. The machine vision-based electrical operation site safety monitoring and intervention system according to claim 6, characterized in that: The automatic intervention module generates sound and light warning signals and video prompts based on the analysis results of the risk identification module to remind operators and auxiliary managers to make decisions, specifically including: Intervention trigger condition determination: Receive the identification results of the risk identification module, including behavioral risk identification, environmental anomaly identification, and equipment failure identification results, establish a risk level mechanism, divide the risk levels into low risk, medium risk, and high risk, and determine the intervention trigger conditions based on the risk level; Sound and light warning signal generation: Activate sound and light alarm devices according to medium or high risk trigger conditions; Video prompt generation: Combine the operator behavior image data collected by the multimodal data acquisition module with the risk identification results to display high-risk areas through visual annotation; Synchronous output of warnings and prompts: Sound and light warning signals are output through on-site equipment to remind operators to pay attention to safety, and video prompts are output through management terminals to assist managers to grasp risk situations in real time and make decisions.

8. The machine vision-based electrical operation site safety monitoring and intervention system according to claim 7, characterized in that: The automatic intervention module generates equipment control instructions and handles equipment failures according to the analysis results of the risk identification module, including: Equipment emergency control trigger condition determination: Receive the equipment fault identification results provided by the risk identification module, and set the emergency control trigger conditions based on the type and severity of the equipment fault; Emergency control command generation: Generate targeted control commands based on trigger conditions; Emergency control execution: The control command is transmitted to the execution terminal through the wireless communication unit, the execution of the control command is monitored in real time, and the execution result is fed back; Post-control safety status assessment: Reassess the safety status of on-site equipment and working environment based on the execution results of emergency control.

9. The machine vision-based electrical operation site safety monitoring and intervention system according to claim 8, characterized in that: The data storage and visualization module stores the collected multimodal data and risk identification results, including: Data classification and processing: receiving the multimodal data collected by the multimodal data acquisition module and the risk level and classification results identified by the risk identification module, and classifying and storing them; Hierarchical storage construction: adopt distributed storage architecture to build three-layer storage areas; Data compression and backup: compress the stored data.

10. The electrical operation site safety monitoring and intervention system based on machine vision according to claim 9, characterized in that: The data storage and visualization module visualizes the multimodal data and risk identification results, including: Dynamic monitoring interface: Provides a dynamic monitoring interface to dynamically display changes in operator behavior trajectories, environmental data, and equipment operating status data; Historical data analysis interface: Provides a historical data analysis interface to provide users with long-term risk analysis and abnormal event statistics by tracing back the behavior trajectory of operators, environmental trends and equipment operating status; User interaction function: The user interaction function provides real-time data annotation, customized query and data export functions, and supports the generation of visual reports.

Images