Electric power operation safety early warning method and system based on space-time attention mechanism

Abstract

The invention relates to the technical field of electric power operation image recognition, and discloses an electric power operation safety early warning method and system based on a space-time attention mechanism, and the method comprises the steps: obtaining a video data stream of an electric power operation site and a dynamic operation parameter data stream of an operator; obtaining a plurality of video clips and a plurality of corresponding dynamic operation parameter clips; obtaining a plurality of video clip time-space features and dynamic operation parameter time sequence features; fusion features are obtained through corresponding fusion; obtaining a safety risk index of the current moment through a pre-trained safety risk assessment model according to the fusion features; and comparing the safety risk index with a preset risk threshold value to judge whether a safety early warning response condition is met, and if so, triggering an early warning response protocol. According to the invention, through multi-modal information fusion, a space-time attention mechanism, space-time correlation analysis and adaptive risk assessment, the intelligent level and early warning accuracy of power operation safety supervision are improved.

Description

Technical Field

[0001] This invention relates to the field of power operation image recognition technology, and more specifically, to a power operation safety early warning method and system based on a spatiotemporal attention mechanism. Background Technology

[0002] With the rapid development of the power industry, safety issues in power operations have become increasingly prominent. The power operation environment is complex, involving various hazardous factors such as high-voltage electrical equipment and working at heights, posing multiple safety risks to workers, including electric shock and falls. Statistics show that human error accounts for a disproportionately large share of safety accidents in the power industry. Current power operation safety supervision mainly relies on manual inspections and post-event analysis, which easily overlooks the integration and utilization of multimodal information, making it difficult to effectively establish the temporal characteristics and spatial correlations of hazardous behaviors, and lacking adaptive adjustment capabilities. This results in significant regulatory blind spots and delayed responses, making it difficult to adapt to diverse operating scenarios.

[0003] Therefore, how to integrate multimodal information to support an adaptive risk assessment-based power operation safety early warning method has become an urgent technical problem to be solved. Summary of the Invention

[0004] This invention provides a method and system for early warning of power operation safety based on a spatiotemporal attention mechanism, which solves the technical problem in the prior art of how to integrate multimodal information to support adaptive risk assessment.

[0005] This invention provides a method and system for power operation safety early warning based on a spatiotemporal attention mechanism, comprising:

[0006] Firstly, a power operation safety early warning method based on a spatiotemporal attention mechanism includes:

[0007] Acquire video data streams from the power operation site and dynamic operation parameter data streams from the operators;

[0008] Based on the preset time window length and sliding step, the video data stream and dynamic operation parameter data stream are segmented to obtain several video segments and corresponding dynamic operation parameter segments.

[0009] Feature extraction is performed on several video segments one by one to obtain the spatiotemporal features of the video segments.

[0010] Feature extraction is performed on several dynamic operation parameter segments one by one to obtain the time-series features of dynamic operation parameters;

[0011] The spatiotemporal features of video clips and the temporal features of dynamic operation parameters are input into the spatiotemporal attention fusion model, and the corresponding fusion features are obtained.

[0012] The fused features are used to obtain the security risk index at the current moment through a pre-trained security risk assessment model;

[0013] The system compares the safety risk index with a preset risk threshold to determine whether the safety warning response conditions are met. If they are met, the warning response protocol is triggered.

[0014] Furthermore, based on a preset time window length and sliding step size, the video data stream and dynamic job parameter data stream are segmented to obtain several video segments and corresponding dynamic job parameter segments, including:

[0015] The video data stream represents video footage of power operation sites captured by multi-view cameras;

[0016] Based on the type of power operation, determine the corresponding time window length and sliding step from the preset time window length mapping table;

[0017] According to the determined time window length and sliding step size, the video data stream and dynamic operation parameter data stream are processed by sliding window segmentation to obtain several video segments and several dynamic operation parameter segments that overlap in time.

[0018] The dynamic operation parameter data stream includes behavioral parameters, dynamic operation parameters, reaction parameters, and body posture parameters of the power workers; the dynamic operation parameter data stream includes at least one of heart rate, blood pressure, body temperature, blood oxygen saturation, and electroencephalogram (EEG).

[0019] Furthermore, feature extraction is performed on each of the video segments to obtain the spatiotemporal features of the video segments, including:

[0020] Feature extraction is performed on the video segments to obtain initial video features;

[0021] The initial video features are input into the self-attention model to calculate the spatial correlation between different regions in the video segment, thereby obtaining the spatial correlation features;

[0022] The spatial correlation feature maps of different levels are superimposed and fused from top to bottom to form a multi-scale fused feature map.

[0023] A region proposal operation is performed on the multi-scale fused feature map to accurately identify small target feature maps and obtain video spatiotemporal features;

[0024] Based on the spatiotemporal features of the video, the wearing status of safety protective equipment and dangerous operation behaviors of power workers are accurately identified, and the behavior identification results are obtained.

[0025] Based on the behavior recognition results and video spatiotemporal features, a security risk assessment feature vector containing spatiotemporal information and behavioral features is generated.

[0026] Furthermore, feature extraction is performed on each segment of the dynamic operation parameter to obtain the temporal features of the dynamic operation parameters, including:

[0027] Extract the baseline and offset segments from the current work data stream of the power workers;

[0028] The dynamic changes of each parameter point in the baseline segment at different timestamps are obtained to obtain the primary deviation;

[0029] When the primary deviation is greater than or less than the preset parameter threshold range, the relevant parameter points are marked as the first abnormal points;

[0030] Record the original timestamps of each parameter point in the offset state segment;

[0031] Based on the temporal matching relationship, the original timestamp is mapped to the target timestamp of the reference state segment;

[0032] Obtain the secondary deviation between the offset fragment parameter point and the target timestamp parameter value;

[0033] When the secondary deviation is greater than or less than the preset parameter threshold range, the parameter points involved are marked as the second abnormal points.

[0034] Based on the ratio of the number of anomalies in the first and second anomalies to the total number of parameter points, the anomaly degree of dynamic operation parameters is quantified; the anomaly degree of dynamic operation parameters is integrated with the temporal change characteristics to construct a dynamic parameter temporal feature vector containing anomaly detection information.

[0035] Multi-scale decomposition of the time-series feature vectors yields time-series feature components in different frequency domains.

[0036] The frequency domain feature components are weighted and fused for processing to obtain the normalized time-series features of dynamic operation parameters.

[0037] Furthermore, the spatiotemporal features of the video clips and the temporal features of the dynamic operation parameters are input into the spatiotemporal attention fusion model, and the corresponding fused features are obtained, including:

[0038] The spatiotemporal attention fusion model includes a temporal attention processing layer and a spatial attention processing layer.

[0039] The attention weights of each time frame in the security risk assessment feature vector are obtained by using a temporal attention processing layer. The security risk assessment feature vector is then weighted to obtain temporally weighted video features.

[0040] The attention weights of each parameter in the temporal features of dynamic operation parameters are obtained by using a spatial attention processing layer. The temporal features of dynamic operation parameters are then weighted to obtain spatially weighted dynamic operation parameter features.

[0041] The time-weighted video features and spatially weighted dynamic operation parameter features are fused across modal features to obtain fused features.

[0042] Furthermore, the time-weighted video features and spatially weighted dynamic operation parameter features are fused across modal features to obtain fused features, including:

[0043] Obtain the cross-correlation matrix between time-weighted video features and spatially weighted dynamic job parameter features;

[0044] Based on the cross-correlation matrix, feature alignment and fusion are performed to obtain cross-modal fused features;

[0045] The cross-modal fusion features are subjected to a nonlinear transformation to obtain the final fusion features.

[0046] Furthermore, the fused features are used to obtain the current-moment security risk index through a pre-trained security risk assessment model, including:

[0047] Acquire environmental data and power equipment status data from the power operation site to construct a multi-source dataset for the operation.

[0048] The environmental data includes temperature, humidity, wind speed, and gas concentration; the power equipment status data includes equipment operating temperature, vibration frequency, and current and voltage parameters.

[0049] By combining multi-source datasets of the task with fusion features, a security risk assessment model for multi-source data fusion is constructed.

[0050] A correlation analysis was performed on the multi-source datasets of the assignments, and an assignment risk warning index was generated using a combination of data-driven and knowledge-driven methods.

[0051] The fused features are input into the pre-trained safety risk assessment model, and combined with the operation risk warning index, the safety risk index at the current moment is obtained.

[0052] A safety risk level classification mechanism is established based on the multi-source dataset of the operation, dividing the power operation risk into multiple levels, and formulating corresponding early warning information and handling strategies for different risk levels;

[0053] Based on the different levels of the current safety risk index, corresponding early warning information and response strategies will be pushed out through various means.

[0054] Furthermore, the safety risk index is compared with a preset risk threshold to determine whether the safety warning response conditions are met. If the safety warning response conditions are met, the warning response protocol is triggered, including:

[0055] The current safety risk index is compared with preset multi-level risk thresholds to determine the current risk level.

[0056] If the security risk index exceeds the preset threshold, the corresponding level of security warning response will be triggered.

[0057] If the security risk index is greater than the high-risk threshold, a high-level security alert will be triggered, and a first alert message containing the results of high-risk behavior identification will be output.

[0058] If the security risk index is greater than the medium risk threshold and less than or equal to the high risk threshold, a medium-level security warning is triggered, and a second warning message containing the results of medium-risk behavior identification is output.

[0059] If the security risk index is greater than the low risk threshold and less than or equal to the medium risk threshold, a low-level security warning is triggered, and a third warning message containing the low-risk behavior identification results is output.

[0060] If the safety risk index is less than or equal to the low-risk threshold, a safety warning will not be triggered.

[0061] Furthermore, acquire historical video data streams and dynamic operation parameter data streams from multiple safety warning responses;

[0062] Statistical analysis and pattern mining are performed on historical video data streams and dynamic operation parameter data streams to identify risk patterns and key risk factors that frequently trigger safety warnings.

[0063] A risk feature matrix is ​​constructed by identifying risk patterns and key risk factors; the risk feature matrix establishes a nonlinear mapping relationship between environmental factors and safety risks through a safety risk assessment model.

[0064] Based on the nonlinear mapping relationship, identify the risk evolution pattern under different environmental conditions, and determine the weight coefficients and threshold ranges of various risk factors;

[0065] Based on the combined characteristics of historical high-frequency safety warning accident data, operation type, environmental parameters and personnel status, the weighting coefficients and threshold ranges are dynamically adjusted to construct safety risk judgment standards and form an adaptive multi-level risk warning mechanism.

[0066] The parameters and structure of the pre-trained security risk assessment model are tuned and optimized based on an adaptive multi-level risk early warning mechanism.

[0067] The optimized security risk assessment model is redeployed into the system to achieve accurate and efficient security risk assessment.

[0068] Secondly, a power operation safety early warning system based on a spatiotemporal attention mechanism is provided for executing a power operation safety early warning method based on a spatiotemporal attention mechanism, comprising:

[0069] The data acquisition module is used to acquire video data streams from the power operation site and dynamic operation parameter data streams from the operators.

[0070] The data segmentation processing module is used to segment the video data stream and the dynamic operation parameter data stream based on a preset time window length and sliding step size, so as to obtain several video segments and several corresponding dynamic operation parameter segments.

[0071] The video feature extraction module is used to extract features from several video segments one by one to obtain the spatiotemporal features of the video segments.

[0072] The dynamic operation parameter feature extraction module is used to extract features from several dynamic operation parameter segments one by one to obtain the time-series features of dynamic operation parameters.

[0073] The spatiotemporal attention fusion model is used to input the spatiotemporal features of video clips and the temporal features of dynamic operation parameters into the spatiotemporal attention fusion model, and then fuse them to obtain the fused features.

[0074] The security risk assessment module is used to obtain the security risk index at the current moment by using the fused features through a pre-trained security risk assessment model.

[0075] The safety warning module is used to compare the safety risk index with the preset risk threshold to determine whether the safety warning response conditions are met. If they are met, the warning response protocol is triggered.

[0076] The beneficial effects of this invention are as follows: By fusing video data streams and dynamic operation parameter data streams, this invention achieves effective integration of multimodal information. By comprehensively considering the physiological state (heart rate, blood pressure, body temperature, etc.) and behavioral characteristics of the workers, it improves the accuracy and comprehensiveness of safety risk identification. Multimodal information fusion can perceive changes in the workers' state from different dimensions, effectively compensating for the limitations of a single information source.

[0077] Secondly, it effectively focuses on critical moments and important areas, improving the accuracy of feature extraction in complex power operation scenarios. By calculating the attention weights at different times and spatial locations, it automatically identifies the most critical information for safety risk assessment, enabling the system to accurately identify dangerous behaviors even in the presence of interference factors such as changes in lighting, occlusion, and multiple targets.

[0078] Furthermore, by modeling the temporal characteristics and spatial correlations of hazardous behaviors in power operations, the system considers not only the current state but also historical information for comprehensive judgment, while analyzing the correlations between operational behaviors in different spatial locations. This enables the system to identify potential safety risks in advance and achieve preventative safety warnings, rather than simply responding after the fact.

[0079] Finally, by constructing an adaptive multi-level risk early warning mechanism, the weighting coefficients and threshold ranges can be dynamically adjusted based on historical data. This adaptive mechanism can automatically optimize evaluation parameters according to different working environments, job types, and personnel statuses, effectively reducing false alarms and missed alarms, and improving the practicality and reliability of the early warning system.

[0080] In summary, this invention improves the intelligence level and early warning accuracy of power operation safety supervision, and has significant technical value and application prospects. Attached Figure Description

[0081] Figure 1 This is a schematic flowchart of a power operation safety early warning method based on a spatiotemporal attention mechanism provided in an embodiment of the present invention;

[0082] Figure 2 This is a schematic diagram of a power operation safety early warning system module based on a spatiotemporal attention mechanism provided in an embodiment of the present invention;

[0083] Figure 3 This is a schematic diagram of data collection for a power operation safety early warning system based on a spatiotemporal attention mechanism, provided in an embodiment of the present invention. Detailed Implementation

[0084] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0085] At least one embodiment of the present invention discloses a power operation safety early warning method and system based on a spatiotemporal attention mechanism, comprising:

[0086] like Figure 1 As shown, a power operation safety early warning method based on a spatiotemporal attention mechanism includes the following steps:

[0087] Step 1: Obtain video data streams from the power operation site and dynamic operation parameter data streams from the operators.

[0088] In this embodiment, the video data stream at the power operation site is acquired in real time by multiple camera devices deployed at the site, and may include high-definition video streams from multiple perspectives. The dynamic operation parameter data stream is acquired by smart devices worn by power workers (such as smart bracelets, smartwatches, smart helmets, etc.), including dynamic parameter data such as heart rate, blood pressure, body temperature, blood oxygen saturation, electroencephalogram (EEG), electromyography (EMG) signals, motion acceleration, position coordinates, and posture angles.

[0089] The dynamic operation parameter data stream also includes behavioral parameters, reaction parameters, and body posture parameters. The behavioral parameters include the operator's location, movement trajectory, tool usage, and risk status of the work area. Specifically, the behavioral parameters include key indicators such as movement speed, operation frequency, dwell time, and movement amplitude. The reaction parameters include reaction time, decision delay, and emergency response speed. The body posture parameters include body posture angle, center of gravity shift, limb coordination, and fatigue indicators.

[0090] Specifically, the camera equipment can include fixed cameras and mobile cameras. Fixed cameras are installed at key locations at the power work site, while mobile cameras can be installed on the safety helmets or work clothes of the workers to provide first-person view of the work.

[0091] Step 2: Based on the preset time window length and sliding step size, the video data stream and dynamic operation parameter data stream are segmented to obtain several video segments and corresponding dynamic operation parameter segments.

[0092] In this embodiment, considering the different characteristics of different types of power operation tasks and the varying time characteristics of hazard occurrence, a mapping table between power operation types and time window lengths is established. Based on the current power operation type, the corresponding time window length and sliding step size are determined from the mapping table, and then the data stream is processed by sliding window segmentation.

[0093] Specifically, step 2 may include the following sub-steps:

[0094] Step 21: Determine the corresponding time window length and sliding step from the preset time window length mapping table according to the type of power operation.

[0095] In this embodiment, the preset time window length mapping table contains the optimal time window length and sliding step parameters corresponding to different types of power operations. For example, for high-voltage line maintenance operations, the time window length can be set to 30 seconds and the sliding step to 10 seconds; for substation equipment installation operations, the time window length can be set to 45 seconds and the sliding step to 15 seconds. These parameter values ​​can be determined through historical data analysis and expert experience, or they can be automatically optimized through machine learning methods.

[0096] Step 22: According to the determined time window length and sliding step size, perform sliding window segmentation on the video data stream and dynamic operation parameter data stream to obtain several video segments and several dynamic operation parameter segments that overlap in time.

[0097] In this embodiment, a sliding window technique is used to divide a continuous data stream into a series of overlapping data segments. The existence of the overlapping parts allows the system to capture cross-segment behavior and dynamic job parameter state changes, avoiding the loss of critical information at segment boundaries.

[0098] Step 3: Perform feature extraction processing on several video segments one by one to obtain the spatiotemporal features of several video segments.

[0099] In this embodiment, the pre-trained video feature extraction model employs an advanced 3D convolutional neural network structure, capable of simultaneously capturing spatial and temporal information from the video. This model was pre-trained on a large-scale power operation safety video dataset and can effectively identify various actions and scene features during power operations.

[0100] Specifically, step 3 may include the following sub-steps:

[0101] Step 31: Extract features from the video clips to obtain initial video features.

[0102] In this embodiment, an improved 3D convolutional neural network is used to extract features from video segments. The network input is a video segment with shape [T, H, W, C], where T represents the number of frames, H and W represent the frame height and width, respectively, and C represents the number of channels. After processing through multiple layers of 3D convolution, pooling, and nonlinear activation functions, the initial video feature representation is obtained.

[0103] Step 32: Input the initial video features into the self-attention model to calculate the spatial correlation between different regions in the video segment and obtain the spatial correlation features.

[0104] In this embodiment, the self-attention model employs a self-attention mechanism similar to that in the Transformer architecture, enabling it to model long-distance dependencies between different regions in a video. By calculating the attention score between each location and all other locations in the feature map, the self-attention model can capture the interactions between key areas (such as high-voltage equipment, work tools, and personnel postures) in power operation scenarios.

[0105] Step 33: Overlay and fuse spatial correlation feature maps of different levels from top to bottom to form a multi-scale fused feature map.

[0106] In this embodiment, a Feature Pyramid Network (FPN) structure is used to fuse spatial correlation feature maps at different levels. Specifically, high-level semantic features are upsampled and fused with low-level detail features to form a fused feature map containing multi-scale information, effectively improving the detection capability for targets at different scales.

[0107] Step 34: Perform region proposal operation on the multi-scale fused feature map to accurately identify small target feature maps and obtain video spatiotemporal features.

[0108] In this embodiment, a Region Proposal Network (RPN) is used to process the multi-scale fused feature map to generate candidate region proposals. Through post-processing operations such as Non-Maximum Suppression (NMS), small target features in the video, such as security equipment and tools, are accurately identified, ultimately yielding video spatiotemporal features containing rich spatiotemporal information.

[0109] Step 4: Perform feature extraction processing on several dynamic operation parameter segments one by one to obtain the time-series features of dynamic operation parameters.

[0110] In this embodiment, the dynamic operation parameter feature extraction model employs a deep learning architecture specifically designed for multi-dimensional time-series data. This architecture effectively captures subtle changes and trends in the dynamic operation parameter status of power workers, which are often early indicators of potential hazards. The model is specifically optimized for behavioral parameters (movement speed, operation frequency, dwell time, and amplitude of movement), reaction parameters (reaction time, decision delay, and emergency response speed), and postural parameters (body posture angles, center of gravity shift, limb coordination, and fatigue indicators), enabling accurate identification of abnormal patterns related to safety risks.

[0111] Specifically, step 4 may include the following sub-steps:

[0112] Step 41: Extract the reference state segment and the offset state segment from the current operation data stream of the power workers.

[0113] In this embodiment, the baseline state segment refers to the dynamic operation parameter data segment of the operator under normal working conditions, typically characterized by a stable state where various physiological indicators and behavioral parameters are within the normal range. The deviation state segment refers to the dynamic operation parameter data segment of the operator under abnormal or potentially dangerous conditions, characterized by certain parameters significantly deviating from the normal range. By comparing and analyzing the baseline and deviation state segments, abnormal changes in the operator's condition can be effectively identified, providing an important basis for safety risk early warning. The extraction of the baseline state segment is based on statistical analysis of historical normal operation data to establish a normal distribution model for each parameter; the identification of the deviation state segment is achieved by real-time monitoring of parameter changes, automatically marking a parameter deviation as a deviation state when it is detected.

[0114] Step 42: Obtain the dynamic changes of each parameter point in the reference state segment at different timestamps to obtain the primary deviation.

[0115] In this embodiment, the primary deviation refers to the degree of fluctuation of each parameter within the baseline segment over time, reflecting the natural variation pattern of the physiological and behavioral parameters of the operator under normal conditions. By calculating the difference between each parameter point and consecutive timestamps, the dynamic trend of parameter change can be obtained.

[0116] The formula for calculating the primary deviation is: ;in, This indicates the initial deviation value. The parameter value represents the current time t. This represents the parameter value at subsequent increment time t+n, where n represents the increment time. Analysis of primary bias helps establish an individualized baseline for normal variation, providing a reference standard for subsequent anomaly detection. The establishment of this dynamic baseline considers the influence of individual differences and environmental factors, improving the accuracy of anomaly detection.

[0117] Step 43: When the primary deviation is greater than or less than the preset parameter threshold range, the relevant parameter points are marked as the first abnormal points.

[0118] In this embodiment, the preset parameter threshold range is a normal variation range derived from statistical analysis of a large amount of historical data. Specifically, for different types of dynamic operation parameters, the preset parameter threshold ranges are as follows: For behavioral parameters, the movement speed threshold range is [0.2m / s, 2.5m / s], the operation frequency threshold range is [5 times / minute, 45 times / minute], the dwell time threshold range is [2 seconds, 30 seconds], and the action amplitude threshold range is [10%, 90%]; For reaction parameters, the reaction time threshold range is [150ms, 800ms], the decision delay threshold range is [200ms, 1500ms], and the emergency response speed threshold range is [0.5 seconds, 3 seconds]. [Seconds]; Among the body posture parameters, the threshold range for body posture angle is [-30°, +30°], the threshold range for center of gravity offset is [-8cm, +8cm], the threshold range for limb coordination is [60%, 95%], and the threshold range for fatigue index is [0%, 25%]; Among the physiological parameters, the threshold range for heart rate is [60 beats / minute, 100 beats / minute], the threshold range for blood pressure is systolic blood pressure [90mmHg, 140mmHg] and diastolic blood pressure [60mmHg, 90mmHg], the threshold range for body temperature is [36.0℃, 37.5℃], the threshold range for blood oxygen saturation is [95%, 100%], and the threshold range for electroencephalogram (EEG) is [75%, 125% of the baseline value].

[0119] The first anomaly marking mechanism can identify abnormal fluctuations in the baseline segment, which may indicate potential changes in the operator's condition. The identification of the first anomaly employs an adaptive threshold mechanism, which can dynamically adjust the threshold range according to different working environments and individual characteristics. For example, in high-temperature environments, the normal range of body temperature and heart rate will be correspondingly expanded; in precision operation tasks, the thresholds for reaction time and limb coordination will be more stringent.

[0120] Step 44: Record the original timestamps of each parameter point in the offset state segment.

[0121] In this embodiment, the original timestamp records the precise acquisition time of each parameter data point in the offset segment, providing a time reference for subsequent time series matching and comparative analysis. The timestamp accuracy reaches the millisecond level, ensuring accurate capture of rapidly changing physiological and behavioral parameters. The original timestamp record also includes metadata information such as parameter type identifier, acquisition device identifier, and data quality identifier, providing support for data traceability and quality control.

[0122] Step 45: Based on the timing matching relationship, map the original timestamp to the target timestamp of the reference state segment.

[0123] In this embodiment, the time-series matching relationship refers to the mapping rule that aligns the time series of the offset state segment with the time series of the reference state segment. Since the reference state and offset state segments may be acquired at different time periods, time-series matching is needed to find the corresponding time points for comparison. The mapping process employs the Dynamic Time Warping (DTW) algorithm, which can handle issues such as inconsistent time series lengths and time offsets. The target timestamp is the time point in the reference state segment corresponding to the offset state segment; time-series matching ensures the validity and accuracy of the comparison.

[0124] Step 46: Obtain the secondary deviation between the offset fragment parameter point and the target timestamp parameter value.

[0125] In this embodiment, the secondary deviation refers to the difference between the parameter value in the offset state segment and the parameter value at the corresponding time point in the reference state segment, reflecting the degree to which the operator's state changes from normal to abnormal.

[0126] The formula for calculating secondary deviation is: ,in, Indicates secondary deviation, The parameter value represents the original timestamp T in the offset segment. This represents the parameter value corresponding to the target timestamp in the baseline state segment. This represents the target timestamp corresponding to the time of the baseline state segment and the offset state segment obtained through temporal matching mapping. The magnitude of the secondary deviation directly reflects the degree of anomaly and is an important indicator for security risk assessment.

[0127] Step 47: When the secondary deviation is greater than or less than the preset parameter threshold range, the parameter points involved are marked as second anomalies.

[0128] In this embodiment, the preset threshold range for marking the second anomaly is determined based on the comparison between the secondary deviation and the preset threshold range. The preset threshold range is set according to different types of dynamic operation parameters: For behavioral parameters, the abnormal threshold for movement speed is set to ±0.5m / s, the abnormal threshold for operation frequency is set to ±30%, the abnormal threshold for dwell time is set to ±10 seconds, and the abnormal threshold for movement amplitude is set to ±25%; For reaction parameters, the abnormal threshold for reaction time is set to ±200ms, the abnormal threshold for decision delay is set to ±500ms, and the abnormal threshold for emergency response speed is set to ±1 second; For body posture parameters, the abnormal threshold for body posture angle is set to ±15°, the abnormal threshold for center of gravity offset is set to ±5cm, the abnormal threshold for limb coordination is set to ±20%, and the abnormal threshold for fatigue index is set to ±15%; For physiological parameters, the abnormal threshold for heart rate is set to ±20 beats / minute, the abnormal threshold for blood pressure is set to systolic blood pressure ±20mmHg and diastolic blood pressure ±15mmHg, the abnormal threshold for body temperature is set to ±1.5℃, the abnormal threshold for blood oxygen saturation is set to ±3%, and the abnormal threshold for electroencephalogram (EEG) is set to ±25%. The second point of abnormality marking also takes into account the duration of the abnormality. Only abnormalities that last for more than a preset time threshold (5 seconds for behavioral parameters, 3 seconds for reaction parameters, 8 seconds for body posture parameters, and 10 seconds for physiological parameters) will be marked, thus avoiding interference from instantaneous noise.

[0129] It can identify parameter points in offset segments that significantly deviate from the normal state. The identification of the second anomaly takes into account the differences in parameter types, using different threshold standards for different parameter types. The marking of the second anomaly also considers the duration of the anomaly; only anomalies lasting longer than a certain time are marked, avoiding interference from transient noise.

[0130] Step 48: Quantify the anomaly degree of dynamic operation parameters based on the ratio of the number of anomalies of the first and second anomalies to the total number of parameter points.

[0131] In this embodiment, the anomaly degree of dynamic operation parameters is a comprehensive index that reflects the degree of abnormality in the overall condition of the operators. The calculation formula is: ,in, Indicating the degree of anomaly, w1 and w2 are the weight coefficients for the first and second anomalies, respectively, with w2 typically greater than w1. and These represent the number of outliers for the first and second outliers, respectively. This represents the total number of parameter points, as the second outlier indicates a more severe anomaly. The anomaly score ranges from 0 to 1, with higher values ​​indicating a greater degree of anomaly. This quantification method transforms complex, multi-dimensional parameter anomalies into a single, comparable indicator, facilitating subsequent risk assessment and decision-making.

[0132] Step 49: Integrate the anomaly degree of dynamic operation parameters with temporal change characteristics to construct a dynamic parameter temporal feature vector containing anomaly detection information.

[0133] In this embodiment, the temporal variation features include time-domain features such as the trend, periodicity, and abrupt change points of parameter changes. By fusing anomaly information with temporal variation features, the constructed feature vector not only contains information on the degree of parameter anomaly but also information on the temporal pattern of anomaly occurrence.

[0134] The fusion process employs a weighted concatenation method: Feature vector = [G × α, Temporal variation features × β], where α and β represent the balancing factors for anomaly severity and temporal variation features, respectively. This fusion method allows the feature vector to simultaneously reflect both the severity and temporal characteristics of the anomaly, providing richer information for subsequent security risk assessments.

[0135] Step 410: Perform multi-scale decomposition on the time-series feature vector to obtain time-series feature components in different frequency domains.

[0136] In this embodiment, multi-scale decomposition employs wavelet transform technology to decompose the time-series feature vector into sub-signals with different frequency components. Low-frequency components reflect the long-term trend and slow changes of parameters, while high-frequency components reflect rapid fluctuations and sudden changes in parameters. Feature components in different frequency domains correspond to different types of safety risks: low-frequency anomalies may indicate fatigue accumulation or chronic health problems, while high-frequency anomalies may indicate sudden dangerous situations or stress responses.

[0137] Multiscale decomposition is represented as: ,in, D represents the multiscale decomposition coefficients. i (t) are the detail coefficients of the i-th layer frequency domain component, and A(t) are the approximation coefficients.

[0138] Step 411: Weighted fusion of each frequency domain feature component is performed to obtain the normalized dynamic operation parameter time series features.

[0139] In this embodiment, the weighted fusion process assigns weights based on the contribution of different frequency domain components to security risk prediction. The weights are determined based on statistical analysis of historical data and feature importance assessment using machine learning methods.

[0140] The fusion formula is: , where w iw is the weighting coefficient of the detail coefficients of the i-th layer frequency domain component. a These are the weighting coefficients of the approximate components. These are the coefficients of the fusion term. Normalization is performed using the Z-score standardization method to ensure that feature components at different scales have the same numerical range during fusion. The resulting temporal features of the dynamic job parameters exhibit good numerical stability and interpretability, providing high-quality input for subsequent spatiotemporal attention fusion.

[0141] Step 5: Input the spatiotemporal features of the video clips and the temporal features of the dynamic operation parameters into the spatiotemporal attention fusion model, and obtain the fused features accordingly.

[0142] In this embodiment, the spatiotemporal attention fusion model adaptively fuses video features and dynamic operation parameter features through attention mechanisms in both time and space dimensions, highlighting the spatiotemporal information that is most critical for safety risk assessment.

[0143] Specifically, step 5 may include the following sub-steps:

[0144] Step 51: The spatiotemporal attention fusion model includes a temporal attention processing layer and a spatial attention processing layer; the attention weights of each time frame in the security risk assessment feature vector are obtained using the temporal attention processing layer, and the security risk assessment feature vector is weighted to obtain time-weighted video features.

[0145] In this embodiment, the temporal attention processing layer first calculates the importance score of each time frame in the video sequence, and then weights the video features based on these scores.

[0146] The formula for calculating temporal attention weights is: ;

[0147] in, This represents the time attention weighting coefficient. It is a feature of the t-th time frame. , These are the first and second learnable time parameters, respectively. and These are the learnable third and fourth time-weight configuration coefficients, respectively. The softmax function ensures that the sum of the weights across all time frames is 1. tanh is the hyperbolic tangent activation function, used to map the input value to the (-1,1) interval, providing non-linear transformation capabilities.

[0148] Through this mechanism, the system can automatically focus on the most critical moments in a video sequence that indicate the safety risks, such as the instant a dangerous action occurs, while ignoring relatively unimportant background frames.

[0149] Step 52: Use the spatial attention processing layer to obtain the attention weights of each parameter in the time series features of dynamic operation parameters, and perform weighted processing on the time series features of dynamic operation parameters to obtain spatially weighted dynamic operation parameter features.

[0150] In this embodiment, the spatial attention processing layer calculates the importance weights of different dynamic operation parameters, because the contribution of each dynamic operation parameter to risk assessment varies under different power operation scenarios.

[0151] Feature extraction is performed on the time-series characteristics of dynamic operation parameters to obtain the parameter feature matrix P=[p1,p2,...,p f ], where p f Let f represent the feature vector of the f-th dynamic task parameter, where f is the total number of parameters; then, the correlation matrix between parameters is calculated using a multi-head attention mechanism. : , , ;in, , and These are the weight matrices for queries, key-value pairs, and numeric values, respectively. Represents a numerical matrix.

[0152] Next, the attention score matrix is ​​calculated. : ;in, is the dimension of the key-value vector, used for scaling to prevent gradient vanishing. The key-value matrix is ​​represented by the feature matrix P of the dynamic job parameters, passed through the weight matrix. Obtained by transformation.

[0153] Furthermore, by combining the contextual information C of the work scenario, attention weights are adjusted through a gating mechanism: ;in, For context weight matrix, The bias term is represented by [,], which indicates the feature concatenation operation, and C represents the context information of the job scenario. Let be the gating weight of the i-th parameter, and sigmoid be the sigmoid activation function.

[0154] Finally, the final spatial attention weights: ;in, S represents the final spatial attention weight of the i-th dynamic task parameter. i This represents the attention score for parameter i. This represents the numerical vector of the j-th parameter, where j represents all dynamic job parameters.

[0155] Through this mechanism, the system can adaptively allocate importance weights to different dynamic operation parameters according to the current operation scenario, highlighting the most critical parameter dimensions for judging the current safety risks.

[0156] For example, changes in blood pressure and heart rate may be more critical in high-altitude operations; body temperature and sweating rate may be more important in high-temperature environments; reaction time, decision delay, and limb coordination may be more critical in precision operations; and fatigue indicators and changes in movement speed may be more important in long-term operations. Spatial attention weights are calculated through a similar mechanism, but the focus is on the relative importance of different dynamic operation parameters.

[0157] Step 53: Perform cross-modal feature correspondence fusion of the time-weighted video features and the spatially weighted dynamic operation parameter features to obtain fused features.

[0158] In this embodiment, cross-modal feature mapping fusion is a crucial step, requiring the processing of heterogeneous data from different modalities (visual and dynamic task parameters). This step can be further subdivided into the following processes:

[0159] Step 531: Obtain the cross-correlation matrix between time-weighted video features and spatially weighted dynamic operation parameter features.

[0160] In this embodiment, the formula for calculating the cross-correlation matrix M is: Where V represents time-weighted video features, P represents spatially weighted dynamic job parameter features, and d represents the feature dimension used for normalization. The cross-correlation matrix captures the relationship between video features and dynamic job parameter features, providing a basis for subsequent feature alignment.

[0161] Step 532: Based on the cross-correlation matrix, perform feature alignment and fusion to obtain cross-modal fused features.

[0162] In this embodiment, a cross-correlation matrix is ​​used for bidirectional feature mapping:

[0163] (Mapping dynamic operation parameter features to video feature space);

[0164] (Mapping video features to the feature space of dynamic job parameters)

[0165] Then, the original features are concatenated with the mapped features and a weighted sum is performed:

[0166] ;

[0167] ;

[0168] in, and These are learnable parameters that control the relative importance of the original features and the mapped features. This represents the features of the fused video. V represents the fused dynamic job parameter features, P represents the spatially weighted dynamic job parameter features, V' represents the dynamic job parameter features mapped to the video feature space, and P' represents the video features mapped to the dynamic job parameter feature space.

[0169] Finally, and The connection yields cross-modal fusion features.

[0170] Step 533: Perform a nonlinear transformation on the cross-modal fusion features to obtain the final fusion features.

[0171] In this embodiment, a multilayer perceptron (MLP) is used to perform a nonlinear transformation on the cross-modal fusion features to enhance the expressive power index of the features. : ; where [,] represents the feature connection operation, and the MLP contains the ReLU activation function of two fully connected layers.

[0172] Step 6: Obtain the security risk index at the current moment by using the fused features through a pre-trained security risk assessment model.

[0173] In this embodiment, the safety risk assessment model employs a multilayer perceptron structure, receiving fused features as input and outputting a safety risk index between 0 and 1. This model has been pre-trained on a large amount of labeled power operation safety event data, enabling it to accurately assess the safety risk level of the current scenario.

[0174] Specifically, step 6 may include the following sub-steps:

[0175] Step 61: Obtain environmental data and power equipment status data at the power operation site to construct a multi-source dataset for the operation.

[0176] In this embodiment, environmental data includes temperature, humidity, wind speed, and rainfall; power equipment status data includes equipment operating temperature, vibration frequency, and current and voltage parameters. This multi-source data provides more comprehensive environmental background information for safety risk assessment.

[0177] Environmental monitoring equipment deployed at the work site collects environmental parameter data in real time. Gas concentration monitoring includes the detection of concentrations of harmful gases such as SF6, CO, and H2S, providing important environmental indicators for operational safety.

[0178] Step 62: Combine the multi-source datasets of the job with the fusion features to construct a security risk assessment model for multi-source data fusion.

[0179] In this embodiment, a hierarchical data fusion architecture is adopted to preprocess, synchronize and align data from different sources, and then construct a comprehensive security risk assessment model through feature layer fusion and decision layer fusion.

[0180] Step 63: Perform correlation analysis on the multi-source datasets of the assignments, and generate an assignment risk warning index by combining data-driven and knowledge-driven methods.

[0181] In this embodiment, statistical analysis and machine learning methods are used to identify the correlation between different environmental factors and safety risks, and combined with the knowledge of experts in the field of power operations, a comprehensive operation risk warning index is generated.

[0182] Step 64: Input the fused features into the pre-trained safety risk assessment model, and combine them with the operation risk warning index to obtain the safety risk index at the current moment.

[0183] In this embodiment, the security risk assessment model The forward computation process is as follows:

[0184] ;in, It is a fusion feature, representing an array of feature parameters for a node. , These are the third and fourth evaluation parameters during model training. and The model training uses the coefficients of the third and fourth configuration terms. σ is the sigmoid activation function, ensuring that the output risk index is between 0 and 1. A value closer to 1 indicates higher risk, and a value closer to 0 indicates lower risk.

[0185] The fusion features are based on feature vectors of environmental parameter nodes, equipment status nodes, personnel behavior nodes, and risk event nodes.

[0186] The complex relationships between nodes are modeled using a security risk assessment model.

[0187] The environmental parameter nodes include temperature T, humidity H, wind speed W, and gas concentration G. The feature vector R1 of the environmental parameter nodes is [T,H,W,G].

[0188] The equipment status node includes the operating temperature T. eq Vibration frequency V eq Current I, voltage U, etc., the feature vector R2 of the device state node is [T eq V eq ,I,U];

[0189] Personnel behavior nodes include movement speed V p Operating frequency F p Fatigue degree F t The feature vector R3 of the personnel behavior node is [V p ,F p ,F t ].

[0190] A data-driven approach is used for statistical learning: a risk prediction model is trained using historical accident data to identify high-risk pattern combinations; at the same time, a knowledge-driven approach is combined with expert rules in the field of power operation safety, such as a risk weight coefficient of 1.5 for the combination of "high temperature + high humidity + long-term operation".

[0191] Ultimately, the current operational risk warning index R t for: ;in, It is the data-driven weight of the xth risk factor. It is the normalized value of the i-th risk factor. These are the corresponding expert knowledge correction coefficients, and R1, R2, and R3 are the operational risk warning driving parameters. The index ranges from 0 to 1, with a higher value indicating a higher risk in the current operational environment.

[0192] Step 65: Establish a safety risk level classification mechanism based on the multi-source dataset of the operation, divide the power operation risk into multiple levels, and formulate corresponding early warning information and handling strategies for different risk levels.

[0193] In this embodiment, a four-level risk classification mechanism is established: low risk (0-0.3), medium risk (0.3-0.6), high risk (0.6-0.8), and extremely high risk (0.8-1.0), with each level corresponding to different early warning information and handling strategies.

[0194] Step 66: Based on the different levels of the current safety risk index, push out corresponding early warning information and response strategies through various means.

[0195] In this embodiment, the early warning information is pushed through various means, including audible and visual alarms, mobile phone vibration, large screen display in the monitoring center, and SMS notifications, to ensure that relevant personnel can receive the early warning information in a timely manner.

[0196] Step 7: Compare the safety risk index with the preset risk threshold to determine whether the safety warning response conditions are met. If the safety warning response conditions are met, the warning response protocol is triggered.

[0197] Step 7 may include the following sub-steps:

[0198] Step 71: Compare the current safety risk index with the preset multi-level risk thresholds to determine the current risk level.

[0199] In this embodiment, the current risk level is determined by comparing the current safety risk index with preset multi-level risk thresholds, providing a basis for subsequent early warning responses.

[0200] Step 72: If the security risk index exceeds the preset threshold, a corresponding level of security warning response will be triggered.

[0201] In this embodiment, a corresponding level of security warning response is triggered based on the different risk levels, including the generation and push of warning information and the execution of handling strategies.

[0202] Step 73: If the security risk index is greater than the high risk threshold, a high-level security warning is triggered, and a first warning message containing the high-risk behavior identification results is generated.

[0203] In this embodiment, high-level safety warnings are triggered simultaneously through multiple methods, including audible and visual alarms, mobile phone vibration, and large-screen display at the monitoring center, to ensure that supervisors can notice them immediately. The first warning message includes a detailed risk description, such as "Dangerous operation detected: Direct contact with high-voltage lines without wearing insulated gloves; please stop work immediately."

[0204] Step 74: If the safety risk index is greater than the medium risk threshold and less than or equal to the high risk threshold, a medium-level safety warning is triggered, and a second warning message containing the results of medium-risk behavior identification is generated.

[0205] In this embodiment, the intermediate-level safety warning is triggered via display at the monitoring center and mobile phone notification. The second warning information includes a risk description and suggested actions, such as "Improper operation detected: The safety belt is not properly fastened. Please adjust and continue working."

[0206] Step 75: If the security risk index is greater than the low risk threshold and less than or equal to the medium risk threshold, a low-level security warning is triggered, and a third warning message containing the low-risk behavior identification results is generated.

[0207] In this embodiment, low-level safety warnings are only recorded and displayed through the monitoring system. The third warning information includes reminders such as "Attention: Working hours have exceeded 4 hours, please take a break."

[0208] Step 76: If the safety risk index is less than or equal to the low risk threshold, a safety warning will not be triggered.

[0209] Step 8: Obtain historical video data streams and dynamic operation parameter data streams from multiple safety warning responses.

[0210] In this embodiment, the system continuously collects and stores historical video data streams and dynamic operation parameter data streams, providing a data foundation for subsequent pattern analysis and model optimization.

[0211] Specifically, step 8 may include the following sub-steps:

[0212] Step 81: Perform statistical analysis and pattern mining on the historical video data stream and dynamic operation parameter data stream to identify risk patterns and key risk factors that trigger safety warnings frequently.

[0213] In this embodiment, data mining techniques are used to analyze patterns and regularities in historical data to identify risk patterns that are likely to lead to safety warnings, such as specific combinations of work behaviors and specific environmental conditions. Key risk factors include, but are not limited to: worker fatigue, approach to hazardous areas, wearing of protective equipment, abnormal changes in physiological parameters, abnormal behavioral parameters (such as excessively fast or slow movement speed, abnormal operation frequency, excessive dwell time, excessively large movements), abnormal reaction parameters (such as prolonged reaction time, increased decision-making delay, decreased emergency response speed), and abnormal postural parameters (such as improper body posture angle, excessive center of gravity shift, decreased limb coordination, and increased fatigue index).

[0214] Step 82: Construct a risk feature matrix from the risk patterns and key risk factors; the risk feature matrix establishes a nonlinear mapping relationship between environmental factors and safety risks through a safety risk assessment model.

[0215] In this embodiment, the risk feature matrix is ​​a multi-dimensional matrix, containing risk features across multiple dimensions, including time, space, environment, and behavior. By employing deep learning methods, a complex nonlinear mapping relationship between environmental factors and safety risks is established, enabling more accurate prediction of safety risks under different environmental conditions.

[0216] Step 83: Identify the risk evolution patterns under different environmental conditions based on the nonlinear mapping relationship, and determine the weight coefficients and threshold ranges of various risk factors.

[0217] In this embodiment, by analyzing the evolution of risks, the weight coefficients and threshold ranges of different risk factors are dynamically adjusted to enable the risk assessment model to adapt to different operating environments and conditions. The weight coefficients are determined based on the correlation analysis between each risk factor and safety accidents in historical data, while the threshold ranges are determined based on statistical analysis results and expert experience.

[0218] Step 84: Based on the combined characteristics of historical high-frequency safety warning accident data, operation type, environmental parameters and personnel status, dynamically adjust the weight coefficients and threshold ranges to construct safety risk judgment standards and form an adaptive multi-level risk warning mechanism.

[0219] In this embodiment, the parameters of the risk assessment model are dynamically adjusted based on the results of historical data analysis to form a multi-level risk warning mechanism that can adapt to different scenarios. The adaptive mechanism includes: (1) parameter adjustment based on job type: different types of power operations have different risk characteristics, and the system automatically adjusts the corresponding weight coefficients according to the current job type; (2) threshold adjustment based on environmental conditions: under severe weather or special environmental conditions, the system will lower the warning threshold and improve the warning sensitivity; (3) personalized adjustment based on personnel status: considering the individual differences such as the experience level and health status of the operators, personalized risk assessment parameters are adjusted.

[0220] Step 85: Based on the adaptive multi-level risk warning mechanism, perform parameter tuning and structural optimization on the pre-trained security risk assessment model.

[0221] In this embodiment, feedback information from the adaptive early warning mechanism is used to continuously fine-tune the parameters and optimize the structure of the security risk assessment model, including adjusting the number of network layers, neurons, and activation functions. The optimization process employs reinforcement learning methods, with the goal of reducing early warning accuracy and false alarm rate.

[0222] Step 86: Redeploy the optimized security risk assessment model back into the system to achieve accurate and efficient security risk assessment.

[0223] In this embodiment, the optimized model is redeployed into the actual power operation safety early warning system to achieve continuous model improvement and performance enhancement. The deployment process includes steps such as model version management, A / B testing and verification, and canary release to ensure system stability and reliability.

[0224] This embodiment, by simultaneously analyzing video data and dynamic operation parameter data, combined with an innovative spatiotemporal attention fusion mechanism, can comprehensively capture safety risk factors in power operations. Compared with traditional methods, this method has the following advantages:

[0225] First, by introducing attention mechanisms in both temporal and spatial dimensions, the system can adaptively focus on the most relevant spatiotemporal information, significantly improving the accuracy of risk identification. The temporal attention mechanism enables the system to capture key moments in video sequences, while the spatial attention mechanism helps the system identify the dynamic combination of operational parameters that best indicates risk.

[0226] Secondly, cross-modal feature fusion technology effectively integrates video data and dynamic operation parameter data, enabling the system to simultaneously consider the external behavior and internal state of operators, thereby providing a more comprehensive assessment of safety risks. This multimodal analysis method can uncover potential risks that are difficult to identify with single-modal analysis.

[0227] Furthermore, the comparative analysis mechanism between the baseline and offset states can accurately identify abnormal changes in the operator's condition. Through dual detection of primary and secondary deviations, it effectively improves the accuracy and reliability of anomaly detection. This hierarchical detection mechanism can distinguish between normal fluctuations and genuine abnormal states, reducing false alarms.

[0228] Finally, the continuous learning mechanism based on historical data enables the system to continuously optimize and adapt, with the accuracy and relevance of warnings constantly improving over time. This self-improving characteristic ensures the system remains highly efficient in long-term use and can adapt to the characteristics of different power operation scenarios.

[0229] like Figures 2-3 As shown, a power operation safety early warning system based on a spatiotemporal attention mechanism includes:

[0230] The data acquisition module is used to acquire video data streams from the power operation site and dynamic operation parameter data streams from the operators.

[0231] The data segmentation processing module is used to segment the video data stream and the dynamic operation parameter data stream based on a preset time window length and sliding step size, so as to obtain several video segments and several corresponding dynamic operation parameter segments.

[0232] The video feature extraction module is used to extract features from several video segments one by one to obtain the spatiotemporal features of the video segments.

[0233] The dynamic operation parameter feature extraction module is used to extract features from several dynamic operation parameter segments one by one to obtain the time-series features of dynamic operation parameters.

[0234] The spatiotemporal attention fusion model is used to input the spatiotemporal features of video clips and the temporal features of dynamic operation parameters into the spatiotemporal attention fusion model, and then fuse them to obtain the fused features.

[0235] The security risk assessment module is used to obtain the security risk index at the current moment by using the fused features through a pre-trained security risk assessment model.

[0236] The safety warning module is used to compare the safety risk index with the preset risk threshold to determine whether the safety warning response conditions are met. If they are met, the warning response protocol is triggered.

[0237] Application example:

[0238] Application Example 1: Safety Early Warning for High-Voltage Transmission Line Maintenance Operations.

[0239] A power company employed the power operation safety early warning system based on the spatiotemporal attention mechanism of this invention during maintenance work on a 220kV high-voltage transmission line. Four 4K high-definition cameras were deployed at the work site, installed in the four corners of the work area, providing 360-degree all-around monitoring. Three maintenance personnel wore smart wearable devices to monitor dynamic work parameters in real time, including heart rate, blood pressure, body temperature, movement speed, operation frequency, reaction time, and body posture angles.

[0240] Based on the characteristics of high-voltage line maintenance work, the system automatically sets a time window length of 30 seconds and a sliding step of 10 seconds. During the operation, when maintenance worker A approached the live equipment, his heart rate surged from a normal 75 beats / minute to 110 beats / minute. Simultaneously, video monitoring showed an abnormal tilt in his body posture, with the center of gravity shift exceeding the safety threshold. The spatiotemporal attention fusion model immediately detected this abnormal pattern, fusing and analyzing key areas in the video (personnel position, live equipment, safety distance) with dynamic operation parameters (abnormal heart rate, abnormal posture), calculating a safety risk index of 0.85 (high risk).

[0241] The system immediately triggered a Level 2 early warning response, alerting on-site personnel with audible and visual alarms, and simultaneously sending a warning message to the monitoring center: "Maintenance personnel A is in a high-risk situation near tower No. 10. It is recommended to immediately stop work and check safety measures." Upon receiving the warning, the on-site safety officer immediately ordered maintenance personnel A to stop work. An inspection revealed that their safety belt buckle was loose. Thanks to the timely warning, a potential fall from height accident was successfully averted.

[0242] Application Example 2: Safety supervision of substation equipment installation operations.

[0243] During the installation of new equipment at a substation, the system of this invention was used for full-process safety monitoring. The work site involved the installation of a large transformer, and the working environment was complex, containing multiple hazards: high-voltage equipment, lifting machinery, and high-altitude operations. The system deployed 6 camera devices and environmental monitoring sensors, and all 5 workers wore smart wearable devices.

[0244] During the transformer hoisting process, the system detected a continuous increase in worker B's fatigue index, rising from a normal 0.3 to 0.7. Simultaneously, his reaction time increased from an average of 1.2 seconds to 2.8 seconds, and his operating frequency decreased significantly. Video analysis revealed that this worker exhibited inattentive behavior while directing the hoisting operation, with unclear hand gestures and abnormally prolonged pauses.

[0245] The spatiotemporal attention fusion model comprehensively analyzes video features and dynamic operation parameter characteristics to identify high-risk patterns of fatigued operation, with a safety risk index reaching 0.78. The system triggers a Level 1 warning, recommending that the person in question immediately rest and that a replacement supervisor be appointed. Simultaneously, the system automatically adjusts the monitoring frequency of operation parameters for other personnel, strengthening safety monitoring of the entire operation team.

[0246] Through early warning intervention, the work team promptly adjusted its personnel arrangements, preventing potential equipment damage and personnel injuries caused by fatigue. The operation was successfully completed without any safety incidents.

[0247] Application Example 3: Emergency safety assurance for power distribution network repair operations.

[0248] A power distribution network failure occurred in a certain area, requiring emergency repairs. Due to the nighttime emergency operation, environmental conditions were harsh, posing a high safety risk. The repair team utilized the mobile safety early warning system of this invention, comprising vehicle-mounted camera equipment, portable environmental monitoring equipment, and smart wearable devices.

[0249] During the emergency repair, the system continuously monitored the status of the four repair personnel. When repairman C was handling the faulty cable, the system detected an abnormal increase in his heart rate and blood pressure (heart rate rose from 80 beats / minute to 125 beats / minute, and blood pressure rose from 120 / 80 mmHg to 150 / 95 mmHg), and his body temperature also showed an upward trend. Video analysis showed that the person's range of motion increased significantly, and his limb coordination decreased, indicating a risk of operational error.

[0250] Meanwhile, environmental monitoring data showed that the electromagnetic field strength in the work area was close to the upper limit of the safety threshold. The spatiotemporal attention fusion model comprehensively analyzed abnormal personnel status, environmental risk factors, and work behavior characteristics to identify a composite risk pattern, with a safety risk index reaching 0.82.

[0251] The system immediately triggered a Level 3 warning (the highest level), requiring an immediate halt to operations and evacuation from the danger zone. Simultaneously, it automatically activated the emergency response protocol, notifying the dispatch center and medical rescue teams. On-site inspection confirmed a potential electrical leakage hazard in the area; continued operations could result in electric shock.

[0252] Thanks to the system's timely early warning and emergency response, a serious electrical accident was successfully averted, ensuring the safety of the repair personnel. After implementing appropriate safety measures, the repair team successfully completed the fault repair work.

[0253] The above three application examples fully demonstrate the practical application effects of the power operation safety early warning method and system based on spatiotemporal attention mechanism of the present invention in different power operation scenarios, and verify the feasibility, practicality and effectiveness of the technical solution.

[0254] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A power operation safety early warning method based on a spatiotemporal attention mechanism, characterized in that, include: Acquire video data streams from the power operation site and dynamic operation parameter data streams from the operators; Based on the preset time window length and sliding step, the video data stream and dynamic operation parameter data stream are segmented to obtain several video segments and corresponding dynamic operation parameter segments. Feature extraction is performed on several video segments one by one to obtain the spatiotemporal features of the video segments; Feature extraction is performed on several dynamic operation parameter segments one by one to obtain the time-series features of dynamic operation parameters; The spatiotemporal features of video clips and the temporal features of dynamic operation parameters are input into the spatiotemporal attention fusion model, and the corresponding fusion features are obtained. The fused features are used to obtain the security risk index at the current moment through a pre-trained security risk assessment model; The system compares the safety risk index with a preset risk threshold to determine whether the safety warning response conditions are met. If they are met, the warning response protocol is triggered.

2. The power operation safety early warning method based on spatiotemporal attention mechanism according to claim 1, characterized in that, Based on a preset time window length and sliding step size, the video data stream and dynamic job parameter data stream are segmented to obtain several video segments and corresponding dynamic job parameter segments, including: The video data stream represents video footage of power operation sites captured by multi-view cameras; Based on the type of power operation, determine the corresponding time window length and sliding step from the preset time window length mapping table; According to the determined time window length and sliding step size, the video data stream and dynamic operation parameter data stream are processed by sliding window segmentation to obtain several video segments and several dynamic operation parameter segments that overlap in time. The dynamic operation parameter data stream includes behavioral parameters, physiological parameters, reaction parameters, and body posture parameters of the power workers; the physiological parameter data stream includes at least one of heart rate, blood pressure, body temperature, blood oxygen saturation, and electroencephalogram (EEG).

3. The power operation safety early warning method based on spatiotemporal attention mechanism according to claim 1, characterized in that, Feature extraction is performed on several video segments one by one to obtain the spatiotemporal features of the video segments, including: Feature extraction is performed on the video segments to obtain initial video features; The initial video features are input into the self-attention model to calculate the spatial correlation between different regions in the video segment, thereby obtaining the spatial correlation features; The spatial correlation feature maps of different levels are superimposed and fused from top to bottom to form a multi-scale fused feature map. A region proposal operation is performed on the multi-scale fused feature map to accurately identify small target feature maps and obtain video spatiotemporal features; Based on the spatiotemporal features of the video, the wearing status of safety protective equipment and dangerous operation behaviors of power workers are accurately identified, and the behavior identification results are obtained. Based on the behavior recognition results and video spatiotemporal features, a security risk assessment feature vector containing spatiotemporal information and behavioral features is generated.

4. The power operation safety early warning method based on spatiotemporal attention mechanism according to claim 1, characterized in that, Feature extraction is performed on several dynamic operation parameter segments one by one to obtain the time-series features of the dynamic operation parameters, including: Extract the baseline and offset segments from the current work data stream of the power workers; The dynamic changes of each parameter point in the baseline segment at different timestamps are obtained to obtain the primary deviation; When the primary deviation is greater than or less than the preset parameter threshold range, the relevant parameter points are marked as the first abnormal points; Record the original timestamps of each parameter point in the offset state segment; Based on the temporal matching relationship, the original timestamp is mapped to the target timestamp of the reference state segment; Obtain the secondary deviation between the offset fragment parameter point and the target timestamp parameter value; When the secondary deviation is greater than or less than the preset parameter threshold range, the parameter points involved are marked as the second abnormal points. Based on the ratio of the number of anomalies in the first and second anomalies to the total number of parameter points, the anomaly degree of dynamic operation parameters is quantified; the anomaly degree of dynamic operation parameters is integrated with the temporal change characteristics to construct a dynamic parameter temporal feature vector containing anomaly detection information. Multi-scale decomposition of the time-series feature vectors yields time-series feature components in different frequency domains. The frequency domain feature components are weighted and fused for processing to obtain the normalized time-series features of dynamic operation parameters.

5. The power operation safety early warning method based on spatiotemporal attention mechanism according to claim 1, characterized in that, The spatiotemporal features of video clips and the temporal features of dynamic operation parameters are input into the spatiotemporal attention fusion model, and the corresponding fused features are obtained, including: The spatiotemporal attention fusion model includes a temporal attention processing layer and a spatial attention processing layer. The attention weights of each time frame in the security risk assessment feature vector are obtained by using a temporal attention processing layer. The security risk assessment feature vector is then weighted to obtain temporally weighted video features. The attention weights of each parameter in the temporal features of dynamic operation parameters are obtained by using a spatial attention processing layer. The temporal features of dynamic operation parameters are then weighted to obtain spatially weighted dynamic operation parameter features. The time-weighted video features and spatially weighted dynamic operation parameter features are fused across modal features to obtain fused features.

6. The power operation safety early warning method based on spatiotemporal attention mechanism according to claim 5, characterized in that, The time-weighted video features and spatially weighted dynamic operation parameter features are fused across modal features to obtain fused features, including: Obtain the cross-correlation matrix between time-weighted video features and spatially weighted dynamic job parameter features; Based on the cross-correlation matrix, feature alignment and fusion are performed to obtain cross-modal fused features; The cross-modal fusion features are subjected to a nonlinear transformation to obtain the final fusion features.

7. The power operation safety early warning method based on spatiotemporal attention mechanism according to claim 1, characterized in that, The fused features are used to obtain the current-time security risk index through a pre-trained security risk assessment model, including: Acquire environmental data and power equipment status data from the power operation site to construct a multi-source dataset for the operation. The environmental data includes temperature, humidity, wind speed, and gas concentration; the power equipment status data includes equipment operating temperature, vibration frequency, and current and voltage parameters. By combining multi-source datasets of the task with fusion features, a security risk assessment model for multi-source data fusion is constructed. A correlation analysis was performed on the multi-source datasets of the assignments, and an assignment risk warning index was generated using a combination of data-driven and knowledge-driven methods. The fused features are input into the pre-trained safety risk assessment model, and combined with the operation risk warning index, the safety risk index at the current moment is obtained. A safety risk level classification mechanism is established based on the multi-source dataset of the operation, dividing the power operation risk into multiple levels, and formulating corresponding early warning information and handling strategies for different risk levels; Based on the different levels of the current safety risk index, corresponding early warning information and response strategies will be pushed out through various means.

8. The power operation safety early warning method based on spatiotemporal attention mechanism according to claim 7, characterized in that, The safety risk index is compared with a preset risk threshold to determine whether the safety warning response conditions are met. If the conditions are met, the warning response protocol is triggered, including: The current safety risk index is compared with preset multi-level risk thresholds to determine the current risk level. If the security risk index exceeds the preset threshold, the corresponding level of security warning response will be triggered. If the security risk index is greater than the high-risk threshold, a high-level security alert will be triggered, and a first alert message containing the results of high-risk behavior identification will be output. If the security risk index is greater than the medium risk threshold and less than or equal to the high risk threshold, a medium-level security warning is triggered, and a second warning message containing the results of medium-risk behavior identification is output. If the security risk index is greater than the low risk threshold and less than or equal to the medium risk threshold, a low-level security warning is triggered, and a third warning message containing the low-risk behavior identification results is output. If the safety risk index is less than or equal to the low-risk threshold, a safety warning will not be triggered.

9. The power operation safety early warning method based on spatiotemporal attention mechanism according to claim 7, characterized in that, Acquire historical video data streams and dynamic operation parameter data streams from multiple safety warning responses; Statistical analysis and pattern mining were performed on historical video data streams and dynamic operation parameter data streams to identify risk patterns and key risk factors that frequently trigger safety warnings. Construct a risk characteristic matrix based on risk patterns and key risk factors; The risk characteristic matrix establishes a nonlinear mapping relationship between environmental factors and safety risks through a safety risk assessment model; Based on the nonlinear mapping relationship, identify the risk evolution pattern under different environmental conditions, and determine the weight coefficients and threshold ranges of various risk factors; Based on the combined characteristics of historical high-frequency safety warning accident data, operation type, environmental parameters and personnel status, the weighting coefficients and threshold ranges are dynamically adjusted to construct safety risk judgment standards and form an adaptive multi-level risk warning mechanism. The parameters and structure of the pre-trained security risk assessment model are tuned and optimized based on an adaptive multi-level risk early warning mechanism. The optimized security risk assessment model is redeployed into the system to achieve accurate and efficient security risk assessment.

10. A power operation safety early warning system based on a spatiotemporal attention mechanism, used to execute a power operation safety early warning method based on a spatiotemporal attention mechanism as described in any one of claims 1-9, characterized in that, include: Data acquisition module: used to acquire video data streams from the power operation site and dynamic operation parameter data streams from the operators; Data segmentation module: Used to segment video data stream and dynamic operation parameter data stream based on preset time window length and sliding step size, to obtain several video segments and corresponding several dynamic operation parameter segments; Video feature extraction module: used to extract features from several video segments one by one to obtain the spatiotemporal features of the video segments; Dynamic operation parameter feature extraction module: used to extract features from several dynamic operation parameter segments one by one to obtain the time-series features of dynamic operation parameters; Spatiotemporal attention fusion model: This model is used to input the spatiotemporal features of video clips and the temporal features of dynamic operation parameters into the spatiotemporal attention fusion model, and then fuse them to obtain the fused features. Security risk assessment module: used to obtain the security risk index at the current moment by using the fused features through a pre-trained security risk assessment model; Safety warning module: It is used to compare the safety risk index with the preset risk threshold to determine whether the safety warning response conditions are met. If they are met, the warning response protocol is triggered.

Images