Power operation site real-time risk assessment method, device and computer equipment

Abstract

The application relates to a power operation site real-time risk assessment method, device and computer equipment. The method comprises the following steps: acquiring real-time multi-modal data of a power operation site, and respectively pre-processing the real-time multi-modal data to obtain pre-processed multi-modal data; respectively extracting risk features from each mode of data to obtain visual risk features, environmental risk features, behavior risk features and text risk features; performing multi-modal feature fusion on the visual risk features, the environmental risk features, the behavior risk features and the text risk features to obtain comprehensive risk features; performing multi-dimensional risk assessment based on the comprehensive risk features to obtain a risk assessment result; the risk assessment result comprises a risk level and a risk type; and based on the risk level, the risk type and corresponding safety regulation information, structured risk warning information is generated. The method can comprehensively and accurately assess the real-time risk situation of the power operation site, and improve the safety of power operation.

Description

Technical Field

[0001] This application relates to the field of power operation safety monitoring technology, and in particular to a method, device and computer equipment for real-time risk assessment at power operation sites. Background Technology

[0002] Safety management at power work sites is a core aspect of safe power production, directly impacting the lives of workers and the stable operation of the power system. Power work encompasses various high-risk types, including working at heights, live-line work, hoisting, confined space work, and hot work, with diverse job types and complex risk factors.

[0003] In related technologies, risk identification and early warning are usually carried out using a single risk factor. However, data from a single modality can only reflect one aspect of the risk and is difficult to comprehensively and accurately assess the real-time risk situation at the work site.

[0004] Therefore, there is an urgent need for a cross-modal fusion mechanism to improve the ability to conduct comprehensive risk assessment of multiple factors at the work site, thereby comprehensively and accurately assessing the real-time risk situation at the power work site and improving the safety of power operations. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, device, computer equipment, computer-readable storage medium, and computer program product for real-time risk assessment of power operation sites that can comprehensively and accurately assess the real-time risk situation at power operation sites and improve the safety of power operations, in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a method for real-time risk assessment at power operation sites, including:

[0007] Real-time multimodal data from the power operation site is acquired, and the real-time multimodal data is preprocessed to obtain preprocessed multimodal data. The preprocessed multimodal data includes video data, environmental sensor data, personnel positioning data, and work order text data.

[0008] Risk features were extracted from each modality of data to obtain visual risk features, environmental risk features, behavioral risk features, and textual risk features;

[0009] Multimodal feature fusion is performed on visual risk features, environmental risk features, behavioral risk features, and textual risk features to obtain comprehensive risk features;

[0010] A multi-dimensional risk assessment is conducted based on comprehensive risk characteristics to obtain risk assessment results. The risk assessment results include risk level and risk type. The multi-dimensional risk assessment includes personnel safety dimension scores, environmental condition dimension scores, and work type dimension scores.

[0011] Based on risk level, risk type, and corresponding safety procedure information, structured risk warning information is generated; the risk warning information is used to trigger alarm responses at the corresponding level.

[0012] In one embodiment, risk features are extracted from each modality of data to obtain visual risk features, environmental risk features, behavioral risk features, and textual risk features, including:

[0013] The video data is input into the visual feature extraction engine to extract visual features and obtain visual risk features.

[0014] Environmental sensor data is input into an environmental feature extraction engine to extract environmental features and obtain environmental risk features.

[0015] Personnel location data is input into a behavior feature extraction engine to extract behavior features and obtain behavior risk features.

[0016] The work order text data is input into the text feature extraction engine to extract text features and obtain text risk features;

[0017] The visual feature extraction engine, environmental feature extraction engine, behavioral feature extraction engine, and text feature extraction engine are deployed on edge devices and executed in parallel. Each of these engines has a lightweight feature extraction model, and the size of the lightweight feature extraction model is smaller than the preset size.

[0018] In one embodiment, multimodal feature fusion is performed on visual risk features, environmental risk features, behavioral risk features, and textual risk features to obtain comprehensive risk features, including:

[0019] Visual risk features, environmental risk features, behavioral risk features, and text risk features are projected onto a unified feature dimension space to obtain aligned visual risk feature vectors, environmental risk feature vectors, behavioral risk feature vectors, and text risk feature vectors.

[0020] The visual risk feature vector, environmental risk feature vector, behavioral risk feature vector, and textual risk feature vector are input into the cross-modal attention module to determine the correlation weights between the features of each modality.

[0021] Based on the correlation weights between various modal features, multimodal feature fusion is performed on visual risk features, environmental risk features, behavioral risk features, and textual risk features to obtain comprehensive risk features.

[0022] In one embodiment, a multi-dimensional risk assessment is performed based on comprehensive risk characteristics to obtain risk assessment results, including:

[0023] Based on comprehensive risk characteristics, personnel safety dimensions are scored to obtain personnel safety risk scores;

[0024] Environmental condition risk scores are obtained by scoring the environmental condition dimension based on comprehensive risk characteristics.

[0025] Based on comprehensive risk characteristics, a risk score for each job type is obtained by scoring the job type dimension.

[0026] The comprehensive risk score is obtained by weighting and summing the personnel safety risk score, environmental condition risk score, and work type risk score; the weights of each dimension of risk score are dynamically adjusted based on the work type and work time stage.

[0027] The risk level is determined based on a comprehensive risk score and a preset risk threshold.

[0028] In one embodiment, the method further includes:

[0029] Obtain preset constraints; preset constraints are determined based on relevant rules in the power safety regulations knowledge base.

[0030] Based on preset constraints and fusion identification results, the risk level is corrected to obtain the corrected risk level; the fusion identification results are determined based on comprehensive risk characteristics.

[0031] In one embodiment, the preset risk thresholds include a first threshold, a second threshold, and a third threshold. Based on the comprehensive risk score and the preset risk thresholds, a risk level is determined, including:

[0032] If the overall risk score is less than the first threshold, the risk level is low.

[0033] If the comprehensive risk score is greater than or equal to the first threshold and less than the second threshold, the risk level is medium risk.

[0034] If the comprehensive risk score is greater than or equal to the second threshold and less than the third threshold, the risk level is high risk.

[0035] If the comprehensive risk score is greater than or equal to the third threshold, the risk level is extremely high.

[0036] In one embodiment, if the risk level is low risk, the corresponding alarm response includes recording and labeling in the monitoring backend;

[0037] If the risk level is medium risk, the corresponding alarm response includes pushing a prompt message to the handheld terminal of the on-site safety supervisor;

[0038] If the risk level is high, the corresponding alarm response includes on-site audible and visual alarms, prominent marking on the monitoring screen, and push notification information to the handheld terminal of on-site safety supervisors;

[0039] If the risk level is extremely high, the corresponding alarm response includes on-site audible and visual alarms, highlighting on the monitoring screen, pushing notification information to the handheld terminals of on-site safety supervisors, and notifying the emergency response center.

[0040] In one embodiment, the method further includes:

[0041] Obtain verification information regarding risk warnings submitted by on-site safety supervisors;

[0042] The handling outcome of this warning event will be determined based on the verification information; if the verification information indicates that the warning is true, the handling outcome will include the handling measures and their effects; if the verification information indicates that the warning is false, the handling outcome will include the reasons for the false alarm.

[0043] The handling results will be used as feedback data for this early warning event. The feedback data will be used for incremental learning of the risk assessment model, which will then be used to conduct multi-dimensional risk assessments.

[0044] Secondly, this application also provides a real-time risk assessment device for power operation sites, comprising:

[0045] The multimodal data acquisition module is used to acquire real-time multimodal data from the power operation site and preprocess the real-time multimodal data to obtain preprocessed multimodal data. The preprocessed multimodal data includes video data, environmental sensor data, personnel positioning data, and work order text data.

[0046] The risk feature extraction module is used to extract risk features from each modality of data to obtain visual risk features, environmental risk features, behavioral risk features, and textual risk features.

[0047] The multimodal feature fusion module is used to fuse visual risk features, environmental risk features, behavioral risk features, and textual risk features to obtain comprehensive risk features.

[0048] The multi-dimensional risk assessment module is used to conduct multi-dimensional risk assessments based on comprehensive risk characteristics and obtain risk assessment results. The risk assessment results include risk level and risk type. The multi-dimensional risk assessment includes personnel safety dimension scores, environmental condition dimension scores, and work type dimension scores.

[0049] The risk warning information generation module is used to generate structured risk warning information based on risk level, risk type and corresponding safety procedure information; the risk warning information is used to trigger alarm responses at the corresponding level.

[0050] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the real-time risk assessment method for power operation sites described in the first aspect above.

[0051] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the real-time risk assessment method for power operation sites described in the first aspect above.

[0052] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the real-time risk assessment method for power operation sites described in the first aspect above.

[0053] The aforementioned real-time risk assessment method, device, computer equipment, storage medium, and computer program product for power operation sites acquire real-time multimodal data from the power operation site and preprocess the real-time multimodal data to obtain preprocessed multimodal data, including video data, environmental sensor data, personnel positioning data, and work order text data. Risk features are extracted from each modal data to obtain visual risk features, environmental risk features, behavioral risk features, and textual risk features. These features are then fused to obtain comprehensive risk features. Based on these comprehensive risk features, a multi-dimensional risk assessment is performed to obtain a risk assessment result, which includes risk level and risk type. The multi-dimensional risk assessment includes personnel safety dimension scores, environmental condition dimension scores, and work type dimension scores. Finally, based on the risk level, risk type, and corresponding safety procedure information, structured risk warning information is generated, which is used to trigger alarm responses at the corresponding level. This method enables the extraction and fusion of risk features from multimodal data at power operation sites. Based on comprehensive risk features, it allows for real-time quantitative assessment of operational risks from dimensions such as personnel safety, environmental conditions, and operation type. Combined with power safety regulations, it provides risk warnings. This approach not only comprehensively and accurately assesses the real-time risk situation at power operation sites, improving the comprehensiveness of risk assessment, but also enhances the compliance and accuracy of risk warnings. It provides comprehensive, reliable, and referable risk warning information for subsequent power supervision and handling, thereby comprehensively improving the operational safety of power operation sites. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a diagram illustrating the application environment of a real-time risk assessment method for power operation sites in one embodiment.

[0056] Figure 2 This is a flowchart illustrating the real-time risk assessment method for power operation sites in the first embodiment;

[0057] Figure 3 This is a flowchart illustrating the real-time risk assessment method for power operation sites in the second embodiment;

[0058] Figure 4 This is a flowchart illustrating the real-time risk assessment method for power operation sites in the third embodiment;

[0059] Figure 5 This is a flowchart illustrating the real-time risk assessment method for power operation sites in the fourth embodiment;

[0060] Figure 6 This is a schematic diagram of the complete process of a real-time risk assessment method for power operation sites in one embodiment;

[0061] Figure 7 This is a structural block diagram of a real-time risk assessment device for power operation sites in one embodiment;

[0062] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] In actual power operation operations, on-site risk management faces the following prominent challenges:

[0065] First, the types of work are diverse, and the risk factors are complex. Different types of work have significantly different risk characteristics and key control points.

[0066] For example, the main risks of working at heights are falls and falling objects; the main risks of working with electricity are electric shock and arc burns; the main risks of lifting and hoisting are falling objects and overturning of cranes; and the main risks of working in confined spaces are suffocation and poisoning. Furthermore, risk factors at the same work site involve multiple levels, including personnel behavior, equipment status, environmental conditions, and management factors, forming a complex, multi-dimensional, coupled risk system.

[0067] Second, there is a gap between real-time perception and semantic understanding. Safety management of power operations requires risk perception and early warning capabilities at the second or even millisecond level.

[0068] In recent years, traditional monitoring solutions have achieved high frame rates for real-time processing in perception tasks such as target detection and behavior recognition, enabling rapid detection of personnel, actions, and objects at the work site. However, a significant gap remains between "perception" and "understanding"—while visual models can efficiently identify "people, actions, and objects," they struggle to understand the semantic context of the work, and are unable to semantically correlate and comprehensively reason with procedures and rules. They cannot determine whether current actions comply with work procedures, constitute violations, or how risk levels dynamically change. This situation prevents efficient perception results from being effectively transformed into deep semantic understanding and intelligent decision-making, hindering the comprehensive intelligentization of risk management at the work site.

[0069] Third, data from multiple sources are isolated and the information is incomplete.

[0070] Modern power work sites typically deploy various sensing devices such as video surveillance cameras, environmental sensors, and personnel positioning terminals. However, the data generated by these devices is processed and analyzed in isolation, lacking an effective cross-modal fusion mechanism. Data from a single modality can only reflect one aspect of the risk, making it difficult to comprehensively and accurately assess the real-time risk situation at the work site.

[0071] Fourth, risk assessment relies on static rules and lacks dynamic quantification capabilities.

[0072] Existing risk assessment methods often rely on pre-set fixed rules or thresholds to determine risk levels, such as stipulating that "wind speeds exceeding a certain threshold are considered high-risk." However, the risk situation at the work site is dynamically evolving, and relying solely on static rules cannot provide a dynamic and quantitative assessment based on actual work progress, environmental changes, and personnel conditions, leading to discrepancies between the assessment results and the actual situation.

[0073] Fifth, knowledge of safety procedures has not been effectively utilized.

[0074] The power safety regulations contain a wealth of expert knowledge on risk identification, risk classification, and risk management. However, existing methods do not deeply embed this knowledge into the risk assessment process, resulting in assessment results that lack interpretability and traceability, making it difficult to support manual review and post-audit.

[0075] Sixth, high latency in multi-channel data processing.

[0076] Power work sites typically deploy multiple video surveillance cameras, simultaneously overlaying environmental sensor data and personnel positioning data for real-time acquisition and processing. A series of computational tasks, including parallel analysis of multiple video streams, sensor data fusion processing, feature extraction, and risk assessment, pose significant challenges to inference latency. Existing solutions struggle to meet the second-level real-time early warning response requirements while ensuring recognition accuracy.

[0077] Existing technologies typically employ a single risk factor for risk identification and early warning. In terms of multi-source data processing, the data is stored separately in their respective systems and only displayed in a simple parallel manner on the final interface, lacking semantic-level correlation and fusion between the data. Current risk detection methods at power operation sites have the following shortcomings:

[0078] First, the existing solutions suffer from a single modality and incomplete risk identification. Currently, video surveillance, environmental sensors, and work permit management operate independently, lacking a unified multimodal collaborative perception framework. Specifically, the video surveillance solution uses visible light or infrared cameras to collect images of the work site and employs target detection models such as YOLO and Faster R-CNN to infer frame-by-frame violations, identifying apparent violations such as helmet wearing, work uniform wearing, and area intrusion. However, it is completely unable to perceive non-visual risk factors such as environmental temperature, humidity, wind speed, and harmful gas concentrations. The environmental sensor solution uses various sensors deployed on-site to collect physical environmental parameters and compares them with preset thresholds to determine if environmental risks exceed limits, but it cannot obtain dynamic information such as personnel behavior and equipment status. The work permit solution relies on static text information entered into the electronic work permit system for pre-determined risk levels, completely disconnected from the dynamic changes during actual operations. Methods relying solely on video surveillance show a significant drop in accuracy under conditions such as obstruction and changes in lighting, and cannot detect risk factors such as abnormal environmental parameters and personnel locations; methods relying solely on sensors cannot identify violations of personnel behavior and abnormal equipment status; and single-modal methods cannot achieve comprehensive perception of multi-dimensional risk factors at the work site.

[0079] Second, cross-modal fusion is insufficient, and the complementary nature of information is not fully utilized. Video, sensor, positioning, and text data are processed and stored independently, and simply spliced ​​together at the output stage. The complementary information contained in each modal data (such as the precise coordinates of people's positions in the video and the correlation between sensor data and people's postures in the video) has not been effectively mined and utilized.

[0080] Third, risk assessments are static and cannot be dynamically updated. Risk level determination relies on preset fixed rules or thresholds, which remain unchanged throughout the entire operation once set. In actual operation, the risk situation evolves dynamically as the operation progresses, environmental conditions change, and personnel status changes; static assessments cannot reflect this dynamic change.

[0081] Fourth, there is a lack of knowledge linkage and insufficient interpretability. The system is not deeply integrated with the power safety regulations knowledge base, resulting in a lack of clear regulatory basis for risk assessment results, making manual review and audit traceability difficult. Furthermore, the system cannot quickly adapt and update when safety regulations change.

[0082] Fifth, the lack of closed-loop optimization hinders continuous capability improvement. The handling results and feedback experience of on-site personnel regarding early warnings are not systematically collected, preventing the model from incrementally learning through feedback. Consequently, the system's recognition accuracy and scenario adaptability are difficult to improve with increased usage time.

[0083] The method proposed in this application aims to solve the following technical problems:

[0084] 1. How to construct a unified risk representation space for work sites that integrates multimodal data such as video, environmental sensors, personnel positioning, and work order text, to achieve cross-modal semantic alignment and feature complementarity, and solve the problem of incomplete information in a single modality;

[0085] 2. How to design a real-time risk identification algorithm for power operation scenarios to achieve multi-task parallel detection of personnel posture, safety equipment wearing, area intrusion, equipment status, and environmental parameters, while ensuring identification accuracy while meeting real-time requirements;

[0086] 3. How to construct a dynamic risk assessment model that integrates multi-source features to achieve real-time quantitative assessment and dynamic updating of risk levels, overcoming the problem that static rule assessment cannot reflect real-time situational changes;

[0087] 4. How to deeply integrate the power safety regulations knowledge base with the risk assessment model, transform the regulations and rules into constraints to verify and correct the identification results, make the assessment results interpretable and traceable, and support hot updates of the regulations and rules.

[0088] 5. How to control inference latency to meet second-level early warning requirements while ensuring assessment accuracy through lightweight deployment at the edge and optimized design of multi-path data parallel processing pipeline;

[0089] 6. How to construct a closed-loop optimization mechanism driven by human feedback, so that the system can continuously perform incremental learning and model updates through feedback from on-site personnel, thereby improving the system's adaptability to different scenarios.

[0090] The real-time risk assessment method for power operation sites provided in this application can be applied to, for example... Figure 1 In the application environment shown, computer device 101 can communicate with a multi-source data system via a network to acquire real-time multimodal data from the power operation site. This allows for comprehensive analysis of the real-time multimodal data to achieve risk assessment of the power operation site. The multi-source data system may include a visual data acquisition system, an environmental data acquisition system, a positioning data acquisition system, and a work permit system. The visual data acquisition system may include visual cameras installed at different locations within the power operation site. The environmental data acquisition system may include environmental sensors installed at different locations within the power operation site, including but not limited to temperature sensors, humidity sensors, gas concentration sensors, and wind speed sensors. The positioning data acquisition system may include location sensors installed at different locations within the power operation site. The work permit system contains work permit text data, including but not limited to work type codes, specified risk levels, and a list of safety measures.

[0091] Computer device 101 can be a terminal device or a server. The terminal device can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The server can be a standalone server or a server cluster composed of multiple servers. The server can be a local server, a cloud server, a remote server, etc.

[0092] In one exemplary embodiment, such as Figure 2 As shown, a real-time risk assessment method for power operation sites is provided, which is then applied to... Figure 1 The following steps, 201 to 205, are used as an example of computer equipment.

[0093] Step 201: Obtain real-time multimodal data from the power operation site, and preprocess the real-time multimodal data to obtain preprocessed multimodal data.

[0094] The preprocessed multimodal data includes video data, environmental sensor data, personnel positioning data, and work order text data.

[0095] For example, computer equipment can acquire video data from a visual data acquisition system, such as video cameras installed at different locations at a power work site that can send the acquired video data to the computer equipment in real time; computer equipment can acquire environmental sensor data from an environmental data acquisition system, such as environmental sensors installed at different locations at a power work site that can send the acquired environmental sensor data to the computer equipment in real time; computer equipment can acquire personnel location data from a positioning data acquisition system, such as location sensors installed at different locations at a power work site that, upon detecting personnel, send the personnel location data to the computer equipment in real time; and computer equipment can extract the work ticket text data corresponding to the current moment from a work ticket system.

[0096] For the acquired real-time multimodal data, corresponding data preprocessing can be performed on each modality. For example, for video data, preprocessing operations such as noise reduction, illumination normalization, and resolution unification can be performed; for environmental sensor data, outlier removal and standardization can be performed; for personnel positioning data, coordinate transformation and filtering smoothing can be performed; and for work ticket text data, structured parsing can be performed to extract key information such as work type, planned duration, specified safety measures, and personnel qualification requirements.

[0097] In addition, the computer equipment can also simultaneously access the power safety regulations knowledge base, providing rule constraints for subsequent feature extraction and risk assessment.

[0098] For example, video data preprocessing may include at least: keyframe filtering or uniform frame sampling based on motion detection, illumination normalization based on adaptive histogram equalization, and resolution unification based on bicubic interpolation. Environmental sensor data preprocessing may include at least: outlier detection and removal based on statistical thresholds, and normalization based on the Z-score method. Personnel positioning data preprocessing may include at least: coordinate system transformation and alignment, and trajectory smoothing based on Kalman filtering. Structured parsing of work order text data may include at least: extraction of key fields such as work type, work location, work time, and safety measures based on named entity recognition.

[0099] Step 202: Extract risk features from each modality of data to obtain visual risk features, environmental risk features, behavioral risk features, and textual risk features.

[0100] For example, a computer device can extract risk features from real-time multimodal data or preprocessed real-time multimodal data to obtain risk features corresponding to each modality.

[0101] Specifically, visual risk features can be extracted from video data, including but not limited to personnel posture features, safety equipment wearing status features, and area intrusion detection features. For example, a lightweight target detection model can be used to identify key targets such as human bodies, safety helmets, work clothes, and safety ropes. Alternatively, a posture estimation algorithm can be used to extract the coordinates of key points on the human body, while an area intrusion detection algorithm can be used to determine whether a person has entered a dangerous area.

[0102] Environmental risk features can be extracted from environmental sensor data, including but not limited to environmental parameters such as temperature, humidity, gas concentration, and wind speed. For example, time-series statistical methods can be used to extract statistical features such as the mean, variance, and rate of change of each environmental parameter.

[0103] Behavioral risk characteristics can be extracted from personnel location data, including but not limited to personnel trajectory, time spent in the work area, and distance from other personnel; for example, trajectory analysis algorithms can be used to identify behavioral patterns such as abnormal stays and boundary crossings.

[0104] Extract text risk features from the work order text data, including but not limited to work type codes, specified risk levels, and lists of safety measures; for example, a pre-trained language model can be used to encode the text information into semantic vectors.

[0105] In one alternative implementation, risk feature extraction can be performed on multimodal data in parallel. For example, corresponding feature extraction engines can be set for different modalities of data, such as visual feature extraction engine, environmental feature extraction engine, behavioral feature extraction engine and text feature extraction engine. Based on this, each feature extraction engine can perform feature extraction operations on the corresponding modal data in parallel, thereby obtaining the risk features of the corresponding modal data.

[0106] For example, video data can be input into a visual feature extraction engine to extract visual features and obtain visual risk features. For each frame of the video data, the following visual tasks can be performed in parallel: a lightweight object detection model can be used to detect key targets such as people, safety helmets, safety ropes, and safety fences in the image, and output the target category, bounding box position, and confidence score; a lightweight pose estimation algorithm can be used to extract the key point coordinates of people and determine whether the person's pose is stable; based on the object detection results, the wearing status of safety equipment can be determined, and whether the person has entered the designated danger zone can be determined.

[0107] Environmental sensor data is input into the environmental feature extraction engine to extract environmental features and obtain environmental risk features. For environmental sensor data, time-series statistical features of each environmental parameter are extracted, including current value, rate of change, etc., and compared in real time with the safety thresholds specified in the regulations to generate environmental risk indication features.

[0108] Personnel location data is input into the behavior feature extraction engine to extract behavior features and obtain behavior risk features. For personnel location data, the spatial relationship between the real-time position of personnel and the preset work area and danger area is calculated, the personnel movement speed and trajectory smoothness are calculated, and it is detected whether personnel stay in abnormal positions for a long time or deviate from the predetermined work area.

[0109] The text data of the work ticket is input into the text feature extraction engine to extract text features and obtain text risk features. For the text data of the work ticket, a pre-trained language model is used to extract text semantic features, and information such as work type, specified risk level, and safety measure list are encoded into semantic vectors to provide prior information for work type perception for subsequent assessment.

[0110] The aforementioned visual feature extraction engine, environmental feature extraction engine, behavioral feature extraction engine, and text feature extraction engine can be deployed on edge devices for parallel execution, thereby improving the extraction efficiency of multimodal risk features. Furthermore, each feature extraction engine can be deployed with a corresponding lightweight feature extraction model. This lightweight model has a smaller model size than a preset size. Using the corresponding lightweight feature extraction model to extract features from different modalities in parallel ensures the requirements for real-time detection. In this example, the real-time guarantee mechanism of edge deployment and lightweight models minimizes risk assessment latency, achieving real-time and timely early warning effects with second-level alerts.

[0111] It should be noted that visual tasks, environmental feature extraction, localization feature extraction, and text feature encoding can be deployed on multiple computing cores for parallel execution to reduce overall inference latency. In this example, by designing a multi-task parallel risk feature extraction method for power operation scenarios, visual detection, environmental analysis, localization tracking, and text encoding are deployed on the edge for parallel execution. Through lightweight model design and pipelined parallel optimization to build a real-time guarantee mechanism, the inference latency can be controlled within seconds, meeting the requirements for real-time early warning.

[0112] Based on this, the server can call the various feature extraction engines deployed in the edge devices to perform lightweight feature extraction on each modality data to obtain the corresponding feature information; or, the server can send the preprocessed modality data to the edge devices so that the edge devices can use the corresponding feature extraction engines to perform feature extraction on the corresponding modality data based on the lightweight extraction model, and then return the obtained feature information to the server for subsequent risk identification and early warning processing.

[0113] Step 203: Perform multimodal feature fusion on visual risk features, environmental risk features, behavioral risk features and textual risk features to obtain comprehensive risk features.

[0114] For example, a multimodal feature fusion model can be used to input visual risk features, environmental risk features, behavioral risk features, and textual risk features into the multimodal feature fusion model for feature fusion processing, and output the fused comprehensive risk features. The multimodal feature fusion model can be an attention-based feature fusion model, used to adaptively determine the fusion weights of the visual dimension, environmental dimension, personnel behavior dimension, and work ticket dimension for different power operation scenarios. This ensures the feature emphasis under different power operation scenarios, improves the matching degree between the comprehensive risk features and the actual power operation scenarios, and provides an accurate basis for subsequent risk assessment.

[0115] For example, when performing multimodal feature fusion, cross-modal feature alignment and fusion can also be performed on the risk features of each modality. Specifically, visual risk features, environmental risk features, behavioral risk features, and text risk features are first projected onto a unified feature dimension space to obtain aligned visual risk feature vectors, environmental risk feature vectors, behavioral risk feature vectors, and text risk feature vectors. Then, the visual risk feature vectors, environmental risk feature vectors, behavioral risk feature vectors, and text risk feature vectors are input into a cross-modal attention module to determine the correlation weights between each modality feature. Then, based on the correlation weights between each modality feature, multimodal feature fusion is performed on the visual risk features, environmental risk features, behavioral risk features, and text risk features to obtain a comprehensive risk feature.

[0116] First, the extracted visual risk features, environmental risk features, behavioral risk features, and textual risk features are mapped to a unified risk feature space through linear mapping, resulting in aligned feature vectors for each modality, achieving cross-modal feature alignment. Then, a cross-modal attention mechanism is used to dynamically calculate the association weights between the features of each modality, achieving weighted fusion of multimodal features. For example, in a high-altitude work scenario, when the video detects that a person's safety rope is not properly suspended, the attention mechanism will increase the weight of the person's position feature in the location data to confirm whether the person is in the edge area.

[0117] The multimodal feature fusion model can include a projection module, a cross-modal attention module, and a fusion module. The projection module projects the risk features of different modalities onto a unified risk feature space to obtain the corresponding risk feature vectors. The cross-modal attention module dynamically calculates the correlation weights between the features of each modality. The fusion module performs weighted fusion of the risk feature vectors of each modality based on the correlation weights of each modality to obtain the final comprehensive risk features.

[0118] In this example, by constructing a unified risk representation space for multimodal work sites that integrates video, environmental sensors, personnel positioning, and work order text, and by achieving semantic complementarity of multimodal information through cross-modal feature alignment and attention fusion, the shortcomings of incomplete single-modal information are overcome, significantly improving the risk detection rate in complex scenarios and reducing the false alarm rate.

[0119] Step 204: Conduct a multi-dimensional risk assessment based on comprehensive risk characteristics to obtain the risk assessment results; the risk assessment results include risk level and risk type.

[0120] Multi-dimensional risk assessment can include personnel safety dimension scoring, environmental condition dimension scoring, and work type dimension scoring.

[0121] For example, based on the integrated risk characteristics, a multi-dimensional risk scoring model can be constructed to dynamically score risks from the dimensions of personnel safety, equipment status, environmental conditions, and management compliance. The dynamic risk score can be updated dynamically according to the real-time situation. The scores of each dimension can be weighted and summed to obtain a comprehensive risk score, which can then be used to determine the risk level. Optionally, the risk level can be divided into at least four levels: low risk, medium risk, high risk, and extremely high risk.

[0122] In one optional implementation, a preset risk threshold can be set, and different risk levels can be determined based on the comprehensive risk score and the preset risk threshold. For example, the preset risk threshold may include a first threshold, a second threshold, and a third threshold, wherein the first threshold is less than the second threshold, and the second threshold is less than the third threshold; if the comprehensive risk score is less than the first threshold, the risk level is low risk; if the comprehensive risk score is greater than or equal to the first threshold and less than the second threshold, the risk level is medium risk; if the comprehensive risk score is greater than or equal to the second threshold and less than the third threshold, the risk level is high risk; and if the comprehensive risk score is greater than or equal to the third threshold, the risk level is extremely high risk.

[0123] Furthermore, the risk type can be determined based on the comprehensive risk features obtained in step 203. For example, computer equipment can directly identify one or more risk events based on the multimodal risk features obtained in step 202. Further, multimodal feature fusion reasoning can be performed based on the comprehensive risk features obtained in step 3 to obtain a composite risk type. For example: visual risk features can identify risks such as not wearing a safety helmet, not wearing a safety belt, personnel crossing boundaries, exposed equipment, open flames, and smoke; environmental risk features can identify risks such as slippery ground, excessive wind speed, insufficient lighting, high temperatures, and flammable / explosive environments; behavioral risk features can identify risks such as violations of regulations, climbing, approaching dangerous areas, and unsupervised single-person operation; and textual risk features can identify risks such as abnormal work order information, discrepancies between work content and the actual situation, and missing safety briefings.

[0124] A single modality can only identify local risks, while fusion can yield more accurate and complete risk types. For example, "personnel + environment + behavior" fusion can identify working at heights without wearing a safety belt, "vision + text + equipment" fusion can identify illegal operations in energized areas, and "behavior + procedure" fusion can identify unqualified personnel performing special operations.

[0125] For example, when identifying risk types based on multimodal risk characteristics, a safety regulations knowledge base can be used to standardize the identification of risk types. For instance, the safety regulations knowledge base can predefine standard risk types and violation types for electrical work sites, such as: risks related to working at heights, electrical operation risks, fire safety risks, safety protection risks, and management and qualification risks. Based on this, computer equipment can match the risk types identified based on comprehensive risk characteristics with the standard risk types in the safety regulations knowledge base, thereby outputting standardized risk types that correspond to specific regulations clauses.

[0126] Step 205: Based on the risk level, risk type and corresponding safety procedure information, generate structured risk warning information; the risk warning information is used to trigger alarm responses at the corresponding level.

[0127] For example, computer equipment can generate structured risk warning information based on risk level, specific risk type identified, and corresponding safety procedure information. This risk warning information can include at least the warning level, risk type, risk description, risk location, timestamp, corresponding safety procedure clause (including clause number and content), and recommended handling measures. The safety procedure information can be obtained from a power safety procedure knowledge base. For instance, the risk level and risk type can be compared and verified against the power safety procedure knowledge base. When the model identifies that a worker at height is not wearing a safety rope, it automatically matches the relevant clauses in the procedure regarding safety equipment for working at height, outputting the corresponding procedure number and clause content, providing a clear basis for the warning.

[0128] Furthermore, different levels of alarm responses can be triggered based on risk warning information. For example, warning information can be pushed and visualized in a tiered manner through on-site audible and visual alarms, mobile terminal push notifications, and large-screen annotations.

[0129] For example, if the risk level is low, the corresponding alarm response may include recording and marking in the monitoring backend; if the risk level is medium, the corresponding alarm response may include pushing a prompt message to the handheld terminal of the on-site safety supervisor; if the risk level is high, the corresponding alarm response may include on-site audible and visual alarms, highlighting on the monitoring screen, and pushing a prompt message to the handheld terminal of the on-site safety supervisor; if the risk level is extremely high, the corresponding alarm response may include on-site audible and visual alarms, highlighting on the monitoring screen, pushing a prompt message to the handheld terminal of the on-site safety supervisor, and notifying the emergency response center.

[0130] In the aforementioned real-time risk assessment method for power operation sites, real-time multimodal data of the power operation site is acquired and preprocessed to obtain preprocessed multimodal data, including video data, environmental sensor data, personnel positioning data, and work order text data. Risk features are extracted from each modality to obtain visual risk features, environmental risk features, behavioral risk features, and text risk features. These features are then fused to obtain comprehensive risk features. Based on these comprehensive risk features, a multi-dimensional risk assessment is performed to obtain a risk assessment result, which includes risk level and risk type. The multi-dimensional risk assessment includes personnel safety dimension scores, environmental condition dimension scores, and work type dimension scores. Finally, based on the risk level, risk type, and corresponding safety procedure information, structured risk warning information is generated, which is used to trigger alarm responses at the corresponding levels. This method enables the extraction and fusion of risk features from multimodal data at power operation sites. Based on comprehensive risk features, it allows for real-time quantitative assessment of operational risks from dimensions such as personnel safety, environmental conditions, and operation type. Combined with power safety regulations, it provides risk warnings. This approach not only comprehensively and accurately assesses the real-time risk situation at power operation sites, improving the comprehensiveness of risk assessment, but also enhances the compliance and accuracy of risk warnings. It provides comprehensive, reliable, and referable risk warning information for subsequent power supervision and handling, thereby comprehensively improving the operational safety of power operation sites.

[0131] In one exemplary embodiment, such as Figure 3 As shown, step 204 above may include steps 301 to 305. Wherein:

[0132] Step 301: Based on the comprehensive risk characteristics, score the personnel safety dimension to obtain the personnel safety risk score.

[0133] For example, a person's safety risk score is calculated by comprehensively considering factors such as the wearing of safety equipment, the stability of the person's posture, and whether they have entered a dangerous area. When safety equipment is complete, the posture is stable, and no dangerous area has been entered, the risk score for this dimension is low; when no safety rope or helmet is worn, the score for this dimension increases significantly.

[0134] Step 302: Score the environmental condition dimension based on the comprehensive risk characteristics to obtain the environmental condition risk score.

[0135] For example, real-time environmental parameters can be compared with safety thresholds specified in regulations to calculate environmental condition risk scores. The score for this dimension dynamically increases as wind speed continues to rise.

[0136] Step 303: Score the job type dimension based on the comprehensive risk characteristics to obtain the job type risk score.

[0137] For example, the inherent risk score of the work type can be calculated based on the work type and the specified risk level in the work ticket, with different basic risk coefficients corresponding to different work types.

[0138] Step 304: Weight the personnel safety risk score, environmental condition risk score, and work type risk score to obtain a comprehensive risk score.

[0139] The weights corresponding to the risk scores of each dimension are dynamically adjusted based on the type of work and the time period of the work.

[0140] The comprehensive risk score is a weighted sum of the scores from the above dimensions, and the weights can be dynamically adjusted according to the type of work and the time stage. For example, in high-altitude operations, the weight of the personnel safety dimension is higher than that of the environmental dimension, and within the environmental dimension, the weight of wind speed in high-altitude work scenarios is significantly higher than that of other environmental parameters.

[0141] Step 305: Determine the risk level based on the comprehensive risk score and the preset risk threshold.

[0142] For example, a computer device can map a comprehensive risk score to four risk levels, such as: low risk when the comprehensive risk score is below a first threshold, medium risk when it is between the first and second thresholds, high risk when it is between the second and third thresholds, and extremely high risk when it is above the third threshold.

[0143] In this embodiment, a multi-dimensional dynamic risk scoring model can be constructed to conduct real-time quantitative assessments from dimensions such as personnel safety, environmental conditions, and job type. The weights of each dimension can be dynamically adjusted according to the job type and time stage. Risk assessment is upgraded from static rule judgment to dynamic quantitative scoring, which can be adaptively updated according to the real-time situation.

[0144] In an exemplary embodiment, for the determined risk level, the risk level can also be compared and verified based on the power safety regulations knowledge base. Based on this, such as... Figure 4 As shown, the above method may further include steps 401 to 402. Wherein:

[0145] Step 401: Obtain preset constraints; the preset constraints are determined based on relevant rules in the power safety regulations knowledge base.

[0146] Step 402: Based on the preset constraints and the fusion identification results, the risk level is corrected to obtain the corrected risk level; the fusion identification results are determined based on comprehensive risk characteristics.

[0147] In this embodiment, the rules in the power safety regulations knowledge base can be transformed into constraints to verify and correct the fusion identification results. The power safety regulations knowledge base includes at least: safety regulations clauses for various types of operations, risk classification standards, matching rules between personnel qualifications and operation types, and requirements for wearing safety equipment. Verification content may include: whether the identified violations correspond to specific clauses in the regulations, whether the risk level of the multimodal fusion assessment is consistent with the risk classification standards in the regulations, and whether the qualifications of the operators meet the requirements of the regulations.

[0148] Regarding risk levels, if the risk level of the integrated assessment is consistent with the risk grading standards in the procedures, and there are additional events that fail the verification, the risk level can be increased / raised based on the original risk level. For example, if a violation is identified that does not correspond to a specific clause in the procedures, the risk level can be raised by one level, such as revising a low-risk level to a medium-risk level.

[0149] It should be noted that the risk level can be increased based on the number of additional events that fail the verification. For example, if there is one additional event that fails the verification, the risk level can be increased by one level. If there are two or more additional events, the risk level can be increased by two levels, until it reaches the extremely high risk level.

[0150] In this embodiment, by deeply embedding the power safety regulations knowledge base into the risk assessment process, the regulations and rules are transformed into constraints to verify and correct the identification results. The output results can be traced back to specific regulations clauses, significantly enhancing the transparency and interpretability of the assessment results and facilitating manual review and audit traceability.

[0151] In an exemplary embodiment, after issuing a risk warning, warning response actions can also be collected and used as feedback data for dynamic training of the model and dynamic updating of the knowledge base. Based on this, and building upon the above embodiments, as follows... Figure 5 As shown, the above method may further include steps 501 to 503. Wherein:

[0152] Step 501: Obtain the review information on risk warning information submitted by on-site safety supervisors.

[0153] Step 502: Determine the handling result of this early warning event based on the review information.

[0154] If the verified information indicates that the warning is true, the handling result includes the handling measures and the handling effect; if the verified information indicates that the warning is false, the handling result includes the reason for the false alarm.

[0155] Step 503: The handling results are used as feedback data for this early warning event. The feedback data is used for incremental learning of the risk assessment model, which is then used to conduct multi-dimensional risk assessment.

[0156] In this embodiment, the output risk warning information can be manually processed and confirmed by on-site safety supervisors, and the processing results (including but not limited to whether the warning is true, the processing measures taken, and the processing effect) can be recorded. On-site safety supervisors can review and confirm the risk warning information. If the warning is true, the processing measures taken and the processing effect can be recorded; if the warning is a false alarm, the reason for the false alarm can be recorded.

[0157] Based on this, computer equipment can use the handling results as feedback data, which can be used periodically for incremental training of the risk assessment model and optimization and updating of the knowledge base. By periodically integrating the feedback data into the training set and incrementally learning the risk assessment model, the model's recognition accuracy and scenario adaptability can be continuously optimized, as well as the system's risk recognition accuracy and scenario adaptability can be continuously improved.

[0158] Through the above processing, real-time intelligent assessment and early warning of risks at power operation sites can be achieved, meeting the real-time requirement of second-level early warning while ensuring the accuracy of multimodal fusion recognition.

[0159] In this embodiment, by constructing a closed-loop optimization mechanism driven by human feedback, the feedback data from on-site personnel continuously optimizes the risk assessment model through incremental learning. The system's recognition accuracy and scenario adaptability continuously improve with the increase of usage time, realizing a positive cycle of "use-feedback-optimization".

[0160] In an exemplary embodiment, addressing issues such as delayed risk perception, strong subjectivity in assessment, and difficulty in fusing multi-source information at power operation sites, a complete implementation of a real-time risk assessment method for power operation sites is provided. This method is designed for multimodal data such as video data, environmental sensor data, personnel positioning data, and work order text data. It constructs a unified risk representation space for the operation site through cross-modal feature extraction and semantic alignment. Based on this, a dynamic risk assessment model for multi-source feature fusion is designed. Combined with a power safety regulations knowledge base, it achieves real-time quantitative assessment and level determination of operational risks, ultimately outputting risk warning information and supporting closed-loop optimization driven by human feedback. Figure 6 As shown, the following steps may be included:

[0161] Step 1: Multimodal data acquisition and preprocessing.

[0162] For example, multimodal data acquisition terminals can be deployed at power work sites, including but not limited to: high-definition cameras for acquiring video images of the work site, environmental sensors for acquiring environmental parameters of the site, and personnel positioning terminals for acquiring real-time location data of workers; at the same time, work ticket information can be read from the electronic work ticket system, and relevant regulations and clauses can be read from the safety regulations knowledge base.

[0163] Taking high-altitude operations as an example, fixed cameras are deployed around the work area, and workers wear helmet-mounted cameras and positioning tags. Anemometers and temperature and humidity sensors are also deployed on-site.

[0164] Frame sampling is performed on video data, and the resolution of the sampled frames is uniformly adjusted to a preset size. Outlier removal is performed on sensor time-series data, and data from all sensors are standardized to the same sampling frequency. Coordinate transformation and trajectory smoothing are performed on positioning data. Structured parsing is performed on the work order text to extract key information such as work type, planned work period, list of required safety measures, and personnel qualification requirements.

[0165] Through the above processing, unified access and time synchronization of multimodal data are achieved, providing an aligned multimodal data foundation for subsequent parallel feature extraction.

[0166] Step 2, cross-modal risk feature extraction.

[0167] Parallel risk feature extraction is performed on the preprocessed multimodal data to obtain visual risk features, environmental risk features, behavioral risk features, and textual risk features.

[0168] Step 3: Multimodal feature fusion and risk assessment.

[0169] The visual risk features, environmental risk features, behavioral risk features, and textual risk features extracted in step 2 are subjected to cross-modal feature alignment and fusion. Based on the fused comprehensive risk features, risk scores are calculated from the dimensions of personnel safety, environmental conditions, and job type. The comprehensive risk score is obtained by weighting and summing the scores of each dimension. Risk levels and risk types are then determined based on the comprehensive risk score.

[0170] Step 4: Risk warning and closed-loop feedback.

[0171] Based on the risk level determined in step 3 and the specific risk type identified, structured risk warning information is generated. Different levels of alarm responses are triggered according to different warning levels. After the warning information is output, it is reviewed and confirmed by on-site safety supervisors. The processing results can be used as feedback data and periodically used for incremental learning of the risk assessment model to continuously optimize the model's recognition accuracy and scenario adaptability.

[0172] Accordingly, a real-time intelligent risk assessment system for power operation sites is also provided in response to the above method. This system may include: a multimodal data acquisition and preprocessing module, a cross-modal risk feature extraction module, a multimodal feature fusion and risk assessment module, and a risk early warning and closed-loop feedback module, wherein:

[0173] (1) Multimodal data acquisition and preprocessing module, used to acquire video data, environmental sensor data, personnel positioning data and work order text data, and to perform preprocessing operations such as format parsing, noise reduction and standardization on various types of data. This module includes a video acquisition and preprocessing engine, a sensor data acquisition engine, a positioning data acquisition engine, a work order parsing engine and a metadata extractor.

[0174] (2) Cross-modal risk feature extraction module, used to extract multi-dimensional risk features from multimodal data. This module includes a visual feature extraction engine (personnel posture recognition, security equipment detection, area intrusion detection), an environmental feature extraction engine, a behavioral feature extraction engine, and a text feature extraction engine. Each engine can work in parallel to meet real-time requirements.

[0175] (3) Multimodal feature fusion and risk assessment module, which is used to map multimodal risk features to a unified risk feature space and perform cross-modal alignment and weighted fusion, and perform multi-dimensional dynamic risk scoring and risk level determination based on the fused features. This module includes a cross-modal feature alignment engine, an attention fusion layer, a multi-dimensional risk scoring engine and a safety procedure knowledge base verification engine.

[0176] (4) Risk warning and closed-loop feedback module, which is used to generate and push warning information according to risk level, record the results of manual handling and drive incremental training of model and optimization and update of knowledge base. This module includes warning generation engine, hierarchical push engine, visualization display engine, feedback collection interface and incremental learning engine.

[0177] The above modules are connected sequentially. The output of the multimodal data acquisition and preprocessing module is connected to the feature extraction engines in the cross-modal risk feature extraction module. The output of each engine in the cross-modal risk feature extraction module is connected to the multimodal feature fusion and risk assessment module. The output of the multimodal feature fusion and risk assessment module is connected to the risk warning and closed-loop feedback module. The risk warning and closed-loop feedback module is connected back to the multimodal feature fusion and risk assessment module through the feedback acquisition interface and the incremental learning engine, realizing a closed loop for the entire process from data acquisition to risk warning and model optimization.

[0178] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0179] Based on the same inventive concept, this application also provides a real-time risk assessment device for power operation sites to implement the aforementioned real-time risk assessment method for power operation sites. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the real-time risk assessment device for power operation sites provided below can be found in the limitations of the real-time risk assessment method for power operation sites described above, and will not be repeated here.

[0180] In one exemplary embodiment, such as Figure 7 As shown, a real-time risk assessment device 700 for power operation sites is provided, including: a multimodal data acquisition module 701, a risk feature extraction module 702, a multimodal feature fusion module 703, a multi-dimensional risk assessment module 704, and a risk warning information generation module 705, wherein:

[0181] The multimodal data acquisition module 701 is used to acquire real-time multimodal data from the power operation site and preprocess the real-time multimodal data to obtain preprocessed multimodal data. The preprocessed multimodal data includes video data, environmental sensor data, personnel positioning data, and work order text data.

[0182] The risk feature extraction module 702 is used to extract risk features from each modality of data to obtain visual risk features, environmental risk features, behavioral risk features and text risk features.

[0183] The multimodal feature fusion module 703 is used to perform multimodal feature fusion on visual risk features, environmental risk features, behavioral risk features and text risk features to obtain comprehensive risk features.

[0184] The multi-dimensional risk assessment module 704 is used to conduct multi-dimensional risk assessments based on comprehensive risk characteristics to obtain risk assessment results. The risk assessment results include risk level and risk type. The multi-dimensional risk assessment includes personnel safety dimension scores, environmental condition dimension scores, and operation type dimension scores.

[0185] The risk warning information generation module 705 is used to generate structured risk warning information based on risk level, risk type and corresponding safety procedure information; the risk warning information is used to trigger alarm response at the corresponding level.

[0186] In one embodiment, the risk feature extraction module 702 includes:

[0187] The visual feature extraction unit is used to input video data into the visual feature extraction engine to extract visual features and obtain visual risk features;

[0188] The environmental feature extraction unit is used to input environmental sensor data into the environmental feature extraction engine to extract environmental features and obtain environmental risk features.

[0189] The behavior feature extraction unit is used to input personnel location data into the behavior feature extraction engine to extract behavior features and obtain behavior risk features.

[0190] The text feature extraction unit is used to input the work ticket text data into the text feature extraction engine to extract text features and obtain text risk features.

[0191] The visual feature extraction engine, environmental feature extraction engine, behavioral feature extraction engine, and text feature extraction engine are deployed on edge devices and executed in parallel. Each of these engines has a lightweight feature extraction model, and the size of the lightweight feature extraction model is smaller than the preset size.

[0192] In one embodiment, the multimodal feature fusion module 703 includes:

[0193] The projection unit is used to project visual risk features, environmental risk features, behavioral risk features and text risk features onto a unified feature dimension space to obtain aligned visual risk feature vectors, environmental risk feature vectors, behavioral risk feature vectors and text risk feature vectors.

[0194] The weight determination unit is used to input the visual risk feature vector, environmental risk feature vector, behavioral risk feature vector and text risk feature vector into the cross-modal attention module to determine the correlation weights between the features of each modality;

[0195] The feature fusion unit is used to perform multimodal feature fusion on visual risk features, environmental risk features, behavioral risk features and text risk features based on the correlation weights between the features of each modality, so as to obtain comprehensive risk features.

[0196] In one embodiment, the multi-dimensional risk assessment module 704 includes:

[0197] The personnel safety scoring unit is used to score personnel safety dimensions based on comprehensive risk characteristics to obtain a personnel safety risk score.

[0198] The environmental condition scoring unit is used to score environmental condition dimensions based on comprehensive risk characteristics to obtain an environmental condition risk score.

[0199] The job type scoring unit is used to score the job type dimension based on comprehensive risk characteristics and obtain the job type risk score.

[0200] The scoring fusion unit is used to weight and sum the personnel safety risk scores, environmental condition risk scores, and work type risk scores to obtain a comprehensive risk score; the weights corresponding to the risk scores of each dimension are dynamically adjusted based on the work type and work time stage;

[0201] The risk level determination unit is used to determine the risk level based on the comprehensive risk score and the preset risk threshold.

[0202] In one embodiment, the device further includes:

[0203] The constraint acquisition module is used to acquire preset constraint conditions; the preset constraint conditions are determined based on relevant rules in the power safety regulations knowledge base.

[0204] The risk level correction module is used to correct the risk level based on preset constraints and fusion identification results to obtain the corrected risk level; the fusion identification results are determined based on comprehensive risk characteristics.

[0205] In one embodiment, the preset risk thresholds include a first threshold, a second threshold, and a third threshold. The risk level determination unit is specifically used to determine the risk level as follows: if the comprehensive risk score is less than the first threshold, the risk level is low; if the comprehensive risk score is greater than or equal to the first threshold and less than the second threshold, the risk level is medium; if the comprehensive risk score is greater than or equal to the second threshold and less than the third threshold, the risk level is high; and if the comprehensive risk score is greater than or equal to the third threshold, the risk level is extremely high.

[0206] In one embodiment, if the risk level is low, the corresponding alarm response includes recording and marking in the monitoring backend; if the risk level is medium, the corresponding alarm response includes pushing a prompt message to the handheld terminal of the on-site safety supervisor; if the risk level is high, the corresponding alarm response includes on-site audible and visual alarms, highlighting on the monitoring screen, and pushing a prompt message to the handheld terminal of the on-site safety supervisor; if the risk level is extremely high, the corresponding alarm response includes on-site audible and visual alarms, highlighting on the monitoring screen, pushing a prompt message to the handheld terminal of the on-site safety supervisor, and notifying the emergency response center.

[0207] In one embodiment, the device further includes:

[0208] The review module is used to obtain review information on risk warning information submitted by on-site safety supervisors;

[0209] The handling result module is used to determine the handling result of this early warning event based on the review information. If the review information indicates that the early warning is true, the handling result includes the handling measures and the handling effect; if the review information indicates that the early warning is false, the handling result includes the reason for the false alarm.

[0210] The feedback module is used to use the handling results as feedback data for this early warning event. The feedback data is used for incremental learning of the risk assessment model, which is used to conduct multi-dimensional risk assessment.

[0211] Each module in the aforementioned real-time risk assessment device for power operations can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0212] In one exemplary embodiment, a computer device is provided, taking a server as an example, and its internal structure diagram can be as follows: Figure 8As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores real-time acquired multimodal data, intermediate process data and result data for risk assessment based on multimodal data, and early warning feedback data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a real-time risk assessment method for power operation sites.

[0213] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0214] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the real-time risk assessment method for power operation sites in any of the above embodiments.

[0215] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the real-time risk assessment method for power operation sites in any of the above embodiments.

[0216] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the real-time risk assessment method for power operation sites in any of the above embodiments.

[0217] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0218] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0219] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0220] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for real-time risk assessment at power operation sites, characterized in that, The method includes: Real-time multimodal data from the power operation site is acquired, and the real-time multimodal data is preprocessed to obtain preprocessed multimodal data; the preprocessed multimodal data includes video data, environmental sensor data, personnel positioning data, and work order text data; Risk features were extracted from each modality of data to obtain visual risk features, environmental risk features, behavioral risk features, and textual risk features; Multimodal feature fusion is performed on the visual risk features, environmental risk features, behavioral risk features, and text risk features to obtain comprehensive risk features; A multi-dimensional risk assessment is conducted based on the comprehensive risk characteristics to obtain risk assessment results; the risk assessment results include risk level and risk type; the multi-dimensional risk assessment includes personnel safety dimension score, environmental condition dimension score, and work type dimension score. Based on the risk level, the risk type, and the corresponding safety procedure information, structured risk warning information is generated; the risk warning information is used to trigger alarm responses at the corresponding level.

2. The method according to claim 1, characterized in that, The process of extracting risk features from each modality of data yields visual risk features, environmental risk features, behavioral risk features, and textual risk features, including: The video data is input into a visual feature extraction engine to extract visual features, thereby obtaining the visual risk features. The environmental sensor data is input into an environmental feature extraction engine to extract environmental features and obtain the environmental risk features. The personnel location data is input into a behavior feature extraction engine to extract behavior features, thereby obtaining the behavior risk features. The work order text data is input into a text feature extraction engine to extract text features, thereby obtaining the text risk features; The visual feature extraction engine, the environmental feature extraction engine, the behavioral feature extraction engine, and the text feature extraction engine are deployed on edge devices and executed in parallel. Each of the visual feature extraction engine, the environmental feature extraction engine, the behavioral feature extraction engine, and the text feature extraction engine is equipped with a lightweight feature extraction model, the size of which is smaller than a preset size.

3. The method according to claim 1, characterized in that, The multimodal feature fusion of the visual risk features, environmental risk features, behavioral risk features, and textual risk features yields a comprehensive risk feature, including: The visual risk features, environmental risk features, behavioral risk features, and text risk features are projected onto a unified feature dimension space to obtain aligned visual risk feature vectors, environmental risk feature vectors, behavioral risk feature vectors, and text risk feature vectors. The visual risk feature vector, the environmental risk feature vector, the behavioral risk feature vector, and the text risk feature vector are input into the cross-modal attention module to determine the correlation weights between each modality feature. Based on the correlation weights between the modal features, the visual risk features, environmental risk features, behavioral risk features, and textual risk features are fused using multimodal features to obtain the comprehensive risk features.

4. The method according to claim 1, characterized in that, The multi-dimensional risk assessment based on the comprehensive risk characteristics yields the following risk assessment results: Based on the comprehensive risk characteristics, a personnel safety dimension score is obtained to determine the personnel safety risk score. Environmental condition risk scores are obtained by scoring the environmental condition dimension based on the comprehensive risk characteristics. Based on the comprehensive risk characteristics, a risk score for each job type is obtained. The personnel safety risk score, the environmental condition risk score, and the job type risk score are weighted and summed to obtain a comprehensive risk score; the weights corresponding to the risk scores of each dimension are dynamically adjusted based on the job type and job time stage. The risk level is determined based on the comprehensive risk score and the preset risk threshold.

5. The method according to claim 4, characterized in that, The method further includes: Obtain preset constraints; the preset constraints are determined based on relevant rules in the power safety regulations knowledge base. Based on the preset constraints and the fusion identification results, the risk level is corrected to obtain the corrected risk level; the fusion identification results are determined based on the comprehensive risk characteristics.

6. The method according to claim 4, characterized in that, The preset risk thresholds include a first threshold, a second threshold, and a third threshold. Determining the risk level based on the comprehensive risk score and the preset risk thresholds includes: If the comprehensive risk score is less than the first threshold, then the risk level is low risk; If the comprehensive risk score is greater than or equal to the first threshold and less than the second threshold, then the risk level is medium risk. If the comprehensive risk score is greater than or equal to the second threshold and less than the third threshold, then the risk level is high risk. If the comprehensive risk score is greater than or equal to the third threshold, then the risk level is extremely high.

7. The method according to claim 1, characterized in that, If the risk level is low, the corresponding alarm response will be recorded and marked in the monitoring backend; If the risk level is medium risk, the corresponding alarm response includes pushing a prompt message to the handheld terminal of the on-site safety supervisor; If the risk level is high risk, the corresponding alarm response includes on-site audible and visual alarms, prominent marking on the monitoring screen, and push notification information to the handheld terminal of on-site safety supervisors; If the risk level is extremely high, the corresponding alarm response includes on-site audible and visual alarms, highlighting on the monitoring screen, pushing notification information to the handheld terminals of on-site safety supervisors, and notifying the emergency response center.

8. The method according to claim 1, characterized in that, The method further includes: Obtain verification information regarding the risk warning information submitted by on-site safety supervisors; The handling result of this early warning event is determined based on the verification information; wherein, if the verification information indicates that the early warning is true, the handling result includes the handling measures and the handling effect; if the verification information indicates that the early warning is false, the handling result includes the reason for the false alarm. The handling results are used as feedback data for this early warning event. The feedback data is used for incremental learning of the risk assessment model, which is used to conduct multi-dimensional risk assessment.

9. A real-time risk assessment device for power operation sites, characterized in that, The device includes: The multimodal data acquisition module is used to acquire real-time multimodal data from the power operation site and preprocess the real-time multimodal data to obtain preprocessed multimodal data. The preprocessed multimodal data includes video data, environmental sensor data, personnel positioning data, and work order text data. The risk feature extraction module is used to extract risk features from each modality of data to obtain visual risk features, environmental risk features, behavioral risk features, and textual risk features. The multimodal feature fusion module is used to perform multimodal feature fusion on the visual risk features, the environmental risk features, the behavioral risk features, and the text risk features to obtain comprehensive risk features; The multi-dimensional risk assessment module is used to perform multi-dimensional risk assessment based on the comprehensive risk characteristics to obtain risk assessment results; the risk assessment results include risk level and risk type; the multi-dimensional risk assessment includes personnel safety dimension score, environmental condition dimension score, and work type dimension score. The risk warning information generation module is used to generate structured risk warning information based on the risk level, the risk type, and the corresponding safety procedure information; the risk warning information is used to trigger alarm responses at the corresponding level.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images