Power distribution station room environment hidden danger diagnosis method and device and nonvolatile storage medium

Abstract

This invention discloses a method, device, and non-volatile storage medium for diagnosing environmental hazards in a substation. The method includes: acquiring a monitoring video stream and data from multiple environmental sensors in the substation, including temperature and humidity sensors; aligning the monitoring video stream and the data from the environmental sensors to obtain aligned image frames; and detecting the aligned image frames based on a preset lightweight visual model, outputting candidate hazards and their information. The preset lightweight visual model is trained on multiple images labeled with hazard types in the substation, and the information includes the hazard category and location. This invention solves the technical problem of high false alarm rates in current lightweight models for diagnosing hazards in complex substation environments, leading to low accuracy in diagnostic results.

Description

Technical Field

[0001] This invention relates to the field of power operation and maintenance technology, and more specifically, to a method, device, and non-volatile storage medium for diagnosing environmental hazards in a power distribution station room. Background Technology

[0002] As a core node in urban power distribution networks, substations are frequently threatened by hazards such as smoke, water accumulation, small animal intrusion, and foreign object buildup. Traditional methods relying on manual inspections or single-sensor alarms suffer from limitations such as limited perception dimensions, delayed response, and high false alarm rates. Existing intelligent detection technologies often employ lightweight visual models for real-time identification, which are susceptible to complex environmental factors such as changes in lighting, ground reflections, and dust interference, leading to frequent false alarms. Introducing large-scale full-scale models for all-weather inference faces bottlenecks such as insufficient edge computing power, response latency, and high energy consumption, and lacks constraints from power operation procedures and physical mechanisms, easily generating "illusory diagnoses" that contradict real-world logic. Furthermore, statically deployed models struggle to adapt to seasonal environmental changes in substations (such as increased humidity, equipment aging, and gradual lighting changes), causing diagnostic performance to continuously decline over time. Currently, there is no effective technical solution that can simultaneously achieve high-precision detection, logical interpretability, and long-term adaptive capabilities while ensuring low-computing deployment.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a method, device, and non-volatile storage medium for diagnosing environmental hazards in substation rooms, thereby at least solving the technical problem of low accuracy of diagnostic results due to high false alarms in lightweight models in complex substation environments.

[0005] According to one aspect of the present invention, an environmental hazard diagnosis method for a power distribution room is provided, comprising: acquiring a monitoring video stream of the power distribution room and data corresponding to multiple environmental sensors, wherein the multiple environmental sensors include temperature and humidity sensors; aligning the monitoring video stream and the data corresponding to the multiple environmental sensors to obtain aligned image frames; detecting the aligned image frames based on a preset lightweight visual model, and outputting candidate hazards and information about the candidate hazards, wherein the preset lightweight visual model is trained based on multiple images labeled with hazard types in the power distribution room, and the information includes hazard category and hazard location; scoring the candidate hazards based on the information of the candidate hazards according to a preset comprehensive evaluation index to obtain a score corresponding to the candidate hazard, wherein the preset comprehensive evaluation index includes hazard type, risk level of hazard location, environmental false alarm characteristics of monitoring position, and whether the monitoring video stream and the data corresponding to the multiple environmental sensors are in good condition. The process involves several steps: First, identifying candidate hazards with scores exceeding a preset threshold as target hazards. Second, determining multimodal data corresponding to the target hazard based on monitoring video streams and data from multiple environmental sensors. This multimodal data includes hazard images, environmental images of the hazard, monitoring location and area type, and sensor state sequences and short-term historical trend summaries within the corresponding time window. Third, inputting the multimodal data into a large multimodal model for cross-modal semantic analysis to output hazard results. These results include hazard status, hazard category and risk level, structured multidimensional evidence description chain, and suggested actions for coordinated inspections. Hazard status results include both real and false alarms. Fourth, verifying hazard status results for those that are real hazards based on the spatial layout of the substation to obtain target hazard diagnosis results. Verification includes checking the legality of the hazard location, whether the hazard conflicts with data from multiple environmental sensors, and whether it conforms to physical evolution patterns.

[0006] Optionally, based on a preset lightweight visual model, the aligned image frames are detected, and candidate potential hazards are output. This includes: inputting the aligned image frames into the preset lightweight visual model, performing forward inference, and outputting multi-scale candidate target prediction results corresponding to the aligned image frames. The multi-scale candidate target prediction results include the class probabilities, bounding box coordinates, and confidence scores of candidate targets at multiple scales. Based on a preset confidence threshold, the multi-scale candidate target prediction results corresponding to the aligned image frames are filtered to obtain filtered candidate targets. The detection is performed to determine whether the bounding boxes corresponding to the filtered candidate targets overlap. If there is positional overlap in the bounding boxes corresponding to the filtered candidate targets, redundant targets are deleted, and candidate potential hazards are output. The redundant targets represent candidate targets that have positional overlap and whose confidence scores do not meet the preset conditions.

[0007] Optionally, based on preset comprehensive evaluation indicators, candidate hazards are scored according to their information to obtain corresponding scores, including: based on preset comprehensive evaluation indicators, determining whether a candidate hazard is a high-risk hazard type according to its corresponding hazard category, and obtaining a first judgment result; based on preset comprehensive evaluation indicators, determining whether a candidate hazard is located in a high-risk area according to its corresponding hazard location, and obtaining a second judgment result; obtaining historical false alarm records of monitoring positions corresponding to candidate hazard; based on preset comprehensive evaluation indicators, determining whether a candidate hazard is in a high-interference environment according to the historical false alarm records of monitoring positions corresponding to candidate hazard, and obtaining a third judgment result; based on preset comprehensive evaluation indicators, for candidate hazard involving water seepage and dripping, extracting contradictory signals between visual detection conclusions and spatial water immersion switch quantities or humidity change rates, and continuously calculating the area growth rate index of the candidate hazard within the corresponding time window, judging whether the candidate hazard is consistent with the change trends of data corresponding to multiple environmental sensors, and obtaining a fourth judgment result; based on the evaluation weights of multiple evaluation indicators in the preset comprehensive evaluation indicators, determining the corresponding score of the candidate hazard according to the first, second, third, and fourth judgment results.

[0008] Optionally, based on the monitoring video stream and data from multiple environmental sensors, the multimodal data corresponding to the target hazard is determined, including: extracting a cropped local image of the area where the target hazard is located based on the monitoring video stream to determine the hazard image; extracting the environmental background image of the monitoring screen where the target hazard is located based on the monitoring video stream to determine the environmental image of the hazard; obtaining the monitoring location corresponding to the target hazard, the type of area it occupies in the substation room space, and a short-term historical evolution trend summary within the corresponding time window, wherein the short-term historical evolution trend summary includes temperature and humidity slope, smoke detector status, and access control records; and determining the multimodal data corresponding to the target hazard based on the hazard image, the environmental image of the hazard, the monitoring location, the type of area, data from multiple environmental sensors, and the short-term historical evolution trend summary.

[0009] Optionally, the multimodal data is input into a multimodal large model for cross-modal association semantic parsing, and the hazard results are output, including: generating a constraint prompt template based on the multimodal data, wherein the constraint prompt template includes key area priors and multi-source consistency check requirements; inputting the multimodal data and constraint prompt template into the multimodal large model for cross-modal association semantic parsing, and outputting the hazard results.

[0010] Optionally, it also includes: storing the hazard status result that is a false alarm and the corresponding multimodal data into a first sample set; storing the hazard status result that is a real hazard and the corresponding multimodal data into a second sample set; and adjusting the preset lightweight visual model based on the first sample set and the second sample set.

[0011] Optionally, based on the hazard status results that are false alarms and the corresponding multimodal data, the deviation pattern is determined according to the characteristics of environmental false alarms; based on the deviation pattern, the adjustment parameters for the initial screening confidence threshold and the adjustment parameters for the gated scoring trigger score line are determined; based on the adjustment parameters for the initial screening confidence threshold and the adjustment parameters for the gated scoring trigger score line, the preset lightweight visual model is updated; based on the hazard status results that are false alarms and the corresponding multimodal data, the constraint prompt template used to constrain the large multimodal model is updated.

[0012] Optionally, based on the spatial layout of the substation, the hazard status results that are real hazards are verified to obtain the target hazard diagnosis results. The verification includes whether it conforms to the laws of physical evolution, including: extracting the area boundary change and trajectory displacement features of the target hazard in continuous frames of a preset duration; based on the area boundary change and trajectory displacement features, and according to the preset differential evolution law of physical entities and optical environment noise, verifying whether the target hazard has stable extension attributes to determine whether it conforms to the laws of physical evolution.

[0013] According to another aspect of the present invention, an environmental hazard diagnosis device for a power distribution room is also provided, comprising: a data acquisition module for acquiring monitoring video streams and data corresponding to multiple environmental sensors in the power distribution room, wherein the multiple environmental sensors include temperature and humidity sensors; an alignment module for aligning the monitoring video streams and data corresponding to the multiple environmental sensors to obtain aligned image frames; a first output module for detecting the aligned image frames based on a preset lightweight visual model and outputting candidate hazards and information about the candidate hazards, wherein the preset lightweight visual model is trained based on multiple images labeled with hazard types in the power distribution room, and the information includes hazard category and hazard location; and a scoring module for scoring candidate hazards based on the information of the candidate hazards according to preset comprehensive evaluation indicators to obtain the corresponding scores of the candidate hazards, wherein the preset comprehensive evaluation indicators include hazard type, risk level of hazard location, environmental false alarm characteristics of monitoring station, and whether the monitoring video stream and data corresponding to the multiple environmental sensors are in a state of readiness. The system comprises the following modules: a first determination module, used to identify candidate hazards whose scores exceed a preset threshold as target hazards; a second determination module, used to determine the multimodal data corresponding to the target hazard based on the monitoring video stream and data from multiple environmental sensors, wherein the multimodal data includes the hazard image, the environmental image of the hazard, the monitoring location and the type of the area, and the sensor state sequence and short-term historical trend summary within the corresponding time window; a second output module, used to input the multimodal data into a multimodal large model for cross-modal association semantic parsing and output the hazard result, wherein the hazard result includes the hazard status result, the hazard category and the hazard risk level, the structured multidimensional evidence description chain, and the action suggestions for joint inspection, and the hazard status result includes real hazards and false alarms; and a verification module, used to verify the hazard results that are real hazards based on the spatial layout of the substation room, to obtain the target hazard diagnosis result, wherein the verification includes whether the hazard location is legal, whether the hazard conflicts with the data corresponding to multiple environmental sensors, and whether it conforms to the physical evolution law.

[0014] According to another aspect of the present invention, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute any of the above-described environmental hazard diagnosis methods for substation rooms.

[0015] According to another aspect of the present invention, a computer device is also provided, the computer device including a processor, the processor being configured to run a program, wherein the program, when running, executes any of the above-described methods for diagnosing environmental hazards in a power distribution room.

[0016] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements any of the above-described methods for diagnosing environmental hazards in a power distribution room.

[0017] In this embodiment of the invention, an environmental hazard diagnosis method for a power distribution room is adopted. This involves collecting monitoring video streams and data from multiple environmental sensors, including temperature and humidity sensors. The monitoring video streams and the data from the environmental sensors are aligned to obtain aligned image frames. Based on a preset lightweight visual model, the aligned image frames are detected, and candidate hazards and their information are output. The preset lightweight visual model is trained on images labeled with multiple hazard types in the power distribution room, and the information includes hazard category and hazard location. Based on a preset comprehensive evaluation index, the candidate hazards are scored according to their information to obtain corresponding scores. The preset comprehensive evaluation index includes hazard type, risk level of hazard location, environmental false alarm characteristics of the monitoring station, and consistency between the monitoring video stream and the data from the environmental sensors. Candidate hazards with scores exceeding a preset threshold are identified as target hazards. Finally, based on the monitoring video streams and the data from the environmental sensors, the target hazards are determined. The system utilizes multimodal data corresponding to potential hazards, including hazard images, environmental images of the hazard, monitoring locations and area types, sensor state sequences within corresponding time windows, and short-term historical trend summaries. This multimodal data is input into a large multimodal model for cross-modal semantic analysis, outputting hazard results. These results include hazard status, hazard category and risk level, structured multidimensional evidence description chains, and suggested actions for coordinated inspections. The hazard status results include both real and false alarms. Based on the spatial layout of the substation, hazard status results that identify real hazards are validated to obtain target hazard diagnostic results. Validation includes verifying the legality of the hazard location, whether the hazard conflicts with data from multiple environmental sensors, and whether it conforms to physical evolution laws. This achieves the goal of accurate diagnosis and adaptive closed-loop evolution of environmental hazards in substations based on hierarchical reasoning using a heterogeneous model, thereby improving the accuracy of hazard diagnosis and solving the current technical problem of low accuracy in diagnostic results due to high false alarm rates in lightweight models in complex substation environments. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0019] Figure 1A hardware block diagram of a computer terminal for implementing an environmental hazard diagnosis method in a substation room is shown.

[0020] Figure 2 This is a flowchart illustrating the environmental hazard diagnosis method for a power distribution room according to an embodiment of the present invention;

[0021] Figure 3 This is a structural block diagram of an environmental hazard diagnosis device for a substation room provided according to an embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] According to an embodiment of the present invention, a method embodiment for diagnosing environmental hazards in a power distribution room is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0025] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal for implementing an environmental hazard diagnosis method in a substation room is shown. Figure 1As shown, the computer terminal 10 may include one or more processors (shown as 102a, 102b, ..., 102n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0026] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10. As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0027] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the environmental hazard diagnosis method for the substation room in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned application method for diagnosing environmental hazard problems in the substation room. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0028] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10.

[0029] Figure 2 This is a flowchart illustrating the environmental hazard diagnosis method for a substation room according to an embodiment of the present invention, as shown below. Figure 2As shown, the method includes the following steps:

[0030] Step S202: Collect the monitoring video stream of the substation room and the data corresponding to each of the multiple environmental sensors, including temperature and humidity sensors.

[0031] In this step, to achieve intelligent sensing of environmental hazards such as smoke, fire, small animal intrusion, and water leakage in the substation operating environment, the system deploys multiple fixed industrial cameras in key areas of the substation to continuously collect real-time monitoring video streams. The cameras are installed in high-risk areas such as the bottom of switchgear, cable trench entrances, power distribution channels, drainage points, and access control areas. The acquired image sequences are output continuously frame by frame at fixed time intervals, forming the raw visual data stream. Due to the complex lighting conditions inside the substation, images are often affected by low illumination, glare from floor tiles, lamp flicker, suspended dust, steam, and sudden changes in light caused by door opening and closing, resulting in significant fluctuations in image quality. This places high demands on the robustness of the visual inspection model.

[0032] Simultaneously, the system connects to multiple environmental physical sensors deployed within the station, including smoke detectors, water immersion sensors, access control switches, and temperature and humidity sensors. Smoke detectors detect whether the smoke concentration in the air reaches the alarm threshold, outputting a binary switch signal. Water immersion sensors, installed in low-lying areas or along cable trench edges, detect liquid contact and also output a switch signal. Access control sensors record the opening and closing status of equipment room doors, providing a basis for determining whether environmental disturbances are caused by personnel activity. Temperature and humidity sensors continuously collect analog values ​​of ambient temperature and relative humidity to reflect the steady-state and dynamic trends of the station's internal microclimate. These sensors are typically deployed with dual-point redundancy to improve data reliability and are uploaded to edge computing nodes via industrial bus or wired / wireless communication protocols.

[0033] All video and sensor data are timestamped locally at the acquisition end and calibrated through a unified time synchronization mechanism to ensure comparability of data from different sources in the temporal dimension. Despite differences in sampling frequencies among devices (e.g., video frame rates of 5-15 FPS and sensor sampling frequencies of 1-5 Hz), the system guarantees that each frame corresponds to a temporally closest set of sensor states, providing a foundation for subsequent multimodal fusion and logical consistency judgment. This acquisition process is independent of manual intervention, supports continuous operation 24 / 7, and the acquired data constitutes the initial perceptual input for the system's hazard diagnosis, serving as the starting point and support for the overall technical approach of this invention.

[0034] Step S204: Align the monitoring video stream and the data corresponding to each of the multiple environmental sensors to obtain aligned image frames.

[0035] In this step, within the operating environment of the substation, the video surveillance system and the environmental sensor network are typically composed of different hardware devices, with differences in data acquisition frequency, communication protocol, and internal clock source. Cameras continuously output image frame sequences at a fixed frame rate, while sensors for temperature and humidity, smoke detection, water immersion, and access control periodically upload status data at a higher sampling frequency. Due to inherent latency, network jitter, buffer delays, and local clock drift between devices, directly matching video frames with sensor data sequentially will cause a temporal misalignment between visual events and physical states, leading to misjudgments, missed judgments, or logical contradictions.

[0036] To ensure the accuracy of subsequent multimodal inference and consistency checks, the system must perform precise timestamp alignment on all data sources. This process is based on a unified global timeline and uses hardware time synchronization protocols, such as PTP (Precise Time Protocol), or software interpolation algorithms to precisely match video frames with sensor data in the time dimension. Specifically, for each frame... The system searches for the set of sensor states that is closest to its timestamp. And by removing asynchronous data points caused by communication delays or cache buildup, a one-to-one aligned data pair is finally constructed:

[0037]

[0038] in, This represents the video frames or set of frames aligned at time t. This represents a multi-source sensor state slice that matches the time window of the frame, including environmental physical quantities such as smoke detection status, water immersion status, access control status, temperature, and humidity values. This alignment process does not change the original data content, but only calibrates its temporal assignment, ensuring that each frame of visual image corresponds to a unique, stable, and interpretable snapshot of the environmental state.

[0039] This alignment mechanism is a key element in the diagnostic process of this invention. For example, when a lightweight model detects a suspected water accumulation area, if the water immersion sensor at the corresponding time point is in an "untriggered" state, but the temperature and humidity values ​​are continuously rising, it can be initially judged as a false alarm due to reflection. Conversely, if the water immersion sensor is simultaneously flipped, the humidity slope is significant, and the area growth rate meets the threshold, the weight of determining actual leakage can be increased. If precise alignment is not performed, such a chain of evidence will become invalid due to time offset, leading to an increased false alarm rate or the omission of key hidden dangers.

[0040] In addition, the aligned image frames Not only does it serve as input for subsequent visual detection, but it also becomes the core visual anchor for constructing the multimodal evidence input matrix. During the review phase, this frame is assembled with the locally cropped image, the global background image, camera semantic labels, and sensor temporal summaries into a structured input for cross-modal semantic parsing by the large multimodal model. If the image and sensor data are out of sync in time, the model will receive "false context," easily generating illusory conclusions that contradict physical mechanisms.

[0041] The alignment mechanism implemented in this step does not rely on the absolute precision of any external clock source. Instead, it achieves highly reliable synchronization suitable for engineering use under existing industrial equipment conditions through relative time difference compensation and a sliding window matching strategy. Its output is an aligned image frame. It is the only visual observation unit in the entire hierarchical reasoning system with a clear time anchor point, providing a traceable, reproducible, and auditable time benchmark for subsequent gating triggering, evidence arrangement, and knowledge verification, and serving as the underlying support for ensuring the long-term stable operation of the system.

[0042] Step S206: Based on the preset lightweight visual model, the aligned image frames are detected, and candidate hazards and their information are output. The preset lightweight visual model is trained on multiple images labeled with hazard types in the substation room, and the information includes hazard category and hazard location.

[0043] In this step, after completing the spatiotemporal alignment of image frames, the system inputs the aligned image into a preset lightweight visual hazard detection model to achieve rapid initial screening of typical environmental hazards in substation rooms. This model is trained on a large amount of labeled real image data of substation rooms. The training samples cover four types of hazards: smoke and flames, animal intrusion, foreign object intrusion, and water leakage. It also covers typical interference scenarios such as low light, strong reflections, dust disturbance, and sudden changes in light intensity when doors open and close, ensuring the model's robustness to real-world engineering environments.

[0044] The model employs a lightweight object detection architecture, preferentially utilizing the YOLOv8 network structure. Its parameter scale and inference efficiency have been optimized for edge deployment, enabling real-time frame-level processing on low-computing-power devices. The model performs forward inference on the input image frames and outputs a set of candidate targets. Each candidate target contains three core pieces of information: hazard category, hazard location, and initial detection confidence.

[0045] The hazard categories are limited to a preset set {smoke and flame, animal intrusion, foreign object intrusion, water seepage and dripping}, ensuring that the output focuses on the key risk types of the substation room; the hazard location is output in the form of two-dimensional bounding box coordinates, which is used to accurately locate the pixel area of ​​the hazard in the image, supporting subsequent area cropping, key area hit judgment and spatial topology verification.

[0046] The initial detection confidence score is a quantitative assessment of the model's reliability of the current detection results. Its value is obtained by mapping the original network output through the Sigmoid function, and the specific calculation method is as follows:

[0047]

[0048] in, This is the raw score output by the network. Map the score to an interval In the substation room scenario, It reflects the uncertainty of the model's judgment under interference conditions such as reflection, shadow, and vapor, and serves as an important input for subsequent gating triggering. This confidence level reflects the model's confidence in judging the target in complex backgrounds and is the core input basis for the subsequent "multi-dimensional threshold gating triggering evaluation" step.

[0049] This step does not output a final diagnostic conclusion, but only a set of candidate potential hazards that meet the minimum confidence threshold, providing a pool of candidate targets for subsequent heterogeneous model verification. The output of this lightweight model provides a structured and traceable visual foundation for subsequent multimodal verification and adaptive iteration.

[0050] Step S208: Based on the preset comprehensive evaluation indicators, the candidate hidden dangers are scored according to the information of the candidate hidden dangers to obtain the corresponding scores. The preset comprehensive evaluation indicators include the type of hidden danger, the risk level of the location of the hidden danger, the environmental false alarm characteristics of the monitoring position, and whether the monitoring video stream and the data corresponding to multiple environmental sensors are consistent.

[0051] In this step, after the lightweight visual model outputs candidate hazards and their basic information, the system needs to further evaluate the credibility and risk level of each candidate hazard to determine whether to trigger the resource-intensive multimodal large model review process. To this end, the system introduces a set of pre-set comprehensive evaluation indicators. These indicators comprehensively consider the physical hazard, spatial location sensitivity, environmental interference tendency, and consistency of multi-source sensor evidence to quantify and score each candidate hazard, forming a review trigger score.

[0052] The scoring process targets each candidate hazard. Construct evaluation vectors Each component corresponds to: the risk factor mapped by the type of hazard, the model uncertainty factor, the hit weight of the key area, the environmental characteristics of false alarms at the location, and the consistency support of visual and sensor data.

[0053] First, risk factors According to the category of hazard The risk is calculated using a weighted average with the spatial area where the bounding box is located. The system pre-defines a set of labels for key areas of the station, including "bottom of switchgear," "cable trench," "near busbar room," "drainage point," and "doorway," and assigns different basic risk weights to different areas. For example, when a candidate hazard is "water leakage" and its location overlaps with the "cable trench" or "bottom of switchgear" area, the system assigns it a high risk value. ,in The label is the hit area determined by image semantic segmentation or a preset region mask; conversely, if the potential hazard occurs in a low-risk area such as a "doorway", the risk value is reduced.

[0054] Secondly, uncertainty factors This measure assesses the fluctuation in the model's confidence level for the current detection. Its calculation depends on the stability of the candidate hazard's confidence level within a short historical window, and is expressed as follows:

[0055]

[0056] in, Display window The confidence sequence corresponding to this event, i.e.:

[0057]

[0058] in Display window The confidence sequence corresponding to this event. Represents the variance operator. These are the weighting coefficients. This is the short-time window length.

[0059] Third, key regional factors A binary or continuous mapping function used to characterize candidate bounding boxes. Whether it falls within a high-risk area. When the spatial overlap between the candidate area and preset key areas such as "cable trench" and "cabinet bottom" exceeds a threshold, this factor takes a high value; otherwise, it takes a low value. This factor does not rely on a formula, but its mapping rules are predefined by the station equipment topology diagram to ensure that the assessment closely matches actual operation and maintenance.

[0060] Fourth, scene factors This is used to characterize the historical false alarm characteristics of a specific camera position in the current time period. For example, a certain camera position may frequently falsely report "water accumulation" due to the reflection of paving stones during the rainy season, or falsely report "foreign objects" due to changes in the angle of the lights during the morning and evening. Based on historical false alarm statistics, the system assigns a dynamic false alarm tendency score to this camera position-time combination as a priori penalty to suppress false alarms.

[0061] Fifth, consistency factor This is used to quantify the degree of matching between visual inspection results and the status of synchronous sensors. When the hazard category is "water accumulation, seepage, and dripping," the system defines the status of the water immersion sensor. The humidity change trend serves as a key verification criterion. If visual detection indicates water accumulation, but the water immersion sensor does not trigger and the humidity does not show an increasing trend, the consistency factor decreases; conversely, if the sensor status supports this, the factor increases. Specifically, to identify false alarms related to leakage, the system introduces contradictory signals:

[0062]

[0063] in This indicates the sensor's support status against water immersion and leakage. Indicates the strength of the contradiction between visual and sensory evidence. Indicates inconsistency. This indicates agreement.

[0064] Finally, the system weights and aggregates the above factors to generate a comprehensive review trigger score for candidate potential hazards:

[0065]

[0066] in, For the gated weight vector, For contradictory signal weights, Weighted by growth rate, To the area growth rate The normalized value is defined as:

[0067]

[0068] in This is the area calculation function. Indicates time The corresponding location area at any given time. To prevent tiny positive numbers with a denominator of 0, the area growth rate and contradictory signals are used together to identify typical false alarm patterns in the substation room, such as residual water marks from mopping, reflective floor tiles, and shadow drift.

[0069] This step enables a scientific and quantitative assessment of candidate potential risks, avoiding the proliferation of false alarms caused by relying solely on a single visual confidence level, and providing a calculable, interpretable, and iterative decision-making basis for the on-demand allocation of heterogeneous model resources.

[0070] Step S210: Identify candidate hazards whose scores exceed a preset threshold as target hazards.

[0071] After completing the comprehensive scoring of each candidate hazard, the system will filter out candidate events with high-risk characteristics or strong evidence contradictions based on preset dynamic trigger thresholds, and formally identify them as target hazards, serving as the sole input for subsequent multimodal large-scale model review and closed-loop handling. This screening process does not use a static uniform threshold, but rather an adaptive gating mechanism that dynamically adjusts based on station location, time period, seasonal environment, and historical false alarm distribution characteristics to cope with environmental drift caused by gradual changes in lighting, aging of ground materials, and adjustments to equipment layout during long-term operation of the substation.

[0072] Specifically, the system provides a comprehensive score for each candidate hazard in the output. With a time-varying dynamic threshold A comparison is made. This threshold function is maintained by the system's online adaptive module, and its value varies depending on the camera number. The system dynamically evolves with the current time t, reflecting the false alarm characteristics of different camera positions at different times. For example, on a rainy morning, a camera facing a tiled area may experience increased reflection, leading to a higher historical false alarm rate. The system will automatically raise the threshold for that camera position to suppress false water accumulation detection caused by reflection. Conversely, in the hot and humid afternoon, if a cable trench camera position experiences multiple contradictory combinations of "water accumulation + increased humidity + no water immersion trigger," the system may lower the threshold for that time period to improve sensitivity to potential real leaks.

[0073] When the comprehensive score of a candidate hidden danger meets the following conditions:

[0074]

[0075] The system then identifies it as a potential hazard and marks it as a high-priority event requiring cross-modal review. This identification means that the event simultaneously possesses at least one of the following characteristics: the hazard type is located in a high-risk area, the model confidence level is low and the temporal fluctuation is large, there is a significant contradiction between visual detection and sensor status, or the water accumulation shows a continuous spreading trend.

[0076] For potential risks that do not meet the threshold, the system does not trigger a large model review, but only makes a light record or directly classifies them as "low-risk events," thus avoiding excessive consumption of computing resources by low-confidence, low-risk false alarms. This mechanism effectively solves the core problem of "full-scale calling of the large model leading to computing overload" in traditional solutions, and achieves precise on-demand allocation of computing resources.

[0077] Only events marked as potential threats will proceed to the subsequent multi-source heterogeneous evidence compilation, multimodal large-model inference, and secondary verification of knowledge rules. Therefore, this step is not only a filtering mechanism but also a core decision point in the entire hierarchical inference architecture, achieving a balance between maximizing resource efficiency and optimizing diagnostic accuracy. Its dynamic threshold mechanism ensures that the system possesses adaptive disturbance resistance capabilities during long-term operation, maintaining high stability and low false alarm diagnostic performance without frequent manual intervention.

[0078] Step S212: Based on the monitoring video stream and the data corresponding to multiple environmental sensors, determine the multimodal data corresponding to the target hazard. The multimodal data includes the hazard image, the environmental image of the hazard, the monitoring location and the type of the area, as well as the sensor status sequence and short-term historical trend summary within the corresponding time window.

[0079] In this step, after identifying the target potential hazard, the system needs to construct a structured and context-complete set of input evidence for subsequent multimodal large-scale model verification. This process does not rely on local information from a single image frame, but rather deeply integrates visual perception and environmental sensing data to construct a multimodal data package with spatial integrity and physical consistency support. This ensures that the large model can reason within a sufficient semantic context and avoids hallucination-like judgments due to missing information.

[0080] First, the system extracts the image of the target hazard, specifically by precisely cropping the bounding box region of the identified candidate hazard to obtain a local image fragment focusing on the hazard itself. This image preserves key visual features such as the texture and diffusion pattern of smoke, the outline details of animals, the shape and structure of foreign objects, or the edge distribution of accumulated water, which are the core basis for the model to determine the authenticity of the hazard. The cropping operation is strictly performed according to the coordinate position of the target hazard in the original image, ensuring no information offset.

[0081] Secondly, the system synchronously extracts the environmental image of the potential hazard, namely the global context image of the image frame containing the target hazard. This image is not cropped and fully preserves background information such as the overall layout of the station, equipment distribution, lighting status, floor tile material, cable routing, and access control location, providing spatial semantic context for the large model. For example, when the hazard is "water accumulation," the global image can help the model determine whether the area is located at the edge of a cable trench, at the bottom of a switch cabinet, or near a drainage point, thus assisting in identifying whether it is actual leakage or reflection from the floor tiles; when the hazard is "animal intrusion," the global image can indicate whether there are gaps in equipment, ventilation openings, or unsealed cable holes, enhancing the rationality of the judgment.

[0082] Third, the system acquires monitoring location information, namely the unique identifier of the camera where the target hazard is located and its deployment semantic tag. This information is provided by the system's preset site topology database, clearly marking the installation location of each camera, such as "view from the cable trench south of power distribution room No. 1", "monitoring the passage directly in front of switch cabinet A", and "viewing the camera position from above the drainage outlet". This tag does not rely on image recognition, but is bound through static configuration during engineering deployment, ensuring that subsequent reasoning can be associated with the specific physical spatial location, providing a precise anchor point for the spatial legality verification of the knowledge rule base.

[0083] Fourth, the system determines the area type, i.e., the functional zoning of the station corresponding to the spatial coordinates of the target hazard. This area type is determined by the spatial overlap between a preset area mask or semantic segmentation map and the hazard boundary box, limiting it to several key operation and maintenance areas such as "cable trench," "bottom of switchgear," "surroundings of busbar room," "doorway passage," "critical flood control area," and "edge of blind zone." This information comes directly from the station equipment layout drawings and equipment location coverage model, and is used to distinguish the physical risk level of the hazard location. For example, the potential hazards of water accumulation in the same form appearing in a cable trench are completely different from those appearing on the floor tiles of a passageway, and must be clearly distinguished.

[0084] The aforementioned images of potential hazards, environmental images, monitoring locations, and area types collectively constitute a multimodal dataset of the target hazard. This dataset does not contain raw video streams or full sensor data, but rather a minimum necessary evidence package that has been filtered, encapsulated, and structured. This satisfies the contextual integrity requirements of large model inference while strictly controlling the data transmission scale to ensure network bandwidth and security compliance requirements at the station end.

[0085] This multimodal dataset is further bound with timestamps and unique identifiers for target hazards during encapsulation, forming a traceable input payload. This payload, along with auxiliary information such as the calculated comprehensive score, conflict signals, and area growth rate, is then transmitted to the platform's multimodal reassessment service. This design effectively suppresses misjudgments and illusions caused by insufficient local information in the model, laying a solid data foundation for subsequent output of interpretable, auditable, and actionable diagnostic conclusions.

[0086] Step S214: Input multimodal data into a multimodal large model for cross-modal association semantic parsing and output hazard results. The hazard results include hazard status results, hazard category and hazard risk level, structured multidimensional evidence description chain, and action suggestions for joint inspection. The hazard status results include real hazards and false hazards.

[0087] In this step, after the structured encapsulation of multimodal data is completed, the system takes evidence information such as local images of potential hazards, global environmental images, semantic labels of camera monitoring locations, and the type of the area as input, and feeds it into a multimodal large model deployed on the platform for joint reasoning. This model, based on a pre-trained vision-language-context fusion architecture, performs cross-modal semantic alignment and deep interaction on the input multi-source heterogeneous data, comprehensively analyzing the physical nature of visual anomalies and their true meaning in the power operation environment.

[0088] The model analyzes smoke textures, animal outlines, foreign object shapes, or water accumulation edge features in local images, combined with equipment layout, lighting patterns, and ground texture in global images, to identify whether an event constitutes a real environmental hazard. Simultaneously, the model incorporates monitoring location tags and area types to determine whether the hazard occurs in high-risk physical areas such as cable trenches or the bottom of switch cabinets, thereby enhancing the business relevance of the judgment.

[0089] Based on this, the model integrates time-synchronized sensor trend summaries to assess whether the visual results are consistent with changes in environmental physical quantities: if there is a contradiction of "visual water accumulation but no water immersion trigger and no increase in humidity", the system tends to judge it as a false alarm; if there is multiple supporting evidence such as "continuous expansion of the boundary, simultaneous increase in temperature and humidity, and location in a cable trench", it is judged as a real hidden danger.

[0090] Ultimately, the model outputs structured hazard results, including three core components: hazard status, hazard category, and hazard risk level. The hazard status is clearly categorized as either "real hazard" or "false alarm," with "uncertain" as an auxiliary identifier when necessary, for situations with insufficient evidence or significant contradictions. Hazard categories are limited to four types: "smoke and flame," "animal intrusion," "foreign object intrusion," and "water leakage," ensuring semantic consistency with the system's previous detection. The hazard risk level is graded based on the hazard category and the operational risk weight of the affected area, outputting "general," "important," or "urgent" to align with the station's alarm response mechanism.

[0091] The output includes a structured chain of evidence, clearly explaining the basis for the judgment, such as "The image shows stable and spreading water stains at the edge of the cable trench, the area is labeled as cable trench, the humidity has increased by 15% in the last 60 seconds, the water immersion sensor did not activate, and the overall judgment is that it is a real leak, risk level: emergency". This provides a traceable and auditable reasoning basis for subsequent knowledge verification and disposal suggestions.

[0092] This step effectively solves the dual challenges of high false alarm rate in lightweight models and susceptibility to illusions in purely data-driven models by leveraging the cross-modal understanding capabilities of large models. It is the core link in achieving "high accuracy, low false alarm, and interpretability" hazard diagnosis.

[0093] Step S216: Based on the spatial layout of the substation room, verify the hazard status results that are real hazards to obtain the target hazard diagnosis results. The verification includes whether the hazard location is legal, whether the hazard conflicts with the data corresponding to multiple environmental sensors, and whether it conforms to the physical evolution law.

[0094] In this step, after the multimodal large model outputs the initial conclusion that the hidden danger status is "real hidden danger", the system does not directly take it as the final diagnosis result. Instead, it further introduces a priori knowledge rule base based on the physical space structure and operation mechanism of the substation room to perform secondary logical verification on the model output, so as to intercept and correct possible "model illusion" or "semantic misreading" and ensure that the diagnosis result strictly conforms to the actual operation logic and safety specifications of the power production site.

[0095] First, the system verifies the spatial legitimacy of the potential hazard location. This process matches and compares the spatial coordinates of the hazard output by the multimodal large model with the pre-constructed equipment topology map of the substation. The equipment layout in the substation has clear spatial constraints: for example, cable trenches are located in low-lying areas, switchgear bottoms are adjacent to grounding metal structures, foreign objects are strictly prohibited from approaching the busbar area, while personnel are allowed to pass through access control areas. The system uses a preset area mask or semantic coordinate mapping table to determine whether the hazard occurs in physically impossible areas such as "directly above the busbar," "inside live terminals," or "inside the enclosed switchgear body." If the model mistakenly identifies the reflection on the cabinet surface as "water accumulation" and locates it above live components inside the cabinet, the system will determine that the location is invalid and directly downgrade it to "false alarm" or "pending verification," avoiding incorrect linkage or misassignment of work orders due to spatial logic errors.

[0096] Secondly, the system performs multi-source sensor state conflict verification to check whether the hazard detection results contradict the changes in physical quantities of environmental sensors. Even if the large model determines it to be a "real hazard," the conclusion is still unreliable if it conflicts logically with the synchronously acquired sensor data. For example, when the model determines that there is "water seepage and dripping," the system retrieves the water immersion sensor status, humidity change rate, and temperature and humidity trend data within the corresponding time window. If the water immersion sensor does not trigger, the humidity does not rise significantly, and the surrounding ambient temperature is stable, the system determines that the visual evidence lacks physical support; conversely, if the water immersion sensor's state reverses before and after the hazard occurs, or the humidity slope continues to rise, the model's judgment is supported. Especially for "smoke and flame" events, if the model output is a flame but the smoke detector does not alarm, or the smoke signal is in a low-level stable state, the system will trigger a conflict flag, forcibly correcting the hazard status to "false alarm" or "requires re-examination for confirmation."

[0097] Third, the system performs physical evolution law verification to identify and eliminate pseudo-dynamic phenomena caused by environmental interference. Real hazards exhibit continuous development characteristics consistent with physical laws: for example, real water accumulation expands slowly due to gravity or seepage pressure, with relatively stable boundaries and a trending increase in area growth rate; while reflective water ripples, shadow drift, or dust disturbances manifest as irregular boundary fluctuations, rapid area increases and decreases, and random trajectory drifts. The system extracts the area growth rate of the target hazard within consecutive frames. With boundary mobility indicators ,in:

[0098]

[0099]

[0100] in It is the intersection-union ratio function. A larger value indicates significant boundary drift, commonly seen in reflection and shadow drift. A smaller value indicates a relatively stable boundary, better reflecting the stable diffusion or fixed form of real accumulated water. The two formulas mentioned above serve as inputs for a secondary verification, used to determine whether the potential hazard possesses the evolutionary characteristics of a real physical entity. If... Approaching zero or fluctuating wildly, and If the value exceeds a preset threshold (e.g., >0.6), the system will determine it as a false dynamic caused by reflection, shadow, or dust disturbance. Even if the large model outputs "real hidden danger", it will be rejected and classified as a false alarm.

[0101] The aforementioned spatial legitimacy, sensor conflict, and evolutionary rationality do not operate independently, but are interconnected through logic and relationships. System definition comprehensive verification validity score:

[0102]

[0103] in, , , These are the Boolean results for the three checks (1 for pass, 0 for fail). A result is given only if all three checks pass. 1. Only the "real hidden dangers" output by the model can be confirmed as the final target hidden danger diagnosis result; if any one fails, the conclusion will be downgraded to "false alarm" or "pending manual review", and a complete chain of evidence will be retained for manual audit.

[0104] This step, by introducing spatial topology, sensor mutual exclusion, and physical evolution rules unique to the power industry, constructs a "knowledge-constrained firewall," effectively correcting unreasonable diagnoses that might arise from training bias, adversarial examples, or semantic misunderstandings in purely data-driven models. Essentially, it encodes "expert experience" and "operational procedures" into computable verification logic, ensuring that AI diagnostic results not only "look real" but are "unlikely to be fake in reality." The final output of the target hazard diagnosis results possesses high business credibility, safety compliance, and engineering feasibility, providing a solid and reliable decision-making basis for subsequent closed-loop handling actions.

[0105] As an optional embodiment, this can be achieved through the following steps: Based on a preset lightweight visual model, the aligned image frame is detected, and candidate potential hazards are output. This includes: inputting the aligned image frame into the preset lightweight visual model, performing forward inference, and outputting multi-scale candidate target prediction results corresponding to the aligned image frame. The multi-scale candidate target prediction results include the class probability, bounding box coordinates, and confidence score of the candidate targets at multiple scales. Based on a preset confidence threshold, the multi-scale candidate target prediction results corresponding to the aligned image frame are filtered to obtain filtered candidate targets. The detection is performed to determine whether the bounding boxes corresponding to the filtered candidate targets overlap. If there is overlap in the bounding boxes corresponding to the filtered candidate targets, redundant targets are deleted, and candidate potential hazards are output. Redundant targets represent candidate targets that overlap in position and whose confidence scores do not meet preset conditions.

[0106] Optionally, the system inputs the spatiotemporally aligned image frames into a preset lightweight visual hazard detection model. This model employs a target detection network based on the YOLOv8 architecture, possessing multi-scale feature extraction and fusion capabilities to adapt to hazard targets of different sizes and shapes in power substation rooms, such as the diffuse outline of smoke, the small bodies of animals, the irregular shapes of foreign objects, and the low-contrast planar areas of stagnant water. During forward inference, the model outputs a set of multi-scale candidate target prediction results. These results contain candidate targets predicted at multiple feature levels. Each candidate target carries three key pieces of information: the category probability of its hazard category, the coordinates of the two-dimensional bounding box representing the target's position in the image, and a score used to measure the detection confidence. This confidence score is mapped from the original logit value output by the network classification branch to the [0,1] interval using the Sigmoid function. Its value reflects the model's certainty regarding the current detection result. The specific calculation method is as follows:

[0107]

[0108] in, Let represent the confidence level of the i-th candidate target at time t, where This is the raw score output by the network. Map the score to an interval .

[0109] After obtaining the multi-scale candidate target prediction results, the system first applies a preset confidence threshold. All candidate targets undergo initial filtering, retaining only those with a confidence level not lower than a certain threshold, forming a filtered set of candidate targets. This threshold is not fixed but is configured differently based on the hazard category and camera environment characteristics: a lower threshold is used for smoke and flame hazards to improve the recall rate of early thin smoke; for animal or foreign object intrusion suspects, the threshold is appropriately increased during door opening and closing or sudden changes in lighting to suppress false detections of shadows; for water seepage targets, the threshold is increased in highly reflective areas based on the ground reflection image of the camera position to reduce false alarm interference. The design principle is "it is better to trigger more re-judgments than to miss critical hazards."

[0110] Subsequently, the system performs non-maximum suppression (NMS) on the filtered candidate targets to eliminate bounding box redundancy caused by multi-scale prediction or local deformation of the targets. This process uses the intersection-overlap ratio (IoU) between bounding boxes as a metric for overlap and sets a suppression threshold. For any two candidate targets, if their bounding box IoU exceeds the threshold, their confidence scores are compared. The target with the higher confidence score is retained, while redundant targets with lower confidence scores and high spatial overlap are removed. Redundant targets are defined as targets that significantly overlap with another candidate target in spatial location, whose confidence scores do not meet the priority retention criteria, and whose existence would interfere with the accuracy of subsequent gating evaluation and multimodal evidence orchestration.

[0111] The final set of candidate vulnerabilities is output, each containing its category, precise 2D bounding box coordinates, and final retained confidence score, serving as the input basis for subsequent multi-dimensional threshold gating evaluation. This process effectively reduces the number of candidate targets and computational redundancy while ensuring high recall, ensuring that the lightweight detection process can still achieve stable, efficient, and low false positive real-time operation under low computing power conditions at the edge, providing high-quality, structured initial detection results for the entire hierarchical inference architecture.

[0112] As an optional embodiment, this can be achieved through the following steps: Based on preset comprehensive evaluation indicators, score candidate hazards according to their information to obtain their corresponding scores, including: Based on preset comprehensive evaluation indicators, determine whether a candidate hazard is a high-risk hazard type according to its corresponding hazard category, obtaining a first judgment result; Based on preset comprehensive evaluation indicators, determine whether a candidate hazard is located in a high-risk area according to its corresponding hazard location, obtaining a second judgment result; Obtain historical false alarm records of the monitoring stations corresponding to the candidate hazard; Based on preset comprehensive evaluation indicators and the historical false alarm records of the monitoring stations corresponding to the candidate hazard, determine... The third judgment result is obtained by determining whether the candidate hidden danger is in a high-interference environment. Based on the preset comprehensive evaluation index, for the candidate hidden danger of water seepage and dripping, the contradictory signals between the visual detection conclusion and the spatial water immersion switch quantity or humidity change rate are extracted, and the area growth rate index of the candidate hidden danger of water seepage and dripping is continuously calculated within the corresponding time window. It is determined whether the candidate hidden danger is consistent with the change trend of the data corresponding to multiple environmental sensors, and the fourth judgment result is obtained. Based on the evaluation weights of the multiple evaluation indicators in the preset comprehensive evaluation index, the score corresponding to the candidate hidden danger is determined according to the first judgment result, the second judgment result, the third judgment result and the fourth judgment result.

[0113] Optionally, after acquiring the category, location, monitoring station information, and corresponding sensor data of candidate hazards, the system constructs a multi-dimensional and quantifiable scoring mechanism based on preset comprehensive evaluation indicators to objectively assess the necessity of triggering subsequent review processes for the candidate event. This scoring process does not rely on a single feature but integrates four core dimensions: the hazard's inherent danger attributes, spatial sensitivity, environmental interference tendency, and multi-source sensor consistency, forming a structured decision-making basis.

[0114] First, the system determines whether a candidate hazard belongs to a preset high-risk hazard type based on its hazard category, thus obtaining a first judgment result. In the substation room scenario, "smoke and flame" and "water leakage" are considered high-risk types because they may directly cause fires or equipment short circuits; "animal intrusion" and "foreign object intrusion" are further evaluated based on their location, but the category itself is assigned a basic risk weight. If a candidate hazard belongs to the above high-risk category, the first judgment result is "yes"; otherwise, it is "no".

[0115] Secondly, based on the spatial location of the candidate hazard, the system determines whether it is located in a pre-defined high-risk area of ​​the station, thus obtaining a second judgment result. This area is predefined by the station equipment topology diagram and includes key locations such as "bottom of switchgear", "cable trench", "around busbar compartment", and "drainage outlet". The system performs spatial overlap analysis between the boundary box of the candidate hazard and the area mask. If its center of gravity or main area falls within the aforementioned high-risk area, the second judgment result is "yes", indicating that the hazard has a higher potential hazard value; if it is located in a low-risk area such as "doorway" or "lighting area", the judgment is "no".

[0116] Third, the system retrieves historical false alarm records for the monitoring station corresponding to the candidate potential hazard and analyzes whether the station frequently generates similar false alarms under similar time periods, lighting conditions, and environmental conditions. For example, if a station consistently falsely reports "water accumulation" after rain due to glare from paving stones, or falsely identifies "foreign objects" due to flickering lights in the early morning or evening, the system marks that station as a "high-interference environment." Based on the distribution of historical false alarm frequencies and types, the system determines whether the current candidate event is located in such a high-interference environment, obtaining a third judgment result. If the event highly matches the historical high-frequency false alarm pattern, the third judgment result is "yes," indicating that it is more likely caused by environmental noise.

[0117] Fourth, the system compares the visual detection results of candidate hazards with the changing trends of data from multiple simultaneously collected environmental sensors to determine whether there is logical consistency between the two, thus obtaining a fourth judgment result. For example, when the candidate hazard is "water seepage and dripping," the system detects whether the water immersion sensor is triggered, whether the humidity rises significantly, and whether the temperature fluctuates abnormally within the corresponding time window. If the visual system detects water accumulation but the water immersion sensor does not activate and the humidity does not change, it is determined to be inconsistent, and the fourth judgment result is "no." Conversely, if the sensor data synchronously supports visual judgment, the result is "yes." This step ensures that the visual output has physical support, avoiding incorrect judgments based solely on image features.

[0118] After obtaining the above four judgment results, the system weights and aggregates the first, second, third, and fourth judgment results according to the weight coefficients corresponding to each dimension in the preset comprehensive evaluation indicators, and finally calculates the comprehensive score of the candidate hidden danger. The weight allocation reflects the operational priority: high-risk hidden danger types and high-risk areas have the highest weights to ensure that critical safety points are not overlooked; the weight of high-interference environments is a negative penalty to suppress false alarms; and the sensor consistency weight serves as a calibration factor to improve diagnostic reliability. The scoring calculation process is as follows:

[0119]

[0120] in For the gated weight vector, For contradictory signal weights, Weighted by growth rate, This is a normalization function used to map the growth rate to a comparable scale.

[0121] Through the above steps, the system achieves a multi-dimensional, interpretable, and traceable quantitative assessment of potential hazards. This not only avoids the rampant misjudgments caused by relying solely on confidence scores, but also transforms the operational experience of the power industry into executable assessment logic, providing a scientific and robust decision-making basis for subsequent dynamic gating triggering mechanisms.

[0122] As an optional embodiment, this can be achieved through the following steps: determining the multimodal data corresponding to the target hazard based on the monitoring video stream and the data corresponding to multiple environmental sensors, including: extracting a local image cropping map of the area where the target hazard is located based on the monitoring video stream to determine the hazard image; extracting the environmental background image of the monitoring screen where the target hazard is located based on the monitoring video stream to determine the environmental image where the hazard is located; obtaining the monitoring location corresponding to the target hazard, the type of area it occupies in the substation room space, and a short-term historical evolution trend summary within the corresponding time window, wherein the short-term historical evolution trend summary includes temperature and humidity slope, smoke detector status, and access control records; determining the multimodal data corresponding to the target hazard based on the hazard image, the environmental image where the hazard is located, the monitoring location, the type of area it occupies, the data corresponding to multiple environmental sensors, and the short-term historical evolution trend summary.

[0123] Optionally, the process of determining the multimodal data corresponding to the target hazard aims to provide rich, multidimensional, and logically consistent evidence input for the re-evaluation of the large multimodal model. Specifically, the system first extracts visual evidence from the monitoring video stream deployed in the substation. For the target hazard initially screened by the lightweight visual model, the system extracts a cropped image of the local area where the hazard is located, thereby forming a hazard image focusing on the details of the hazard itself, such as clearly showing the edge shape of water accumulation, the diffusion texture of smoke, or the outline features of intruding foreign objects, so that the large model can perform fine-grained visual discrimination. At the same time, in order to obtain a more complete scene context, the system extracts a background image containing the target hazard and its surrounding environment from the same monitoring screen, determining the environment image where the hazard is located. This image can provide key information such as the overall layout of the substation, lighting conditions, ground material (such as tiles, epoxy flooring, or metal covers), and the relative position of equipment, helping the large model to combine prior environmental knowledge to distinguish between real hazards and environmental interferences (such as reflections and shadows).

[0124] While acquiring visual evidence, the system further incorporates spatial semantic information and temporal evolution features to enhance the physical meaning of the data. The system queries the digital topology map of the substation to determine the monitoring location markers corresponding to the target hazard and its location within the substation space. For example, the hazard is located in specific key areas such as "cable trench," "bottom of switchgear," or "substation passageway." Different areas have significantly different risk level definitions for the same visual phenomenon. Furthermore, the system extracts a short-term historical evolution trend summary within the time window corresponding to the trigger time of the target hazard. This summary is generated through statistical analysis and feature compression of multi-source sensor data, specifically covering temporal features such as temperature and humidity change slopes, smoke sensor status sequences, and access control opening and closing records. For example, the temperature and humidity slope can determine whether there is an abnormal temperature rise or a sudden increase in humidity; the smoke sensor status can confirm fire risk; and access control records can correlate with personnel activity or sudden changes in lighting, thus compensating for the deficiency of single-frame images in reflecting temporal evolution patterns.

[0125] Finally, the system aligns and encapsulates the extracted hazard images, environmental images of the hazard, monitoring locations, area types, real-time data from multiple environmental sensors, and short-term historical evolution trend summaries to construct multimodal data corresponding to the target hazard. This multimodal data assembly method has significant technical effects: on the one hand, the combination of local images and global background achieves dual complementarity of details and context, enabling the large model to focus on the hazard's intrinsic features while filtering high-frequency false alarms such as metallic reflections and old watermarks by incorporating environmental background; on the other hand, the introduction of spatial semantic tags and sensor time-series data transforms single visual perception into a multidimensional evidence chain supported by physical mechanisms, allowing the large model to make comprehensive inferences based on business logic such as "the risk of water accumulation in cable trenches is higher than water stains at the door" and "a sudden increase in humidity confirms the authenticity of water accumulation," effectively suppressing the hallucination output of purely data-driven models under complex working conditions, and significantly improving the accuracy, interpretability, and adaptability of hazard diagnosis to long-term environmental evolution.

[0126] As an optional embodiment, multimodal data is input into a large multimodal model for cross-modal association semantic parsing, and potential hazard results are output. This includes: generating a constraint hint template based on the multimodal data, wherein the constraint hint template includes prior knowledge of key areas and multi-source consistency check requirements; inputting the multimodal data and constraint hint template into the large multimodal model for cross-modal association semantic parsing, and outputting potential hazard results.

[0127] Optionally, in this embodiment, the core of the process of performing cross-modal semantic parsing and outputting potential hazard results in the multimodal large model lies in injecting the professional business logic and physical prior knowledge of the substation into the reasoning process of the large model through dynamically generated constraint prompt templates, thereby guiding the model to make rational judgments that conform to industry standards.

[0128] Specifically, the system first dynamically generates a constraint prompt template based on the input multimodal data and its associated contextual information. This constraint prompt template is not static text, but a set of inference instructions tailored to the current candidate potential hazard event. Its content includes at least prior information on key areas and multi-source consistency check requirements. Among them, the prior information on key areas is pre-set based on the spatial semantic labels of the target hazard (such as high-risk areas such as cable trenches, bottom of switch cabinets, busbar rooms, etc., or low-risk areas such as doorways and passageways), used to prompt the large model on the differences in the tolerance of different areas to the hazard. For example, it prompts the model that "the water accumulation at the bottom of the cable trench has an extremely high risk of insulation failure and should be confirmed first." The multi-source consistency check requirements explicitly instruct the large model to cross-validate the logical relationship between visual images and sensor data during inference. For example, it requires the model that "if the visual recognition is water accumulation, but the water immersion sensor does not trigger and the humidity does not change significantly, then the possibility of false alarms such as reflections or old water marks should be investigated."

[0129] Subsequently, the system inputs multimodal data, including visual evidence (local cropped images, global environmental images) and sensor evidence (real-time sensor data, short-term historical evolution trend summaries), along with the aforementioned constraint prompt templates, into a multimodal large-scale model for cross-modal semantic parsing. During the parsing process, the multimodal large-scale model first uses a visual encoder to extract semantic features from the images, while simultaneously using a text encoder to parse the logical rules in the constraint prompt templates. A cross-modal attention mechanism is then used to achieve deep interaction between image features and text instructions. Based on prior knowledge of key regions, the model assigns different risk weights to potential hazards in different spatial locations. Furthermore, in accordance with multi-source consistency checks, it rigorously compares the consistency between visual detection results and sensor states. For example, when visual detection detects suspected smoke but the smoke sensor is functioning normally, the model combines the environmental background image to determine whether it is dust or steam interference; when visual detection detects suspected water accumulation but the boundary is stable and there is no humidity change, the model determines it as a false alarm due to reflection or static watermarks.

[0130] Through deep cross-modal reasoning, the multimodal large model outputs the final hazard assessment results. These results not only include classification decisions for "real hazards," "false alarms," ​​or "uncertainty," but also simultaneously generate detailed hazard categories, quantified risk levels, and a structured, multi-dimensional chain of evidence. The technical advantage of this process lies in effectively constraining the "illusion" phenomenon that large models are prone to when lacking domain knowledge constraints by introducing key area priors and multi-source consistency checks. This ensures that diagnostic conclusions conform to the physical mechanisms and operational procedures of power distribution operation, significantly improving the accuracy and interpretability of hazard identification in complex environments, and enabling the output results to directly support subsequent operation and maintenance decisions.

[0131] As an optional embodiment, it can be achieved through the following steps: further including: storing the hazard status result that is a false alarm and the corresponding multimodal data into a first sample set; storing the hazard status result that is a real hazard and the corresponding multimodal data into a second sample set; and adjusting the preset lightweight visual model based on the first sample set and the second sample set.

[0132] Optionally, after the system identifies the target hazard, to ensure that the subsequent inference of the multimodal large model has sufficient contextual support and physical basis, the system does not rely on the local information of a single image frame. Instead, it extracts multi-layered and multi-perspective visual evidence from the monitoring video stream and combines it with the spatial semantic information of the station to construct a structured and interpretable multimodal input dataset. This dataset consists of four core elements: hazard image, image of the environment in which the hazard is located, monitoring location, and type of area.

[0133] First, the system extracts a cropped image of the local area where the target hazard is located from the monitoring video stream, serving as the hazard image. This cropping operation is performed based on the precise bounding box coordinates of the target hazard in the original image, retaining only the hazard itself and its immediate surrounding microenvironment, focusing on key visual features such as the diffusion texture of smoke, the outline of animal limbs, the material shape of foreign objects, or the edge continuity of accumulated water. This image does not contain irrelevant background, aiming to provide the model with high-information-density evidence of the fault itself, serving as a direct basis for determining the authenticity of the hazard.

[0134] Secondly, the system extracts a global environmental background image from the same monitoring frame as the environmental image of the potential hazard. This image is an uncropped, complete view, fully preserving spatial contextual information such as the layout of indoor equipment in the substation, lighting status, floor material, cable routing, cabinet structure, and the location of doors and windows. For example, when the hazard is "water accumulation," the environmental image can reveal whether the area is located near the edge of a cable trench, the bottom of a switch cabinet, or a metal drainage ditch; when the hazard is "animal intrusion," the environmental image can help determine whether there are ventilation openings, unsealed cable holes, or equipment gaps. This image provides key semantic clues for the model to understand "why this anomaly occurs here," effectively suppressing misjudgments caused by local reflections, shadows, or dust interference.

[0135] Third, the system acquires the monitoring location corresponding to the target hazard, namely the unique identifier of the camera associated with the event and its preset semantic description. This information is fixed in the configuration during the station engineering deployment phase, such as "view camera location overlooking the cable trench on the south side of power distribution room No. 1" and "monitoring the passage directly in front of switchgear group A," etc., without relying on image recognition, ensuring the accuracy and stability of spatial positioning. This location tag anchors the visual anomaly to the specific physical installation point, providing accurate coordinate reference for subsequent spatial topology verification.

[0136] Fourth, the system matches the bounding box coordinates of the target hazard with the pre-defined spatial semantic zoning map of the substation to determine its location within the substation area. This area type is clearly defined by the operation and maintenance procedures and includes several key functional areas such as "cable trench," "bottom of switchgear," "surroundings of busbar room," "doorway passage," "critical flood control area," and "edge of blind zone." Through spatial overlap analysis, the system identifies the intersection between the hazard and the masks in each area, thereby determining whether it is located in a high-risk area. This information is directly related to power safety regulations and is the core basis for assessing the potential hazard level—for example, the same type of water accumulation has a significantly higher risk level if located in a cable trench than if it appears on the floor tiles of a passageway.

[0137] Ultimately, the system standardizes and encapsulates the above four types of information—hazard images, environmental images of the hazard, monitoring location, and area type—to form a structured multimodal data input package. This data package does not contain the original video stream or all sensor data; it retains only the minimum necessary evidence most discriminative for inference. This satisfies the contextual integrity requirements of large models while strictly controlling bandwidth usage and data security risks. This data package is bound to auxiliary information such as timestamps, unique hazard identifiers, and gating scores, serving as the sole input for subsequent cross-modal inference. This ensures that the system, during the inference process, "not only sees the anomaly, but also understands where the anomaly occurred, why it occurred, and whether it is reasonable," thereby significantly improving the accuracy and interpretability of the diagnosis.

[0138] The system first performs feature clustering and distribution analysis on the false alarm samples in the first sample set to identify frequently occurring environmental false alarm features. These features include, but are not limited to: during low-light twilight hours, the reflection of floor tiles creates a water-like artifact under the wide-angle lens of the camera; when the ventilation system is activated, dust particles drifting with the airflow are misjudged as smoke; during periods when access control is frequently opened, the shadow of the door frame shifts rapidly with changes in light, which is identified as "foreign object intrusion"; or in a humid environment after rain, the local reflection of epoxy flooring is highly similar to the actual water accumulation, leading to continuous misjudgments by the model. Through statistical modeling of these false alarm patterns, the system identifies systematic deviations in false alarms across time, space, and environmental dimensions. For example, at a certain camera position, the false alarm rate increases significantly by 300% between 18:00 and 20:00 daily, when the ground is a high-gloss epoxy floor and the ambient humidity is >75%; or at a certain cable trench viewpoint, under the combination of cloudy weather and on lights, reflective false alarms are concentrated in the lower 1 / 3 of the image area.

[0139] After identifying the deviation pattern, the system dynamically calculates the threshold adjustment parameters for each monitoring station based on this pattern. These parameters guide the adaptive optimization of the initial detection threshold and multi-dimensional gating trigger threshold of the lightweight vision model. The system employs an exponential sliding update mechanism, independently calculating the specific confidence threshold correction and gating score threshold offset for each camera. For example, if a camera frequently misidentifies "reflection as water accumulation" in historical false alarms, the system will set a positive adjustment offset for the confidence threshold of that camera during humid periods, thus increasing its filtering threshold and reducing the frequency of low-confidence suspected water accumulation outputs. Simultaneously, the penalty weights for "scene factors" and "area growth rate" are strengthened in the gating score, making the re-judgment trigger conditions for that camera more stringent.

[0140] The adjustment parameters are not globally uniform, but rather configured in a fine-grained, zoned manner based on aircraft location, time period, weather pattern, and ground material. This ensures that the threshold adjustments accurately reflect the actual interference environment of each area. The adjusted threshold parameters are written into the system's operational configuration and dynamically applied to the hazard detection process in the next cycle, achieving a closed-loop evolution of "false alarm-driven—pattern discovery—threshold optimization."

[0141] Through this mechanism, the system can continuously learn the long-term evolution characteristics of the substation environment without human intervention, such as seasonal changes in sunlight, increased reflectivity due to equipment aging, and material differences caused by wear and tear on the floor coating. This gives the diagnostic system "adaptive immunity" during long-term operation. This embodiment overcomes the inherent defect of traditional static threshold models in terms of performance degradation under environmental drift, transforming the substation hidden danger diagnosis system from a "one-time deployment" to a "continuously evolving" intelligent agent, significantly improving the system's stability and reliability across seasons and operating conditions.

[0142] As an optional embodiment, based on the hazard status result being a false alarm and the corresponding multimodal data, the deviation pattern is determined according to the environmental false alarm characteristics; based on the deviation pattern, the adjustment parameters for the initial screening confidence threshold and the adjustment parameters for the gated scoring trigger score line are determined; based on the adjustment parameters for the initial screening confidence threshold and the adjustment parameters for the gated scoring trigger score line, the preset lightweight visual model is updated; based on the hazard status result being a false alarm and the corresponding multimodal data, the constraint prompt template used to constrain the large multimodal model is updated.

[0143] Optionally, an online adaptive iteration of the hazard diagnosis model is achieved through a closed-loop feedback mechanism, aiming to address the performance degradation caused by the dynamic evolution of the substation environment. This process mainly includes two parallel optimization paths: one is threshold tuning for the lightweight visual initial screening model, and the other is updating the prompt template for the inference logic of the multimodal large model.

[0144] First, the system uses verified samples of potential hazards that have been confirmed as false alarms, along with their corresponding multimodal data, to analyze deviation patterns. Specifically, the system extracts environmental false alarm characteristics from these samples, such as specific lighting conditions (e.g., strong light reflection, low illumination), ground material properties (e.g., highly reflective tiles, metal covers), time period characteristics (e.g., periods of drastic changes in light intensity between morning and evening), and sensor status inconsistencies (e.g., visual indication of water accumulation but failure of the water immersion sensor to trigger), thereby identifying high-frequency interference patterns or deviation patterns that lead to false alarms. Based on the identified deviation patterns, the system calculates adjustment parameters for the initial screening confidence threshold for specific monitoring locations or time windows, as well as adjustment parameters for the comprehensive scoring trigger line in the multidimensional gating evaluation mechanism. For example, if a monitoring location is found to be prone to falsely reporting reflection as water accumulation under strong afternoon light, the system increases the initial screening confidence threshold for that location during that time period, or raises the threshold score for entering the large model for further review, to reduce unnecessary computational consumption and filter out low-confidence noise. Subsequently, the system applies these adjustment parameters to update the confidence threshold of the candidate hazard detection of the preset lightweight visual model, and simultaneously updates the comprehensive scoring trigger line of the gating evaluation mechanism, thereby achieving dynamic fine-tuning of the sensitivity of the initial screening stage to better match the real-time environmental status of the current station.

[0145] Secondly, based on the confirmed false alarms and corresponding multimodal data, the system analyzes the root causes of the large model's misjudgments. If the false alarm is due to the large model ignoring specific physical common sense or business rules (e.g., failing to consider that the reflective properties of metal covers are not fluid, or failing to recognize the typical texture of cable trench covers), the system generates targeted constraint instructions. These instructions are used to update the constraint prompt templates used to constrain the multimodal large model. Specifically, the system transforms the analyzed root causes of misjudgments into prior rules or exclusion conditions in natural language form and inserts or replaces them in the large model's input prompt template. For example, constraints such as "If the target area is a metal cover and has highlights, please prioritize determining it as non-water accumulation" or "If visually identified as a foreign object but access control records show no personnel entering or leaving and no signs of movement, please determine it as a static object false alarm" are added to the prompt template. By updating the constraint prompt template, the large model can automatically apply these newly added business rules in subsequent reasoning processes, thereby correcting its logical illusions and improving its ability to identify specific interference scenarios.

[0146] Through the aforementioned dual-path adaptive iteration, the system not only optimizes the detection threshold of the lightweight front-end model to balance recall and false alarm rates, but also enhances the business logic constraint capabilities of the large back-end model. This enables the entire heterogeneous inference system to adapt to the long-term evolution of complex environments in power distribution rooms (such as seasonal changes in lighting, equipment aging, and ground wear) without human intervention, effectively ensuring the diagnostic performance and stability of the system throughout its entire lifecycle.

[0147] As an optional embodiment, based on the spatial layout of the substation, the hazard status result is verified for the actual hazard to obtain the target hazard diagnosis result. The verification includes whether it conforms to the physical evolution law, including: extracting the area boundary change and trajectory displacement features of the target hazard in continuous frames of a preset time; based on the area boundary change and trajectory displacement features, and according to the preset differential evolution law of physical entities and optical environment noise, verifying whether the target hazard has stable extension properties to determine whether it conforms to the physical evolution law.

[0148] Optionally, in this embodiment, to further eliminate the logical illusions that may arise from the multimodal large model and ensure that the diagnostic results conform to the actual physical operating mechanism of the substation, the system, after performing spatial topology verification, also needs to perform a secondary verification based on the physical evolution law for the target hazards confirmed as real hazards. The core of this verification process lies in using the dynamic features in the video sequence to distinguish the essential differences between real physical hazards and environmental noise (such as light and shadow drift, electrostatic interference, etc.).

[0149] Specifically, the system first extracts a sequence of continuous video frames of the target hazard within a preset time period, and then calculates the area boundary change characteristics and trajectory displacement characteristics of the hazard area for each frame. The area boundary change characteristics reflect the expansion or contraction trend of the hazard area over time. For example, the spread of real water accumulation usually shows that the area increases monotonically or remains relatively stable over time, while the spread of real smoke shows irregular and non-rigid morphological changes. The trajectory displacement characteristics record the movement path of the target's center point in the image, used to determine whether the target has active movement or natural flow characteristics affected by gravity.

[0150] Subsequently, based on the extracted area boundary changes and trajectory displacement features, and according to the preset differential evolution laws of physical entities and optical environmental noise, the system verifies the stability of the target hazard. The preset physical evolution laws define the temporal continuity, mass conservation, and natural mobility characteristics that a real physical hazard should possess, while optical environmental noise (such as moving shadows formed by sunlight passing through blinds, light spot drift caused by light flicker, and random shaking of airborne particles) usually manifests as irregular rapid flickering, unnatural sudden deformation, or instantaneous appearance without mass accumulation. By comparing the evolution characteristics of the target hazard with the above laws, the system determines whether it possesses "stable extension properties." For example, if the boundary of the target area shows a smooth and continuous expansion trend in consecutive frames, and the trajectory displacement conforms to the characteristics of gravity or hydrodynamics, it is determined that it has stable extension properties, conforms to the laws of physical evolution, and is confirmed as a real hidden danger; conversely, if the target area undergoes drastic deformation in a very short time, the boundary jumps frequently, or the displacement trajectory violates the common sense of physics (such as rising against gravity, passive movement), it is determined that it does not have stable extension properties, belongs to false features caused by optical environmental noise, and is thus intercepted and corrected to a false alarm or pending verification state.

[0151] By introducing a verification mechanism based on the laws of physical evolution, this invention can effectively identify and filter "illusionary" diagnostic results that, although passing visual recognition and sensor consistency checks, still violate common sense physics. This verification method based on dynamic temporal characteristics compensates for the shortcomings of static image analysis and instantaneous sensor data in representing physical processes in the time dimension, significantly improving the credibility and robustness of hazard diagnosis results, and ensuring that the final output of the target hazard diagnosis result conforms to both visual perception facts and the underlying safety physical logic of the substation.

[0152] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that the environmental hazard diagnosis method for substation rooms according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0154] According to embodiments of the present invention, an environmental hazard diagnosis device for a substation room is also provided for implementing the above-described environmental hazard diagnosis method for a substation room. Figure 3 This is a structural block diagram of an environmental hazard diagnosis device for a substation room provided according to an embodiment of the present invention, such as... Figure 3 As shown, the environmental hazard diagnosis device for the substation includes: a data acquisition module 302, an alignment module 304, a first output module 306, a scoring module 308, a first determination module 310, a second determination module 312, a second output module 314, and a verification module 316. The environmental hazard diagnosis device for the substation is described below.

[0155] The acquisition module 302 is used to acquire the monitoring video stream of the substation room and the data corresponding to multiple environmental sensors, including temperature and humidity sensors.

[0156] Alignment module 304 is used to align the monitoring video stream and the corresponding data from multiple environmental sensors to obtain aligned image frames.

[0157] The first output module 306 is used to detect aligned image frames based on a preset lightweight visual model and output candidate hazards and their information. The preset lightweight visual model is trained based on multiple images labeled with hazard types in the substation room, and the information includes hazard category and hazard location.

[0158] The scoring module 308 is used to score candidate hazards based on the information of the candidate hazards according to the preset comprehensive evaluation indicators, and obtain the corresponding score of the candidate hazard. The preset comprehensive evaluation indicators include the hazard type, the risk level of the hazard location, the environmental false alarm characteristics of the monitoring position, and whether the monitoring video stream is consistent with the data corresponding to multiple environmental sensors.

[0159] The first determining module 310 is used to determine candidate hidden dangers whose scores exceed a preset threshold as target hidden dangers.

[0160] The second determining module 312 is used to determine the multimodal data corresponding to the target hidden danger based on the monitoring video stream and the data corresponding to multiple environmental sensors. The multimodal data includes the hidden danger image, the environmental image of the hidden danger, the monitoring location and the type of the area, as well as the sensor status sequence and short-term historical trend summary within the corresponding time window.

[0161] The second output module 314 is used to input multimodal data into the multimodal large model and output hazard results. These results include hazard status, hazard category, and hazard risk level. The hazard status includes actual hazards and false alarms. Specifically, this module receives the multimodal input package encapsulated by the second determining module and inputs it, along with a dynamically generated prompt constraint template, into the multimodal large model inference engine deployed on the platform, triggering a cross-modal joint semantic parsing and risk perception inference process.

[0162] The verification module 316 is used to input multimodal data into a multimodal large model for cross-modal association semantic parsing and output hazard results. The hazard results include hazard status results, hazard category and hazard risk level, structured multidimensional evidence description chain, and action suggestions for joint inspection. The hazard status results include real hazards and false alarms.

[0163] It should be noted that the aforementioned acquisition module 302, alignment module 304, first output module 306, scoring module 308, first determination module 310, second determination module 312, second output module 314, and verification module 316 correspond to steps S202 to S216 in the embodiments. Multiple modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should also be noted that the aforementioned modules, as part of the device, can run on the computer terminal 10 provided in the embodiments.

[0164] Embodiments of the present invention may provide a computer device. Optionally, in this embodiment, the computer device may be located in at least one of a plurality of network devices in a computer network. The computer device includes a memory and a processor.

[0165] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the environmental hazard diagnosis method and device for substation rooms in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned environmental hazard diagnosis method for substation rooms. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0166] The processor can access the information and application programs stored in the memory via the transmission device to execute any of the steps in the above-mentioned environmental hazard diagnosis method for the substation room.

[0167] This invention provides a scheme for diagnosing environmental hazards in a power distribution room based on hierarchical reasoning using a heterogeneous model. The scheme involves collecting monitoring video streams and data from multiple environmental sensors in the power distribution room, including temperature and humidity sensors. The monitoring video streams and the data from the environmental sensors are aligned to obtain aligned image frames. Based on a preset lightweight visual model, the aligned image frames are detected, and candidate hazards and their information are output. The preset lightweight visual model is trained on images labeled with multiple hazard types in the power distribution room, and the information includes hazard category and location. Based on a preset comprehensive evaluation index, the candidate hazards are scored according to their information to obtain their corresponding scores. The preset comprehensive evaluation index includes hazard type, risk level of hazard location, environmental false alarm characteristics of the monitoring station, and consistency between the monitoring video stream and the data from the environmental sensors. Candidate hazards with scores exceeding a preset threshold are identified as target hazards. The process is repeated based on the monitoring video streams and the data from the environmental sensors. The system uses data from each sensor to determine the multimodal data corresponding to the target hazard. This multimodal data includes the hazard image, the environmental image of the hazard, the monitoring location, and the type of the area. The multimodal data is input into a large multimodal model, which outputs hazard results. These results include the hazard status, hazard category, and hazard risk level. The hazard status results include both actual hazard and false alarms. Based on the spatial layout of the substation, the hazard status results for those that are actual hazard are verified to obtain the target hazard diagnosis result. Verification includes checking the legality of the hazard location, whether the hazard conflicts with data from multiple environmental sensors, and whether it conforms to physical evolution laws. This achieves the goal of accurate diagnosis and adaptive closed-loop evolution of environmental hazards in substations based on hierarchical reasoning using a heterogeneous model, thereby improving the technical accuracy of hazard diagnosis and solving the current technical problem of low accuracy in diagnosis results due to high false alarms in lightweight models in complex substation environments.

[0168] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a non-volatile storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0169] Embodiments of the present invention also provide a non-volatile storage medium. Optionally, in this embodiment, the aforementioned non-volatile storage medium can be used to store the program code executed by the environmental hazard diagnosis method for substation rooms provided in the above embodiments.

[0170] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0171] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing any of the above-described methods for diagnosing environmental hazards in a power distribution room.

[0172] Embodiments of the present invention also provide a computer program product, including a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it can implement any of the above-described methods for diagnosing environmental hazards in a power distribution room.

[0173] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0174] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0175] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0176] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0178] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0179] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for diagnosing environmental hazards in a substation, characterized in that, include: The system collects monitoring video streams from the substation room and data from multiple environmental sensors, including temperature and humidity sensors. The monitoring video stream and the data corresponding to each of the multiple environmental sensors are aligned to obtain aligned image frames. Based on a preset lightweight visual model, the aligned image frames are detected, and candidate hazards and their information are output. The preset lightweight visual model is trained based on multiple images labeled with hazard types in the power distribution room, and the information includes hazard category and hazard location. Based on the preset comprehensive evaluation indicators, the candidate hidden dangers are scored according to the information of the candidate hidden dangers to obtain the corresponding scores of the candidate hidden dangers. The preset comprehensive evaluation indicators include the type of hidden danger, the risk level of the location of the hidden danger, the environmental false alarm characteristics of the monitoring position, and whether the monitoring video stream is consistent with the data corresponding to each of the multiple environmental sensors. Candidate hazards whose scores exceed a preset threshold are identified as target hazards; Based on the monitoring video stream and the data corresponding to each of the multiple environmental sensors, the multimodal data corresponding to the target hazard is determined. The multimodal data includes the hazard image, the environmental image of the hazard, the monitoring location and the type of the area, as well as the sensor status sequence and short-term historical trend summary within the corresponding time window. The multimodal data is input into a multimodal large model for cross-modal association semantic parsing, and the output hazard results are generated. The hazard results include hazard status results, hazard category and hazard risk level, structured multidimensional evidence description chain, and action suggestions for joint inspection. The hazard status results include real hazards and false alarms. Based on the spatial layout of the substation, the hazard status results that are real hazards are verified to obtain the target hazard diagnosis results. The verification includes whether the hazard location is legal, whether the hazard conflicts with the data corresponding to the multiple environmental sensors, and whether it conforms to the laws of physical evolution.

2. The method according to claim 1, characterized in that, The step of detecting potential problems in the aligned image frames based on a preset lightweight visual model and outputting candidate potential issues includes: The aligned image frame is input into the preset lightweight visual model, forward inference is performed, and the multi-scale candidate target prediction result corresponding to the aligned image frame is output. The multi-scale candidate target prediction result includes the class probability, bounding box coordinates and confidence score of the candidate target at multiple scales. Based on a preset confidence threshold, the multi-scale candidate target prediction results corresponding to the aligned image frame are filtered to obtain the filtered candidate targets. Detect whether the bounding boxes corresponding to the filtered candidate targets have overlapping positions; If there is positional overlap in the bounding boxes corresponding to the filtered candidate targets, redundant targets are deleted and the candidate potential hazards are output. The redundant targets represent candidate targets that have positional overlap and whose confidence level does not meet the preset conditions.

3. The method according to claim 1, characterized in that, The process of scoring the candidate hazards based on pre-set comprehensive evaluation indicators and their information to obtain a score for each candidate hazard includes: Based on the pre-set comprehensive evaluation indicators, and according to the hazard category corresponding to the candidate hazard, it is determined whether it is a high-risk hazard type, and a first judgment result is obtained; Based on the preset comprehensive evaluation indicators, and according to the location of the candidate hidden danger, it is determined whether the candidate hidden danger is located in a high-risk area, and a second judgment result is obtained; Obtain the historical false alarm records of the monitoring positions corresponding to the candidate potential hazards; Based on the preset comprehensive evaluation indicators and the historical false alarm records of the monitoring positions corresponding to the candidate hidden dangers, it is determined whether the candidate hidden dangers are in a high interference environment, and a third judgment result is obtained. Based on the preset comprehensive evaluation index, for the candidate hidden dangers of water seepage and dripping, the contradictory signals between the visual detection conclusion and the spatial water immersion switch quantity or humidity change rate are extracted, and the area growth rate index of the candidate hidden dangers of water seepage and dripping is continuously calculated within the corresponding time window. It is then determined whether the candidate hidden danger is consistent with the change trend of the data corresponding to the multiple environmental sensors, and a fourth judgment result is obtained. Based on the evaluation weights corresponding to each of the multiple evaluation indicators in the preset comprehensive evaluation indicators, the score corresponding to the candidate hidden danger is determined according to the first judgment result, the second judgment result, the third judgment result and the fourth judgment result.

4. The method according to claim 1, characterized in that, The step of determining the multimodal data corresponding to the target hazard based on the monitoring video stream and the data corresponding to each of the multiple environmental sensors includes: Based on the monitoring video stream, a local image cropping image of the area where the target hazard is located is extracted to determine the hazard image; Based on the monitoring video stream, the environmental background image of the monitoring screen where the target hidden danger is located is extracted to determine the environmental image of the hidden danger; The monitoring location corresponding to the target hidden danger, the type of the area in the substation room, and a summary of the short-term historical evolution trend within the corresponding time window are obtained. The summary of the short-term historical evolution trend includes temperature and humidity slope, smoke detection status, and access control records. Based on the hazard image, the environmental image of the hazard, the monitoring location, the type of the area, the data corresponding to each of the multiple environmental sensors, and the short-term historical evolution trend summary, the multimodal data corresponding to the target hazard is determined.

5. The method according to claim 1, characterized in that, The step of inputting the multimodal data into a large multimodal model for cross-modal association semantic parsing and outputting potential hazard results includes: Based on the multimodal data, a constraint hint template is generated, wherein the constraint hint template includes key region priors and multi-source consistency check requirements; The multimodal data and the constraint prompt template are input into the multimodal large model for cross-modal association semantic parsing, and the hidden danger results are output.

6. The method according to claim 1, characterized in that, Also includes: The hazard status results that are false alarms and the corresponding multimodal data are stored in the first sample set; The hazard status result that is a real hazard and the corresponding multimodal data are stored in the second sample set; The preset lightweight visual model is adjusted based on the first sample set and the second sample set.

7. The method according to claim 1, characterized in that, Also includes: Based on the hazard status results being false alarms and the corresponding multimodal data, the deviation pattern is determined according to the environmental false alarm characteristics; Based on the aforementioned deviation pattern, determine the adjustment parameters for the initial screening confidence threshold and the adjustment parameters for the gated scoring trigger score line; The preset lightweight visual model is updated based on the initial screening confidence threshold adjustment parameters and the gated scoring trigger score line adjustment parameters. Based on the hazard status result being a false alarm and the corresponding multimodal data, the constraint prompt template used to constrain the multimodal large model is updated.

8. The method according to claim 1, characterized in that, Based on the spatial layout of the substation room, the hazard status results are verified for those that are actual hazards to obtain the target hazard diagnosis result. The verification includes checking whether the hazard conforms to physical evolution laws, including: Extract the area boundary changes and trajectory displacement features of the target hazard within consecutive frames of a preset duration; Based on the changes in the area boundary and the trajectory displacement characteristics, and according to the pre-defined differential evolution law between physical entities and optical environmental noise, it is verified whether the target hazard has stable extension properties to determine whether it conforms to the physical evolution law.

9. An environmental hazard diagnosis device for a substation room, characterized in that, include: The acquisition module is used to acquire the monitoring video stream of the substation room and the data corresponding to multiple environmental sensors, including temperature and humidity sensors. The alignment module is used to align the monitoring video stream and the data corresponding to the multiple environmental sensors to obtain aligned image frames. The first output module is used to detect the aligned image frame based on a preset lightweight visual model and output candidate hazards and information about the candidate hazards. The preset lightweight visual model is trained based on multiple images labeled with hazard types in the power distribution room. The information includes hazard category and hazard location. The scoring module is used to score the candidate hidden dangers based on the information of the candidate hidden dangers according to the preset comprehensive evaluation indicators, and to obtain the score corresponding to the candidate hidden dangers. The preset comprehensive evaluation indicators include the hidden danger type, the risk level of the location of the hidden danger, the environmental false alarm characteristics of the monitoring position, and whether the monitoring video stream is consistent with the data corresponding to each of the multiple environmental sensors. The first determination module is used to determine candidate hidden dangers whose scores exceed a preset threshold as target hidden dangers; The second determining module is used to determine the multimodal data corresponding to the target hidden danger based on the monitoring video stream and the data corresponding to each of the multiple environmental sensors. The multimodal data includes the hidden danger image, the environmental image where the hidden danger is located, the monitoring location and the type of the area, as well as the sensor status sequence and short-term historical trend summary within the corresponding time window. The second output module is used to input the multimodal data into a multimodal large model for cross-modal association semantic parsing and output the hidden danger results. The hidden danger results include the hidden danger status results, hidden danger category and hidden danger risk level, structured multidimensional evidence description chain and joint inspection handling action suggestions. The hidden danger status results include real hidden dangers and false alarms. The verification module is used to verify the hazard status results that are real hazards based on the spatial layout of the substation room, and obtain the target hazard diagnosis result. The verification includes whether the hazard location is legal, whether the hazard conflicts with the data corresponding to the multiple environmental sensors, and whether it conforms to the physical evolution law.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the non-volatile storage medium to perform the environmental hazard diagnosis method for the substation room as described in any one of claims 1 to 8.

11. A computer device, characterized in that, include: Memory and processor The memory stores computer programs; The processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, the processor performs the environmental hazard diagnosis method for the substation room as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the environmental hazard diagnosis method for the substation room as described in any one of claims 1 to 8.

Images